3D-reconstruction of hierarchical nanoporous structures using multi-modal deep learning models trained on synthetic multi-voltage FIB tomography data

Abstract text (incl. figure legends and references)

3D reconstruction of hierarchical nanoporous structures requires high-resolution imaging data in the range of a few nanometers. Focused ion beam (FIB) tomography can be combined with a scanning electron microscope (SEM) to collect high-resolution images of nanostructures in the range of 1 nm in the SEM plane and 10 nm in the out-of-plane direction [1]. FIB tomography collects data layer-wise, where a FIB milling process follows each imaging process by an SEM. However, these consecutive series of cross-sectional images have artefacts, such as the "shine-through effect" [2]. Due to this effect, some structures become visible from posterior regions of the currently milled plane, and it is not straightforward to map pixel intensities with the material phase uniquely. Therefore, such images cannot be segmented accurately using conventional methods like thresholding or the k-means algorithm. Earlier, [3] presented a machine learning-based approach to reconstruct hierarchical nanoporous materials accurately. [3] also demonstrated a method to generate synthetic FIB-SEM images of any complex nanoporous structures to overcome the lack of training data. In this study, we present two ways to improve 3D reconstruction results. The first approach is to use a multi-modal machine learning method to take advantage of more information from the same region. In this case, we train a machine learning model using synthetic multi-voltage FIB tomography data generated by changing the accelerating voltage and simulating images of the same regions (Figure 1). Figure 2 shows pixel intensity profiles of images simulated using different accelerating voltages and the actual structure. The second approach is to make synthetic FIB-SEM data more similar to the real electron microscopy images using machine learning-based domain adaptation techniques. We show that using these approaches, our segmentation methods can suppress the "shine-through effect" and improve upon the methods presented in [3].

[1] Knott et al., Journal of Neuroscience, 28(12):2959–2964 (2008)

[2] Prill et al., J Microsc., 250(2):77-87 (2013)

[3] Sardhara et al., Frontiers in Materials, 9 2296-8016 (2022)