Video frame interpolation neural network for FIB-SEM tomography

Abstract text (incl. figure legends and references)

Introduction

Focused-Ion-Beam Scanning-Electron-Microscope (FIB-SEM) tomography is a destructive imaging technique useful to investigate the 3D internal structure, and therefore the properties, of various materials composites. This technology is often employed for the study of nanomaterials, such as graphene nanosheet networks, whose mechanical and electrical properties are highly dependent on the material morphology. Through serial milling and imaging hundreds of network cross-sections, a high-fidelity 3D reconstruction of a nanostructured network can be produced and analysed [1].

The typical slice thickness is about 10-20 nm, while the resolution in the cross section is about 5 nm. It would be advisable to have the same resolution along the milling direction, this will result in a 3D reconstruction made of cubic voxels, which facilitate the studies of certain morphological properties. However, most imaging systems do not allow to produce thinner slices of the material and the process would be more expensive. In addition thinner slices have the disadvantage that the damage produced during one cut can propagate down to the next one.

Objectives

The solution proposed in this work [2] is to use a neural network developed for video frame interpolation, namely RIFE [3], to generate the missing slices between the collected images, and therefore obtain cubic voxels.

Materials and methods

The first application of the model was performed on a dataset made of graphene nanosheet networks, with pixel size of 5 nm and slice thickness of 17 nm.

Along with computer vision metrics, such as mean squared error (MSE) and structural similarity index (SSIM), the evaluation of the network porosity in each frame was used to quantitatively compare the results to the ground truth and a simple linear interpolation method.

Results

The results show that the proposed model successfully generates the missing frames and outperforms the linear interpolation method on the graphene dataset. The application was extended to datasets of materials of a different scale generated using various imaging techniques, such as micro CT scans and serial block face SEM.

Conclusion

This work demonstrates that the resolution along the cut direction of FIB-SEM data, or other imaging techniques, can be improved by employing a neural network developed for video frame interpolation. Thanks to this model, it is possible to obtain cubic voxels in the 3D reconstruction despite the acquisition of thick slices, which facilitates the imaging process.

[1] Gabbett, C. Electrical, Mechanical & Morphological Characterisation of Nanosheet Networks. Diss. Trinity College Dublin, 2021.

[2] Gambini, L., Gabbett, C., Coleman, J. N., Jones, L., Sanvito S. Video frame interpolation Neural Network for FIB-SEM tomography. In preparation.

[3] Huang Z., Zhang T., Heng W., Shi B., Zhou S. Real-time intermediate flow estimation for video frame interpolation. In: Proceedings of the European Conference on Computer Vision (ECCV), 2022.