Authors: Fredrik Skärberg
Source: Gothenburg University Publications Electronic Archive (Gothenburg University) (article, 2020-11-26)
Field: Materials Science — Surfaces, Coatings and Films

Convolutional neural networks for semantic segmentation of FIB-SEM volumetric image data Focused ion beam scanning electron microscopy (FIB-SEM) is a well-established microscopy\ntechnique for 3D imaging of porous materials. We investigate three porous\nsamples of ethyl cellulose microporous films made from ethyl cellulose and hydroxypropyl\ncellulose (EC/HPC) polymer blends. These types of polymer blends are used\nas coating materials on various pharmaceutical tablets or pellets and form a continuous\nnetwork of pores in the film. Understanding the microstructures of these porous\nnetworks allow for controlling drug release. We perform semantic segmentation of the\nimage data, separating the solid parts of the material from the pores to accurately\nquantify the microstructures in terms of porosity. Segmentation of FIB-SEM data is\ncomplicated because in each 2D slice there is 2.5D information, due to parts of deeper\nunderlying cross-sections shining through in porous areas. The supposed shine-through\neffect greatly complicates the segmentation in regards to two factors; uncertainty in\nthe positioning of the microstructural features and overlapping grayscale intensities\nbetween pore and solid regions.\nIn this work, we explore different convolutional neural networks (CNNs) for pixelwise\nclassification of FIB-SEM data, where the class of each pixel is predicted using a\nthree-dimensional neighborhood of size (nx; ny; nz). In total, we investigate six types\nof CNN architectures with different hyperparameters, dimensionalities, and inputs.\nFor assessing the classification performance we consider the mean intersection over\nunion (mIoU), also called Jaccard index. All the investigated CNNs are well suited\nto the problem and perform good segmentations of the FIB-SEM data. The so-called\nstandard 2DCNN performs the best overall followed by different varieties of 2D and\n3D CNN architectures. The best performing models utilize larger neighborhoods, and\nthere is a clear trend that larger neighborhoods boost performance. Our proposed\nmethod improves results on all metrics by 1.35 - 3.14 % compared to a previously\ndeveloped method for the same data using Gaussian scale-space features and a random\nforest classifier. The porosities for the three HPC samples are estimated to 20.34,\n33.51, and 45.75 %, which is in close agreement with the expected porosities of 22,\n30, and 45 %. Interesting future work would be to let multiple experts segment the\nsame image to obtain more accurate ground truths, to investigate loss functions that\nbetter correlate with the porosity, and to consider other neighborhood sizes. Ensemble\nlearning methods could potentially boost results even further, by utilizing multiple\nCNNs and/or other machine learning models together. Convolutional neural network ; Artificial intelligence ; Segmentation ; Computer science ; Pattern recognition (psychology) ; Semantics (computer science) ; Image (mathematics) ; Image segmentation ; Computer vision ; Natural language processing Electron and X-Ray Spectroscopy Techniques ; Industrial Vision Systems and Defect Detection ; Non-Destructive Testing Techniques