# Deep-Learning Multiscale Digital Holographic Intensity and Phase Reconstruction
## 2 Principles and Methods
### Mimo-Net Network Structure  
Deep-learning-based hologram reconstruction methods have received widespread attention [27; 28; 29], among which the Y-Net is a particularly significant class [30]. The Mimo  
Figure 1: Digital hologram system model. Net proposed in this paper consists of the Y-Net, a multiscale feature-extraction module, and multiscale reconstruction fusion. The network structure is symmetric and can satisfy different requirements. As shown in Figure 2, the Mimo-Net consists of one down-sampling path, two up-sampling paths (one for intensity and one for phase), and skip links. (1) The down-sampling path consists of a multiscale feature-extraction module and a max-pooling layer, which can gradually reduce the size of the feature maps while increasing the level of feature abstraction. (2) Multiscale feature-extraction module. As shown in Figure 3, it consists of three parallel convolutional layers and cascades of small convolutional kernels for feature extraction. Figure 2: Mimo-Net architecture. There are three inputs on the left and six outputs (intensity and phase) on the right. The pink arrow represents the \(3\times 3\) convolution, the purple arrow represents the double \(3\times 3\) convolution, and the pink arrow represents the triple \(3\times 3\) convolution. Each block has the number of channels and the size of the feature map. Three feature maps are extracted using single \(3\times 3\) convolution, double \(3\times 3\) convolutions, and triple \(3\times 3\) convolutions. Batch normalization (BN) [31] and activation function [32] are applied to reduce internal covariate shift and speed up network training. Choosing Leaky Rectified Linear Unit (LReLU) as the activation function can reduce the oscillation and better fit the model. Finally, the feature maps from the three branches are concatenated. In the convolution layer, the series of two \(3\times 3\) convolution layers is equivalent to one \(5\times 5\) convolution layer, the series of three \(3\times 3\) convolution layers is equivalent to one \(7\times 7\) convolution layer, and the receptive field size of three \(3\times 3\) convolution layers is equivalent to one \(7\times 7\) convolution layer. However, the number of parameters of three \(3\times 3\) convolution layers is only about half that of \(7\times 7\), and the former can have three nonlinear operations, while the latter has only one nonlinear operation, which makes the former have stronger feature learning ability [33]. The multiscale feature-extraction module deepens the model with increased non-linearity and improved learning capacity. This results in a more extensive representation space and more vital feature-extraction capability, efficiently extracting features of different sizes in holograms. (3) Multiscale reconstruction. In the leading hologram feature-extraction network, three input branches are sequentially constructed to extract multiscale features and perform feature fusion, as shown in Figure 3. In this network, holograms of three different scales are selected: \(256\times 256\), \(128\times 128\), and \(64\times 64\). The \(256\times 256\) scale hologram is used as the input for multiscale feature extraction in the leading network. After the maximum pooling layer with a kernel size of \(2\times 2\), the scale is reduced by half. Another branch is added, with the same input channels, where the \(128\times 128\) hologram is merged with the down-sampled image for further multiscale feature extraction and max-pooling. Another branch is added, with the same input channels, where the \(64\times 64\) hologram is merged with the down-sampled image for further down-sampling learning. Each down-sampling and corresponding up-sampling performs feature fusion. The up-sampling output undergoes multiple max-pooling and convolution operations, resulting in three outputs that simultaneously produce intensity and phase information at \(256\times 256\), \(128\times 128\), and \(64\times 64\) scales.