# Detecting Multiple Mixed Bacteria Using Dual-Mode Hyperspectral Imaging and Deep Neural Networks
## 2 Materials and Methods
### Deep Learning Architecture  
Aiming at dual-mode spectral features (transmission spectra and fluorescence spectra), we customized a mixed bacteria measurement network (MB-Net) with multiple DL architecture based on spectral feature fusion. The detailed framework of MB-Net is shown in Figure 3. Two convolutional neural network (CNN) architectures with different optimized structure parameters were selected to process the transmission/fluorescence spectral images, respectively. Before model training, image cropping and down sampling were performed to reduce data redundancy and prevent overfitting. The input images were transformed into a form with a spatial dimension of 10 and a spectral dimension of 180. The spectra data were input into a convolution layer with parameters of (1, 8, 2 \(\times\) 5), where 1 is the number of input channel, 8 is the number of output channel, and 2 \(\times\) 5 is the size of the convolution kernel. The strides of the all convolution layers were set to 1. Different from conventional CNNs, here, we utilized the rectangular convolution kernel for feature extraction. In the hyperspectral dimension, the size of the convolution kernel is larger, which can help extract more comprehensive spectral features. Regarding spatial dimension, spatial information is not the primary focus of the network. Therefore, only small-scale convolution kernels are used here to perform basic filtering operations on spatial data. The utilization of rectangular convolution kernels enables targeted processing of information across different dimensions, extracting pertinent features and reducing inference from redundant information. This leads to improved operational efficiency. After convolutional operation, max pooling operation was employed to reduce the spatial dimensionality of an input volume by selecting the maximum value within each region, helping to extract the most dominant features. The parameters of the max pooling layer are (4 \(\times\) 5, 1, 1), where 4 \(\times\) 5 is the kernel size, the middle 1 is the horizontal stride, and the last 1 is the vertical stride. The input data were transformed into a new data after processing by the two layers, with dimension of 6\(\times\)176 and channel numbers of 8, marked as (6, 176, 8) in Figure 3.  
After a series of convolutional and max pooling operations, the dimensions of feature maps output by the final module were 1 \(\times\) 27, with 128 output channels. The feature maps were processed by a dropout layer in order to alleviate overfitting, and were flattened into a vector. To analyze transmission and fluorescence simultaneously, we employed a feature fusion strategy. The outputs of the two CNN models (one is for the process of transmission hyperspectral images, and the other is for the process of fluorescence  
Figure 3: The framework of MB-Net.  
hyperspectral images) were directly concatenated together, forming a dense layer with a total of 6528 nodes, where the latent transmission spectral features and fluorescence spectral features each contribute equally. In addition, another fully connected dense layer consisting of 6528 nodes was employed for the purpose of feature sorting. This was then followed by a fully connected layer comprising four nodes, outputting the regression results corresponding to concentration ratio of four types of bacteria.  
The MB-Net model was trained in the computer equipped with GTX4090 GPU (NVIDIA, Santa Clara, CA, USA), I9-13,900K CPU (Intel, Santa Clara, CA, USA), and 32 GB of memory. To prevent the MB-Net from overfitting, so as to validate its performance, we randomly divided the datasets into training set and validation set with ratios 0.80/0.20. The activation function is ReLu function, training with Adam optimizer. The loss function is defined as the mean square error (MSE) between the network output and the sample label. The training goal of the MB-Net is to minimize the MSE.