# Microbial Colony Detection Based on Deep Learning
## 3 Materials and Methods
### Swin Transformer  
The pipeline of the backbone is depicted in Figure 2a, illustrating the base version (Swin-B). To handle 2D images, we restructure the image \(x\in R^{H\times W\times C}\) into a series of flattened 2D patches, where \(H\) stands for the image's height, \(W\) stands for its width, and \(C\) stands for its number of channels. Figure 1: The pipeline of microbial dataset preparation for deep learning analysis. In the top panel, the dataset contains different kinds of microbial media plate and blank plate images; the colony images are extracted from the plate images and subjected to style transfer to generate various microbial patch images along with their corresponding segmentation masks. In the bottom panel, the generated microbial image dataset is used to achieve the object detection effect of the colonies using different deep learning models (the figure shows the model training process of Cascade Mask R-CNN with Swin-B). The image is separated into distinct, non-overlapping patches via a patch-splitting module before being sent to the backbone. Each patch is used as a \"token\", and its feature is created by joining the RGB values of its individual raw pixels. This feature is used as a linear embedding layer to input the rawest feature information into the network. In our implementation, the Swin Transformer network consists of four stages, and the feature maps extracted in different stages are of varying sizes, capturing different levels of feature granularity at the semantic scale. Each stage consists of patch merging and Swin Transformer blocks. Patch merging works similarly to the pooling layer of the down-sampling process of a CNN. The architecture processes the input image by treating each patch as an individual \"token\", as illustrated in Figure 2a. A set patch size of \(4\times 4\) is initially selected. A linear embedding layer uses the feature map's eigenvalues to project them to the C dimension. These tokenized patches are then subjected to the Swin Transformer block. This set of procedures is known as \"Stage 1\". The output feature map that is produced is \(\frac{H}{4}\times\frac{W}{4}\) in size, which is a down-sampling of four. Patch merging layers reduce the number of tokens as the network becomes deeper to create a hierarchical representation. Before a linear layer is applied to the 4C-dimensional concatenated features, features from each set of \(2\times 2\) adjacent patches are concatenated in the first patch merge layer. As a result, the output dimension is set to 2C, and the number of tokens is decreased by a factor of 4 (equal to two times the resolution being down-sampled). After that, Swin Transformer blocks are used to modify the features while keeping the resolution at \(\frac{H}{8}\times\frac{W}{8}\). The first phase of feature transformation and patch merging is known as \"Stage 2\". For \"Stage 3\" and \"Stage 4\", the procedure is then performed again, producing output resolutions of \(\frac{H}{16}\times\frac{W}{16}\) and \(\frac{H}{32}\times\frac{W}{32}\), respectively. Together, these steps produce a hierarchical representation with feature map resolutions comparable to those of ordinary convolutional networks. Figure 2a illustrates the architecture of Swin-Base (Swin-B), featuring 128 channels in the first hidden layer and a total of 18 layers in Stage 3. In Section 4, various model sizes are employed to  
Figure 2: (**a**) The architecture of a Swin Transformer. (**b**) The successive Swin Transformer Blocks (notation presented with Equations (4)–(7)). W-MSA and SW-MSA are Multi-Head Self-Attention for replacing conventional MSA structures. The symbols between blocks represent plus. delve into the impact of the backbone's computational complexity. Among these, Swin-Tiny (Swin-T) and Swin-Small (Swin-S) comprise 6 and 18 layers in Stage 3, respectively, with the initial hidden layer containing 96 channels. This adjustment results in variations in their respective theoretical computational complexities (Flops).