# Microbial Colony Detection Based on Deep Learning
## 1 Introduction  
In clinical microbiology laboratories, we often extract secretions or blood from patients and perform bacterial cultures. The treatment is based on the results of the drug sensitivity tests [1] of the isolated pathogenic organisms. In this process, we usually use strains collected from different patients, isolate and culture them in agar or other plates, and then manually morphologically identify the colonies after they are formed and select independent, uncontaminated colonies to configure samples at appropriate concentrations for drug sensitivity tests. However, the entire process of sample collection, incubation, and final plate evaluation, including microbial colony identification, requires the full participation of the laboratory staff. The manual identification of colony types in plates [2] requires not only the experience of the laboratory personnel, but also a lot of time. Therefore, we hope to solve this problem of microbial plate identification with the help of the latest Artificial Intelligence (AI) technology. In the last few years, picture recognition has benefited greatly from data-based AI training techniques. Since the advent of AlexNet [3], Convolutional Neural Networks (CNNs) [4; 5; 6] have been the standard architecture for Computer Vision (CV) for a long time. This is followed by CNN architecture models such as VGGNet [7], Resnet [8], and EfficientNet [9]. Although CNN models play well in feature information extraction, these models have difficulties in learning long-range and global semantic data. Compared with the traditional CNN models, the recognition accuracy of the Transformer [10; 11], which introduces a self-attentive mechanism [12], is higher. It was originally applied in Natural Language Processing (NLP) [13] to solve the sequential structure of language. Subsequently, the application of the Transformer [14; 15] was extended to tasks in thefield of image and video. Vision Transformers (ViTs) [16] split an image into patches and process them sequentially by flattening or rearranging them into a sequence, which is then used as an input for an encoder-decoder architecture. Utilizing the patch sequence's extracted image feature data, the object of the input image can be detected and localized. However, ViTs have difficulty dealing with long sequence data, which requires significant computational power. A Shifted Window (Swin) Transformer [17; 18] introduces a localized attention mechanism that limits attention to patches, reduces computational complexity, and extends the sensory field of the model by window shifting to better capture contextual information. Therefore, we apply it as a backbone to the Cascade Mask R-CNN [19] model for our microbial colony object detection task. When it comes to microbial image recognition [20], the first problem is figuring out how to obtain a high-resolution image dataset for model training. The Annotated Germs for Automated Recognition (AGAR) dataset [21], comprising over 18k annotated Petri dish pictures with five microbiological groups, serves as the raw material for our experiments. We use 100 images provided by the authors in [21] from the higher-resolution AGAR subset and employ style transfer [22] as a data augmentation technique, resulting in 4k additional images, each with a resolution of 512 \(\times\) 512 pixels. To ensure that style transfer does not produce undesirable image labels, we constrain the number of iterations and adjust the hyperparameters. Additionally, we conduct manual evaluations of the generated results. These images are used to train deep learning models for object detection. In this work, our main contributions are as follows:  
1. We propose using style transfer to augment the microbial colony dataset, addressing challenges in acquisition due to contamination and unclear images, and reducing the manual effort required for handling a large number of colonies in the images. 2.