###### Abstract  
In many medical and pharmaceutical processes, continuous hygiene monitoring relies on manual detection of microorganisms in agar dishes by skilled personnel. While deep learning offers the potential for automating this task, it often faces limitations due to insufficient training data, a common issue in colony detection. To address this, we propose a simple yet efficient SAM-based pipeline for Copy-Paste data augmentation to enhance detection performance, even with limited data. This paper explores a method where annotated microbial colonies from real images were copied and pasted into empty agar dish images to create new synthetic samples. These new samples inherited the annotations of the colonies inserted into them so that no further labeling was required. The resulting synthetic datasets were used to train a YOLOv8 detection model, which was then fine-tuned on just 10 to 1000 real images. The best fine-tuned model, trained on only 1000 real images, achieved an mAP of 60.6, while a base model trained on 5241 real images achieved 64.9. Although far fewer real images were used, the fine-tuned model performed comparably well, demonstrating the effectiveness of the SAM-based Copy-Paste augmentation. This approach matches or even exceeds the performance of the current state of the art in synthetic data generation in colony detection and can be expanded to include more microbial species and agar dishes. color=yellow!10, inner=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer=, outer= outer=, outer=, outer=, outer= outer=, outer=, outer= outer=,  
the detection process. Several open-source tools [1, 2] use classical computer vision methods, such as image filters and intensity-based variations, to distinguish colonies from the agar medium. However, these approaches rely heavily on manually crafted features, making them labor-intensive and difficult to use. Neural networks, such as Faster R-CNN and Cascade R-CNN [3, 4], offer a more promising solution for automated colony detection due to their higher accuracy and robustness compared to traditional methods. More recently, transformer networks [5, 6] have surpassed convolutional models in various computer vision tasks, including colony detection [7, 8]. For colony segmentation, techniques such as U-Net [9] and Mask R-CNN [10] are also used. While these deep learning models offer considerable advantages, they require large amounts of training data to reach state-of-the-art performance. Despite their potential, the lack of sufficient training data poses a significant challenge in implementing deep learning for colony detection. One of the biggest obstacles to collecting additional training data is the high cost of expert personnel required for accurate data labeling, which is considerably more expensive than in other fields like autonomous driving. Another complication arises when new bacterial contaminants emerge that were not present during the network's initial training, posing unknown challenges for the model. To overcome the shortage of training data, traditional data augmentation techniques, such as image manipulation [11], are frequently employed. Additionally, generative models like GANs [12, 13] are used to produce synthetic images that enlarge the dataset. However, the generation of synthetic data can sometimes introduce artifacts into the artificial images, which may affect the performance of the subsequent trained detection model. In this work, we propose using a SAM-based Copy-Paste technique [14] as a straightforward yet effective data augmentation method for colony detection. The method is based on real colonies that are cut out of existing images using the Segment Anything Model (SAM) [15] and inserted into new, empty agar dish images.