# Development of a Prototype Solution for Reducing Soup Waste in an Institutional Canteen
## 2 Prototype
### Software Component
#### 2.3.2 ResNet-50 and Faster R-CNN Models  
CNNs, or convolutional neural networks, are specifically designed to process grid-structured data such as images and have excelled in computer vision tasks such as image classification, detection, and segmentation. Models based on CNNs are made up of convolutional, pooling, and fully connected layers, which extract and represent features from the input image [24]. In the architecture of a CNN, shown in Figure 9, the convolutional layer is responsible for extracting features from the image using trainable filters, which identify patterns such as edges, textures, and shapes. The resulting feature maps are passed through the ReLU layer, which introduces non-linearity, allowing the network to learn complex tasks [24; 25; 26]. Figure 8: Dataset split. Figure 7: Image annotation in Roboflow. Figure 9: Typical architecture of a CNN. Following on from the work in [5] by the same authors of this paper and the conclusions observed, it was decided to combine two models: ResNet-50 and Faster R-CNN. These two models combine two different approaches in the field of computer vision and deep learning for object detection. ResNet-50 is a deep neural network architecture known as a \"residual network\", designed to overcome training challenges in deep networks [27]. Faster R-CNN describes an approach that divides the image into regions and uses a CNN to identify objects in those regions. By uniting these two approaches, as illustrated in Figure 10, it is possible to take advantage of ResNet-50's ability to extract complex features from images, while Faster R-CNN handles object detection. The model was trained on the Google Colab platform [28], using a notebook available at [29]. The notebook, with Python code, was adapted to allow the dataset prepared for this purpose to be imported, using the Roboflow API [22]. The Google Colab platform offers free hardware resources, such as Graphics Processing Units (GPUs) and Tensory Processing Units (TPUs), support for various languages, and integration with Google Drive [30] and GitHub [31]. The machine provided by the platform in the free plan has the following characteristics: NVIDIA T4 graphics card with 16 GB of VRAM and 13 GB of RAM for the system. In addition to training the model, this notebook also allows validation of the model, which consists of analyzing the validation images of the dataset to assess performance. Figure 11 shows an example image of the validation results carried out directly on the notebook, showing that the model accurately identifies the location of the bowl. The model was trained several times with different versions of the dataset, and the number of training epochs was adapted to avoid overfitting. Overfitting represents a significant challenge when training CNNs. It occurs when a model is over-trained to the point where it memorizes specific details of the training data. As a result, the model performs excellently on the training data, but fails when dealing with new data [32]. The model used was trained for 19 epochs. The decision to stop training was made based on several factors. When it is found that the model's performance on the dataset is not improving, i.e., it is stagnating or getting worse, indicating an increase in errors, training is stopped [33]. In this scenario, where all the bowls are the same and therefore the images of the trays are very similar, extending training for too many epochs would not improve the model's performance and could even make it worse. Figure 10: Architecture of the ResNet-50 and Faster R-CNN models. The performance metrics used to evaluate the model were mean average precision (\(mAP\)) and loss. Average precision (\(AP\)) is a metric commonly used in the context of binary classification and information retrieval to summarize the precision-retrieval curve. It provides a single value that represents the quality of retrieval results classified to a specific class or category, especially in tasks such as object detection [34]. The calculation is carried out according to Equation (1).