# Egyptian Hieroglyphs Segmentation with Convolutional Neural Networks
## 3 Materials and Methods
### Detectron2  
The problem of multiple instances segmentation in an image by using Deep Learning approaches has been extensively studied in recent years, as evidenced in several review papers [28; 29]. For the specific task of image segmentation, many CNN architectures have been proposed, such as the popular U-Net [12], mainly devoted to semantic segmentation, Fast R-CNN and Faster R-CNN [10; 11]. Mask R-CNN [24], an extension of the Faster R-CNN, is well suited for instances segmentation.  
In this study, we used Detectron2 [25; 26], a framework developed by the Facebook AI Research Group (FAIR). One of the main reasons for choosing Detectron2 is that it represents a \"state of the art\" framework for the task of image segmentation, achieved as an evolution of previous networks. Detectron2 includes implementations for many object detection algorithms, such as the above-mentioned Mask R-CNN, Faster R-CNN and Fast R-CNN. An advantage of Detectron2 is its flexibility in integrating different networks as the backbone to be used for subsequent processing steps, e.g., classification (which, however, were not considered in this study). We decided to use Mask R-CNN as object detection algorithm in the Detectron2 framework, relying on the results coming from benchmark studies, such as [29; 30; 31; 32], which demonstrated the high performances of Mask R-CNN in comparison to other networks.  
More specifically, region-based CNNs are a family of Convolutional Neural Networks that tries to extend classical detection and recognition tasks to the case of images with multiple instances of objects of interest [10; 11]. In a nutshell, the system consists of two modules. The first module is the Region Proposal Network (RPN), which is responsible for generating category-independent _region_ proposals, i.e., areas of the input image that are the candidate to contain an object of interest. These regions are also referred to as _Regions of Interest_ (RoI). An _objectness score_ is associated with each RoI: this score quantifies the probability that an area contains an object belonging to one of the classes of interest _vs._ background. The second module classifies the proposed regions. The two modules share some convolutional layers in order to optimize calculations and rely upon a single deep CNN (the _backbone_), which is responsible for generating the feature maps. In this study, we selected as a backbone network the well-known ResNet [8]; more specifically, we used a model that was pre-trained on the COCO dataset [33] and that was finely tuned over the dataset of segmented hieroglyphs (described in the following), exploiting the _transfer learning_ paradigm. The output of the segmentation network is a set of rectangular bounding boxes, hopefully one for each instance of an object belonging to one of the classes of interest, along with a label for the predicted class. Parallel to the classification task, Mask R-CNN also produces for each RoI, in a parallel branch, a _binary mask_ that, in the ideal case, should be perfectly superimposed to the object contained in the bounding box. This binary mask is the actual outcome of the segmentation task we are interested in: it is worth noticing that this is performed independently from both bounding-box regression and classification [24].