# A Photo Identification Framework to Prevent Copyright Infringement with Manipulations
## 3 Copyright Photo Identification Framework
### Image Rol Detection  
In this step, the image collage is divided into a set of single images and then the corresponding image Rol representing a salient object in an image is determined for every single image. To detect the single image in an image collage and objects, we adopt two separate deep detector networks. The network architecture is determined by benchmarking three state-of-the-art object detection network: Faster R-CNN [14], Yolo v3 [17], and EfficientDet [12]. Based on the benchmark presented in Table 1, our framework uses EfficientDet (D1) architecture for each detector, which outperforms the other detectors in predicting the RoIs from geometrically manipulated images. We trained these two networks with classification and localization losses. For a class \(t\) and corresponding network prediction \(p_{t}\in[0,1]\) with ground truth \(p^{*}\), the classification loss is defined by the alpha balanced focal loss [44] which is defined as weighted binary cross-entropy,  
\[L_{cls}(p_{t},p^{*})=-\big{\{}\alpha(1-p_{t})^{\gamma}p^{*}\log{(p_{t})}+(1- \alpha)p_{t}^{\gamma}(1-p^{*})\log{(1-p_{t})}\big{\}}, \tag{1}\]  
where \(\gamma\in[0,5]\) and \(\alpha\in[0,1]\) are the focusing weight for modulating easy/hard examples and balancing weight for positive/negative examples, respectively. The localization loss is smooth L1 loss [14] with predicted bounding box \(t^{u}=\{t_{x},t_{y},t_{w},t_{h}\}\) and ground truth bounding box \(t^{*}\),  
\[L_{loc}(t^{u},t^{*})=\sum_{i\in\{x,y,w,h\}}\text{smooth}_{L_{1}}(t_{i}^{u}-t^{ *}), \tag{2}\]  
where \(\text{smooth}_{L_{1}}(x)\) is defined as  
\[\text{smooth}_{L_{1}}(x)= \begin{cases}0.5x^{2}&\text{if }|x|<1,\\ |x|-0.5&\text{otherwise.}\end{cases}  \tag{3}\]  
The total loss function is expressed as a weighted summation of the classification loss and localization loss. Since the image detection is less dependent on the detection label, we put a larger weight on localization loss for the image detector while the key object detector is trained on evenly weighted losses. Since there are many publicly available datasets for object detection such as MS-COCO [45], the object detector can be trained by using a public dataset. However, for the best of our knowledge, there is no dataset for detecting the image collage. Thus, we develop a data generation method to create collage images by using pristine image data from public image dataset. For a rectangular frame with a predefined resolution, the proposed method recursively partitions the frame and inserts the pristine images into each subframe. To avoid an unnatural image resizing, an image to be inserted into the subframe is selected based on the similarity of aspect ratio between them. Before insertion, the selected image \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Detector** & \multicolumn{2}{c}{**Average IoU**} & \multicolumn{1}{c}{**Avg. Time (s)**} \\ \hline \hline Yolo v3 [17] & 0.75 & 0.02 \\ \hline Faster R-CNN [14] & ResNet50 & 0.76 & 0.3 \\ without FPN [19] & ResNet101 & 0.86 & 0.3 \\ \hline Faster R-CNN [14] & ResNet50 & 0.82 & 0.17 \\ with FPN [19] & ResNet101 & 0.83 & 0.17 \\ \hline  & D0 & 0.92 & 0.06 \\ EfficientDet [12] & D1 & **0.98** & **0.08** \\  & D2 & 0.98 & 0.11 \\  & D3 & 0.99 & 0.15 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Image detection performance with image collage inputs applied to object detectors. is center-cropped to make it fit the subframe. From an empty frame with a predefined resolution, the recursive partitioning is performed on horizontal or vertical axes.