# A Photo Identification Framework to Prevent Copyright Infringement with Manipulations
## 3 Copyright Photo Identification Framework
### Image Hashing  
SimSiam [35], a self-supervised scheme for Siamese networks, is employed for _Image Hashing_. The image hashing network is composed of Backbone, Projection MLP, and Prediction MLP as shown in Figure 7. The original photo \(\mathbf{I}_{o}\) is augmented randomly and sampled into \(\mathbf{I}_{1}^{{}^{\prime}}\) and \(\mathbf{I}_{2}^{{}^{\prime}}\). This augmentation includes geometric manipulations such as random aspect ratio and random cropping and color manipulations such as color distortion, grayscale, and Gaussian blur. This augmentation helps to learn common feature representation between the augmented image inputs by generating diverse pixel distributions for the same image. Without this augmentation, similar pixel distributions can cause overfitting and degrade the performance of the network. The backbone network is pretrained to utilize generalized features by ImageNet, which can help to prevent overfitting with fewer dataset images. The weights of the Projection MLP and the Prediction MLP are initialized randomly, and trained to minimize negative cosine similarity loss by fitting the copyright photos. The negative cosine similarity \(D\) is defined as:  
\[D(p_{1},z_{2})=-\frac{p_{1}}{\left\|p_{1}\right\|_{2}}\cdot\frac{z_{2}}{\left\| z_{2}\right\|_{2}^{2}}, \tag{4}\]  
where \(p\) and \(z\) are the outputs of the Prediction MLP and the Projection MLP, respectively. Since there is no negative sample, the image hashing network collapses with \(D\) and all the  
Figure 6: Example of generating an image collage frame with recursive partitioning with \(L=2\). In this example, the partitioning occurs in the node notated in the red circle (\(\mathbb{1}\), \(\mathbb{2}\), and \(\mathbb{3}\) ) while the others do not. In this example, the partition axis for \(\mathbb{1}\), \(\mathbb{2}\), and \(\mathbb{3}\) are horizontal, vertical, and horizontal, respectively. For each partition level, the partitioning would not occur when it is in the maximum level of partition (after \(\mathbb{3}\)) or the decision of partition occurrence is false (in \(\mathbb{3}\)).  
Figure 7: Image hashing network, where MLP is a Multi-Layer Perceptron.  
outputs have same value. To prevent the collapse, the _stopGrad_ operation is applied to \(z\) and \(D\) is defined again as:  
\[D(p_{1},stopGrad(z_{2})), \tag{5}\]  
The _stopGrad_ operation prevents the gradients of network weights from being updated and the network is trained to be closer to the constant tensor _stopGrad_(\(z\)). For symmetry, negative cosine similarity loss \(L_{ncs}\) is defined as:  
\[L_{ncs}=\frac{1}{2}(D(p_{2},stopGrad(z_{1}))+D(p_{1},stopGrad(z_{2}))), \tag{6}\]  
To generate a binary descriptor, sign function is applied to the output of the Prediction MLP and the binary descriptor \(b\) is defined as:  
\[b=\frac{1}{2}(sign(p)+1), \tag{7}\]  
The binary descriptor can search the database very quickly by using bit operations. The index of the closest photo to the query is defined as:  
\[i_{match}=\operatorname*{arg\,min}_{i}(b_{Q}\oplus b_{DB}^{i}), \tag{8}\]  
where \(b_{Q}\), \(b_{DB}^{i}\), and \(\oplus\) are the binary descriptor of the query, \(i^{th}\) binary descriptor of the database, and XOR operation, respectively. In order to minimize information loss due to the dimension reduction in output features from the backbone network and the binarization, the similarity preservation loss \(L_{sp}\) is defined as:  
\[L_{sp}=|\frac{1}{L}b_{1}^{T}b_{2}-D(F_{1},F_{2})|^{2}, \tag{9}\]  
where \(F_{1}\), \(F_{2}\), \(b_{1}\), and \(b_{2}\) are the output features from the backbone network and the binary descriptors, respectively. The final loss \(L\) is defined as:  
\[L=L_{ncs}+L_{sp}. \tag{10}\]  
The image hashing network is trained to minimize the final loss \(L\).