# Evaluating Image DNA Techniques for Filtering Unauthorized Content in Large-Scale Social Platforms
## 4 Experiment Results
For the bitmap-based method, certain equivalent images fail to be detected even at thresholds close to zero, indicating that its ability to distinguish image features is weaker compared to image processing-based methods. However, the TDR decreases more gradually compared to other techniques, resulting in a broader effective threshold range. Notably, when the DNA resolution is high, the gap between the TDR and FDR widens, making it easier to determine an optimal threshold. Figure 9 shows the trends in the True Detection Ratio (TDR) and False Detection Ratio (FDR) for the image processing-based methods SIFT and ORB. Both methods used the Matched Vector Ratio (MVR) to determine thresholds, and the results indicate that FDR remained exceptionally low across all threshold values. This is because image processing-based methods generate hundreds of feature vectors per DNA, and the likelihood of feature vectors from non-equivalent DNA matching those of transformed DNA is extremely rare. However, compared to bitmap-based methods, the TDR for image processing-based methods decreases more sharply as the threshold increases, making it relatively more challenging to determine an optimal threshold for deployment. Figure 10 depicts the TDR and FDR for deep learning-based methods. The thresholds on the \(x\)-axis are based on L2 distances. Among the methods, Autoencoder performed well in detecting the 20 transformed images it was specifically trained on. However, overall TDR was lower than that of other approaches, and the lightweight MobileNet model exhibited a higher FDR than TDR, making it impractical for real-world use. Although VGG achieved higher detection rates compared to MobileNet, its performance was inadequate for practical equivalence detection. This limitation is attributed to its dependence on class-probability  
Figure 8: Detection rates of bitmap-based Nabla methods. Figure 9: Detection rates of image processing-based methods. vectors, which are specifically designed for image classification tasks. These results indicate that deep learning models have significant room for improvement in addressing the image equivalence detection problem, particularly for untrained arbitrary images, by shifting the focus from similarity detection to equivalence detection. The poor performance of deep learning approaches can largely be attributed to the architecture itself. While current deep learning architectures demonstrate strong performance in terms of \"image similarity\", they are less effective in determining \"image equivalence\". If a model were trained on a sufficiently diverse dataset of entirely distinct images, it could establish an appropriate threshold while maintaining a low false negative rate. However, training on a limited dataset may perform well in image classification problems but remains less effective in distinguishing whether an image is a derivative of another. Figure 10 clearly illustrates that a well-trained Autoencoder, which learns a compact representation of the data, achieves a higher detection rate than classification-based models such as MobileNet or VGG. Figure 11 illustrates the maximum achievable TDR for each method across the 20 types of image transformations in Table 2. As shown in the figure, image processing-based methods achieved the highest detection rates for all image cases we considered. In contrast, low-resolution Nabla (x4d4x8) and the lightweight MobileNet network achieved TDRs below 20%, making them unsuitable for filtering systems. However, high-resolution Nabla (x8d8x16) achieved a TDR of 80%, demonstrating the potential of bitmap-based methods despite their simplicity. Figure 12 compares the minimum FDR across 161,062 non-equivalent images. As shown, the image processing-based ORB method exhibited the lowest FDR among all techniques. The extraction of hundreds of 128-dimensional feature vectors in image processing  
Figure 11: Maximum true detection rates for 20 types of COCO-transformed images. Figure 10: Detection rates of deep learning-based methods.