SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Correspondence on diverse internet images. Same as Fig. 1, in Figs. 15 and 16 we show more correspondence prediction on various image groups that share similar semantics. For each target image, the  $\mathrm{DIFT}_{sd}$  predicted point will be displayed as a red circle, together with a heatmap showing the per-pixel cosine distance calculated using  $\mathrm{DIFT}_{sd}$ . We can see it works well across instances, categories, and even image domains, e.g., from an umbrella photo to an umbrella logo.
Semantic correspondence comparison among off-the-shelf features on SPair-71k. Same as Fig. 3, we show more comparison in Fig. 17, where we can see DIFT works well under challenging occlusion, viewpoint change and intra-class appearance variation.
Same Instance
Cross Instance
Cross Category
Figure 14: Visualization of the first three PCA components of  $\mathrm{DIFT}_{sd}$  on the segmented instance pairs (same instance, cross instance, cross category). Each component matches a color channel. We can see the same object parts share similar DIFT embeddings.
Cross-category semantic correspondence. Same as Fig. 4, in Fig. 18 we select an interesting image patch from a random source image and query the image patches with the nearest  $\mathrm{DIFT}_{sd}$  features in the rest of the test split but with different categories. We see that DIFT is able to identify reasonable correspondence across various categories.
Failure Cases on SPair-71k. In Fig. 19, we select four examples with low PCK accuracy and visualize  $\mathrm{DIFT}_{sd}$  's predictions along with ground-truths. We can see that, when the semantic definition of key points is ambiguous, or the appearance/viewpoint change between source and target images is too dramatic, DIFT fails to give correct predictions.
Image editing propagation. Similar to Fig. 6, Fig. 20 shows more examples on edit propagation using our proposed  $\mathrm{DIFT}_{sd}$ . It further demonstrates the effectiveness of DIFT on finding semantic correspondence, even when source image and target image are from different categories or domains.
Geometric correspondence. Same as Fig. 7, in Fig. 21 we show the sparse feature matching results using  $\mathrm{DIFT}_{sd}$  on HPatches. Though not trained using any explicit geometry supervision, DIFT still works well under large viewpoint change and challenging illumination change.
Temporal correspondence. Similar to Fig. 8, Fig. 22 presents additional examples of video instance segmentation results on DAVIS-2017, comparing DINO,  $\mathrm{DIFT}_{adm}$  and Ground-truth (GT). We can see  $\mathrm{DIFT}_{adm}$  could create instance masks that closely follow the silhouette of instances.
$\mathrm{DIFTS_d}$  Predicted Target Points
Figure 15: DIFT can find correspondences on real images across instances, categories, and even domains, e.g., from a photo of statue of liberty to a logo.
Source Point
$\mathrm{DIFT}_{sd}$  Predicted Target Points
Figure 16: DIFT can find correspondences on real images across instances, categories, and even domains, e.g., from a photo of an aeroplane to a sketch.
Figure 17: Semantic correspondence using various off-the-shelf features on SPair-71k. Circles indicate correct predictions while crosses for incorrect ones.
Figure 18: Given the image patch specified in the leftmost image (red dot), we use  $\mathrm{DIFT}_{sd}$  to query the top-5 nearest image patches from different categories in the SPair-71k test set. DIFT is still able to find correct correspondence for object parts with different overall appearance but sharing the same semantic meaning, e.g., the leg of a bird vs. the leg of a dog.
Figure 19: Failure cases of semantic correspondence on SPair-71k. Circle denotes correct predictions while cross for wrong ones. When the semantic definition of key points is ambiguous, or the appearance/viewpoint change between source and target images is too dramatic,  $\mathrm{DIFT}_{sd}$  fails to predict the correct corresponding points.
Figure 20: Edit propagation using  $\mathrm{DIFT}_{sd}$ . Far left column: edited source images. Right columns: target images with the propagated edits. Note that despite the large domain gap in the last row,  $\mathrm{DIFT}_{sd}$  still manages to establish reliable correspondences for correct propagation.
Figure 21: Sparse feature matching using  $\mathrm{DIFT}_{sd}$  on HPatches after removing outliers. Left are image pairs under viewpoint change, and right are ones under illumination change. Although never trained with geometric correspondence labels, it works well under both challenging changes.
Figure 22: Additional video label propagation results on DAVIS-2017. Colors indicate segmentation masks for different instances. Blue rectangles show the first frames. GT is short for "Ground-Truth".