# Optimizing Contrastive Learning with Semi-Online Triplet Mining
## 6 Results  
Regarding the main dataset, we observed a change in mean pairing accuracy from 98.95% to 99.17%: a minor but statistically significant (\(p\) = 0.0004) improvement. This is a reduction in the pairing error of 20.95%. Cleansing the main dataset in conjunction with our method resulted in a mean pairing accuracy of 99.35% (\(p\) = 0.00009) and a further drop in pairing error of 21.68%. The cumulative error reduction is 38.09%, which is relevant in business terms. Figure 7 shows the results of the proposed method applied to the main dataset. Experiments on the synthetic dataset demonstrate similar trends. As the task was to pair the test dataset with highly similar examples (the generation order was preserved), the task was highly challenging. Regular triplet learning with online (semi-)hard mining is trained on a shuffled training dataset; therefore, on average, only a low number of similar identities end up in each mini-batch, which renders the mining phase ineffective. Despite the initial shuffling of the dataset, our method managed to reorder it so that many more similar identities ended up in the same mini-batch, which enabled the hard-triplet mining step to build harder triplets. The third approach of running triplet learning without shuffling the dataset, therefore feeding it to training in its original order, ideally ordered by similarity, is possible only because the dataset is generated. In the real world, a training  
Figure 6: Performance of different reordering schedules as well as vanilla online (semi-)hard-triplet mining (shuffled). The more frequent the reordering is, the fewer epochs it takes to converge. Despite a lower epoch count, the training time tends to be longer due to growing reordering overhead. dataset does not come ordered. In most cases, our method yields results similar to those of that ideal but artificial control case and better than the random order in every case. Our experiment on the generated dataset demonstrates the importance of hard triplets clearly. In this setup, both the training and test sets are built from consecutive, therefore similar (difficult to distinguish) identities. During training, the dataset gets shuffled and batched. This process results in random, nonsimilar identities getting batched together. Triplet mining working at the batch level is unable to mine enough hard triplets for training to succeed (this explains the poor result for the \"shuffled\" case presented in Figure 8). Because the identities in the batch are so easily distinguishable, the network has no incentive to include fine-grained features in the resulting embedding--instead settling for more coarse-grained features, as they are enough on average for training on mini-batches. Examples of the high-level features in this case might be overall (average) color or brightness instead of more complex features that consider colors and shapes simultaneously. This can be overcome by the use of a larger mini-batch (with bigger samples, there is a greater probability of selecting similar identities) or using the proposed method. The advantage of our method is that it requires no additional GPU memory. The results of our second experiment concerning the distribution of distances from the closest example (with a different identity) within a mini-batch align with our predictions. Our method yields distributions shifted toward lower distances, thus including many more hard negatives for online (semi-)hard-triplet mining to find. Figure 9 suggests that the difference is more pronounced for smaller mini-batches (left column) and larger datasets (bottom row). In a small-mini-batch scenario without reordering, there is little probability of similar-but-different identities meeting. A larger dataset simply means a higher chance of hard-negative existing (globally)--and when it does, the proposed method capitalizes on that fact by increasing the probability of it falling into a proper mini-batch.