Paper: Contrastive losses as generalized models of global epistasis (ICLR-2024)

Reviewer comment (Evaluation):
All of the experiments evaluate fitness prediction in terms of spearman correlation. It's not surprising that training using a BT loss improves spearman correlation vs. MSE, since the BT loss can be seen as a differentiable relaxation of the negative spearman correlation. Therefore, I didn't find this result particularly exciting. It is a common observation in machine learning that a particular downstream metric can be improved by using a loss function that approximates it, and the learning-to-rank framework described in this paper is well established.

Author response:
We agree with the reviewer that the improved performance of the BT loss over the MSE loss in terms of Spearman correlation is intuitive. However, the BT loss has not been used for fitness prediction previously and we were able to achieve improved performance on a metric that is common in the field (Spearman correlation). This is still an important contribution, even if the result is intuitive in retrospect.

Further, the improved performance in benchmark tasks of models trained with the BT loss is not the only contribution of the paper. We have demonstrated that the BT loss is an effective technique for modeling global epistasis that allows one to recover a latent fitness function. Recovering such a latent fitness function is a problem that has been approached by a number of previous papers (Sailer & Harms, 2017 and Otwinowski et al., 2018 being two prominent examples), and we have shown that our method can do so while making fewer assumptions about the form of the global epistasis nonlinearity than these previous works (which are assumed to be power transforms or 3-splines in the aforementioned papers). The task of recovering a latent fitness function is important because it removes the spurious, or “phantom”, epistatic interactions that appear in a fitness function that has been affected by global epistasis, allowing for proper interpretation of the primary interactions that drive fitness. The latent fitness function is often interesting biologically as it can be a better measure of biophysical properties when the global non-linearity is caused by an artifact of the measurement process. Effectively solving this problem is a contribution of our paper, outside of the performance on benchmark tasks. We will clarify this in the introduction and discussion of our paper.

Please see the following comments for the rest of our response to the reviewer.

Reviewer comment (Evaluation):
I don't agree with the assertion that spearman correlation is a good surrogate metric for a model's usefulness for protein engineering. Suppose that fitness values appear in the range [0, 10] and that the best fitness seen so far in a protein engineering project is 6. Since we want to use this model to design new proteins, we don't care if it predicts 3 for a protein that actually has fitness 2. On the other hand, predicting 5.5 for a protein that has true fitness 6.5 could lead to a missed opportunity for ML-guided design. Good eval metrics should think asymmetrically about precision vs. recall of finding proteins with good fitness, etc. Finally, spearman correlation is confusing because it's unclear what the optimal value is. Given the noise level of the assay, I know the optimal MSE. What's the optimal spearman correlation, though?

Author response:
We have used test set Spearman correlation as our primary metric for assessing model performance because it is commonly used in benchmarking fitness prediction methods (see the [FLIP](https://benchmark.protein.properties) and [ProteinGym](https://www.proteingym.org/home) benchmarks for two prominent examples that employ Spearman correlation as the primary metric). We will clarify this in the paper, as it is not explicitly stated as our motivation for using Spearman correlation. 

However, we agree with the reviewer that the primary concern for protein engineering is tail performance rather than bulk correlation. In order to strengthen the results of the paper, we have now done an initial comparison of the tail performance of models trained with the BT and MSE losses on the GB1 predictions tasks in the FLIP benchmark. We follow [Gelman et al., 2021](https://www.pnas.org/doi/10.1073/pnas.2104878118) and consider the recall of the models in classifying the top sequences in the test set. In particular, we calculate the top 10% recall as the fraction of the sequences in the true top 10% of sequences in the test set that are in the top 10% of model predictions on the test set. For all splits except the Sampled split, models trained with the BT loss achieve statistically significant (p < 0.05) improvements over models trained with the MSE loss. In the case of the Sampled split, the MSE loss performs slightly better, but the difference is not statistically significant. The results of this analysis are summarized in the table below, where entries in the table correspond to the mean +/- std.dev. of top 10% recall over 10 random restarts of the models:

| Split   | Bradley-Terry | MSE |
| ----------- | ----------- | -----------|
| Low-vs-High | 0.442 +/- 0.024  | 0.381 +/- 0.028 | 
| 1-vs-Rest | 0.138 +/- 0.051| 0.097 +/- 0.030 | 
| 2-vs-Rest | 0.282 +/- 0.008| 0.250 +/- 0.030 | 
| 3-vs-Rest | 0.664 +/- 0.014| 0.539 +/- 0.084 | 
| Sampled | 0.815 +/- 0.010| 0.823 +/- 0.010 |

We will add these results, as well as corresponding results for the AAV and Meltome FLIP benchmarks to the Appendix and discuss the importance of metrics such as the top 10% recall for protein engineering in the main text.

In the paper’s noiseless simulations, the optimal spearman is 1. Given a noise level of an assay, it is possible to find the optimal spearman by parametric resampling of the noise on the test set. In realistic fitness prediction scenarios, the correct noise model is unknown and complex so one cannot know either the optimal Spearman nor MSE. The reviewer’s concern of course applies to all papers that use Spearman correlation as a metric, and it is outside the scope of our paper to resolve this broader concern.

Please see the following comments for the rest of our response to the reviewer.

Reviewer comment (Theory):
The paper's goal is to fit models on experimental data to estimate a latent fitness function. However, the exposition assumes that these observations are noiseless, which is quite unrealistic. Noise is often heteroskedastic, where the noise level can be assumed to depend on g(f()). For example, the read-out from many assays is based on counts from DNA sequencing, and these are subject to Poisson noise. The BT loss strikes me as being not particularly robust to noise, since it could easily flip the binary fitness(A) > fitness(B) label if the noise is large relative to the difference between the fitness of A and B.

Author response:
Thank you for the follow-up comments. 

> In my initial review I asked for some clarification about how your proposed results on recovery of the latent fitness function extend to a practical setting where there is noise in the assay. It would be easy to add noise in your simulation environment. How does that change things?

We have now completed simulations identical to those whose results are shown in Figure 2c but with homoskedastic noise Gaussian noise added to the observed fitness function. We tested two noise settings, with standard deviations of 0.025 and 0.1. These were selected such that the ranking of sequences would be substantially affected; the Spearmans between the latent and noisy fitness function were about 0.6 and 0.4 in the stddev=0.025 and 0.1 cases, respectively. In both of these cases, the Bradley-Terry loss maintains its advantage over the MSE loss in terms of Spearman correlation at all training set sizes. For example, at a training set size of 200, the average Spearman over simulation replicates are:

| Noise std. dev.   | Bradley-Terry | MSE |
| ----------- | ----------- | -----------|
|0.025 | 0.513  |0.448 | 
| 0.1 |0.296 | 0.278 | 

In the revised manuscript, the complete curves analogous to those shown in Figure 2c for these simulations will be included in the Appendix and referenced in the Discussion section.

> In particular, noise occurs in two places. The latent fitness function is noisy (e.g., cells can be at different points in their life cycle and this affects all sorts of genotype-fitness relationships) and the experimental readout is noisy (due to NGS, masspec, etc).

The reviewer's point about the multiple layers of noise is interesting and well-taken. Unfortunately a simulation of this situation would likely contain too many parameters to yield useful conclusions, so we rely on the results on the FLIP benchmark to report on the BT losses's performance in realistic scenarios. 

> Also, I don't see a satisfactory definition of what it means to 'recover' the latent fitness function. Can you please formally define this?

The implicit definition used in Figure 1 is that the estimated fitness function is equal to the latent fitness function up to an affine transformation (hence our use of an R^2 metric in Figure 1c). In our view, this is appropriate definition because the optimal solution maintains the relative importance of interactions in the epistatic domain. We will make this clear in the revised manuscript.

Reviewer comment (Theory):
Section 3.2 on the 'fitness-epistasis uncertainty principle' is grounded in some results from the compressed sensing literature. While it provides interesting background on CS, the conclusions from the section are too informal to be useful.

Author response:
This section is meant to provide intuition and hypotheses for two observations. First, it has been observed in a number of papers that global epistasis results in the appearance of spurious or “phantom” epistatic terms (see, e.g., Sailer & Harms, 2017 and Baez-Centurion et al., 2019 for two examples). The fitness-epistasis uncertainty principle provides a mathematical explanation for this phenomenon, by demonstrating that the density of the epistatic representation must increase as the fitness domain becomes more concentrated (which is the effect of many common nonlinearities, as we show in our simulations). Second, we observe in our simulations that fitness functions affected by global epistasis require more data in order to effectively estimate with MSE loss. The uncertainty principle and the connection to the CS literature provide a potential hypothesis to explain this phenomenon; we have been careful to specify that this is only a hypothesis rather than a rigorous explanation. We believe that providing reasonable hypotheses to explain our observations is a valuable first step in moving towards a better understanding of these phenomena, even if these hypotheses are not proven in this paper.

Reviewer comment (Theory):
The paper is not sufficiently grounded in standard statistical terminology about model estimation. There is a need to discuss the identifiability of the global epistasis model and whether the change of loss function changes identifiability. The paper claims that the BT loss will result in better sample complexity, but maximum likelihood estimation has optimal sample complexity for parameter identification. The suitability of the BT loss should be discussed in terms of whether the BT model reflects the observation process for the data. Clarification is needed on whether the goal is to identify the latent function f(), and what that means when f() is only identifiable up to a monotonic transformation (since the inverse transformation could be absorbed into g).

Author response:
We understand the desire for more standard statistical terminology, as we wrote this paper primarily from the perspective of signal processing. Our focus was on the practical advantages in performance and the effective identifiability of the latent fitness. BT loss is not invariant to monotonic transformations of f(x), (it is invariant to monotonic transformations of the data y). Clearly our results in Fig 1 and Fig 4 show effective identifiability of the latent fitness function with BT loss, whereas MSE loss has no such capability. Identifiability of the parameters of f(x) is impractical and unnecessary for neural networks, so we do not attempt this. We hypothesize that the BT loss is closer to the data generating process in the sense that some real global epistasis exists in these artificial and real fitness landscapes, and this results in better sample complexity.  We would welcome a formal treatment of the global epistasis problem; however, this is outside the scope of our paper, which we see as a complement to the existing literature on modeling global epistasis.

Reviewer comment (Evaluation):
What statistical test was used to measure the significance of improvements in Figure 3? I am asking this because in some splits the performance of BT loss is almost the same as MSE loss, but the reported p-value is significant (examples: samples and 7-vs-rest in AAV).

Author response:
We used a standard independent t-test with Bonferroni correction for multiple comparisons. Despite the bars having the visually similar heights, the differences between the methods are significant because the error bars over the random restarts for many of the splits are very small. For example, in the AAV “Sampled” split, the standard deviations of spearman correlations for the MSE and Bradley Terry losses are about 7e-4 and 4e-4, respectively. We will add a Table to the Appendix containing the numerical means and standard deviations for all of the bars shown in Figure 3, in order to clarify why the differences are significant.

Reviewer comment (Theory):
In the splits where BT loss provides better performance (Figure 3), it is hypothesized that **it could be partially due to the corruption of fitness function with global epistasis**. I am curious to know how this can be proved.

Author response:
We do not know of a straightforward path towards proving or disproving this hypothesis. The hypothesis was based on the analogy between the improved performance of the Bradley-Terry loss over the MSE loss in the simulated case when global epistasis was present, and the improved performance of the Bradley-Terry loss over the MSE loss in the case of real data. There may be multiple reasons for the improved performance in real data, and we do not immediately know how we could prove that the improved performance is due to the presence of global epistasis. Therefore, we will remove this sentence from the text in order to avoid making an untestable hypothesis.

Reviewer comment (Presentation):
In Figure 1, are the coefficients in the epistatic domain normalized? The scale of $\hat{f}$ does not match f, which is expected.

Author response:
Yes, the coefficients in the epistatic domain are normalized such that the sum of the squared magnitudes sum to one. We will correct the axis label and figure caption to clarify this.

Reviewer comment (Evaluation):
Where do you expect ranking-based losses not to perform as good as MSE losses for protein fitness prediction?

Author response:
A scenario in which we can expect ranking losses to perform worse than MSE for fitness prediction tasks is when the ranking of sets of sequences is determined by noise rather than underlying fitness values. In this case, we can expect the ranking losses to overfit to the noise, while the MSE loss will predict similar output values for all sequences in the set. However, we have found the Bradley-Terry loss to be surprisingly robust to noise, even in artificially pathological scenarios. In particular, in Appendix D we show a simulation in which the observed fitness values for sequences are given by y_i = I(x_i) + epsilon, where I(x) is a 0/1 indicator function and epsilon is Gaussian noise. We can expect this situation to be pathological for ranking losses, because the observed ranking within a class is entirely driven by noise. Unsurprisingly, we find that a model fit with an MSE loss on this data outperforms a model fit with the Bradley-Terry loss at classifying a test set between the 0/1 classes (AUC=0.977 and AUC=0.932 for the MSE and Bradley-Terry losses, respectively). Notably, the Bradley-Terry loss still performs quite well in this task. This Appendix D was mistakenly not referenced in the main text, but we will add a reference to it in the Discussion section and highlight that this is a regime where we can expect the ranking losses to perform worse than the MSE loss.

Reviewer comment (Experiments):
The empirical evaluation has focused only on a single benchmark, FLIP. It would strengthen the paper if the approach was validated with even more datasets.

Author response:
ProteinGym (https://www.proteingym.org/home) is another benchmark besides FLIP that can be used to validate sequence-to-fitness prediction models for proteins. This benchmark contains 94 curated datasets that relate sequences to experimentally-determined fitness values for a wide variety of proteins. ProteinGym is larger than FLIP; however, FLIP has a number of advantages that make it more appropriate for our use case:
1. FLIP curates specific dataset splits that are relevant for protein engineering (e.g. the two-vs-many split in which one predicts the fitness of sequences with 3 or more mutations using data only from sequences 2 or fewer mutations). These splits can be far more challenging for supervised models than a standard uniform split. In contrast, ProteinGym only curates complete datasets which the user must split themselves (usually by a standard uniform split). Therefore, FLIP provides a more challenging and diverse set of prediction tasks than ProteinGym, despite being an overall smaller benchmark.
2. ProteinGym was curated primarily to benchmark zero-shot prediction from pre-trained Protein Language Models. Therefore, the benchmark results that are presented on the ProteinGym webpage are not directly comparable to results from training a supervised model on the data. In contrast, FLIP was curated specifically to benchmark supervised sequence-to-fitness models for proteins, and therefore our results are directly comparable to those presented in the FLIP paper.

Despite these advantages of FLIP, we have now downloaded the ProteinGym benchmark data and begun to compare models trained with the MSE and Bradley-Terry losses on this data. At this time, we have only been able to run this comparison on one of the ProteinGym datasets: CAPSD_AAV2S_Sinai_substitutions_2021. For this dataset, we created 10 uniform 80/20 train/test splits and trained CNNs on each of the training datasets, using either the MSE or Bradley-Terry loss and all of the same model and training parameters as is used in the FLIP benchmark. We then calculated the Spearman correlation between the test set labels and model predictions in each corresponding test set. In this case, the models trained with the Bradley-Terry loss achieved a small but statistically-significant performance gain over the MSE loss (Spearman=0.9089 +/- 0.0038 and Spearman=0.9048 +/- 0.0037 for the Bradley-Terry and MSE losses, respectively, with p=0.027). If the reviewer thinks it will strengthen the paper, then we are happy to run this test for more of the ProteinGym datasets and add a table to the appendix displaying the results, though we will be unlikely to test all 94 datasets due to time and compute constraints.

Reviewer comment (Experiments):
Is there any way to extend the empirical evaluation beyond the FLIP benchmark?

Author response:
ProteinGym (https://www.proteingym.org/home) is another benchmark besides FLIP that can be used to validate sequence-to-fitness prediction models for proteins. This benchmark contains 94 curated datasets that relate sequences to experimentally-determined fitness values for a wide variety of proteins. ProteinGym is larger than FLIP; however, FLIP has a number of advantages that make it more appropriate for our use case:
1. FLIP curates specific dataset splits that are relevant for protein engineering (e.g. the two-vs-many split in which one predicts the fitness of sequences with 3 or more mutations using data only from sequences 2 or fewer mutations). These splits can be far more challenging for supervised models than a standard uniform split. In contrast, ProteinGym only curates complete datasets which the user must split themselves (usually by a standard uniform split). Therefore, FLIP provides a more challenging and diverse set of prediction tasks than ProteinGym, despite being an overall smaller benchmark.
2. ProteinGym was curated primarily to benchmark zero-shot prediction from pre-trained Protein Language Models. Therefore, the benchmark results that are presented on the ProteinGym webpage are not directly comparable to results from training a supervised model on the data. In contrast, FLIP was curated specifically to benchmark supervised sequence-to-fitness models for proteins, and therefore our results are directly comparable to those presented in the FLIP paper.

Despite these advantages of FLIP, we have now downloaded the ProteinGym benchmark data and begun to compare models trained with the MSE and Bradley-Terry losses on this data. At this time, we have only been able to run this comparison on one of the ProteinGym datasets: CAPSD_AAV2S_Sinai_substitutions_2021. For this dataset, we created 10 uniform 80/20 train/test splits and trained CNNs on each of the training datasets, using either the MSE or Bradley-Terry loss and all of the same model and training parameters as is used in the FLIP benchmark. We then calculated the Spearman correlation between the test set labels and model predictions in each corresponding test set. In this case, the models trained with the Bradley-Terry loss achieved a small but statistically-significant performance gain over the MSE loss (Spearman=0.9089 +/- 0.0038 and Spearman=0.9048 +/- 0.0037 for the Bradley-Terry and MSE losses, respectively, with p=0.027). If the reviewer thinks it will strengthen the paper, then we are happy to run this test for more of the ProteinGym datasets and add a table to the appendix displaying the results, though we will be unlikely to test all 94 datasets due to time and compute constraints.

Reviewer comment (Experiments):
More results are needed in order to delineate the regimes where the presented results hold true

Author response:
The reviewer is correct that the benefits of the Bradley-Terry loss extend to the case of additive models that have been affected by global epistasis. This is an important problem because even additive models affected by global epistasis contain spurious epistatic interactions in the observed data, so recovering the underlying additive fitness function enables proper interpretation of the nature of the interactions (or lack thereof) that drive fitness. We will make sure to clarify this in the text. 

Further, our noiseless simulations in Fig 1 and 2 demonstrate that the effects of global epistasis and the ability of the Bradley-Terry loss to mitigate these effects is independent of noise, and therefore applies to both Gaussian and non-Gaussian outcomes.

Reviewer comment (Experiments):
Absence of simple baselines for comparative analysis

Author response:
In the case of the “complete data” simulations whose results are shown in Fig 1 and that we referenced in the previous section, training a model with MSE will result in a model that perfectly recapitulates the observed data (there is no test data so there can be no overfitting). Therefore, for this initial test of the baseline it would be redundant to fit a model with MSE after performing the quantile transform. However, for the follow up experiments on this baseline with incomplete data (analogous to those described in Section 3.3) that we will perform, we will follow the reviewer’s proposed method and fit a model to the transformed data with MSE. 

We appreciate the reviewer pointing out a flaw in our arguments about simplicity. Indeed, the Bradley-Terry loss represents a paradigm shift despite its simplicity of implementation. We are committed to testing this against the more traditional baseline proposed by the reviewer. However, we would like to point out that the proposed baseline is somewhat inappropriate for modeling global epistasis in that the nonlinearity assumed by the method is not adaptive. In particular, the quantile transformation is a deterministic preprocessing step that assumes a particular nonlinearity. This is contrast to both the Bradley-Terry loss and previous methods for modeling global epistasis (such as the methods in Sailer and Harms, 2017 and Otwinowski et al., 2018) where the nonlinearity is modeled jointly with the latent fitness function. This joint modeling is crucial because the form of global epistasis nonlinearities are not generally known *a priori*.

Reviewer comment (Writing):
Clarity on "Corrupting Data": While recognising that the term "corrupting data" might be specific jargon within the authors' field, I find its usage potentially confusing. It typically suggests that data has been made less accurate. In contrast, from a machine learning standpoint, the issues you're addressing appear to be related to the complexity introduced by non-linear relationships, which need complex models that may overfit, particularly when models are trained to predict the exact observed values (y).

Author response:
1. **On “corruption” language**: it is helpful to know that this is confusing language. We had meant to use it to refer to the effect that global epistasis produces a dense epistatic representation (similar to the effect of noise), which results in higher sample complexity. The reviewer’s point is well taken that this is a distinct negative outcome from the effects of noise and therefore we should not use “corruption” to refer to global epistasis. We will update the manuscript to remove this language.

Reviewer comment (Experiments):
It would be highly beneficial to demonstrate the specific regimes in which your observations about the Bradley-Terry loss are valid. Specifically, it's important to determine whether the improvements attributed to the BT loss over the MSE loss are unique to the complex models with epistatic interactions, or if similar benefits could be observed with simpler models, such as a linear model subjected to the same monotonic warping (i.e., the nonlinearity introduced by global epistasis). To address this, I recommend varying the degree of interaction order in your simulations.

Author response:
2. **On testing our methods with varying degrees of interaction orders**: We agree that testing our method for different degrees of interaction order is useful. We repeated the simulation described in Section 3.1 (“Recovery from complete data”) with K=1 (i.e. an additive latent fitness function) and K=3 (up to third-order interactions), in addition to the K=2 setting used in the main text; the results of these additional simulations are shown in Appendix B of the initial submission, which was mistakenly not referenced in the main text (we will add a reference in Section 3.1). These results show that the complete data recovery result is maintained in for all of these interaction order regimes. Further, we have now repeated the simulation results described in Section 3.3 (“Simulations with incomplete data”) for K=1 and K=3 and these results will be added to the Appendix. We have created analogous figures to Figure 2b and 2c using the results from these K=1 and K=3 simulations. Comparing these to Figure 2b and 2c (which are based on simulations using K=2) we find that:

   * The results in Figure 2b are qualitatively similar to the analogous results for the K=1 and K=3 simulations. In particular, the MSE loss shows a clear drop in performance as the entropy of the observed fitness function increases, while the Bradley-Terry loss is more robust to this increase in entropy.

   * The results in Figure 2c are qualitatively similar to the analogous results for the K=1 and K=3 simulations. In the K=1 case, the distinction between the MSE and Bradley-Terry losses is not as large as in the K=2 and K=3 cases, because both losses are able to achieve high performance at small training set sizes (i.e. spearman > 0.9 at 100 training samples). However, if we increase the intensity of the nonlinearity by using a setting of alpha=20 instead of alpha=10, then the distinction between the losses is similar to that observed in Figure 2c. This indicates that the Bradley-Terry loss can still be useful for modeling additive latent fitness functions under global epistasis, but the nonlinearity of global epistasis must be more extreme to observe large performance gains over the MSE loss. The results of these simulations for the alpha=20 setting will also be added to the Appendix.

Reviewer comment (Experiments):
I would recommend including a comparison with a more straightforward baseline to further validate the proposed approach. Specifically, it would be informative to see how a simple quantile transformation of the outcome data (to uniform or Gaussian distributions) performs in conjunction with Mean Squared Error (MSE) loss. This could serve as a more direct way to deal with the non-linear transformations introduced by global epistasis and might provide a competitive baseline to the BT loss approach.

Author response:
3. **On comparing to a baseline method based on quantile transformations**: We thank the reviewer for suggesting this baseline method. We have tested this baseline method in a simulation analogous to that described in Section 3.1 (“Recovery from complete data”); in particular we tested the ability of quantile transformation to recover a latent fitness function from complete data that has been affected by global epistasis. We found that quantile transformations do perform quite well at recovering the latent fitness function (R^2 =0.951 and R^2=0.963 for quantile transformations to normal and uniform distributions, respectively), but not as well as the Bradley-Terry loss (R^2=0.999, as seen in Figure 1c in the main text). Further, we are committed to testing the baseline method in the simulations described in Section 3.3 (“Simulations with incomplete data”) in order to compare the baseline to the Bradley-Terry loss in a more realistic scenario. 
We note that the baseline method described by the reviewer makes stronger assumptions about the nature of the latent fitness function than the Bradley-Terry loss, and this may be the reason for the difference in performance in the complete data case. In particular, the baseline method assumes the latent fitness function follows either a uniform or normal distribution, while the Bradley-Terry loss makes no such assumption. Considering that these methods are similar in their implementation difficulty (both require fewer than 10 lines of code), the Bradley-Terry loss is preferable because it requires fewer assumptions and appears to provide better performance.

Reviewer comment (Writing):
Minor: To avoid confusion with unsupervised contrastive learning methods, it would be beneficial to explicitly state that the contrastive learning approach employed is supervised. (e.g., you could say "supervised contrastive learning loss")

Author response:
4. **On clarifying that we use supervised contrastive losses**: It is helpful to know that this is a point of confusion. We will add a sentence to the Methods that clarifies that we use supervised rather than unsupervised contrastive losses, and reinforce this point by referring to “supervised contrastive losses” wherever appropriate.  

R2: Thank you for this clarification; we will make sure to specify that we use supervised contrastive losses in the abstract and introduction.