Paper: Fairness-Aware Attention for Contrastive Learning (ICLR-2024)

Reviewer comment (Writing):
The paper is poorly written. There are notation problems (e.g. in section 2 the matrix $U$ of similarity scores is overloaded, in section 3.3 there is no $z_i$ appearing in the loss even if it is sampled), incorrect math statements (like in section 3.1 that $\phi(g(y))$ is used to estimate $\mathbb{E}_{y|z} \phi(g(y))$), missing important citations (like [1] for the derivations in page 5 of attention as a kernel-based similarity), and often times definitions (such as for the dataset used and the “bias removal” evaluation metric) are not self-contained in the paper.

Author response:
**Notation in section 2 for matrix U is overloaded**

**Answer**

We thank the reviewer for drawing our attention to this potential source of confusion. We noticed there is a potentially connected typo in which we wrote $U = [e^{f(x_i, y_j)}]_{ij} $ . As U is a matrix with $(i,j)$ element given by these similarity scores, this should of course be that the $[U]_ij = e^{f(x_i, y_j)}$. Regarding overloaded notation, the two references to the definition of U is simply first a general representation of U followed by an instantiation of what U is for our purposes. Specifically, the matrix U is introduced as the data matrix such that when transformed by the values projection matrix W_V, we obtain the values V. That is, $V = UW_V$. Given our FARE set up, the untransformed values in our case is the matrix of similarity scores. Hence in our model U is the matrix whose $(i,j)$ element is given by $e^{f(x_i, y_j)}$. In other words, naming U as the matrix of similarity scores $e^{f(x_i, y_j)} $ is just a specification of the data that takes the place of the values for our model. 


**In section 3.3, there is no $z_i$ appearing in the loss**

**Answer** 

We thank the reviewer for pointing this out and have fixed the notation. In particular FARE is now defined as an operator that takes as argument the whole batch, $FARE( \{ (x_i, y_i, z_i) \}^b_{i=1} )$. This clarifies the fact that FARE computes a finite sample estimate of the conditional similarity score between $(x_i, z_i)$ and $y$ for $y \sim P_{Z = z_i}$ by taking in information over the whole batch and weighing each sample according to their protected attribute.
		
**There are missing citations, for example [1], on page 5 for the derivation of attention as kernel-based similarity, for example**

**Answer**

We agree that [1] is highly relevant related work and indeed we include reference to it in our Related Work section. However, [1] is not cited here because our derivation doesn’t rely on theorems or results from [1] or other work, aside from the use of the kernel density estimator approach to estimate the joint and marginal densities which is cited. We also note that [1] is different to our work in the sense that [1] takes the attention mechanism and decomposes it into parts that, by replacing with various kernels, derive new and enriched attention mechanisms. Our work, on the other hand, doesn’t involve decomposing attention nor replacing any component of attention with kernel similarity metrics. Our work shows instead that the estimation of the similarity score (when conditioning on some additional attribute) in the contrastive objective can be approximated by an attention mechanism. In this sense we don’t perform any decompositions and we go straight from conceptualising the similarity score as an inner product to deriving an attention mechanism without appealing to kernel similarity metrics in between. For these reasons we agree that [1] is a relevant work to understand some of the research community’s work in enriching attention, however we do not cite it on this page due to the differences mentioned.

**There is an incorrect math statement in section 3.1 that $\phi(g(y))$ is used to estimate $E_{y | z} [\phi(g(y)))] $**

**Answer**

We apologise for the confusion regarding notation. To clarify, we don’t use $\phi(g(y))$ to estimate $E_{y | z} [\phi(g(y)))] $. Rather, we use $E_{y | z} [\phi(g(y)))] $  to estimate $\phi(g(y))$. $\phi(g(y))$ is the unknown quantity of interest and we use its expectation to estimate it, expanding the definition of the expectation using joint and marginal densities which we then estimate with kernel density estimators. This allows us to derive an estimator of $\phi(g(y))$.

[1]: Yao-Hung Hubert Tsai, Shaojie Bai, Makoto Yamada, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2019. Transformer Dissection: An Unified Understanding for Transformer’s Attention via the Lens of Kernel. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).

Reviewer comment (Novelty):
Novelty concerns: the FAREContrast objective function is essentially the same as the one described at [2].

Author response:
**What are the differences between FAREContrast and the objective in [2]**

**Answer**

The key difference between the objective in [2] and FAREContrast is that FAREContrast contains learnable attention scores as conditioning coefficients on the sample similarity scores whereas [2] does not. [2] therefore is used only to train in the MLP and encoder where FAREContrast trains the MLP, encoder, and attention projection matrices in one end-to-end loop.

Reviewer comment (Novelty):
Sparsifying the attention is poorly motivated, and it incurs a considerable implementation cost for the induced performance benefit over the considered baselines.

Author response:
**Is sparseFARE sufficiently well motivated and does its induced performance benefit justify the implementation cost?**

**Answer**

We argue that the benefits of sparseFARE do indeed justify its implementation cost. In particular, we note that in colorMNIST sparseFARE Pareto dominates all models and is significantly more computationally more efficient than the kernel models. For CelebA, it Pareto dominates or exhibits a more favourable fairness-accuracy tradeoff than the baselines. We argue that this is because sparisification makes intuitive sense within this framework, whereby samples that are sufficiently distant in terms of bias-status (and therefore likely to bias the representations) may as well be dropped and allow attention to be focussed over the remaining samples that help learn debiased and meaningful representations. 

Secondly, attention sparsification schemes typically exhibit a tradeoff between computational complexity and model performance (in this case taking both accuracy and fairness together) as sparisifation sacrifices the capacity of the attention mechanism for speed. SparseFARE, however, exhibits higher performance than the baselines while being faster as well and so again doesn’t exhibit the normal tradeoff we observe in the literature.

Hence we believe that since sparsity, in this setting, is able to offer improvements in all these areas simultaneously, it’s implementation cost is justified.

Reviewer comment (Experiments):
Table 1 shows the result for CCLK [1] when using Cosine kernel, but [1] also provides a result for CCLK when Laplacian kernel is applied, showing Top-1 Accuracy of $85.0 \pm 0.9$ and MSE of $72.8 \pm 13.2$. Then, I'm not sure whether FARE indeed alleviates a significantly larger amount of bias compared to the baseline methods.

Author response:
It’s true that when implementing CCLK with a Laplacian kernel, the baseline model achieves bias removal close to SparseFARE. We note that our method nonetheless provides 3 simultaneous benefits over the Laplace method in particular, and over the baselines more generally:
Using the Laplacian method can indeed remove similar bias to our proposed models, but it does so at the expense of accuracy, shown by the drop in accuracy of 1.4%. Our SparseFARE does not suffer from this tradeoff, producing the same high bias removal with high accuracy concurrently. In this sense, we find that sparseFARE still Pareto dominates CCLK and the other baseline models.
An additional benefit of our approach is ease of implementation. In order to discover the best performing kernel baseline method, the user would have to exhaustively search over candidate kernels, each with their own hyperparameters, where there is in general little intuitive guidance in terms of knowing prior to the problem which kernel is suitable. Our approach does not need to consider a host of similarity metrics since the attention mechanism can adaptively learn it given the task, and so in this sense our model is easier to implement and fine-tune. 
Lastly, we also refer to the benefits in computational complexity of our approach in reducing cubic computational complexity to $O(b^2)$ or $O(b log b)$. 

Overall, we agree that there probably will exist situations in which a pre-defined kernel performs close to our proposed method in terms of debiasing. In such situations we note that our methods can and do indeed match that debiasing performance without sacrificing accuracy, are faster to implement and fine-tune, and less computationally complex.

Reviewer comment (Experiments):
Given that the performance gain doesn’t seem to be significant, it is not yet clear to me the benefits of the attention-based approach compared to the kernel-based approach. The kernel-based method relies on choosing an appropriate kernel, whereas the attention-based method focuses on training the model using data. However, it seems that more justification is required for the proposed method. It would be beneficial to include additional intuitive explanations on why attention-based methods are more effective than kernel-based methods for calculating the similarity of sensitive attributes, along with experimental results to support this. For instance, in Adult dataset, if ‘age’ is selected as the sensitive attribute, one could consider showing experimentally that the attention score tends to be higher when two individuals have similar ages, whereas this may not be the case with kernel-based methods.

Author response:
An alternate way of seeing the benefit of attention is that it is trained in an end-to-end manner alongside the encoder whereas any given kernel is simply pre-specified. In this way, the attention mechanism helps to focus attention on samples that learn meaningful representations. For example, suppose in a contrastive step that in a given batch there are n samples with highly similar sensitive attributes to the anchor. A kernel would indiscriminately assign all these n samples high weights irrespective of those samples’ effect on the loss. Attention, by taking in information from the loss, is adapted to not only measure similarity over protected attributes, but be adapted by information coming from the gradients to better focus on samples that help minimize the loss and thereby learn meaningful representations.

Reviewer comment (Evaluation):
In Table 1, the result for SpareFARE appears to use the adjacent bucket scheme, but it differs from the result in Table 2 of the appendix. Which is correct?

Author response:
We thank the reviewer for picking this up and we apologize for the oversight. The original number in the main body is correct - we’ve made the correction.

Reviewer comment (Theory):
In Eqn. (12), does FARE use a feature map associated with the Cosine kernel for $\phi$?

Author response:
$\phi$ is the feature map associated with the exponentiation of the cosine kernel, which is what we need to aim for to estimate the similarity score in the objective. You can refer to equation 3 on page 3 to see the definition.