SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Ablation of parameters. In this experiment, we perform ablations with several parameters in KRaM and observe how that affects the concept erasure performance. First, we ex
Table 5: Ablations with kernel functions: we observe that KRaM achieves similar performance using different kernel functions.
Method	Acc (a) ↓	Ak ↑
Original	100.0	1.0
MIFR [55]	68.3	0.58
CCL [56]	56.9	0.56
ICVAE [42]	51.5	0.50
KRaM	52.6	0.65
Table 6: Comparison of KRaM with MI-based baselines on GloVe dataset. We observe that KRaM achieves a good balance between concept erasure and retaining original information.
Figure 8: Ablation experiments to study the effect of parameters  $\lambda$  (Eqn. 4),  $r$  (a scaling factor in  $R(\mathcal{Z}) = rb$ ), and  $\sigma$  (parameter in gaussian kernels) on the performance of concept deletion.
periment with gender removal from GloVe embeddings to understand the impact of  $\lambda$  (Eqn. 4). In Figure 8 (left), we observe that as  $\lambda$  increases, concept erasure worsens ( $\uparrow$  gender accuracy). This is expected as the erasure function  $f$  is penalized for  $|R(\mathcal{Z}) - b|$  term more than maximizing  $R(\mathcal{Z}|\mathbf{K})$  (which helps in erasure). The alignment scores  $A_{k}$  stay mostly stable with a minor drop at high  $\lambda$  values. We believe this happens as  $f$  aims to match the rate-distortion constant, possibly neglecting the underlying representation structure. Second, in the same setup, we modify the equality constraint to be:  $|R(\mathcal{Z}) - rb|$  and ablate  $r$  (shown in Figure 8 (center)). We observe that both alignment scores and gender accuracy increase with an increase in  $r$ , which demonstrates the importance of this constraint. Even though  $R(\mathcal{Z}|\mathbf{K})$  is maximized, if the overall feature space expands (high  $r$ ), the concept variable can still become distinguishable (high gender accuracy). Third, in Figure 8 (right), we report the MSE scores on the synthetic dataset for varying  $\sigma$  (the parameter in the Gaussian kernel). In all setups within Figure 8 (right), we notice the same pattern of increasing MSE  $(a)$  scores followed by a decrease. We believe this drop happens with higher  $\sigma$  values because distances become very small and kernel values are similar. This results in the kernel ignoring the similarity of instances in the concept space.
E Broader Impact & Limitations
In this section, we discuss the broader societal impact and limitations of our framework, KRaM.
Limitations. While erasing sensitive concept attributes can reduce bias and improve privacy, it may also result in the loss of potentially useful information for the task at hand. This could negatively impact the utility of the model. The definition of what constitutes a sensitive concept attribute can vary greatly depending on the cultural, ethical, and legal context. This work assumes that these sensitive attributes can be clearly defined and agreed upon, which might not always be the case. Therefore, developers using such erasure frameworks should take care of the societal impact before utilizing them in the wild.
Negative Usage. KRaM s intended to be used in scenarios where the user is already aware of the concept attribute to be erased. KRaM can only be trained on data where concept labels are annotated either as categorical, continuous, or vector-valued attributes. One potential misuse of KRaM would be to define relevant features for a task (e.g., experience for a job application) as a concept to be erased. In such cases, the classification system may be forced to rely on sensitive demographic information for predictions. It is possible to flag systems in these cases by evaluating the statistical parity when the concept attributes have changed.
In general, we hope that our proposed concept erasure framework, KRaM, would encourage others to develop more robust concept erasure systems that can simultaneously retain a lot of information from the original representations.