Paper: Discrimination-free Pricing with Privatized Sensitive Attributes (ICLR-2024)

Reviewer comment (Experiments):
Since noise levels have a significant effect on risk-LDP, I am curious about the effect of overestimating and underestimating noise on fairness and the impact on different sensitive groups, especially in the case of unevenly distributed groups.

Author response:
To answer the reviewer's question on the impact of noise rate estimation error on Risk-LDP, we added an empirical study. A high-level result of the study is that when $\pi$ is far away from $\frac{1}{|\mathcal{D}|}$, Risk-LDP is robust against estimation error even if the error is large (regardless of overestimation or underestimation). However, when $\pi$ gets close to $\frac{1}{|\mathcal{D}|}$, Risk-LDP suffers from both underestimation (more severe) and overestimation even if the error is small. Please see Section 5.3, Appendix E for figures and a more detailed discussion.

Reviewer comment (Experiments):
Several challenges are associated with a multi-party framework, for example, information leakage, computation overhead, etc.. I am curious how the proposed method would be comparatively better than those settings where fairness is computed on (single-party) fully differentially private data (X,D) or where causal inference (and other methods) is used to perform fairness in the absence of sensitive attributes.

Author response:
All algorithms that involve multiple parties suffer from the shortcomings the reviewer listed (i.e. information leakage, computation overhead, etc..) But, one merit of our framework is that it is not limited to the multi-party setting, multi-party training process is introduced as an alternative when the insurer cannot directly access all the data needed to train a discrimination-free pricing model. We mentioned two settings where this multi-party training process will be reduced to a regular single-party training process.:

1) The insurer is able to obtain the privatized sensitive attributes from a third party and apply the proposed algorithm directly. 
2) The insurer collects information on both non-sensitive and sensitive attributes and sends this information to a third-party vendor to execute the pricing algorithm.

Therefore, even if we assume that the insurer has direct access to all the data needed, meaning that our framework is reduced to single-party training, we do not think our method is directly comparable to methods that compute fair predictions satisfying the fairness notion defined in the conventional algorithmic fairness literature because we are aiming at achieving different fairness notions. 

Since discrimination-free pricing in insurance is a new topic, there are only a few papers that give insights from the causal inference perspective, however, all of them require direct access to the true sensitive attributes, and no discrimination-free pricing algorithms that consider sensitive attributes privatization has been developed in the discrimination-free insurance pricing literature to our best knowledge.

Reviewer comment (Experiments):
Experimental setup and results. There are other (single-party) fair decision-making with private data methods and noisy sensitive attributes that the authors could have compared their work with. Although authors show different error rates with varied privacy budgets, it would have been informative to see how the method compares to other (similar) methods. Additionally, authors say they couldn't find insurance-like data, but there are at least 50 insurance datasets on Data World (and other platforms).

Author response:
As mentioned in some parts of $\textbf{A1, A3}$. We added another empirical experiment to test the performance of our proposed algorithm in a regression setting and found the results also in support of our theoretical guarantees (please see Section 5 for a more detailed discussion). We'd like to point out that we are the first to propose algorithms that train discrimination-free models that rely solely on privatized sensitive attributes in the actuarial science literature. Therefore, we might not be able to find an appropriate paper for a direct comparison of the results from empirical experiments.

Reviewer comment (Novelty):
This paper focused on the ‘actuarial fairness’ definition formulated in an earlier paper. It is unclear whether this formulation is practical, and whether it is a broadly accepted formulation. Further literature review on its usage and potentials in practice will be helpful.

Author response:
We first present the definition of actuarial fairness: The premium is considered actuarially fair if it accurately reflects the expected cost of the coverage provided to the policyholder.  It is a practical and broadly accepted formulation by the insurance industry. However, in recent years regulators have begun to question whether an actuarially fair premium should discriminate against policyholders based on sensitive attributes. As a result, insurers are required to demonstrate that premiums are not discriminative w.r.t. sensitive attributes. Under this backdrop, we focused on the discrimination-free premium, a concept recently proposed in the actuarial science literature. The discrimination-free premium satisfies the notion of fairness from a causal inference perspective [1]. It is free from both direct and indirect discriminations linked to sensitive attributes.

Reviewer comment (Writing):
The term ‘actuarial fairness’ was used in the introduction paragraph without defining what it is. While it is formulated later in the mathematical definition, it would be helpful to first explain how the insurance industry defines ‘actuarial fairness’ on a conceptual level.

Author response:
We first present the definition of actuarial fairness: The premium is considered actuarially fair if it accurately reflects the expected cost of the coverage provided to the policyholder.  It is a practical and broadly accepted formulation by the insurance industry. However, in recent years regulators have begun to question whether an actuarially fair premium should discriminate against policyholders based on sensitive attributes. As a result, insurers are required to demonstrate that premiums are not discriminative w.r.t. sensitive attributes. Under this backdrop, we focused on the discrimination-free premium, a concept recently proposed in the actuarial science literature. The discrimination-free premium satisfies the notion of fairness from a causal inference perspective [1]. It is free from both direct and indirect discriminations linked to sensitive attributes.

Reviewer comment (Theory):
For the derivation of theoretical results, it would be useful to know whether and how the fairness, the noise, or the multi-party training flow leads to challenges.

Author response:
In our work, we employed the concept of discrimination-free premium for pricing insurance contracts. As the reviewer noted, the notion of fairness in this context is different from the concept of fairness in the conventional algorithm fairness literature. However, the discrimination-free premium is a conceptual framework and is not immediately applicable in practice due to regulatory constraints. More precisely, due to regulatory requirements, insurance companies are either prohibited from directly accessing sensitive attributes or are limited to accessing only a noised version of the sensitive attributes. Our research aims to introduce a methodology enabling insurers to derive discrimination-free premiums by effectively addressing the challenges imposed by regulatory requirements. In terms of how noise leads to challenges in the derivation of theoretical results, we added an empirical study on the robustness of Risk-LDP against the noise rate estimation error. A high-level result of the study is that Risk-LDP is robust against estimation error even if the error is large when $\pi$ is far away from $\frac{1}{|\mathcal{D}|}$. However, it suffers from both underestimation (more severe) and overestimation even if the error is small when $\pi$ gets close to $\frac{1}{|\mathcal{D}|}$. Please see Appendix E for figures and a more detailed discussion.