Paper: Regulation Games for Trustworthy Machine Learning (ICLR-2024)

Reviewer comment (Theory):
There are several (simplifying) assumptions made (which can take away the practical feasibility of the work). For example: "We assume that regulators are able to give penalties for violations of their respective objective which they formulate as a utility (or value) function."

Author response:
- There are several (simplifying) assumptions made (which can take away the pratical feasibility of the work). For two examples,
  > We assume that regulators are able to give penalties for violations of their respective objective which they formulate as a utility (or value) function.  

	- Despite nascent privacy laws, there are already [many instances](https://www.enforcementtracker.com) of monetary penalties levied for violation of data privacy laws such as GDPR. Therefore, modeling such fines to be proportional to the privacy loss (as we do) is not a limitation beyond the fact that we take differential privacy as our notion of privacy—a choice that we believe is presently justified because differential privacy is the dominant privacy notion for machine learning models.

Reviewer comment (Writing):
In Section 1: "This is because nowadays ML models are trained, maintained, and audited by separate entities—each of which may pursue their own objectives." Are there references or real-world use-cases where this is true or implemented?

Author response:
- In Section 1
> This is because nowadays ML models are trained, maintained, and audited by separate entities—each of which may pursue their own objectives.  
Are there references or real-world use-cases where this is true or implemented?
	- In California, the Consumer Privacy Act in California stipulates that the Attorney General enforces the privacy of employment background screening.  However, the Employment Non-Discrimination Act is enforced by the Department of Justice. These are two separate governmental entities. We believe cross-agency cases such as this should be more prevalent as the use of automated decision making systems grows. Having said that, we are not trained in law so we refrain from making a case reference.

Reviewer comment (Theory):
In Section 1: "...that assumes shared knowledge of a pre-calculated PF between agent objectives." Can this PF be realized in practice? i.e., how to accurately obtain it? and if only a somewhat inaccurate one can be obtained, what are its implications?

Author response:
- In Section 1,
 > ... that assumes shared knowledge of a pre-calculated PF between agent objectives.  
Can this PF be realized in practice? i.e., how to accurately obtain it? and if only a somewhat inaccurate one can be obtained, what are its implications?
	- Yes, it is possible to develop a cryptographic zero-knowledge-proof (ZKP) version of ParetoPlay where the agents not only share models but also proof of trustworthiness guarantees  (see Shamsabadi et al. 2022 for an example of a ZKP proof of fairness). We expand on this idea in Section J in the appendix, however, this is beyond the scope of the present work which lays the game theoretic foundations for ML Regulation.

	- For a discussion on the shared Pareto frontier, please see the general response.

Reviewer comment (Writing):
What is \( i\in I \) in Definition 1?

Author response:
- What is  $i \in I$ in Definition 1?

	- $I$ is the set of objectives.  This was left out by mistake and has been added to Definition 1.

Reviewer comment (Theory):
In Section 3: "In this work, we do not consider a competition between regulatory bodies since both are assumed to be governmental agencies." Even though the regulatory bodies are not set out to compete with each other, the inherent tension between the objectives can lead to competitive strategy profiles and actions, right? In that case, what is the significant distinction of "not considering a competition between the regulatory bodies"?

Author response:
- In Section 3,
> In this work, we do not consider a competition between regulatory bodies since both are assumed to be governmental agencies.  
Even though the regulatory bodies are not set out to compete with each other, the inherent tension between the objectives can lead to competitive strategy profiles and actions, right?

	- Yes, that is correct. But the regulators being *governmental* agencies, the higher level agent (the government) can make a decision regarding the fairness and privacy trade-off. We capture this scenario in the first stage of a regulator-led SpecGame in the 2nd paragraph of Section 4.1.

- In that case, what is the significant distinction of "not considering a competition between the regulatory bodies"

	- An alternative game formulation, for example, can consider the regulatory bodies as independent agents (not controlled by a 3rd agent, such as the government).

	- In this setting, the regulators can fight against each other to improve their own metric, possibly to the detriment of the general good. For instance, low-privacy low-fairness can be an equilibrium of such a game, which does not benefit either party. In similar 2-player games, a solution to this is to use a third party mediator to ensure a better equilibrium (see Section 10.7.3 in Shoham and Leyton-Brown 2009 for instance).

	- Note that in our formulation, we already have a natural candidate for such a mediator which is the the government. We have thus shown that a solution to the joint game and subgame reduces to our original game formulation.

Reviewer comment (Writing):
What is \( \{c_{i}^{(t)}\}^t \) in the overall discounted loss in Section 3.2?

Author response:
- What is $B: =\left\\{c_i^{(t)}\right\\}^t$   in the overall discounted loss in Section 3.2 ?

	- This is a factor that depends on how the agent perceives its current loss w.r.t its past losses. In the most general case can change over time, $B = B(t)$ ; which is what the expression above models. We have reformulated this general loss as $L_i=\\sum_{t=1}^{\\infty}\\left\\{\\prod^{t}_{i=1}c_i^{(t)}\\right\\} L_i^{(t)}$ which should make this point clearer. Note that, as before, we assume $B$ is constant with time  $c_i^{(t)} = c$ which is a common assumption in modeling of repeated games.

Reviewer comment (Theory):
In Section 4: "However, ...the highly non-convex nature of agent losses in \(\texttt{SpecGame}\)". How is the non-convexity addressed?

Author response:
- In Section 4,
> However, ...the highly non-convex nature of agent losses in  
How is the non-convexity addressed?

	- By approximating and interpolating the agent losses on Pareto frontier (as we discuss in paragraph titled **Approximation and calibration** in Section 4.1).

Reviewer comment (Theory):
In Section 4: "for \( t>1 \), we assume a mapping from the penalty values in S to trustworthy parameters values \( s_{reg} = (\gamma, \epsilon) \)". (How) can this assumption be satisfied in practice?

Author response:
- In Section 4,
> for $t>1$ , we assume a mapping from the penalty values in $\mathrm{S}$ to trustworthy parameters values $s_{, u y}=(\gamma, \epsilon)$  
(How) can this assumption be satisfied in practice ?

	- What this assumption says is that for a given pair of parameters $\boldsymbol{s}=(\gamma, \epsilon)$ we can find a penalty to enforce it. Note that this is trivially true for a large enough penalty $C_\text{fair}$ ( $C_\text{priv}$ ) ensuring that fairness (privacy) losses dominate the model builders loss, and loss update in equations 3 and 4. Therefore, for any feasible solution to the joint optimization of training loss under fairness and privacy constraints, this assumption holds; and therefore does not limit the applicability of our algorithm in practice.

Reviewer comment (Presentation):
What do the colors represent in Figure 4a?

Author response:
- What do the colors represent in Figure 4a?

	- The colors in Figure 4a represent different $C_{fair}$ values. The lightest shade correspond to the lowest $C_{fair}$ , and the darkest the highest. We have adjusted the figure legend and its caption to make this more clear.

Reviewer comment (Evaluation):
Are the experimental results averaged over multiple trials? If so, is there an analysis of the variation?

Author response:
- Are the experimental results averaged over multiple trials? If so, is there an analysis of the variation?

	- We run the games on pre-computed Pareto frontiers which we have calculated over many runs of the private and fair learning models we have adopted (FairPATE and DPSGD-Global-Adapt) to ensure we have an accurate Pareto frontier. As a result,  the only source of randomization in running ParetoPlay would be the calibration step, where the agents train a model to re-compute the Pareto surface. This pre-calculation of trustworthiness hyper-parameters makes for a relatively smooth (but non-convex) Pareto surface (see Figure 6 in the appendix). As a result we, we have found out that calibration often does not produce a new point on the Pareto surface  and therefore we have not observed varied convergence results. The different convergence results we see in the experiments are due to having different starting points and different regulator penalty scalar $C_{fair}$ and $C_{priv}$ .

Reviewer comment (Novelty):
I think the main contribution of the paper is to model the dynamic interaction between the model builder and the regulator. Accordingly, it is more reasonable to think of only fairness or privacy or to possibly even abstract/lump both issues into one. I don't see how having these two considerations has added to the model. One can also consider the safety of the model or its robustness to adversarial manipulations as part of the regulator's concern for example.

Author response:
We agree with the reviewer that our framework is general and can include other trustworthy objectives in future work that builds on our framework.

	- The privacy and fairness regulators can be lumped into one. In fact, we do so at the start of a regulator-led game (in Section 4.1). There we note that considering a joint regulator  not resolve the inherent conflicts between the two objectives (fairness and privacy).
	  **In practice ML models may be audited by separate entities. Therefore, our goal in keeping the regulators separate was to consider broader and more realistic scenarios.** For instance, the Consumer Privacy Act in California stipulates that the Attorney General enforces the privacy of employment background screening. However, the Employment Non-Discrimination Act is enforced by the Department of Justice. A joint regulator tacitly assumes that authorities would coordinate in every decision which is an impractical assumption.

Reviewer comment (Theory):
The section “Making a uniform strategy space.” on page 6 is very unclear. What does "consistent" mean? It seems to suggest that the strategies are fixed. Further, it seems that the horizon is n. If that is the case then why does the utility in section 3.2 sum to infinity?

Author response:
By consistent, we mean that the domain of strategies for each stage of the game is the same. Furthermore, the SpecGame is in the class of infinitely-repeated Stackelberg games; therefore $n \rightarrow \infty$ .

	- We have added clarification for both comment to the main text.

Reviewer comment (Theory):
Why is a correlated equilibrium and correlation device well-motivated in this setting? Also the paper mentions that “This is known as a correlation device. If playing according to the signal is a best response for every player, we can recover a correlated equilibrium (CE). We leave the theoretical proof of this conjecture to future work.” But doesn’t Theorem 1 prove that you have a correlated equilibrium so is it a conjecture?

Author response:
We apologize for the confusion. Indeed the sentence "We leave the theoretical proof of this conjecture to future work." was mistakenly left in the section from a prior submission where we did not have a formal proof).

Reviewer comment (Writing):
Throughout the main body, the authors sometimes refer to notations that were not defined previously. For example, in Section 3.2, notation c_i^{(t)} and L_i^{(t)} are the first time the "(t)" superscript is used. In Equation 4, the notation \nabla_s is used without a definition.

Author response:
- Throughout the main body, the authors sometimes refer to notations that were not defined previously. For example, in Section 3.2, notation $c_i^{(t)}$ and $L_i^{(t)}$ are the first time the "(t)" superscript is used. In Equation 4, the notation $\nabla_s$ is used without a definition.

	- We have added the clarification of all the aforementioned cases (marked in blue in Section 3.2 and Eq. 4)

Reviewer comment (Evaluation):
The discussion of the experiments in Section 5 is confusing. At the end of Section 1, the author claimed that the experiments would highlight the suboptimality of studying trustworthy ML in a single-agent framework. However, in Section 5, the first research question shows that multi-agent setup leads to sub-optimalities.

Author response:
- The discussion of the experiments in Section 5 is confusing. At the end of Section 1, the author claimed that the experiments would highlight the suboptimality of studying trustworthy ML in a single-agent framework. However, in Section 5, the first research question shows that multi-agent setup leads to sub-optimalities.

	- We agree with the reviewer that wording of RQ1 was confusing. We have re-written it for clarity to:
	  > (RQ1) if regulators produced specifications for trustworthy parameters $\boldsymbol{s}$ using a single-agent-optimized model, would that recommendation be respected in the multi-agent setting?  
	    
	  and the description of the results has also been updated to:  
	  > Single-agent-optimized specifications lead to sub-optimalities in the multi-agent settings

Reviewer comment (Presentation):
Figure 1 does not have a legend. Overall, most figures are hard to read and the captions do not provide sufficient description of the experiments.

Author response:
- Figure 1 does not have a legend. Overall, most figures are hard to read and the captions do not provide sufficient description of the experiments.

	- We have addressed the reviewer's comment and re-wrote the captions for all experiment figures for clarity.

Reviewer comment (Writing):
Minor typos: \ell_build(w) instead of \ell_b(w) on page 5 under Equation 3.

Author response:
- Minor typos: ell_build(w) instead of ell_b(w) on page 5 under Equation 3.

	- We thank the reviewer for their attention. We have fixed the typo.