Regulating algorithmic filtering on social media
Authors: Sarah Cen
Devavrat Shah
Venue: NeurIPS 2021 Spotlight (2021)
Keywords: social media
regulation
audit
filtering algorithm
performance cost
content diversity
counterfactual
hypothesis testing
minimum-variance unbiased estimator

Abstract: By filtering the content that users see, social media platforms have the ability to influence users' perceptions and decisions, from their dining choices to their voting preferences. This influence has drawn scrutiny, with many calling for regulations on filtering algorithms, but designing and enforcing regulations remains challenging. In this work, we examine three questions. First, given a regulation, how would one design an audit to enforce it? Second, does the audit impose a performance cost on the platform? Third, how does the audit affect the content that the platform is incentivized to filter? In response to these questions, we propose a method such that, given a regulation, an auditor can test whether that regulation is met with only black-box access to the filtering algorithm. We then turn to the platform's perspective. The platform's goal is to maximize an objective function while meeting regulation. We find that there are conditions under which the regulation does not place a high performance cost on the platform and, notably, that content diversity can play a key role in aligning the interests of the platform and regulators.

Regulating algorithmic filtering on social media
Sarah H. Cen
Devavrat Shah MIT EECS devavrat@mit.co edu
MIT EECS
shcenCmit.
Abstract
By filtering the content that users see, social media platforms have the ability to influence users' perceptions and decisions, from their dining choices to their voting preferences. This influence has drawn scrutiny, with many calling for regulations on filtering algorithms, but designing and enforcing regulations remains challenging. In this work, we examine three questions. First, given a regulation, how would one design an audit to enforce it? Second, does the audit impose performance cost on the platform? Third, how does the audit affect the content that the platform is incentivized to filter? In response to these questions, we propose method such that, given a regulation, an auditor can test whether that regulation is met with only black-box access the filtering algorithm. We then turn to the platform's s perspective. The platform's 's goal is to maximize an objective function while meeting regulation. We find that there are conditions under which the regulation does not place high performance cost on the platform and, notably, that content diversity can play key role in aligning the interests of the platform and regulators.
Introduction
In recent years, there have been increasing calls regulate how social media platforms algorithnically filter the content that appears on a user's feed. For example, one may ask that the advertisements a user sees not be based (explicitly or implicitly) on their sexual orientation [65 or that content related to public health (e.g., COVID-19) does not reflect a user's 's political affiliation
35
However, translating regulatory guideline into auditing procedure has proven difficult. Develop- ing such method is the focus of this work. As our main contribution, we provide a general procedure such that, given regulation, auditor can test whether the platform complies with the regulation.
Providing a general procedure is important because, without it, auditing destined be reactive: auditors must design ways to enforce each regulation as they arise, and there is an inevitable delay between design and enforcement in order to test the proposed solutions.
On the other hand, designing a general procedure is challenging because audits can have unintended side effects the social media ecosystem and its many stakeholders. As a result, we consider two additional questions in this work:
1. How does the proposed auditing procedure affect the platform's 's bottom line? We consider whether the procedure imposes a high performance cost on the platform.
2. How does the auditing procedure affect the user's content? We consider what type of content the platform is incentivized to show the user when the platform complies with the regulation.
Our main contributions are summarized as follows.
Auditing procedure As our main contribution, we provide procedure such that, given regulation, an auditor can test whether the platform complies with the regulation (Section 3}. We restrict our attention regulations that can be written in counterfactual form, including the two examples given at the top of this Introduction. Namely, the procedure applies to any regulation that can be written as:
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

"the filtering algorithm F should behave similarly under inputs x and x' for all (x,x') E S". How to quantify "similarity" under x and x' is crucial, and our second main contribution is to provide a precise notion of "similaritt" in the context of algorithmic filtering (Section 2
Operationally, the auditing procedure has several desirable properties. First, it needs only black-box access to the algorithm and therefore holds even if the filtering algorithm F changes. Second, it does not require access to the platform's 's users or their personal data. Third, its parameters are interpretable and easy to tune. Finally, the procedure is modular, which allows for many possible configurations.
Provable guarantees. To audit counterfactual regulation, we begin by observing that algorithmic filtering is powerful (and often harmful) because information influences decisions: the content that a user sees can affect how they vote, whether they choose to receive a vaccine, what restaurants they frequent, and more. Therefore, if one seeks to enforce a counterfactual regulation, the notion of "similarity" that is enforced should be with respect the outcome of interest: the users' decisions.
Suppose that the auditor seeks to audit content of type (e.g., the advertisements or the entire feed). Suppose the filtering algorithm F generates content Z of type I when given inputs and Z' when given x'. In Theorem I we prove that, if F passes the audit, the decision-making of any user shown Z and their decision-making if shown Z' instead are (asympiotically) indistinguishable. This guarantee-walr
we call decision robustness and formalize in Section 2 is powerful because holds for any user and their decision- personal data (c.g., gender) or knowledge about the user's ' decision decision-making tendencies (c.g.. how easily user is influenced by restaurant advertisements). Providing such strong guarantee without access to users is made possible by two well-known concepts from decision and learning theory: the hypothesis test and canimum--variace unbiased estimator. Combining these tools is of our main technical insights and discussed in Section W
even though the audit does not have access to the users'
Cost of regulation. In Section 5. we study how the audit affects a platform's 's ability to maximize an objective function Rtwhich we refer to as rewarl-and find that being audited does not necessarily place a high performance cost the platform. Studying the cost of regulation is important because there are concerns that regulations can hurt innovation or profits, and our findings surface conditions under which the analysis in Section 5.1 is applicable to any R. As examples, R could measure time spent on the platform, the number of clicks on posts, or combination of these factors.
trade-off does not exist. Because we leave R unspecified,
Content diversity. In Section 5.2 we turn our attention to how an audit would affect the users' content and discover unexpected connection to content diversity. We find that, under regulation, social media platforms are incentivized to add doses of content diversity. Specifically, when faced with regulation, it is in the platform's 's interest to ensure that the content of type T that it shows users is sufficiently diverse along the dimension by which x and x' differ for all x') E S. Because content diversity is not part of the regulatory test by design, this result is unexpected and suggests that content diversity plays a key role in aligning the interests of regulators and platforms.
All proofs are given in the Appendix as well as toy example and further discussion of the audit.
2 Problem statement
Consider a system with two agents: a social media platform and an auditor.
1. The platform selects the content that is shown to its users using a filtering algorithm F X + Z such that Z = F(x) is the feed produced by F given inputs x e X. Here, a feed is collection of content that is shown to a user, and x captures any inputs that the platform uses to filter, such as the user's interaction history, social network, available content sources, and so on. Each feed Z = {z1..... 2m) consists of m pieces of content, where z, E R for all i E [m]. We assume that J can be written as a generative model such that F. generates the content in Z by drawing
samples from distribution (=(:; 0(x)), where 0(x) E θ E R" is unknown]] | Note that if one is only concerned with a certain type of content (e.g., the auditor only wishes to monitor advertisements), then one can take Z to be the content of type T that is filtered by F rather than the entire feed.
'This representation is without loss of generality, For example, any deterministic mapping from x to Z can be achieved by letting nm, 0(x) (Zi..... ...., 2,.) and sequentially generating 2; from the entries of (xx).

---

2.
The auditor is given a regulatory guideline that they wish to enforce. The auditor's s goal is check whether the platform's filtering algorithm F is in compliance with the given regulation. We assume that the auditor has black-box access to F. In other words, the auditor can run on a set of inputs (x,) ) and observe its outputs { = =[(x))) Note that the inputs x and x' need not correspond to real users and could represent hypothetical users.
In this work, we restrict our analysis counterfactual regulations. Specifically, the auditor is given a regulation in the form: "The filtering algorithm F should behave similarly on content of type T under inputs x and x' for all (x, x') S." In the remainder of this work, we omit references to type T and refer directly to content Z. We give three examples of counterfactual regulations.
Example 1. A regulation that prohibits targeted advertisements from being based on user's indicated sexual orientation [65] can be written as "the advertisements shown by F should be similar when given two users who are identical except for their sexual orientations"
Example 2. A regulation that requires articles containing medical advice on COVID- 19 be robust to whether the user is left- or right-leaning can be framed as "the articles that are selected by F and provide medical advice on COVID-19 should be similar for left- and right-leaming users".
Example 3. A regulation prescribing that a platform not sway voting preferences beyond serving as a social network can be framed as "content about political candidates that are injected by F should be similar to the content user would see from its social network without algorithmic filtering."
The goal of this work is to enforce a regulation of the form "F should behave similarly under x and x' for all (x, x') E S The question remains: What is appropriate notion of "similarity"?
2.1 Decision robustness
We begin by observing that algorithmic filtering is powerful (and often harmful) because information influences decisions: the content that a user sees can affect how they vote, whether they get vaccinated, what restaurants they frequent, what items they purchase, and more. Stated differently, if algorithmic filtering did not influence users' decisions, then there would be no desire to regulate it.
Therefore, if one seeks to enforce a counterfactual regulation, the notion of "similarity" that is enforced should be with respect to the outcome of interest: the users' decisions. However, an auditor does not and should not have access to the users their decisions (e.g.. whether they get vaccinated). As such, the problem that the auditor faces can be stated as follows.
Consider a (hypothetical) user. Suppose that there are two identical worlds except that the user is shown F(x) in one world and F(x') in the other. Because the worlds are otherwise identical, both users subsequently face an identical set of queries Q (e.g.. where to eat, whether to get vaccinated, what to wear). Let DD and D' denote the (hypothetical) decisions that the first and second users make given queries Q. Then, the auditor enforces similarity by ensuring decision robustness as follows:
F is decision-robust to (x, x') if and only if, for any user and any Q, one cannot determine with high confidence that x & x' from the decisions D and D'.
Decision robustness guarantees that the decision-maker is indistinguishable with respect to x'). However, ensuring decision robustness is challenging because the auditor does not have access to users or their decisions. Our objective is to provide an auditing procedure that guarantees decision robustness given only S and black-box access to F.
behavior of any user under F(x) and F(x')
2.2 Formalizing the auditor's goal
Recall that J is decision-robust to (x, x')-and therefore complies with the regulation- for any user and any Q, one cannot determine with high confidence that x # x' from D and D'. In this section, we show that decision robustness can be expressed as binary hypothesis test.
Formally, consider a pair of inputs (x, x') ) E S and set of queries Q. To "determine whether x * x' from D and " is equivalent using D and DD' decide between the following hypotheses:
Ho : (x) = (xx)
H1 (x) + 0(x')
2 This example is simplified to illustrate a simple counterfactual regulation. To protect against more nuanced effects, such as proxy variables, one could modify both sexual orientation and its proxies, Producing counterfactual inputs is out of the scope of this work. We direct interested readers other texts (58 41 52 43
3

---

To see this equivalence, observe that we can write the Markov chain x 0(x) + - - D. In other words, decisions D depend on inputs x only through the parameters (xx). If one cannot determine that (x)= + 0(x') from D and D', then one also cannot determine that x + x'.
Let H E (Ho, H1) denote the true (unknown) hypothesis Let H E (Ho. H1} denote the hypothesis that is chosen, where the outcome H H1 is equivalent to determining that (xx) * 0(x').
We say that test ff is (1-e))confice) conffidentt that 0(x) + (xx') if ft H1 and P(H Hi Ho) < e E [o, I While one would like the test to (1 - ))-coffident, a trivial test that always chooses Ho is (1 e)-confident but has 100 percent error when H H1. Therefore, one would also like the test to satisfy P(H Hol H Hi) I for some small [o, 1], but not all a [0. may be achievable while retaining the property of (1 - -c-offddeeece..
To this end, we turn to the uniformly most powerful unbiased (UMPU) test 21 which one can think of as follows: if the UMPU test cannot confidently determine that ((x) + ((x'), then no other reasonable test
Formally, suppose that one would like to find test that maximizes the true positive rate (TPR) while ensuring the false positive rate (FPR) is at most e such that the test solves:
max i P(H Hi|H Hi)
S.t.
P( Hi Ho) se
test H solves for all ((x), 0xx) E O, then it is the uniformly most powerful (UMP) test. The UMPU test is the UMP test among all unbiased tests, where a test H is unbiased if P( - H = H1) 2 8 2 P(H = H = Ho) for some 8 E [0, 1]
Let H! denote the UMPU test, if it exists, and P( = the test that is best at detecting when F not decision-robust (i.e., it maximizes the TPR) while making sure that it rarely falsely accuses F of not being decision-roke roobuot (i.e., its FPR is at most c) among all reliable (i.e., unbiased) tests. Given the UMPU test, as follows. For C, [0, 1] and when the UMPU test exists,
= P(t:
Hol = H1). Intuitively, H; is
can be formalized
F is (e, o))dde-csiion-oouus
(x,x') > for any Q, P(H: Ho| H1) S..
The goal of the auditing procedure. Therefore, determining whether a platform's filtering algorithm F complies with counterfactual regulation comes down determining whether, for all Q and (x,x') E S, the UMPU test cannot confidently reject Ho given D and D'. However, this task is not straightforward because the auditor's goal is to provide guarantee on how F affects users' decisions without access to the users or their decisions (i.c., without Q, D, or D'). In this work, we show that is possible to guarantee approximate asymptotic pecision-robustned given only S and black-bol access to F using insights from statistical learning and decision theory.
3 Auditing procedure
In this section, we present a procedure such that, given a regulation on algorithmic filterin that is expressed in counterfactual form, auditor can test whether the platform' filtering algorithm is in compliance with the regulation. Section we show that, if F passes the audit, then F is approximately asymptotically decision-robust. In Section [ we study the cost of regulation and find that there are conditions under which the audit does not place a performance cost on the platform.
3.1 Notation and definitions
Before proceeding, we require some notation and definitions. Recall that F generates Z by drawing m samples from Pz (x)), where (x) is unknown. In statistical inference [46), an estimator is
mapping L Z θ such that C(Z) is an estimate of the parameters 0 that generated Z.
Definition 1. An estimator L Z - θ is unbiased if and only if Ep ()) (())) = O for all 0 E O.
and knows whether x * x', cannot determine this fact from D and D'. Therefore, the hypothesis test treats x and x' as unknown.
Although auditor has access
requires that one
*I is taken with respect px (-: (x)) and (: (::;((x')))
"Inuiiivvly. an unbiased test A ensures that the probability that fl chooses x o x' is always higher when Hi is true than when Ho x' is true.

---

Algorithm 1: Modular version of auditing procedure
Input: Regulation parameter E [0, 1]; model family C Re; Mack-box access to the filtering algorithm F X' + 2; pair of counterfactual inputs (x, x') E S.
Result: H. Hu if the test is passed; H. H1, otherwise.
(F((x)))
0 < (-
3 (0- - (T) (() 0') x12(1 - -) then
Return H. = H1;
end
6 Return H. = Ho:
Definition 2. When it exists, the connimum-variancee unbiased estimator (MVUE) C+ z - θ is an estimator that is unbiased and has the lowest variance among all unbiased estimators, i.e,, satisfies for all unbiased C: Z - and all O.
(p _) (Z) - 0)²] Ep.(.0)
Let X denote the chi-squared distribution with r degrees of freedom and (²) be defined such that F(u (g))
where X². Lastly, let (() E "2"x" denote the Fisher information matrix at O. An exact definition of (0) is given in the Appendix. Intuitively, (0) captures how well an estimator can learn 0 from Z. As simple example, suppose z, are drawn i.i.d. from ((,, o²), where O² is known, r = 1, and = μ.. It takes more samples to accurately estimate μ when the variance o² is large, and, as expected, the Fisher information scales with 1/0².
3.2 The audit
In this section, we present the auditing procedure. Recall that a counterfactual regulation requires that F behave similarly under and x' for all (x, x') E S. Algorithm [ provides a test for determining whether F complies with the given regulation for pair of inputs (x. x') E S. To test other pairs in S, simply repeat Algorithm| and modify x and x' accordingly. If H. = H2 for any pair, then the platform does not pass the audit. Below, we list and explain several characteristics of the audit.
Modularity. Algorithm| iss designed to be modular. Modularity allows the auditor to construct the audit as they wish. For example, the auditor may wish to add more pairs S or repeat the audit every month. Alternatively, the auditor may require not that F behave similarly under x and x' for all (x, x') S but for at least (1 a) E [0, 1] of them To do so, the auditor over S and, the number of times it returns H, exceeds a S), then the platform does not pass the audit. Modularity also allows the auditor to see the pairs (x, x') for which F fails the test.
Algorithm I
Tunable parameter. One benefit of the procedure is that the tunable parameter e has an intuitive meaning. We see in Section 4. that is maximum FPR. Capping the FPR ensures that the auditor is not distracted by red herrings and prevents the auditor from investing the resources needed to investigate (or bring case against) the platform unless they are at least (1 e) ecconfiddnt that the platform violates the regulation. Decreasing e E [0, 1] reduces the number of false positives while increasing makes the regulation more strict (at the risk of receiving more false positives).
Other advantages. In addition to the benefits regarding modularity and e discussed above, this procedure has two additional advantages. First, the procedure does not require access to users or their personal data. Second, it requires only black-box access to F, which means that an auditor does not need to know the wnrkings of F (there is often resistance to giving auditors full access to F) and, perhaps more importantly, the procedure works even when F changes internally.
When the MVUE does not exist, use the MLE. Recall that - denotes the MVUE. We will see that there is a theoretical justification for using the MVUE (Proposition 2 However, there are cases in which the MVUE does not exist but the maximum likelihood estimator (MLE) does. The MLE is good substitute for the MVUE because the MVUE and MLE are often asymptotically equivalent (59]. When this asymptotic equivalence holds, the main theoretical guarantee -namely, Theorem ! holds when L+ taken to be the MLE instead of the MVUE.
Here, [0, 1] would correspond to the maximum allowable false negative rate (FNR).
5

---

Symmetry. Algorithm is not symmetric with respect to θ and 0' (or, equivalently, x and x'). This can be useful if the aucitor would like to have a baseline input x and run Algorithm over different x'. If the auditor would like symmetry, they may wish to run Algorithm [ [wice, swapping the order of x and x', or to alter the Fisher information matrix in Line to be I((@ + 22), if it exists.
Choice of O. Recall that O captures the set of possible generative models. In choosing the model family O the auditor may find that a simple θ is more tractable and interpretable while a complex O is more general. As explained in Section 4. O can also be viewed as the set of possible cognitive models that users employ when making decisions. Therefore, the auditor may wish to choose θ to be just rich enough to mirror the complexity of common cognitive models.
4 Explaining the procedure and its theoretical guarantees
Recall from Section 2 that a filtering algorithm F complies with counterfactual regulation if F is decision-robust. In Section 4.1 we show that, if the platform passes the audit in Algorithm | then JF is guaranteed to be approximately asymptotically rrobuitn-robust. In Section 4.2 we provide insights on the role of the MVUE. All proofs are given in the Appendix.
4.1 Guarantee on the audit's effectiveness
Theorem 1. Consider ①1 Let 0* (0(x) ((xx))/2, Suppose that z, and z are drawn i.i.d. from
pl:; (x)) and pl:; ((x xx)) }), respectively, for all i E [mm] and P = {px(:: 0) : 0 E O) is a regular
exponential family that meets the regularity conditions stated in Appendix B If H is defined as:
2 2 x2 - -),
A = H1 = (C+ - (1))))- -2)00(=+(2)))
(3)
m
then P(t = HI Ho) S E as m - co. Ifr = 1, then limm-+o0 P(t = Ho) H = = (t).
Understanding the result. Recall that the goal of an auditor to determine whether the platform's filtering algorithm F is compliant with a given regulation by determining whether F is decision- robust. Theorem I confirms that the audit in Algorithm D enforces approximate asymptotic decision robustness. To see this connection, observe that the test 33 is identical to the test in Algorithm I
with one substitution- (Z) is replaced by wwhic implies that the test in B is asymptotically equivalent to the audit. Therefore, Theorem establishes that, if Algorithm returns H. = H1, then the auditor is (1 -ccondddett that J is not decision-robust as m x0
Intuitively, if the platform passes the audit in Algorithm I then the decision-making of any user that is shown F(x) is guaranteed to be c-indistnnguishalle from their decision- makingg should they have been shown F(x') instead. Importantly, this guarantee is provided without access to users or their decisions with the help of insights from statistical learning theory (see proof of Theorem DI.
few remarks. First, is false positive rate (FPR). Decreasing increases the confidence that the auditor would like to have should it pursue action against the platform (see Section B) Second, recall from Section that Algorithm D is modular and that, in most cases, one may wish to repeat it several times with different inputs. In that case, the result of Theorem I holds for each individual run. Lastly, Theorem provides an asymptotic guarantee on the audit. In the next section, we show that, by using the MVUE, Algorithm [ ensures a notion of decision robustness even for finite
4.2 Insight on the MVUE
Recall that the auditor's is goal is to provide guarantee with respect to users' decisions, but the auditor does not have access to the users or their decisions. In this section, we provide intuition for how the auditor enforces decision robustness without this information, and we explain how the use of the MVUE in Algorithm allows the auditor to enforee a notion of decision robustness for finite m. One can think of the MVUE as providing an "upper bound" on how much content Z can influence user's 's decisions. This result is useful because it allows the auditor to reason about users' decisions without access to users their decisions, both which can be expensive or unethical to obtain.
We say that, if F passes the audit, it is approximately decision-robust because H. is not the UMPU test, defined in Section Obtaining a UMPU test is generally difficult for r >> 1. but the test H, is not far from the UMPU test, as demonstrated by the fact that it is the UMPU test when = 1.

---

User model. A user's precision-making process proceeds in three steps: the user observes information, updates their internal belief based on this information, then uses their belief to make decisions. Let
Pz(:;0) ;) denote the belief of a (hypothetical) user who is shown content Z. where 0- θ Let the
influence that Z has on a user's ss belief be denoted by L 2 + 2 such that Ô = ( P
Example 4. As highly simplified example, suppose θ = [-1,, x [0, 00) and P2(:; 0) is a Gaussian
distribution with mean 01 and variance 02. where 0, = -1 implies that the user does not believe that vaccines are effective, 01 = 1 implies the opposite, and 6g scales with the user's 's uncertainty in their belief. If a user is easily inftuenced, then they might develop the belief ((Z) = (-0.8, 0.1) after being shown content Z with anti-vaccine content. Alternatively, the user could be confidently pro-vaccine and very stubborn so that no matter what content they see, [(-) = (1,0).
Suppose the user is given a query Q E Q. for which the user decides between two options: Ag and A,, e.g., whether or not to get vaceinated. (Note that any decision between finite number of options can be written as series of binary decisions.) Internally, the user places a value on each choice such that, if user were given 0, the user would choose Ag if 0o (0) 2 11 (0) and A1, otherwise.
However, 0 is unknown. Instead, the user has a belief i that they use to infer whether vo(0)= 2 in (0). For example, a user does not know the ground truth values of being vaccinated versus unvaccinated, which may depend on many factors, such as the user's underlying medical conditions. In the absence of these values, the user forms belief about the value of receiving vaccine based on information that they ingest. The user's decision precision-making process is therefore a hypothesis test between:
Go : v 2 01 (0)
G1 vv) > (0)
(4)
The following result motivates the use of the MVUE C+ in the audit by demonstrating that the MVUE enforces a sample version of decision robustness.
Proposition 2. Consider 4 Let G E (Go; G1} denote the true (unknown) hypothesis. Suppose
that to. : θ - T. are affine mappings and there exists u : RD -> U such that, for F(x) = {zi...., }. one can write u(z,) ~ Pu(:; U1 (0(x)) vo (0(x))) for all i E [m]. Then, if the UMP (or UMPU) test with a maximum FPR of p exists, it is given by the following decision rule: reject Go
(choose A1) when the sinimum-vvrrinnee unbiased estimate θ = L (Z) satisfies ei vo > 1pp
where ((11(0) (0) > Gp = Go) = pr otherwise, accept Go (choose Ag).
Interpretation and implications. The auditor is interested how users react to their content, which is captured by L. However, L may difficult or even unethical to obtain. For example, an auditor may wish to infer how advertisements affects a user's behavior, but doing so may require access to the user's personal data. Proposition says that, under the stated conditions, if one wishes to study the impact of content on users' decisions, one can focus on the MVUE L because, among all possible users, the one whose decisions are most influenced by their content is the hypothetical user given by the C+. One can think of this hypothetical user as the "most gullible user". Because Line 3 in Algorithm| 1 requires L+ (Z) and L (Z') are sufficiently close, the audit enforces approximate decision robustness by ensuring that the counterfactual beliefs L (Z) and (z') and therefore
the counterfactual decisions DD and of the most gullible user are indistinguishable.
Understanding the MVUE. For intuition on why the MVUE the "most gullible user", recall that the MVUE is the unbiased estimator with the lowest variance. Suppose that a user's estimate Z(Z) differs from L (Z). By definition, this estimate is biased or has higher variance. When biased, the user's estimate is consistently pulled by some factor other than Z. For example, user who remains pro-vaccine no matter what content they see has a biased estimator. When the user's estimate has higher variance than the MVUE, it is an indication that the user places less confidence than the MVUE in what they glean from Z. For example, the user could be skeptical of what they see on social media scrolling very quickly and only reading headlines. In this way, the MVUE corresponds to whose decisions are most affected by their content 2.
the user who "hangs on every
'The use of distributions to represent beliefs is common the cognitive sciences 2> Note that, although representing heliefs as distributions is borrowed from Bayesian inference [0.. our representation does not require that people are Bayesian (update their beliefs according to Bayes" rule), which is contested [.]
"A user's belief may be impacted by information other than Z (c.g.. the user's previous belief, news that they receive from friends, or content that they view on other platforms). This can be modeled by letting L: 2 x J + θ where J E I captures off platform information. Because it does not change our results (see the Appendix), we use L: Z θ for notational simplicity.
7

---

5 Cost of regulation and the role of content diversity
In this section, we turn our attention to how the auditing procedure affects (a) the platform's ability to maximize an objective function R and (b) the type of content the platform is incentivized to filter when compliant with a regulation. In Section 5.1 we find that there are conditions under which the audit does not place performance cost the platform and, intuitively, this occurs when the platform has enough degrees of freedom with which to filter. We show in Section [.22 that one of the ways the platform can increase R while complying with the regulation is to incorporate sufficient content diversity. Because diversity does not appear in the audit by design, this result suggests that content diversity align the interests of regulators and platforms. All proofs are given in the Appendix.
5.1 Cost of regulation
Suppose that the platform's goal is to maximize an objective function R: 2 x X + R--hhich we call reward-while passing the audit. For example, R could be a measure of user engagement, user satisfaction, content novelty, or a combination of these and other factors. We leave R unspecified, which means that our analysis holds for any choice of R, unless otherwise stated [00
Recall that 2 denotes the set of all possible feeds (or collections of type-T content). Complying with a counterfactual regulation defined by S is equivalent to restricting the platform's choice of content to subset (SS) C Z, which we call the feasible set under S. Intuitively, the stricter the regulation, the smaller the feasible set (S)). If there is no regulation, then Z(S) = Z. As such, the platform's 's goal to maximize R given inputs while complying with the regulation can be expressed as:
Z arg max R(W, x). WEZ(3)
Platforms are often interested in whether regulation imposes performance cost. To make this notion precise, we define the cost of regulation as follows.
Definition 3. The cost of regulation inputs x is: C' = maxwez R(W,x) - max R(W, x).
low cost of regulation implies that the platform can meet regulation without sacrificing much reward, while high cost of regulation implies that there is strong performance-regulatie trade-off.
Performance- trade-off. Suppose that the feasible set shrinks from 2 to Z(S). Then, that all decreases, and the cost of regulation increases. Alternatively, if the feasible set shrinks but at least one neward-maximizing
the maximum achievable reward is affected in one of two ways. If the feasible set shrinks such
solutions in Z are not contained in ((S), then the maximum reward
solution in Z is contained in ZS)). then the maximum reward stays the same, and the cost of regulation does not increase. Therefore, as long as Z(S) preserves at least one (near) optimal solution, the cost of regulation is low. The following result formalizes this notion. For this result, we overload the notation R such that R(C (Z). x) denotes R(Z,x) x).
Theorem 3. Suppose there exists Ω C [r] where 1 < | < such that R(01,x) R(02,x) x) if 01,1 02 for all & Q. Suppose that, for any 0 E θ, > 0 and v E R", there exist a vector On where 9 = 0 for all & Z2 and constant K. >> such that v 1(0+ =00) and 0 kōn E θ. Then, if m < cc, there exists a set Z such that the cost of regulation for x under Algorithm is 0.
Interpretation of the result. Imposing a regulation restricts the platform's 's feasible set, which may place performance cost on the platform. However, a high cost of regulation is not inevitable. Indeed, the feasible set contains at least one (near) optimal solution, then the cost of regulation is low. Theorem 3 provides set of conditions under which the cost of regulation is low. Intuitively, the result states that, when R is independent of at least one element in the parameter vector @ and that element has sufficient leverage over the Fisher information, then as long as the amount of content in Z is finite (i.e., m < oo) and the available content 2 is expressive enough, then the platform can always construct Z from Z that passes the audit without sacrificing reward.
One may ask whether the conditions in Theorem B are fcasible. To illustrate that the conditions are achievable, consider the following highly simplified example.
There are settings in which R is varying e.g., when platform's 's sources of revenue change with time. Making R time-varying does not change our analysis or findings. Therefore, leave H static for simplicity.
8

---

Example 5. Suppose θ -= 02, 6 - - R2o. and - (N(u, :p,,0 ). In other words, we consider the family of I-D Gaussian distributions. If R is a function of the mean μ but not the variance o², then Theorem 3 Japplies To see this, observe that Q = (2) and let Og = (0. 1). Recalling that ((1,,²)) = diag(a
12), the entries of ((0) can be made arbitrarily small by [ the smaller the entries of T(Ø), the easier it is for the platform to pass the audit. Therefore, if " is high-reward but not in the feasible set, the platform can still achieve R(Z", x) ,x) by increasing the content's 's variance to obtain a new collection of content Z that is in the feasible set. As long as Z and Z" share μ,, R(Z, x) = R(Z', xx).
increasing 02 = o². By Line 3 in Algorithm|
Theorem provides conditions under which there is no cost of regulation. These conditions can be relaxed if we are interested in scenarios for which the cost of regulation is low but not zero. We build this intuition in the next section, studying one of the ways a platform can achieve high reward while remaining compliant with regulation.
5.2 Content diversity
In this section, we show that one way that the platform can increases its reward while complying with the regulation is to ensure that the filtered content is sufficiently diverse. These results suggest that content diversity may help to align the interests of regulators and platforms.
We first formalize content diversity, then show how it relates to passing the proposed audit.
Definition 4. For v E IR" and Zo Zi E Z, content Zo is more diverse than Z1 along if the Fisher information matrices at (7o) and L+ (71) satisfy: v T (I))+ - (()))))) v > 0.
Interpretation. This definition says that the "smaller" the Fisher information, the higher the content diversity. The Fisher information matrix I [(0) can be viewed as measure of how easy it is to learn O from the content that is generated by (= Consequently, when Z and Z' have low content diversity along (0(x)= ((x') )), an auditor can, without much effort, learn that ((x) is different from (x') and therefore that x o x'. Recall from Section 2.1 that being able to say with high confidence that x A x' implies that F is not decision- robusst and therefore does not comply with regulation. In this way, low content diversity reduces the likelihood that F passes the audit 1T Note that the Fisher information matrix captures two notions of diversity- the diversity of topics in Z and the diversity of perspectives on each given topic in Z-simultaneously.
Connection between content diversity and the cost of regulation. Recall from Algorithm I that F passes the audit when (0- 0') (()) -0) is below some threshold. The platform can therefore pass the audit by ensuring that [(0) is sufficiently "small". By Definition 4 whether the Fisher information is "small" is precisely an indication of content diversity.
In this way. increasing content diversity gives the platform more leeway when filtering. By "shrinking" the Fisher information, the platform obtains more flexibility in setting (Ø - 0'). Stated differently, if I(0) is "large", then the platform is more constrained because (Ø 0') must be very small in order for the platform to pass the audit. Therefore, if the platform has a high-reward Z that does not pass the audit (i.e., is not in the feasible set), then the platform can generally maintain a high reward while complying with the regulation by adding content diversity 12
the Fisher information
Because content diversity is not part of the audit by design, this result is unexpected. It states that the audit naturally incentivizes the platform to include a sufficient amount of content diversity with respect to x x'. Returning to Example [ if regulators require that medical advice COVID-19 be robust to whether a user is left- or right-leaning, then the differences between the medical advice shown to across the political spectrum is captured by x'. Adding content diversity along this dimension means that right-leaning users receive medical advice on COVID-19 not only from right-leaning news outlets, but also from left-leaning ones, and vice versa.
"l "Content diversity can also be understood in the context of Section 4.2 Suppose that the content diversities of 7 and 7' are very low. For example, suppose that 7. contains only pro-vaccine content and Z' contains only anti-vaccine content. Then, the MVUE would leam strong relationship between vaccines with positive outcomes from 7. and vice versa for z'. If Q contains a query about vaceines, 'D and D'' would reflect these strong beliefs. In this way. J is less likely to be decision-robust when the content diversity is low. On the other hand, if the content for all users contains both pro- and anti- vaccine content, then D and D' are more similar.
12e platform does not increase content diversity indefinitely because the platform must also ensure that
other terms in Line 3 specifically, (0- 0¹) do not cause the platform to fail the audit.

---

Background & related work
Algorithmic filtering 28 has the potential to greatly improve the user experience, but it can also yield unwanted side effects, like the spread of fake news 19 23 25
over-representation of polarizing opinions due to comment ranking [1]. amplification of echo chambers due to filter bubbles (34, 56], or advertising of products based on discriminatory judgments about user interests [62
Although the severity of these effects contested- for example, some studies argue that political polarization and echo chambers are not always products of internet use or data-driven algorithms
6 44
40
soccil media platforms are under rising scrutiny.
In response, some platforms have begun to self-regulate [5] [11 For example. Facebook has estab- lished an internal "Supreme Court" that reviews the company's 's decisions [55]. Twitter has banned accounts associated with the spread of conspiracy theories ( YouTube has removed videos it
views as encouraging violence [64]. and so on. Self-regulatory practices offer various benefits, such as the ability to adapt quickly to a changing social media ecosystem and the comparatively greater information access afforded to internal auditors than external ones. However, many argue that self-regulation is insufficient and that governmental regulations are necessary in order to ensure that audits are executed by independent bodies. Several regulations exist, such as the EU's General Data Protection Regulation [57] and Germany's ' Network Enforcement Act [19]. There are also ongoing efforts, such as the push to review Section 230 of the U.S. Communications Decency Act [
Designing such regulations and auditing procedures remains a challenging problem. in part due to the number of stakeholders. For one, there are various legal and social obstacles facing regulations including concerns that regulations might damage free speech or public discourse; violate personal rights or privacy; transfer agency away from users to technology companies or governmental bodies; draw subjective lines between acceptable and unacceptable behavior; or set precedents that are difficult to reverse. In light of the thriving exchange of goods between users, platforms, advertisers and influencers that is facilitated by social media, many also fear that regulations may hurt innovation, lead to worse personalization, or block revenue sources
45
17 9
54
Current efforts to audit content moderation generally focus on specific issues, such as whether content inappropriate (c.g.. posts that contain hate speech or bullying 7 26) ): discriminatory (e.g., race- based advertising
3 44) ); divisive (e.g., comment ranking algorithms that favor polarizing
comments 61); insulating (e.g.. filter bubbles 34 ), or misleading (e-g.. fake news 49 23
These works generally use one of the following strategies: increasing content diversity (e.g.. adding heterogeneity to recommendations [6 BB)); ); drawing a line in the sand (e.g., determining whether discrimination has occurred by thresholding the difference between two proportions (24)); or finding the origin of the content (e.g., reducing fake news by whitelisting news sources (9). this work, we provide general procedure such that, given a regulation in counterfactual form, auditor can test whether the platform's filtering algorithm is compliant. Our aim is to audit with respect to the outcome of interest in order to avoid unwanted side effects. Of particular note is that the proposed procedure does not require access to users or their personal data.
We also consider how an audit affects the platform's 's ability to maximize an objective function as well as the content that the platform is incentivized filter for a user. Our formulation an instance of
constrained optimization and bears resemblance to robust optimization [1] 57 For instance, our
definition of the cost of regulation mirrors the "price" of robustness studied in other works [ [12.
Similarly, the performance-regul regullaiin trade-off that we discuss echoes the trade-offs that appears in
other problems which there are fairness [12
33 42 and privacy 20 39 constraints. Our findings
that there are conditions under which content diversity aligns the interests of regulators and platforms adds to the conversation on presenting different viewpoints on social media. For instance, Levy 4% finds that users respond well when presented with multiple political viewpoints- content-ever ones from an opposing political party. Increasing content diversity is also at the heart of other methods, including those for bursting filter bubbles [16] improving comment ranking 37 It is worth a note that our definition of content diversity captures two notions: diversity in the topics as well as diversity in the viewpoints each topic.
Levy terms
counter-attitudine
As a final remark, our analysis has parallels with differential privacy 29 30 in that it compares outcomes under different interventions [66]. However it differs in the techniques required to study similarity. Our work also touches on aspects of but is distinct from social learning and opinion in that we study how information affects the belief's of individuals.
dynamics 2 6
5
10

---

Acknowledgments and Disclosure of Funding
We would like to thank our anonymous reviewers and area chair for their time and suggestions. We would also like to thank the many people who provided feedback along the way, including but certainly not limited to Martin Copenhaver, Hussein Mozannar, Aspen Hopkins, Divya Shanmugam, and Zachary Schiffer. This work was supported in parts by the MIT-IBM project on "Representation Learning as a Tool for Causal Discovery", the NSF TRIPODS Phase grant towards Foundations of Data Science Institute, the Chyn Duog Shiah Memorial Fellowship, and the Hugh Hampton Young Memorial Fund Fellowship.
References
[1] 47 U.S. Code 230 Protection for private blocking and screening of offensive material.
[2] Democracy Under Threat: Risk and Solutions in the Era of Disinformation and Data Monopoly.
[3] Daron Acemoglu, Munther A. Dahleh, Ilan Lobel, and Asuman Ozdaglar. Bayesian Learning in Social Networks. The Review of Economic Studies, 78(4):1201-122 2011.
[4] Julia Angwin, Ariana Tobin, and Madeleine Varner. Facebook (Still) Letting Housing Advertis- ers Exclude Users by Race, 2017.
[5] R. R. Bahadur. On Fisher's Bound for Asymptotic Variances. The Annals of Mathematical Statistics, 3554511545-1552 1964.
[6] Abhijit Banerjee. A Simple Model of Herd Behavior. The Quarterly Journal of Economics, 107(3):977-817, 1992.
[7] BBC News. Social Media: How Can Govermments Regulate It? April 2019.
[8] BBC News. Twitter suspends 70,000 accounts linked to QAnon. January 2021.
[9] Hal Berghel. Lies, Damn Lies, and Fake News. Computer, Computer,50(2):80-88 2017. 2017.
[10] J.M. Bernardo and A.F.M. Smith. Bayesian Theory. Wiley Series in Probability and Statistics. Wiley, 2009.
[11] Dimitris Bertsimas, David B. Brown, and Constantine Caramanis. Theory and Applications of Robust Optimization. SIAM Review, 53(3):464-5011 10164-51,,10 2011.
[12] Dimitris Bertsimas, Vivek F. Farias, and Nikolaos Trichakis. The Price of Fairness. Operations Research, 59(1):17331, 2011.
[13] Dimitris Bertsimas and Melvyn Sim. The Price of Robustness. Operations Research, 52(1):35- 53, 2004.
[14] Levi Boxell, Matthew Gentzkow, and Jesse M Shapiro. Greater internet use is not associated with faster growth in political polarization among us demographic groups. Proceedings of the National Academy of Sciences, 114(40):061221066
2017.
[15] Engin Bozdag. Bias in algorithmic filtering and personalization. Ethics and Information Technology, 15(3):209-22) 2013.
[16] Engin Bozdag and Jeroen van den Hoven. Breaking the filter bubble: democracy and design. Ethics and Information Technology, 17(4):249-2) 2665,,6,,,,, 2015.
[17] Valerie C. Brannon. Free Speech and the Regulation of Social Media Content, 2019.
[18] Sébastien Bubeck and Nicolo Cesa-Bianchi, Regret analysis of stochastic and nonstochastic multi-armed bandit problems. arXiv preprint arXiv: /24 2012 2012.
[19] German Bundestag. Act to Improve Enforcement of the Law in Social Networks (Network Enforcement Act), 2017.
11

---

[20]
James Campbell, Avi Goldfarb, and Catherine Tucker. Privacy Regulation and Market Structure. Journal of Economics Management Strategy, 24(1):47-73, 2015.
[21] George Casella and Roger L Berger. Statistical inference. Cengage Learning, 2021.
[22] Nick Chater, Joshua B Tenenbaum, and Alan Yuille. Probabilistic models of cognition: Concep- tual foundations. Trends in cognitive sciences, 10(7):287-291, 2006.
[23] Bobby Chesney and Danielle Citron. Deep Fakes: Looming Challenge for Privacy, Democ- racy, and National Security. California Law Review, 107:1753-1820, 2019.
[24]
Alexandra Chouldechova. Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments. Big Data, 5(2):153-163, 2017.
[25] Damian Collins, Clive Efford, Julie Elliott, Paul Farrelly, Simon Hart, Julian Knight, Ian C. Lucas, Brendan O' O'Hara, Rebecca Pow, Jo Stevens, and Giles Watling. Disinformation and 'fake news": Final Report, February 2019.
[26] Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. Automated Hate Speech Detection and the Problem of Offensive Language. In Proceedings of the Eleventh International AAAI Conference on Web and Social Media. AAAI, 2017.
[27] Morris H. DeGroot. Reaching Consensus. Journal of the American Statistical Association, 69(345):118-11,, 1974.
[28] Michael A. Vito,o, Darren Gergle, and Jeremy Birnholtz. "Algorithms Ruin Everything" #RIPTwitter, Folk Theories, and Resistance to Algorithmic Change in Social Media. In Proceedings of the 2017 CHIl Conference Human Factors in Computing Systems, CHI 17, pages 3163-3174. Association for Computing Machinery, 2017.
[29] Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating Noise to Sensi- tivity in Private Data Analysis. In Shai Halevi and Tal Rabin, editors, Theory of Cryptography, pages 265-284- Springer, 2006.
[30] Cynthia Dwork and Aaron Roth. The Algorithmic Foundations of Differential Privacy. Founda- tions and Trends in Theoretical Computer Science, 9(3-4):211-40) 2013.
[31] Frederick Eberhardt and David Danks. Confirmation in the cognitive sciences: The problematic case of bayesian models. Minds and Machines, 21(3):389-410, 2011.
[32] Benedict Evans. Regulating technology. https: / /www. www.bben-eeee evans. com/beneeicctevass/
July 2020.
[33] Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkata- subramanian. Certifying and Removing Disparate Impact. In Proceedings of the 21 ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '15, pages Association for Computing Machinery, 2015.
259-268..
[34] Seth Flaxman, Sharad Goel, and Justin M. Rao. Filter bubbles, echo chambers, and online news consumption. Public Opinion Quarterly, 10281))29,-3 March 2016.
[35] Federico Germani and Nikola Biller-Andorno, The anti- vaccination infodemic on social media: A behavioral analysis. PLOS ONE, 16(3):1-14, 03 2021.
[36] Ali Ghobadzadeh, Sayed Jalal Zahabi, and Ali A. Tadaion. The role of MVU estimator and CRB in binary composite hypothesis test. In 2009 IEEE International Symposium on Information Theoory pages 973-976. IEEE, 2009.
[37] Giorgos Giannopoulos, Marios Koniaris, Ingmar Weber, Alejandro Jaimes, and Timos Sellis. Algorithms and criteria for diversification of news article comments. Journal of Intelligent Information Systems, 44(1):1-47, 2014.
[38] Natali Helberger, Kari Karppinen, and Lucia D' Acunto. Exposure diversity as a design principle for recommender systems. Information, Communication & Society, 2112), 2018
12

---

[39] Dennis D. Hirsch. The Law and Policy of Online Privacy: Co-rRegulation. Seattle University Law Review, 34:439, January 2011.
Regulation, Self- Regulation, or
[40] Homa Hosseinmardi, Amir Ghasemian, Aaron Clauset, David M Rothschild, Markus Mobius, and Duncan J Watts. Evaluating the scale, growth, and origins of right-wing echo chambers on youtube, arXiv preprint (ex.i:20111.28444 2020.
[41] Lily Hu and Issa Kouler-Hausmann. What's sex got to do with machine learning. arXiv preprint arXiv:2006.01770 2020.
[42] Toshihiro Kamishima, Shotaro Akaho, and Jun Sakuma. Fairness-aware Learning through Regularization Approach. In Proceedings of / Ith International Conference on Data Mining Workshops, pages 653-650. IEEE, 2011.
[43] Atoosa Kasirzadeh and Andrew Smart. The use and misuse of counterfactuals in ethical machine learning. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 228-236, 2021.
[44] Pauline T. Kim and Sharion Scott. Discrimination in Online Employment Recruiting. St. Louis University Law Journal, 63:93, 2018.
[45] Kate Klonick. The New Governors: The People, Rules, and Processes Governing Online Speech. Harvard Law Review, 131:1598-1611 1670, 2017.
[46] E. L. Lehmann and George Casella. Theory of Point Estimation. Springer-Veriie New York, NY, USA, 2nd edition, 1998.
[47] Erich Leo Lehmann, Joseph Romano, and George Casella. Testing statistical hypotheses. volume 3. Springer, 2005.
[48] Ro' ee Levy. Social media, news consumption, and polarization: Evidence from a field experi- ment. American economic review, 111(3):831-7,, 2021.
[49] Becca Lewis and Alice E. Marwick. Media Manipulation and Disinformation Online, New York: Data & Society Research Institute, 2017.
[50] Alexander Ly, Maarten Marsman, Josine Verhagen, Raoul Grasman, and Eric-Jan Wagenmakers. Tutorial on Fisher Information. Journal of Mathematical Psychology, 80:40-55, 2017.
[51] Rotem Medzini. Enhanced self-regulation: The case of facebook's s content governance. New Media & Society, page 1461444821989352. 2021.
[52] Alan Mishler, Edward H Kennedy, and Alexandra Chouldechova. Fairness in risk assessment instruments: Post- Post-processing to achieve counterfactual equalized odds. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 386-400, 2021.
[53] Pooya Molavi, Alireza Sahbaz-Salehi and Ali Jadbabaie. A Theory of Non- Non-Bayesian Social Leamiing Econometrica, 86(2):445-490 March 2018.
[54] Jonathan A. Obar and Steven S. Wildman. Social Media Definition and the Governance Challenge. Telecommunication. Policy, 39(9):45--750 2015.
[55] Oversight Board. Ensuring respect for free expression, through independent judggmenn March 2020.
[56] Eli Pariser. The Filter Bubble: How the New Personalized Web Is Changing What We Read and How We Think. Penguin Publishing Group, 2011.
[57] The European Parliament and the Council of the European Union. General Data Protection Regulation.
[58] Pearl, M. Glymour, and N.P. Jewell. Causal Inference in Statistics: Primer. Wiley, 2016.
[59] Stephen Portnoy. Asymptotic efficiency of minimum variance unbiased estimators. The Annals of Statistics, pages 522-529, 1977.
13

---

[60] Bodhisattva Sen. Theoretical statistics-ii. ii. 2021.
[61] Stefan Siersdorfer, Sergiu Chelaru, Jose San Pedro, Ismail Sengor Altingovde, and Wolfgang Nejdl. Analyzing and Mining Comments and Comment Ratings on the Social Web. ACM Trunsactions on the Web, 8(3), July 2014.
[62] Till Speicher, Muhammad Ali, Giridhari Venkatadri, Filipe Nunes Ribeiro, George Arvanitakis, Fabrício Benevenuto, Krishna P. Gummadi, Patrick Loiseau, and Alan Mislove. Potential for Discrimination in Online Targeted Advertising. In Sorelle A. Friedler and Christo Wilson, editors, Proceedings of the 1st Conference on Fairness, Accountability and Transparency, volume 81 of Proceedings of Machine Learning Research, pages 5-19, New York, NY, USA, 23-24 Feb 2018. PMLR.
[63] Latanya Sweeney. Discrimination in Online Ad Delivery. Queue, 11(3):10-29 March 2013.
[64] The Independent. Amid Capitol violence, Facebook, YouTube remove Trump video. January 2021.
[65] Sandra Wachter. Affinity profiling and discrimination by association in online behavioural advertising. Berkeley Technology Law Journal, 35(2), 2020.
[66] Larry Wasserman and Shuheng Zhou, A Statistical Framework for Differential Privacy, Journal of the American Statistical Association, 105(489):375-38,, 2010.
[67] Wolfram Wiesemann, Daniel Kuhn, and Melvyn Sim. Distributionally Robust Convex Opti- mization. Operations Research, 62(6):1358 158 1376, 2014.
14

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] We describe the scope of our
work in Section O We also discuss the limitations our work throughout Sections
We make a few additional remarks in Appendix D
(c) Did you discuss any potential negative societal impacts of your work? [Yes] We discuss the potential negative impacts of the work Appendix and in Section S
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes] All proofs are given the Appendix.
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [N/A] We did not run experiments.
(b) Did you specify all the training details (e.g.. data splits, hyperparameters, how they were chosen)? [N/A]
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [N/A]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [N/A]
4. If you using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [N/A] We do not use existing assets or curate/release assets.
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if
applicable? [N/A] We did not use crowdsourcing or conduct research with human subjects.
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
15


Reviews:
[{"review_id": "aHMqA4WnRc-", "reviewer": "Reviewer_AKdk", "summary": "\nIn this paper, the author studies the adjustment of content filtering algorithms on social platforms and proposes a system of \"one platform, one user and one auditor\" to study these problems in response to two problems. The author proposes a Regulatory procedure, and on this basis, studies the impact of content on users, supervision costs, and content diversity. Finally, analysis shows that content diversity plays an important role in coordinating the interests of regulatory agencies and platforms.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nIn general, I think the work is well-written and quite interesting - solving an emerging and important problem from a new perspective. However, both the hypothesis and the technical solution are lacking enough contributions of novelty. For example,  besides censorship and profitability, factors that affect content filtering include issues such as influence on the platform, user level, and user age. I would like to suggest the authors either make the solution actually practical or focus more on the algorithmic filtering on social networks.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "\nN/A", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "_Mm68yC_dRe", "reviewer": "Reviewer_iJ9f", "summary": "This paper studies the problem of auditing whether algorithmic filtering implementations respect a given regulation. The authors propose a simple hypothesis test that regulators could use to determine compliance given black-box access to a filtering algorithm. They show that their algorithm has desirable asymptotic properties. Finally, the authors show that under their framework platforms are incentivized to provide diverse content to their users.", "questions": "", "limitations": "", "rating": 9, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This is a very strong paper on regulating algorithmic filtering. The results are simple and elegant. The authors do a great job in explaining the intuition behind each of their results, making the paper a very pleasant read.\nI especially enjoyed the framing of users as statistical estimators, the result on the minimum variance unbiased estimator (MVUE) being equivalent to the most gullible user, and the associated implications for compliance testing are likely to be extremely useful for the field moving forward. From my own personal experience, it is very hard to come up with interaction models whose assumptions are both statistically tractable and also realistic enough to be useful when considering areas that involve human interaction, the authors have done a commendable job in threading that needle.\n\nI do have a few comments for improvement:\n - It would be nice to see some simple experiments, while I understand the paper is theoretical in nature I feel experiments would further strengthen an already strong paper, even if the experiments only involve toy data. \n - Theorem 2 is an asymptotic result, have the authors also tried providing a finite sample analysis? As it stands I don't have a good intuition for how well Algorithm 1 works for different values of m in practice, I understand that the authors have finite time but it would be good to see either experiments or a finite sample analysis (ideally both) in the final version of the paper for this reason.\n - I feel the framing of the effects of algorithmic filtering in the related work section should mention the fact that a lot of these effects are still contested. For example, this paper did not find any evidence that echo chambers were caused by recommender systems on Youtube: https://arxiv.org/pdf/2011.12843.pdf while this paper states that greater internet use does not seem to be associated with political polarization within the US https://www.pnas.org/content/114/40/10612\n\nFinally here are some low importance nitpicks: \n - In Section 4.1, the argument for why the MVUE is equivalent to the most gullible user, and hence is the only user that needs to be considered when ascertaining whether a platform meets a regulation could be clarified. I think it's pretty clear after thinking about it for a bit, but I had to pause for a bit on my first pass through the paper.\n - Given the frequent use of footnotes to justify assumptions, I would recommend the authors simply have a separate section that discusses their assumptions. This is just a stylistic preference, so the authors can ignore this point if they wish as it did not impact my review score.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors generally did a good job of discussing the societal impacts and limitations of their work, see my main review for my one comment regarding the impacts of algorithmic filtering in the related work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "SepN_OKyKxw", "reviewer": "Reviewer_qxt3", "summary": "This paper focuses on the issue of (legally) regulating filtering of content in algorithmic feeds. It assumes that feeds containing certain sets of items violate a regulation, and develops a statistical estimator which allows an external auditor to determine whether a platform complies with the regulation by comparing user feeds. Is shows theoretically that, under certain conditions, it is possible to find feeds that comply with the regulation while minimising the platform's loss (e.g. in terms of the revenue). If further shows that platforms that diversify user feeds might have an easier time complying with the regulation.\n", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "My relevant expertise for judging this paper lies in interpreting and computationally modelling regulations and measuring societal impacts of technologies. I am not closely familiar with statistical learning theory and would defer to other reviewers for comments on this part.\n\nOriginality: \n***\nA mathematical angle on social media content filtering might provide solid foundations for practical approaches, and I found the ideas developed by the paper intriguing.\n***\n\nQuality: \n***\nA backbone of the paper is the modelling of various social and legal phenomena, and there is room for improvement when it comes to the rigour and precision of different modelling aspects. Here are a few issues and suggestions:\n\n1. First of all, I am not sure whether the procedure developed in the paper in practice audits what it intends to audit: \n> \"If an auditor is unable to detect a difference in the most gullible user’s decision making when shown the feed produced by [...], then the platform should not be penalized for the difference in those feeds.\"\n\nThe definitions developed early in the paper focus on what is displayed in the feed, while the auditing procedure seems to focus on the (assumed) impact a feed has on the user. These appear to be two very different concepts.\n\n\n2. Currently, it is difficult to tell how certain crucial modelling choices were made: are they assumptions, are they grounded in literature outside of ML? Issues of this kind include:\n- (Line 166) \"Decision-making pipeline. Decisions can be reduced to a simple pipeline: (a) an individual observes information; (b) that information informs their beliefs, and (c) based on those beliefs, they make a decision.\" \n- (Line 22) \"as a user browses through their content, they implicitly learn a model of their world by regressing on the data\"\nIt would be good to specify whether these are assumptions the paper makes or whether there exists supporting evidence in e.g. psychology literature.\n \n3. The contributions of the paper do not accurately reflect the claims from the abstract introduction. For example, the paper does not tackle this question from the abstract at all: \"In this work, we examine two questions. First, how does the filtered content affect a user’s learning and decision-making and is there a principled way to regulate this effect?\". Overall, the paper would benefit by more clearly describing that the contribution lies in mathematical analysis of feed selection rather than impacts on users and their learning or decision-making.\n\nI will let other reviewers comment on the statistical-learning-theory-related technical execution. \n***\n  \nClarity: \n***\nOverall, the paper is pleasant to read and well-structured. The authors clearly describe and embed their results within the story of the paper. A significant clarity issue, however, is that the paper never formally defines what a regulation is, even though it's a central concept in the paper. There are a few paragraphs that talk about the concept (lines 127 and 202), but these are either very high-level or defined through examples. \n***\n\n\nSignificance: \n***\nI believe that, if the modelling quality issues described above are fixed, the paper will provide useful theoretical insights on the feasibility of feed curation as well as straightforward technical means of compliance. A few further suggestions for improving accessibility to a wider audience:\n- The paper limits the analysis to regulations that can be expressed as counterfactual tests. In practice, how often do content feed regulations belong to this category?\n- How many observations are needed for Algorithm 1 to confidently distinguish (non-)compliant feeds?\n\nThe result formally showing that increasing diversity might be a straightforward way to comply with content feed regulations might also have a big impact. Yet, since I don't see how exactly diversity is tied to the attribute around which regulation is defined, I am left wondering if adding diversity here is a means of complying with regulation or 'sneaking around' the regulation?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "As described in the main review, the paper will benefit from \n* clearly articulating the modelling assumptions,\n* better grounding the modeled concepts in relevant literature,\n* more accurately describing its contributions in the abstract and introduction.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "3WABT2y5K1U", "reviewer": "Reviewer_ePHk", "summary": "In this paper, the authors study issues on regulating algorithmic filtering in social networks. They describe a system involving three actors — user, auditor, and social media platform. They prove a result claiming that a regulator only needs to test regulation for the \"most gullible user\"; then, for a specific class of regulations, they give an algorithm that they claim tests the regulation for said user. Finally, they give some technical conditions on when regulations do not affect the reward function of the social media platform.", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n- Regulation of algorithmic filtering is a very important topic, and authors propose a seemingly novel model/have interesting technical results\n\nWeaknesses:\n\n- Writing is *very* confusing, to the point that the the model/problem statement/technical results are impossible to understand\n- Significance of technical results seem somewhat unrelated to qualitative explanations (eg Prop 1 vs \"most gullible user\")\n\nOverall, while I very much like the topic of algorithmic filtering in social networks + regulation, as well as the approach to model/analyze it using a simple 3 actor system, I find the actual technical details impossible to understand. It is not clear to me what the model even is, or what specific problem is being solved. Instead, the authors seem to go back and forth between broad definitions of regulators and social network feeds, and seemingly unrelated technical statements about hypothesis testing. While part of this is probably due to my lack of familiarity with the much of the cited literature, I do think the writing could be greatly improved. \n\nI do sincerely think the work is interesting, so below I list some questions/comments that I hope the authors can address during the rebuttal period. I will give the paper a temporary \"reject\" for now, but I hope the rebuttal will make me improve my score.\n\nI only write my comments for Sections 3 and 4, since Section 5 seems to build off the models in these sections. I also will note that I did not read the proofs carefully, although I take a look at them.\n\n------\nUPDATE: After reading the author response and the helpful summary by one of the reviewers, I have identified some misunderstandings I had in my initial review of the paper. I quite like the results in the paper now, and I have accordingly raised my score to an \"accept\". However, I do think the authors should provide much more context for their setting/model (eg as they have done in their responses to me and reviewer qxt3). It might also help to include an explanation like Reviewer iJ9f's for those unfamiliar with the literature (at least in the supplement).\n\n===============\n\nBefore I get into specifics, I have to say that the fundamental issue I have with the writing is that it is never explicitly written what exact problem the auditor is trying to solve. I guess lines 223-225 attempt to, but it involves some new $f_k, f_l$ variables which don't seem connected to any of the previous definitions? (It also involves only the \"most gullible user\" and not all users.)\n\nIf I had to write out a specific condition for passing regulation, it would probably be something like: \n\n>a platform passes regulation if, for each user u, their belief $\\theta^{(t)}_u(Z_1, \\dots, Z_t)$ at time $t$ after observing the feeds $Z_1, \\dots, Z_k$ (under some user belief model) is the same as their belief $\\theta^{(t)}_u(W_1, \\dots, W_t)$ after observing the counterfactual feeds $W_1, \\dots, W_k$.\n\nI'm not sure how to define \"counterfactual feeds\" (lines 215-219 say it's out of the scope of the paper), or how the user belief function $\\theta^{(t)}$ works, but this already is a cleaner statement than the hand-waving in Sections 3,4. \n\nNow, it would be good if Prop 1/Thm 2 combined could prove a result like this. But it is not clear to me if they can. It would be disappointing if they cannot.\n\n===============\n\nSection 3:\n\n— The description of what it means for whether \"a platform meets regulation\" is far too minimal. \n\n>The auditor’s goal is to test whether the platform meets regulation.\n\nThis seems fine, but it's not a very specific statement. From the following \"Platform\" paragraph, it makes it seem like passing regulation means testing whether a feed $Z^{(t)}$ is in the list of allowed feeds $\\mathcal{Z}^{(t)}$, which I don't think is the case. \n\nInstead, a clear description of the types of regulation you consider and the exact definition of what it means to meet regulation should be defined here (eg see above attempt)\n\nSection 4.1:\n\n— The \"user learning\" section makes no sense to me, as much as I try re-reading it.\n\nFirst, in line 172 you say \"let \\mathcal{P} ... denote the set of possible beliefs\". But isn't \\mathcal{P} a set of probability distributions? So is a belief a probability distribution? And $\\theta$ is a parameter the user seeks to learn? \n\nBut then later on (lines 174-175) you say $\\hat{\\theta}^{(t)}$ is the user's belief at time $t$. So now $\\theta$ is the user's belief... which they seek to learn? (Also what would a user be learning on a social media platform? I think you mean something about updating the user's beliefs? But then why is there an unknown belief $\\theta$ and a learned belief $\\hat{\\theta}$?) \n\nI was also confused by the learning function $\\mathcal{L}$. This does not seem to depend on past beliefs (eg I would expect $\\mathcal{L} : \\mathcal{Z} \\times \\Theta \\rightarrow \\Theta$, or something like $\\mathcal{L}(Z^t, \\theta^{(t-1)}) = \\theta^{(t)}$).\n\nBecause there are no citations, it's impossible to read another resource to figure out these details. The writing makes it seem as though this is an established model of user opinions, but does no citations mean that it is a novel model? \n\nOn a related note, the model does seem somewhat similar to opinion dynamics models, which I am quite familiar with, but which the paper does not cite. I think these models should be cited and compared against, since they often appear in machine learning/data mining fields, eg see references [1,2,3,...] below.\n\n— I also don't understand the broader significance of Proposition 1.\n\nFirst, why does it mention a user choosing between options? This was not mentioned in the model. One issue is that the regulatory model (which talks about counterfactuals) is not yet properly defined in the writing, so this result has no context. \n\nBut even if the regulatory model from Sec 4.2 was defined, I'd still find this result confusing. What does an option mean in the context of a news feed? What is the meaning of the \"value\" $v_0, v_1$ of an option (eg lines 184-186)? \n\nI also don't really understand why Proposition 1 has anything to do with the \"most gullible user\"  (why does the MVUE correspond to such a user?). I also don't really understand why it means you only have to \"test\" said user and not every user. Isn't the user fixed in Proposition 1? (line 184 says \"the user\")\n\nAlso, statements like\n\n>if the platform meets regulation for the MVUE, then it meets regulation for all other \\mathcal{L}\n\nseem vacuous since the regulatory model is not yet defined.\n\nSection 4.2:\n\n— One thing I found confusing is that Section 3 says the auditor only has access to the algorithm $\\mathcal{F}$, but Algorithm 1 requires other things as inputs (eg MVUE feeds/counterfactual feeds). This is minor, but it is indicative of the regulatory model not being well defined.\n\n— Where do these \"conditions\" $f_k, f_l$ (eg lines 206, 222) come from? I guess it makes sense the regulation $\\mathcal{F}$ is a function, but it's never stated what its arguments are. \n\n— Theorem 2 sjhould either be a standalone result, or explicitly reference Algorithm 1 (eg saying \"assume Algorithm 1 was run\". Using Algo 1 notation/results but not referencing it is confusing\n\nThere are also some weird circularity issues, since Algo 1 references the null/alt hypotheses defined in Theorem 2.\n\nAlso I notice the Theorem 2 statement involves \\mathcal{P}, whose role in your model I do not understand (see above).\n\n— I find the discussion of \"similar outcomes\" in lines 248-251 to be confusing. Isn't Theorem 2 saying something stronger — eg that the user beliefs (or whatever theta is) are the *same*, not just similar?\n\n— Line 258:\n\n>In practice, the auditor may wish to run this test over many users’ feeds\n\nVery confused, because I thought we only needed to run this test once for the \"most gullible user\"? Why many users' feeds?\n\n— line 265: the first reason has nothing to do with the MLE?\n\nNits\n\n- In Sec 3, should state more clearly that \\mathcal{Z} consists of m-tuples of pieces of content\n- line 126: \"denoted\" → \"is denoted\"\n- line 179: \"an an\" → \"as an\"\n- line 181: \"as an\" → \"is an\"\n\n[1] Minimizing Polarization and Disagreement in Social Networks, Musco et al WWW 2018\n[2] Quantifying and minimizing risk of conflict in social networks, Chen et al KDD 2018\n[3] Adversarial perturbations of opinion dynamics in networks, Gaitonde etal EC 2020\n[4] Opinion Dynamics with Varying Susceptibility to Persuasion KDD 2018\n[5] Analyzing the Impact of Filter Bubbles on Social Network Polarization, Chitra and Musco WSDM 2020", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "11"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper studies the problem of auditing whether algorithmic filtering implementations respect a given regulation from a theoretical perspective. The authors propose a simple hypothesis test that regulators could use to determine compliance given black-box access to a filtering algorithm. They show that their algorithm has desirable asymptotic properties. Finally, the authors show that under their framework platforms are incentivized to provide diverse content to their users. The topic is very timely and the presented theory/methodology is a strong contribution.

Initially, some of the reviewers had concerns regarding the presentation of the results. The authors' reply as well as especially the follow-up explanation/description by one of the reviewers, who enthusiastically championed the paper, cleared up these concerns to some extent. However, it is very important that the authors incorporate the reviewers' suggestions in the final version of the paper, including the running example provided by one of the reviewers, so that a wider audience can appreciate the contribution.
Decision: Accept (Spotlight)