Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: In this paper, the authors study issues on regulating algorithmic filtering in social networks. They describe a system involving three actors — user, auditor, and social media platform. They prove a result claiming that a regulator only needs to test regulation for the "most gullible user"; then, for a specific class of regulations, they give an algorithm that they claim tests the regulation for said user. Finally, they give some technical conditions on when regulations do not affect the reward function of the social media platform.
main_review: Strengths:

- Regulation of algorithmic filtering is a very important topic, and authors propose a seemingly novel model/have interesting technical results

Weaknesses:

- Writing is *very* confusing, to the point that the the model/problem statement/technical results are impossible to understand
- Significance of technical results seem somewhat unrelated to qualitative explanations (eg Prop 1 vs "most gullible user")

Overall, while I very much like the topic of algorithmic filtering in social networks + regulation, as well as the approach to model/analyze it using a simple 3 actor system, I find the actual technical details impossible to understand. It is not clear to me what the model even is, or what specific problem is being solved. Instead, the authors seem to go back and forth between broad definitions of regulators and social network feeds, and seemingly unrelated technical statements about hypothesis testing. While part of this is probably due to my lack of familiarity with the much of the cited literature, I do think the writing could be greatly improved. 

I do sincerely think the work is interesting, so below I list some questions/comments that I hope the authors can address during the rebuttal period. I will give the paper a temporary "reject" for now, but I hope the rebuttal will make me improve my score.

I only write my comments for Sections 3 and 4, since Section 5 seems to build off the models in these sections. I also will note that I did not read the proofs carefully, although I take a look at them.

------
UPDATE: After reading the author response and the helpful summary by one of the reviewers, I have identified some misunderstandings I had in my initial review of the paper. I quite like the results in the paper now, and I have accordingly raised my score to an "accept". However, I do think the authors should provide much more context for their setting/model (eg as they have done in their responses to me and reviewer qxt3). It might also help to include an explanation like Reviewer iJ9f's for those unfamiliar with the literature (at least in the supplement).

===============

Before I get into specifics, I have to say that the fundamental issue I have with the writing is that it is never explicitly written what exact problem the auditor is trying to solve. I guess lines 223-225 attempt to, but it involves some new $f_k, f_l$ variables which don't seem connected to any of the previous definitions? (It also involves only the "most gullible user" and not all users.)

If I had to write out a specific condition for passing regulation, it would probably be something like: 

>a platform passes regulation if, for each user u, their belief $\theta^{(t)}_u(Z_1, \dots, Z_t)$ at time $t$ after observing the feeds $Z_1, \dots, Z_k$ (under some user belief model) is the same as their belief $\theta^{(t)}_u(W_1, \dots, W_t)$ after observing the counterfactual feeds $W_1, \dots, W_k$.

I'm not sure how to define "counterfactual feeds" (lines 215-219 say it's out of the scope of the paper), or how the user belief function $\theta^{(t)}$ works, but this already is a cleaner statement than the hand-waving in Sections 3,4. 

Now, it would be good if Prop 1/Thm 2 combined could prove a result like this. But it is not clear to me if they can. It would be disappointing if they cannot.

===============

Section 3:

— The description of what it means for whether "a platform meets regulation" is far too minimal. 

>The auditor’s goal is to test whether the platform meets regulation.

This seems fine, but it's not a very specific statement. From the following "Platform" paragraph, it makes it seem like passing regulation means testing whether a feed $Z^{(t)}$ is in the list of allowed feeds $\mathcal{Z}^{(t)}$, which I don't think is the case. 

Instead, a clear description of the types of regulation you consider and the exact definition of what it means to meet regulation should be defined here (eg see above attempt)

Section 4.1:

— The "user learning" section makes no sense to me, as much as I try re-reading it.

First, in line 172 you say "let \mathcal{P} ... denote the set of possible beliefs". But isn't \mathcal{P} a set of probability distributions? So is a belief a probability distribution? And $\theta$ is a parameter the user seeks to learn? 

But then later on (lines 174-175) you say $\hat{\theta}^{(t)}$ is the user's belief at time $t$. So now $\theta$ is the user's belief... which they seek to learn? (Also what would a user be learning on a social media platform? I think you mean something about updating the user's beliefs? But then why is there an unknown belief $\theta$ and a learned belief $\hat{\theta}$?) 

I was also confused by the learning function $\mathcal{L}$. This does not seem to depend on past beliefs (eg I would expect $\mathcal{L} : \mathcal{Z} \times \Theta \rightarrow \Theta$, or something like $\mathcal{L}(Z^t, \theta^{(t-1)}) = \theta^{(t)}$).

Because there are no citations, it's impossible to read another resource to figure out these details. The writing makes it seem as though this is an established model of user opinions, but does no citations mean that it is a novel model? 

On a related note, the model does seem somewhat similar to opinion dynamics models, which I am quite familiar with, but which the paper does not cite. I think these models should be cited and compared against, since they often appear in machine learning/data mining fields, eg see references [1,2,3,...] below.

— I also don't understand the broader significance of Proposition 1.

First, why does it mention a user choosing between options? This was not mentioned in the model. One issue is that the regulatory model (which talks about counterfactuals) is not yet properly defined in the writing, so this result has no context. 

But even if the regulatory model from Sec 4.2 was defined, I'd still find this result confusing. What does an option mean in the context of a news feed? What is the meaning of the "value" $v_0, v_1$ of an option (eg lines 184-186)? 

I also don't really understand why Proposition 1 has anything to do with the "most gullible user"  (why does the MVUE correspond to such a user?). I also don't really understand why it means you only have to "test" said user and not every user. Isn't the user fixed in Proposition 1? (line 184 says "the user")

Also, statements like

>if the platform meets regulation for the MVUE, then it meets regulation for all other \mathcal{L}

seem vacuous since the regulatory model is not yet defined.

Section 4.2:

— One thing I found confusing is that Section 3 says the auditor only has access to the algorithm $\mathcal{F}$, but Algorithm 1 requires other things as inputs (eg MVUE feeds/counterfactual feeds). This is minor, but it is indicative of the regulatory model not being well defined.

— Where do these "conditions" $f_k, f_l$ (eg lines 206, 222) come from? I guess it makes sense the regulation $\mathcal{F}$ is a function, but it's never stated what its arguments are. 

— Theorem 2 sjhould either be a standalone result, or explicitly reference Algorithm 1 (eg saying "assume Algorithm 1 was run". Using Algo 1 notation/results but not referencing it is confusing

There are also some weird circularity issues, since Algo 1 references the null/alt hypotheses defined in Theorem 2.

Also I notice the Theorem 2 statement involves \mathcal{P}, whose role in your model I do not understand (see above).

— I find the discussion of "similar outcomes" in lines 248-251 to be confusing. Isn't Theorem 2 saying something stronger — eg that the user beliefs (or whatever theta is) are the *same*, not just similar?

— Line 258:

>In practice, the auditor may wish to run this test over many users’ feeds

Very confused, because I thought we only needed to run this test once for the "most gullible user"? Why many users' feeds?

— line 265: the first reason has nothing to do with the MLE?

Nits

- In Sec 3, should state more clearly that \mathcal{Z} consists of m-tuples of pieces of content
- line 126: "denoted" → "is denoted"
- line 179: "an an" → "as an"
- line 181: "as an" → "is an"

[1] Minimizing Polarization and Disagreement in Social Networks, Musco et al WWW 2018
[2] Quantifying and minimizing risk of conflict in social networks, Chen et al KDD 2018
[3] Adversarial perturbations of opinion dynamics in networks, Gaitonde etal EC 2020
[4] Opinion Dynamics with Varying Susceptibility to Persuasion KDD 2018
[5] Analyzing the Impact of Filter Bubbles on Social Network Polarization, Chitra and Musco WSDM 2020
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 9: Top 15% of accepted NeurIPS papers, strong accept
review_content: summary: This paper studies the problem of auditing whether algorithmic filtering implementations respect a given regulation. The authors propose a simple hypothesis test that regulators could use to determine compliance given black-box access to a filtering algorithm. They show that their algorithm has desirable asymptotic properties. Finally, the authors show that under their framework platforms are incentivized to provide diverse content to their users.
main_review: This is a very strong paper on regulating algorithmic filtering. The results are simple and elegant. The authors do a great job in explaining the intuition behind each of their results, making the paper a very pleasant read.
I especially enjoyed the framing of users as statistical estimators, the result on the minimum variance unbiased estimator (MVUE) being equivalent to the most gullible user, and the associated implications for compliance testing are likely to be extremely useful for the field moving forward. From my own personal experience, it is very hard to come up with interaction models whose assumptions are both statistically tractable and also realistic enough to be useful when considering areas that involve human interaction, the authors have done a commendable job in threading that needle.

I do have a few comments for improvement:
 - It would be nice to see some simple experiments, while I understand the paper is theoretical in nature I feel experiments would further strengthen an already strong paper, even if the experiments only involve toy data. 
 - Theorem 2 is an asymptotic result, have the authors also tried providing a finite sample analysis? As it stands I don't have a good intuition for how well Algorithm 1 works for different values of m in practice, I understand that the authors have finite time but it would be good to see either experiments or a finite sample analysis (ideally both) in the final version of the paper for this reason.
 - I feel the framing of the effects of algorithmic filtering in the related work section should mention the fact that a lot of these effects are still contested. For example, this paper did not find any evidence that echo chambers were caused by recommender systems on Youtube: https://arxiv.org/pdf/2011.12843.pdf while this paper states that greater internet use does not seem to be associated with political polarization within the US https://www.pnas.org/content/114/40/10612

Finally here are some low importance nitpicks: 
 - In Section 4.1, the argument for why the MVUE is equivalent to the most gullible user, and hence is the only user that needs to be considered when ascertaining whether a platform meets a regulation could be clarified. I think it's pretty clear after thinking about it for a bit, but I had to pause for a bit on my first pass through the paper.
 - Given the frequent use of footnotes to justify assumptions, I would recommend the authors simply have a separate section that discusses their assumptions. This is just a stylistic preference, so the authors can ignore this point if they wish as it did not impact my review score.
limitations_and_societal_impact: The authors generally did a good job of discussing the societal impacts and limitations of their work, see my main review for my one comment regarding the impacts of algorithmic filtering in the related work.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper focuses on the issue of (legally) regulating filtering of content in algorithmic feeds. It assumes that feeds containing certain sets of items violate a regulation, and develops a statistical estimator which allows an external auditor to determine whether a platform complies with the regulation by comparing user feeds. Is shows theoretically that, under certain conditions, it is possible to find feeds that comply with the regulation while minimising the platform's loss (e.g. in terms of the revenue). If further shows that platforms that diversify user feeds might have an easier time complying with the regulation.

main_review: My relevant expertise for judging this paper lies in interpreting and computationally modelling regulations and measuring societal impacts of technologies. I am not closely familiar with statistical learning theory and would defer to other reviewers for comments on this part.

Originality: 
***
A mathematical angle on social media content filtering might provide solid foundations for practical approaches, and I found the ideas developed by the paper intriguing.
***

Quality: 
***
A backbone of the paper is the modelling of various social and legal phenomena, and there is room for improvement when it comes to the rigour and precision of different modelling aspects. Here are a few issues and suggestions:

1. First of all, I am not sure whether the procedure developed in the paper in practice audits what it intends to audit: 
> "If an auditor is unable to detect a difference in the most gullible user’s decision making when shown the feed produced by [...], then the platform should not be penalized for the difference in those feeds."

The definitions developed early in the paper focus on what is displayed in the feed, while the auditing procedure seems to focus on the (assumed) impact a feed has on the user. These appear to be two very different concepts.


2. Currently, it is difficult to tell how certain crucial modelling choices were made: are they assumptions, are they grounded in literature outside of ML? Issues of this kind include:
- (Line 166) "Decision-making pipeline. Decisions can be reduced to a simple pipeline: (a) an individual observes information; (b) that information informs their beliefs, and (c) based on those beliefs, they make a decision." 
- (Line 22) "as a user browses through their content, they implicitly learn a model of their world by regressing on the data"
It would be good to specify whether these are assumptions the paper makes or whether there exists supporting evidence in e.g. psychology literature.
 
3. The contributions of the paper do not accurately reflect the claims from the abstract introduction. For example, the paper does not tackle this question from the abstract at all: "In this work, we examine two questions. First, how does the filtered content affect a user’s learning and decision-making and is there a principled way to regulate this effect?". Overall, the paper would benefit by more clearly describing that the contribution lies in mathematical analysis of feed selection rather than impacts on users and their learning or decision-making.

I will let other reviewers comment on the statistical-learning-theory-related technical execution. 
***
  
Clarity: 
***
Overall, the paper is pleasant to read and well-structured. The authors clearly describe and embed their results within the story of the paper. A significant clarity issue, however, is that the paper never formally defines what a regulation is, even though it's a central concept in the paper. There are a few paragraphs that talk about the concept (lines 127 and 202), but these are either very high-level or defined through examples. 
***


Significance: 
***
I believe that, if the modelling quality issues described above are fixed, the paper will provide useful theoretical insights on the feasibility of feed curation as well as straightforward technical means of compliance. A few further suggestions for improving accessibility to a wider audience:
- The paper limits the analysis to regulations that can be expressed as counterfactual tests. In practice, how often do content feed regulations belong to this category?
- How many observations are needed for Algorithm 1 to confidently distinguish (non-)compliant feeds?

The result formally showing that increasing diversity might be a straightforward way to comply with content feed regulations might also have a big impact. Yet, since I don't see how exactly diversity is tied to the attribute around which regulation is defined, I am left wondering if adding diversity here is a means of complying with regulation or 'sneaking around' the regulation?

limitations_and_societal_impact: As described in the main review, the paper will benefit from 
* clearly articulating the modelling assumptions,
* better grounding the modeled concepts in relevant literature,
* more accurately describing its contributions in the abstract and introduction.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: 
In this paper, the author studies the adjustment of content filtering algorithms on social platforms and proposes a system of "one platform, one user and one auditor" to study these problems in response to two problems. The author proposes a Regulatory procedure, and on this basis, studies the impact of content on users, supervision costs, and content diversity. Finally, analysis shows that content diversity plays an important role in coordinating the interests of regulatory agencies and platforms.
main_review: 
In general, I think the work is well-written and quite interesting - solving an emerging and important problem from a new perspective. However, both the hypothesis and the technical solution are lacking enough contributions of novelty. For example,  besides censorship and profitability, factors that affect content filtering include issues such as influence on the platform, user level, and user age. I would like to suggest the authors either make the solution actually practical or focus more on the algorithmic filtering on social networks.
limitations_and_societal_impact: 
N/A
review_title: null

Meta-review: This paper studies the problem of auditing whether algorithmic filtering implementations respect a given regulation from a theoretical perspective. The authors propose a simple hypothesis test that regulators could use to determine compliance given black-box access to a filtering algorithm. They show that their algorithm has desirable asymptotic properties. Finally, the authors show that under their framework platforms are incentivized to provide diverse content to their users. The topic is very timely and the presented theory/methodology is a strong contribution.

Initially, some of the reviewers had concerns regarding the presentation of the results. The authors' reply as well as especially the follow-up explanation/description by one of the reviewers, who enthusiastically championed the paper, cleared up these concerns to some extent. However, it is very important that the authors incorporate the reviewers' suggestions in the final version of the paper, including the running example provided by one of the reviewers, so that a wider audience can appreciate the contribution.
Decision: Accept (Spotlight)