Paper:
Over the last few years deep multi-agent reinforcement learning (DMARL) has become an increasingly active area of research with hundreds of papers submitted to top machine learning conferences every year. However, so far there have been few real world use cases that benefited from the progress in the field. We use DMARL to develop a flexible computational model of fake news on social networks in which agents act according to learned best response functions. We achieve this by extending an information aggregation game to allow for fake news and by representing agents as recurrent deep Q-networks (DQN). In the game, agents repeatedly guess whether a claim is true or false taking into account an informative private signal and observations of actions of their neighbors on the social network in the previous period. We incorporate fake news into the model by adding an adversarial agent, the attacker, that either provides biased private signals to, or takes over, a subset of agents. The attacker can follow either a hand-tuned or trained policy. Our model allows us to tackle questions that are analytically intractable in fully rational models, while ensuring that agents follow reasonable best response functions. Our results highlight the importance of awareness, privacy and social connectivity in curbing the adverse effects of fake news and open an entire new real world application area for DMARL.

Review:
Update: I thank the authors for their response and for improving the paper. However, I maintain my position that the paper lacks a significant technical contribution to learning algorithms and that the applicability of the proposed approach is remains questionable in the current state.  
Summary:
The paper proposes the  use of deep multi agent reinforcement learning (DMARL) for modelling fake news propagation and detection in social networks. The agents observe an informative yet noisy private signal and the actions of their neighbors (in the social network graph) and have to guess whether a claim (related to the received signal) is true or false. The fake news is modelled as an adversarial attack to the graph that either provides a hand-coded biased private signal to one of the agents or replaces one of  the agents with an RL policy trained to minimize the total reward of the agents in the graph (i.e. social network).
Main Comments:
I lean towards rejecting this paper because I do not find the methodological contribution to be significant enough to be published at ICLR, given that the main contribution is applying current techniques to a novel toy domain. While this paper attempts to apply DMARL to a new domain with real-world relevance, the authors only consider a toy example and make strong assumptions that are likely to break in the real world. Hence, it is not at all clear whether or how the conclusions of this paper would translate to more realistic scenarios of fake news in social networks. While strong assumptions and toy examples are reasonable for showing algorithmic improvements, this paper does not propose any improvement to core DMARL algorithms, but merely applies current methods to a new toy domain. 
I am also concerned about the lack of comparison with other approaches to information aggregation in social networks. While I admit I am not familiar with that literature, I would still find it useful to provide some comparisons with non-DMARL (e.g. heuristic or game theoretic) approaches or at least some motivation for not comparing against those methods. The authors qualitatively describe those methods and their shortcomings, but the experimental section does not support those claims due to the lack of comparison. Despite all these concerns, the paper does indeed open-up numerous research directions and I can imagine follow-up papers being written that relax some of the current assumptions. 
Other Questions / Comments:
1. Can you provide  some motivation for choosing to model the social network as a Barabsi Albert graph and why this is a reasonable modelling choice?
2. What happens if instead of a clustered or balanced graph, you have some combination of the two? It seems to me like that would be a more realistic scenario (i.e. a large graph containing subgraphs with different structures). Can the framework generalize to that? How would the conclusions change?
3. Is there any evidence that the conclusions supported by the experiments in this paper hold in real-world social networks and model some realistic aspects of social network dynamics?  Without such evidence, it is difficult to assess the relevance of this work for the  real-world application. Since the behavior of the agents in the graph is not guaranteed to be optimal / a best-response or even stable, is it at least a good approximation to human behavior in social networks? It would also be useful to show more comparisons against best-response or heuristic agents.
4. What is  the motivation behind considering a fixed budget of bias? Why not instead have a fixed number of agents that you will be biased? I think it would be informative to compare against applying beta = 3 to two citizens,  along with the focused and spread scenarios.
The legend in Figure 2 A & B is slightly confusing. I’d suggest using different styles for “private signal optimal” and “all signals optimal”. 
5. Can you include error bars in Figure 2 D?
6. Can you provide results with a heuristic attacker that always lies about the claim? I read your intuition of why  you believe this wouldn’t be stronger than an attacker that is trained with RL together with the other agents, but is it actually true in practice, do the agents really learn to easily detect the “lying” attacker and distrust it? Based on your results, doesn’t it mean that one can find a heuristic attacker that has an equivalent behavior to the learned one ? Can one build even stronger hand-tuner attackers  based on heuristics?