## 3 Data and Methodology  
Fakenewslab was presented as a test accessible via Web, where volunteers were exposed to 20 news articles that have been actually published online in Italian news outlets; for each news item, users were asked to tell which were true or fake. Every user read the same 20 news items in a random order, and they were randomly redirected to one out of five different experimental environments, which we call \"virtual rooms\", in which the news are presented quite differently:  
* **Room 1** showed, for every news, the bare headline and a short excerpt as they would appear on social media platforms, but was devoid of any contextual clues. * **Room 2** showed the full text of the articles, as they were presented on the original website, again without any clear references to the source. * **Room 3** showed the headline with a short excerpt and the source of the article, as it would appear on social media but devoid of social features. The article source was clickable, and the article could be read at its original source. * **Room 4** showed the headline with a short excerpt, as well as the percentages of users that classified the news as true or false. We will refer to this information as \"social ratings\" from now on. * **Room 5** was similar to Room 4, but with randomly generated percentages. We will refer to these ratings as \"random ratings\" from now on. All the rooms were just variations of Room 1, which was designed as an empirical baseline for comparative purposes. Additionally, Room 5 has been introduced as a \"null model\" to evaluate the social influence effect in comparison with Room 4, which displays actual values. Figure 1 displays a graphical render of how a news item looked like in each room. During the test, we monitored some activities in order to have better insight on the news evaluation process. We collected data such as the timestamp of each answer (which allowed us to reconstruct how much time each answer took), what room the user was redirected to, and what social ratings or random ratings they were seeing in Room 4 and Room 5. We also measured if users left the test tab of the browser to open another tab during the test, an activity we interpreted as a signal of attempted fact checking: searching online in order to assess the veracity of an article was not explicitly encouraged nor forbidden by the test rules, and we left to the users the chance to behave as they would have during everyday web browsing. We are aware that the mindset of a test respondent may have discouraged many users from fact checking the articles, as it may have been perceived to be cheating the test, but our argument is that the very same mindset lead the users to pay attention to news they would not have noticed otherwise, thus raising an alert on possible deception hiding in each news item: explicitly allowing for web searches out of the test would have probably lead to disproportionate, unrealistic fact-checking activity. We noted that eventually, 18.3% of users fact-checked the news at least once on their own initiative, a percentage in accordance with existing literature [38; 39]. However, estimating how many users would have fact-checked the articles is outside the goals of this experiment: we only looked at those users who possibly did it, and at how fact-checking impacted their perception and rating of the news. Along with information related to the test, we asked participants to fill an optional questionnaire about demographics and personal information. We collected users' genre, age, job, education, political alignment, and time and number of daily newspapers read, each of them on an optional basis. A total of 64% of users responded to at least one of the previous questions. This personal information helped us to link many relevant patterns in news judgement with demographic segments of the population. Details about such relationships between test scores and some demographic information are discussed in Appendix A.2.