## 1 Introduction  
Information disorder [1, 2], a general term that includes different ways of describing how the information environment is polluted, is a reason of concern for many national and international institutions; in fact, incorrect beliefs propagated by malicious actors are supposed to have important consequences on politics, finance and public health. The European Commission, for instance, called out a mix of pervasive and polluted information that has been spread online during the COVID-19 pandemic as an \"infodemic\" [3], which risks jeopardising the efficacy of interventions against the outbreak. In particular, social media users are continuously flooded by information shared or liked by accounts they follow, and news that is diffused is not necessarily controlled by editorial boards that, in professional journalism, are in charge of ruling out poor, inaccurate or unwanted reporting. However, the emergence of novel news consumption behaviours is not negligible: according to research from the Pew Research Center [4], 23% of United States citizens stated that they receive their information from social media often, while 53% stated they receive information from social media at least sometimes. Unfortunately, even if social media may actually increase media pluralism in principle, the novelty comes with some drawbacks, such as misinformation and disinformation, which are spreading at an amazingly high rate and scale [5, 6, 7, 8]. If the reliability of the source is an important issue to be addressed, the other side of the coin is an individual's intentionality of spreading low quality information; in fact, although the distinction between misinformation and disinformation might seem blurry, the main difference between the two concepts lies in the individual's intention, since misinformation is false or inaccurate information deceiving its recipients that could spread it even unintentionally, while disinformation refers to a piece of manipulative information that has been diffused purposely to mislead and deceive. Focusing on misinformation, we may wonder how and why users can be deceived and pushed to share inaccurate information; the reasons for such behaviour is usually explained in terms of the inherent attractiveness of fake news, which are more novel and emotionally captivating than regular news [7], repeated exposure to the same information, amplified by filter bubbles and echo chambers [9], or the ways all of us are affected by many psycho-social and cognitive biases, e.g., the bandwagon effect [10], confirmation bias [11], and the vanishing hyper-correction effect [12], which apparently makes debunking a very difficult task to be achieved. However, much remains to be understood on how a single user decides that some news is true or false, how many individuals fact-check information before making such decision, how the interaction design of the Web site publishing the news can contribute to assisting or deceiving the user, and to what extent social influence and peer-pressure mechanisms can activate a person into believing something against all the evidence supporting the contrary. Our main objective is to investigate the latent cognitive and device-induced processes that may prevent us from distinguishing true from false news. To this purpose, we created an artificial online news outlet where 7298 volunteers were asked to review 20 different news and to decide if they were true or false. The target of the experiment is manifold: we aim to assess if individuals make these kinds of decisions independently, or if the \"wisdom of the crowd\" may play a role; if decisions made as a function of limited information (the headline and a short excerpt of the article) lead to better or worse results than choices made upon richer data (i.e., the full text, the source of the article); and if users' better familiarity with the Web or other self-provided information can be linked to better annotations. The results of the experiment and their relationships with related work are described in the rest of the paper.