These chime with the evidence outlined earlier on the costs and benefits of expert and non-expert opinion in collective decision-making. First, there is real value in non-expert opinion when it comes to complex forecasting tasks. In a famous example, Tetlock convened a group of 1,000 amateur forecasters and entered them in a prestigious forecasting tournament run by the US intelligence agencies (IARPA). 36  This team beat the control group by  60% in year 1 and 78% in year 2. It not only beat expert academics from MIT to Colombia, but professionals from the US intelligence services with access to classified information. This is what Tetlock calls Super-forecasting. The incremental value of non-expert judgements has been shown in a wide array of settings and for a wide range of complex problems. What about judgments on the complex dynamics of the economic and financial system? While we do not have a structured experiment on which to draw, we do have illustrative evidence on the relative forecasting performance of different groups. 34 35 36  For example, Roubini and Mihm (2011) and Janis (1972). Shafik (2017) discusses the impact of the rise of social media. Tetlock and Gardner (2015). 13  All speeches are available online at www.bankofengland.co.uk/speeches  13  Chart 3 considers the forecast errors of five groups – three “expert” (MPC, a panel of professional forecasters and financial markets) and two “non-expert” (companies and households). For each group we consider their forecasts for UK inflation 1, 2 and 3 years ahead.

Gruen, N (2018), ‘An Ancient Greek idea could foil Brexit’s democratic tragedy’, article in The Guardian, 16 January 2018. Gürkaynak, R, Sack, B and Swanson, E (2005), ‘Do Actions Speak Louder Than Words? The Response of Asset Prices to Monetary Policy Actions and Statements’, International Journal of Central Banking, Vol. 1, No.1, pp. 55-93. 25  All speeches are available online at www.bankofengland.co.uk/speeches  25  Gürkaynak, R, Levin, A and Swanson, E (2010), ‘Does Inflation Targeting Anchor Long-Run Inflation Expectations? Evidence from the U.S., UK and Sweden’, Journal of the European Economic Association, Vol. 8, No. 6, pp. 1208-1242. Haldane, A (2016), ‘The Sneetches’, speech available at https://www.bankofengland.co.uk/speech/2016/the-sneetches Haldane, A (2017a), ‘A Little More Conversation, A Little Less Action’, speech available at https://www.bankofengland.co.uk/speech/2017/a-little-more-conversation-a-little-less-action Haldane, A (2017b), ‘Everyday Economics’, speech available at https://www.bankofengland.co.uk/speech/2017/andy-haldane-speech-during-regional-visit Haldane, A (2018a), ‘How Monetary Policy Affects Your GDP’, speech available at https://www.bankofengland.co.uk/speech/2018/andy-haldane-david-finch-public-lecture-melbourne Haldane, A (2018b), ‘Climbing the Public Engagement Ladder’, speech available at https://www.bankofengland.co.uk/speech/2018/andy-haldane-royal-society Haldane, A and McMahon, M (2018), ‘Central Bank Communications and the General Public’, AEA Papers and Proceedings, Vol. 108, pp. 578-83. Hennig, B (2017), The End of Politicians: Time for a Real Democracy, Unbound Digital. Hong, L and Page, S (2004), ‘Groups of diverse problem solvers can outperform groups of high-ability problem solvers’, Proceedings of the National Academy of Sciences of the United States of America, Vol. 101, No. 46, pp. 16385-16389. Janis, I (1972), Victims of Groupthink, Houghton, Mifflin. Kahneman, D (2012), Thinking, Fast and Slow, Penguin.