Norges Bank Main navigation ALL TOPICS NEWS & EVENTS SEARCH SEARCH Breadcrumb Home News & events News Papers Working Papers Non-standard errors WORKING PAPER Non-standard errors In statistics, samples are drawn from a population in a datagenerating process (DGP). Standard errors measure the uncertainty in sample estimates of population parameters. In science, evidence is generated to test hypotheses in an evidencegenerating process (EGP). We claim that EGP variation across researchers adds uncertainty: non-standard errors. To study them, we let 164 teams test six hypotheses on the same sample. We find that non-standard errors are sizeable, on par with standard errors. Their size (i) co-varies only weakly with team merits, reproducibility, or peer rating, (ii) declines significantly after peer-feedback, and (iii) is underestimated by participants. Author: Albert J. Menkveld et al. Norges Bank co-authors: Saskia Ter Ellen and Hans Christian Wika. Series: Working Paper Number: 13/2021 Working Paper 13/2021 (pdf) Norges Bank’s working papers present research projects and reports that are generally not in their final form. Other analyses by Norges Bank’s economists are also included in the series. The views and conclusions in these documents are those of the authors. Norges Bank’s Working Papers are also distributed by RepEc, SSRN and BIS Central Bank Research Hub. ISSN 1502-8190 (online) PUBLISHED 23 November 2021 15:00 PRINT Did you find what you were looking for? Give us feedback TOPICS Monetary policy Financial stability Notes and coins Liquidity and markets Statistics Government debt Norges Bank's settlement system About the Bank NEWS News items Publications Calendar Blog Video NORGES BANK Contact Knowledge bank Subscribe FAQ Follow us: Facebook Twitter YouTube LinkedIn Flickr RSS Privacy and cookies Disclaimer Fraud and scams