# Trial and error in policy making

I’m working in government in the UK, and interested in rationality in policy making: at the individual level, but also how we can build systems (in terms of requirements during policy creation, creation of independent bodies, incentive structures for officials/ministers…) that encourage more rational policies. This is the context for the stuff below.

I recently went to a talk on the use of trial and error in policy making. It was hosted by the Institute for Government (a non-political think tank advising ministers and civil servants on good government), and included the Undercover Economist, Tim Harford. Details including a recording can be found here: [http://www.instituteforgovernment.org.uk/our-events/105/government-by-trial-and-error-a-discussion-with-tim-harford](http://www.instituteforgovernment.org.uk/our-events/105/government-by-trial-and-error-a-discussion-with-tim-harford)

Ben Goldacre was also attending in the audience, and has written about this: [http://www.badscience.net/2011/05/we-should-so-blatantly-do-more-randomised-trials-on-policy/](http://www.badscience.net/2011/05/we-should-so-blatantly-do-more-randomised-trials-on-policy/)

The basic message of the talk was that much of policy could benefit from being trialled more effectively, whether in the form of formalised, controlled trials: following the medical example inasmuch as that is practically possible. This approach seems to be being seriously considered by this government in several areas: one example of how it might work in theory can be found here: [http://www.straightstatistics.org/article/what-works-criminal-justice-time-find-out](http://www.straightstatistics.org/article/what-works-criminal-justice-time-find-out)

Of course, the reality is often messier, and ministers are looking for both swifter results and more security. Other forms of ‘experimentation’ involve devolving power to local councils (Localism Bill) or schools (Free Schools) etc. In these cases there aren’t the same formal comparison methods but you at least get lots of ideas and some ability to judge which ones have gone well. An underlying issue is the quality of assessments: a policy that has clear criteria for appraisal built into its design will be much easier to judge, whereas one that is simply acted upon and studied later might suffer from poor evidence and be more open to being interpreted in line with preconcieved or desired outcomes.

A final approach is ‘payment by results’, where tasks such as preventing reoffending or getting people off drugs is paid for on the basis of successes achieved rather than services delivered, the idea being that this provides a fiscal incentive to get it right. A critical view of one such scheme is here

[http://www.straightstatistics.org/article/flaws-mojs-reoffending-study](http://www.straightstatistics.org/article/flaws-mojs-reoffending-study)

I was wondering what people here felt about the idea of trial-and-error in policy. I think it seems like a good idea, with the following limitations

- a) some policies probably can’t be trialled on a national scale (can you have half the country making cuts while the other half keeps spending?)

- b) some cases will have ethical issues as with medicine

- c) some cases might not allow clear-cut results, eliminating the ability to learn and simply creating a chaotic mess of different approaches

- d) the biggy: I don’t know if the political culture is open to it.

Any thoughts?