Benchmarking & Evaluation

Heidi Bellamy - Director, Culture First The purpose of today is to: ‘enable you to better evaluate performance of the archive service and gain realistic comparisons of potential’. The topics

**Bench marking**: measuring performance against other comparable services to identify realistic outcomes and targets

**Developing an evaluation process**: how to design an effective evaluation process What is Benchmarking?

A systematic process for comparing performance More than a rank in a league table – track performance, plan and advocate It differs from other improvement techniques: • focuses on excellence; • involves detailed comparisons; • pools shared information to benefit all contributors; • encourages front line managers and staff in seeking out and implementing change Benchmarking in an archive context

Benchmarking data is usually quantitative so it’s important to support with more qualitative statements.

It can be put to best use when illustrating ‘stories’

• how well an exhibition went (higher visitor numbers than in the same period of a previous year) • how you are contributing to social agendas (how many volunteers and the hours they give) • how effectively you are spending public/charitable money (the amount of match £ and things you’ve been able to deliver as a result) What is available for archives?

Archive Service Accreditation - is a UK wide quality standard which offers a general benchmark PSQG data - survey of visitors to UK archives every 18 months Allows you to compare groups or apply filters. It covers: • Purpose of visit • Frequency of visit • Satisfaction with staff, services and facilities • Outcome of visit Headline report and benchmarking template for October 14 is available from the ARA website What is available for archives?

CIPFA data - comparative profiles enable you to examine how your service costs and performance compare to your peers/nearest neighbours. The profiles look at the following key aspects of archive service provision:

- numbers of access points and service users/visitors

- costs of running archives services

- levels of staffing and volunteers

- availability of resources to the public

- holdings and storage capacity levels

- levels of funding levered in

- user satisfaction How to use the benchmarking data

- Strategic Decision makers - Corporate level, Board of Trustees, Vice Chancellor etc

- Councillors

- Funders

- Commissioners

- Media

- Other? Corporate level

Good organisations will be assessing their performance at Corporate level. Demonstrating that archives offer value is vital. Compelling benchmarking data:

- £ funding secured in past years,
- user numbers have grown despite reduced staff or through volunteer support.

Worth benchmarking the range of functions carried out for the parent body. Benchmarking data can be used to grab attention! Statistics go down well, but make a simple statement, such as ‘Archives users up by 10%’

Some key things to bear in mind:

• Be accurate – make sure the data can be substantiated • Be clear about why you are communicating the information and what you expect as a result • Be concise - politicians are presented with a lot of facts so make yours stand out, it needs to bite-sized and memorable • Make it personal approach – tailor message to their individual interest or give it a local constituency focus Benchmarking data is good for demonstrating ‘how many’ or ‘how much’

Doesn’t illustrate benefit to the individual, community or place, so supplement with user testimony, surveys and interviews

Funders are likely to require additional reporting measures Commissioners

Archives can contribute to wider outcomes and a range of local priorities. You need to have an understanding of your unit costs. It helps to compare the proposed activity with alternative interventions or describe the proxy savings. Benchmarking data is a great ‘hook’ for media communications

Use a key fact such as an increase in user numbers or the financial value of your volunteers

Then enrich with narrative:

• How did you achieve the increase – new service/exhibition? • How does volunteering bring the community together? • What did the users enjoy/learn/benefit? Summary

Benchmarking data will provide a bedrock of evidence

Basic principles of using your Benchmarking data for advocacy:

• Know what message(s) you want to get across • Concentrate on delivering few key points well • Be clear about what Benchmarking data can and can’t do • Be sure of your conclusions Evaluation – a powerful tool for learning What is evaluation?

A structured way of thinking about what happens and why

Evaluation is based on the following ideas

• Evaluation involves making judgements, based on evidence, about the value and quality of a project • Evaluation is open and clear and involves all partners, including the people taking part • Evaluation helps with decision-making during a project and for future projects Why evaluate?

Evaluation has two main purposes:

• to improve practice during the project and for future projects • to show what happened as a result of a project

It can be simple or complex

Internal assessment or External evaluator Five stages of evaluation

Stage 1: Planning Stage 2: Collecting evidence Stage 3: Interpreting Stage 4: Moving forwards Stage 5: Reporting & Sharing Stage 1 - Planning to evaluate

Think about evaluation at the beginning and include in the project schedule/project budget

Set clear aims, objectives and measures of success to inform the evaluation

Consider:

• Who will have overall responsibility for evaluation? • Who else will have a role in collecting evidence? • How much will evaluation cost in time and money? • How will you collect the evidence you need? • When will you review the project? Stage 2 - Collecting evidence

Collect evidence before, during and at the end of a project This will help establish a baseline and enable you to illustrate change The evidence collected should help you judge if the aims and objectives have been achieved and capture any unplanned outcomes Quantitative and qualitative evidence Stage 2 - Choose the right methods

- Will the methods meet funders’ requirements?

- Flexible enough to reveal unexpected outcomes?

- Are they ‘user friendly’ for everyone taking part?

- Do they take account of equal opportunities, ethical and safeguarding issues for CYP and vulnerable adults?

- Will evidence be likely to give different points of view?

- Can evidence be collected without disrupting project?

- Do you have the resources to manage the chosen methods? Pros & Cons of evaluation methods

- Comments’ boxes and books

- Questionnaires

- Small group discussions

- Individual interviews

- Recordings

- Displays at the end of a project

- Observation

- Online websites/chatrooms

- Photography Stage 3 – Interpreting and analysing

Turning raw data into evidence can be time consuming. Quantitative - Survey monkey and similar speed this up. Qualitative evidence may not be as easy to interpret. The following techniques may help you:

• Look back at your original measures of success to help pick out what you need. • Initially sort comments into positive and negative groups. • Check that your judgements are based on evidence from the range of people involved with the project. • Focus on evidence which shows change, progress and development. • Try to establish the majority view. • Look for unexpected outcomes. • ‘Quotable quotes’ Stage 4 – Reflecting & moving forwards

A meeting at the end of the project should not be the end of the evaluation process!

• What were the key findings from the evaluation for the archive service and/or for other partners? • How can strengths and opportunities be built on? • How can weaknesses be reduced? • Is the project likely to have longer-term outcomes, as yet unknown? If so, how will you find out about them? Are they likely to be significant for future work? • What decisions and changes should now be made? Stage 5 – Reporting & Sharing

A written project report is useful for:

• building up organisational knowledge • sharing ideas and good practice with others • helping with future funding applications • showing people what you do

Other ways of reporting may be more appropriate?

In presenting the evidence consider safeguarding and confidentiality issues

Remember to share finding with participants! Stage 5 – Written Reports

Avoid the pitfall of too much description and not enough analysis, conclusions and future action points

• aim to make your report as short as possible • use tables and bullet points to summarise • remember that photographs/diagrams save on words

And keep asking these questions: Are we just telling the story of the project? Have we interpreted the evidence? Are future recommendations based on conclusions?