Explore Your Archive

Summative evaluation report

Steve Slack June 2016 Introduction

Background The National Archives (TNA) commissioned me, on behalf of its partnership with the Archives and Records Association (UK & Ireland) (ARA), to undertake a piece of summative evaluation in relation to the Explore Your Archive (EYA) campaign. This involved an analysis of some existing pieces of quantitative research carried out by both organisations, plus some new, independent qualitative research.

Research criteria The research aims were

- to secure credible quantitative and qualitative data from the sector on the campaign’s impact, or lack of impact; and
- to develop an evidence base to enable a decision on whether to continue with the campaign and, if so, any adjustments that should be made.

Methodology In order to answer these, and to enable an effective analysis of the information gathered, I created an Evaluation Framework based on the original aims and objectives of the project. This document set out a range of objectives we wished to evaluate and also detailed which evaluation inputs I would use to see to what extent those objectives had been realised.

Following this stage I conducted a series of in-depth, 30-minute telephone interviews with archives and record services across the UK and Ireland. Each interview followed a similar ‘script’ and allowed for a large amount of qualitative data to be gathered. Participants were assured that their names would not be linked to their comments, enabling them to give honest and frank feedback.

This new data was then combined with the existing research material and analysed.

Evaluation themes The original project objectives and outcomes were analysed and grouped into four themes:

- **Strategy and Planning** Preparing the groundwork for the campaign and providing individual archives with the tools they need

- **Audience and Engagement** Attracting new audiences to archives, both onsite and online, and engaging with existing users in new ways

- **Publicity** Providing tools which enable archives to communicate with new and existing audiences through a joined-up campaign

- **Perception** Developing the public understanding and appreciation of the role of archives in modern society Report structure The main chapters of this report relate to the objectives of the project, which are laid out in the Evaluation Framework under the four headings above. These objectives have been listed at the start of each section in grey boxes, with associated evidence and discussion of each one detailed below.

There is an introductory chapter about EYA activities and also a supplementary chapter about the potential future development of the campaign.

The report ends with ten conclusions and recommendations to be considered about the future of the campaign. There is also an appendix which details quantitative data from the most recent online surveys to have been undertaken with archives and record services (2015 and 2016).

Telephone interview research sample Using some names/email addresses of warm contacts provided by the ARA and some people who had previously agreed to take part in future research from a TNA online survey, I emailed around 40 people, inviting them to participate in the research.

There were 17 responses to our call for interviewees, 12 of which we were able to schedule 30-minute interviews with over the course of one week in May 2016. One person was unwell during the fieldwork stage so I ended up interviewing 11 people.

I also had conversations with the Chair and CEO of the ARA. Nobody from TNA was interviewed as part of this formal research process.

The 11 respondents were broken down as follows.

Gender

| Gender | Count | |------------|-------| | Male | 5 | | Female | 6 |

Location

| Location | Count | |---------------------|-------| | North of England | 3 | | Midlands | 1 | | South of England | 1 | | London | 3 | | Wales | 1 | | Scotland | 1 | | Ireland | 1 |

Type of archive

| Type of archive | Count | |-----------------|-------| | Local authority | 5 | | Independent | 3 | | Business | 2 | | Academic | 1 | Executive summary

The archives and records sector sees Explore Your Archive (EYA) as a strong campaign. There is plenty of evidence to suggest that EYA should continue into the future and that the current branding should remain in place for the time being.

Archives particularly like the modular format of EYA, allowing them to choose to what extent they want to get involved. Each participates in a way that is appropriate for their remit, size, budget or timescale. Some simply offer a few tweets, while others programme a range of events and activities.

A real strength of the campaign to date has been its interaction with social media. Individual archives are pleased to take part in online activity and enjoy using the daily hashtags.

The sector has offered some constructive criticism to the organisers of the campaign. They believe there is scope to simplify or rationalise the existing toolkit – which is now three years old – and they would like some more time to prepare their activities in advance of the annual launch week. But in general they feel supported by the campaign organisers.

There is a question mark over the impact EYA has on the understanding and appreciation of archives amongst the general public. In part this is due to a lack of direct evidence from users and the public, but there is also a concern that the campaign is, at present, reaching existing users, easy-to-reach audiences and the archive sector itself, rather than on new audiences.

With considerable buy-in from the archive sector and a strong campaign infrastructure already in place, EYA is now in position to take the project to the next level. The conclusions and recommendations at the end of this report suggest that TNA and the ARA might revisit the aims and objectives of the campaign in order to maximise on the achievements to date. There is every hope that EYA might become even stronger and more successful in the future. Campaign activity

Although the exact ways in which archives engaged with the EYA was not part of the initial evaluation framework, it would be useful to describe the kinds of activities that were delivered as part of the campaign.

There was a wide range of types of programmed activity, including:

- exhibitions/displays
- talks/lectures
- social media activity
- film screenings
- drop-in advice sessions
- archive tours
- creative activities
- sending out story boxes
- open days

The two most popular ways to get involved with the campaign were to hold events and to engage with social media. Section AE3 describes the kinds of activity that took place in more detail.

Some archives chose to get involved with just one element. Others put together a programme with more than one type of activity.

“We just did social media. Some tweets. It’s all we had time to do, really.”

“A whole week of events. One thing every day.”

“Great War Bake Off, building tours, film show and exchange of experience on research data management and archives.”

“Campaign on social media to promote the archives and especially what happens ‘behind the scenes’.”

Some people told us they were planning to hold events in November anyway and simply tagged the EYA branding onto their existing activities.

“We were planning to do [some activities] anyway, but actually used the EYA branded stuff and used pencils and badges.”

For others it was a catalyst to do something.

“Gave us an impetus to increase presence on social media and this resulted in much more engagement and an increase in followers.”

“We’ve never done anything like that before so a good impetus.”

For a handful of archives, EYA gave them the opportunity to be creative with their programming or for a chance to try something new.

“We borrowed costumes and got dressed up in Tudor dress.”

“We printed the EYA logo onto biscuits. They went down a treat!” “The events that worked best were the ones which had a local appeal and relevance - they got people who had never been into the archive before to come in for the first time. We chose famous local historical moments and planned our events around those. The public seemed to like it and engage with them.” Strategy and planning

| Objective SP1 | |---------------| | Create a campaign that archive services want to get involved with and something in which they see value |

This objective has definitely been achieved. There is plenty of evidence that the archive and record services we spoke with want to engage with the campaign. They talk about the campaign in positive terms. Over 80% of archives surveyed agreed that EYA was successful both for their service and also for the archives and records sector overall.

It is clear from looking at the range of research material that EYA has gathered momentum over the three years it has operated so far. More archives are reporting taking part in EYA each year. Of the people that were questioned in the most recent online survey around 40% took part in the 2013; around 50%-60% took part in 2014 and around 70% took part in 2015.

Around half of respondents to one survey said EYA launch week in 2015 was better than in previous years.

It is also evident that archives see EYA as getting better and better, in particular the social media campaign.

A majority of archives said they felt the campaign was a success for their individual archive services. They see EYA as something which has value and want to continue to be associated with it (see FD1).

Archives like the EYA brand.

“The brand is strong and clear.”

“It’s something we are proud to associate with.”

“We liked the brand and realised there was an opportunity for giving out free things.”

They also see EYA as an improvement on previous campaigns.

“These sorts of campaigns used to look a bit WI, or Blue Peter. But now the visual clarity in the campaign is great.”

“Much better than the old Archives Awareness Week.”

EYA launch week is seen as a moment of focus and a drive for activity.

“Good that it focuses everyone’s efforts over the course of one week, so the noise that each individual archive makes is amplified by everything else that’s happening.” “Allows us to focus and give attention to something in November. If it was even more dispersed around the year it wouldn’t be as attractive to us.”

Archives think the **public-focused aims and objectives** of EYA are sound.

“It gives you the chance to show off the archives and make people aware that they have this local service they can come and visit.”

The people I spoke with like feeling **involved and engaged** in the campaign. It makes them feel part of a wider network across the sector. It is also seen as a vehicle for **celebrating archive services** and gave them a **sense of pride** in the work they do.

“Celebrating alongside other archives across the UK and reinforcing that we are a part of a national network.”

“It allows us to feel part of a community.”

“Using the Explore your Archive campaign has helped us nudge our archive stories up the list of priorities for our corporate [communications] team. It serves us well when we are able to remind our local authority that we form part of a wider archive community. They like that!”

There are, however, some **barriers to archives participating in EYA** and they were not shy about offering constructive criticism of the campaign, which is detailed in later chapters of this report.

**Time** is a constraining factor for many archives, as is **funding**.

“Given the prevailing financial situation we find it difficult to create events just for the purpose of advertising/promoting archives.”

“We have an archive service to run here.”

There is concern from some that the campaign launch week is held at the **wrong time of year**. But this is not a universal finding; there are some archives who like celebrating EYA in November.

For some there is an issue of **not having enough time to plan for participation** in EYA. This is discussed more in section SP5. Objective SP2

Create a stand-alone campaign brand and strategy for archive services to engage with EYA

This objective has been achieved. The archives we spoke with are very aware of the campaign and the brand of EYA. Although we must remember that this research was undertaken with people who had previously engaged with EYA – we didn’t speak to the archives sector as a whole.

Archives are slightly less aware of the strategy that drives EYA than they are of the main brand.

Part of the strategy for the campaign was to undertake a full evaluation of the project, collecting vox pops at events across the country, encouraging archives to undertake their own evaluation of events, counting numbers of participants and providing short exit questionnaires. There was also to be some analysis of media responses and of the social media engagement through monitoring the daily hashtags. It was suggested that the number of ambassadors be counted as well.

Archives are not aware that this was part of the original plan and it seems from what I can establish so far that these planned activities did not happen, making it difficult for us now to fully evaluate the programme, especially from the public’s point of view. But using the data that we have been able to gather recently we have managed to get a deep insight, at least, into the perspectives of individual archives’ experiences and their observations and memories of what happened over the three years EYA has run.

It should be noted that for the majority of archives EYA is seen as a series of events that take place in November, rather than as a year-long campaign.

Objective SP3

Create a community discussion forum for archives to use to discuss their ideas, stories and plans for the campaign

This objective was not achieved.

Of the people we spoke with, nobody used the online forum. In fact, the majority had never heard of it. A few archives told us that they felt quite isolated when they were planning their events, and that having some support or chance to speak with other archives going through the same experience would be seen as positive.

“This might already happen and I could be ignorant of it - perhaps some sort of regional networking. For example, xxxxxx were also holding an event on the same theme as us and on the same day as us. We were able to promote their afternoon talk at our morning event. However, it was only that the archivist contacted me to ask if we would distribute flyers that I know this was happening at all.”

**Objective SP4**

Create a campaign that is clear to understand and with a simple call to action and realistic in its objectives

This objective has been largely achieved. The people we spoke with thought the EYA campaign was **articulated to them clearly** and that the prospect of taking part in it was **realistic**.

“We were given enough idea of what they were looking for, but also enough scope to find something interesting as well.”

“It was all quite manageable and achievable.”

Archives like that there is plenty of guidance available, to help them deliver their EYA activities.

It seems, however, that there is a concept among some archives that EYA is intended to take place **just for one week**, rather than a year-long campaign that has a launch week in November and they are invited to be a part of it all year. People told us about their participation in ‘Explore Your Archive Week’, which is a misconception as this does not technically exist. It may be that this is a residual idea from the previous awareness campaign.

**Thematic content**

Over 90% of the archives represented in this evaluation liked that in 2015 the **campaign did not have one central theme**.

“Specifying [a theme] can be restrictive.”

“For the mini Explore Your Archive event in summer 2015 on the theme of democracy we had no relevant material so could not get involved.”

A small group of people thought the **overarching themes should be retained**. But these could potentially be people who just need a little more support and inspiration, rather than a specific overarching theme.

“We feel overall it’s good to have a central theme or structure to hold it all together.”

“I feel that having a theme concentrates the mind and helps us to think creatively about our collections.”

Something people really liked were the daily twitter themes. These were welcomed and seemed to encourage people to want to participate.

“We found the daily themes on Twitter a great way to get involved. The more fun ones, e.g. #archiveanimals, worked the best. They get a wider audience interacting with them and the social media team are more likely to work with a theme like that, rather than more specific ones such as #archiveslive and #yearinarchives (which we also felt were quite similar).”

Cost benefits and values There is no clear consensus on whether participation in EYA offers cost benefit to archives. In one survey around 60% of archives and record services see a cost benefit in terms of time and money spent, and for these people the costs associated with taking part in EYA are worth it.

“Publicity is invaluable especially at a time when local councils are still making cuts to services. So it’s good to be a part of.”

“Not much cost was involved so potentially yes.”

While for others it is seen as an expense that might not have paid off.

“At the moment it takes a lot of time to organise and there are costs in terms of creating captions for displays and marketing. We don’t usually see any particular increase in donations to offset the costs.”

It is fair to say that while all archives don’t necessarily see a direct cost benefit in taking part in EYA, they do see the value in taking part in the campaign.

“It is time-consuming to compile social media content alongside regular work and the time/money spent did not physically result in more money. But it did raise awareness of our archive.”

“Creating an exhibition takes time and despite advertising it through the local media and to interested groups, few come specially to see it. However I think it is worth doing.”

Others were more concerned that there was no way of measuring what impact the activity they had planned and delivered had had on users or visitors.

“This is difficult to answer, as we don’t have any measurable points.”

“Hard to say without seeing the balance of books or knowing how many people visit the website.” Resources Archives like having a central pool of online resources that they can refer to and describe them in generally positive terms. They appreciate having a toolkit that is always available and the guidance notes are well used and seem to have an impact on how people deliver their activities.

“Thinking up hashtags in advance would be really time intensive for me, but having some options makes it really easy. Advance notice really helps too.”

“We used the branding to push ourselves into places that we wouldn't normally go to. It gave us the impetus to make approaches - something as simple as the poster design that looked that professional gave us a certain confidence to take our ideas to new audiences.”

Findings from some of the testing methodologies suggest that the majority of archives find the branding material easy to use. They are suitable for archives’ needs. There are, however, some comments about how these are accessed that TNA and the ARA should be made aware of. These are detailed in section P2.

The support provided to archives from TNA and the ARA is generally seen in positive terms with very few people saying it needed improvement.

| Objective SP5 | |----------------| | Provide sufficient time for archives to respond to how they will create their local campaign |

There is concern from many who we spoke with about the lead-in period for the campaign and the amount of time that leaves them to prepare their activities.

“We felt the information about the social media campaign came out quite late, and did not give us a lot of time to prepare content. Our events are planned months in advance (up to a year) and this makes it very difficult for us to do an EYA event when we don't know dates, if there will be a theme, and what sort of event to plan.”

“A bit more notice about [the social media themes] would be good.”

“If you could get the toolkit for each year out earlier, that would be great.”

It seems that local authorities need longer to prepare their activities than business or academic archives, which can respond faster. They are often more agile organisations and are used to moving quickly. One local authority archive told us that they plan their activity April-March and need to submit their plans for approval in December for that period. That would mean knowing the themes and ideas for EYA as soon as one November event is completed, to plan for the next year. This is an extreme example, however. Most people agreed that having information about the campaign eight to nine months in advance would be appropriate.

For some, the time of year isn’t right.

“The campaign at present runs very close to most archives’ shut-down periods, so this seems a bit at odds with trying to generate awareness of archives because a lot will be closed or have reduced services not long afterwards.” Audience and engagement

There are examples of archives engaging with just a handful of people as part of the EYA campaign, while others claim to have attracted over 250 people to their events. But as the individual archives and their events are so different, an analysis of user engagement numbers would not be comparing like with like.

The sample size and testing regime used for the online surveys means we cannot make more meaningful statements about the number of members of public who engaged with EYA. But numbers are not the only way to evaluate the campaign’s success. The evidence that has been gathered can still tell us something about how people engaged with this campaign.

| Objective AE1 | |---------------| | EYA engages with existing archive audiences in a new way, deepening their understanding and levels of support for using archives and creating more sustainable relationships |

EYA activities organised by individual archives have certainly engaged with their existing user bases. Although there are no hard numbers to go off, the impression is that more existing archive users engage with EYA activities than new users. This is perhaps to be expected, given that this audience is already going into archives to use them. But that’s not necessarily a bad thing.

“We were pleased with the turn out for the talk and hope to build on this in the future.”

“We got to promote to our usual users of the library and say ‘hey look, there’s more in this archive than you might think of in a regular library.”

There is no direct evidence to suggest that existing users are engaged in new ways or that they have deepened their understanding or changed the nature of the archive-user relationship. EYA has certainly attracted some new people to archives. While the numbers are not known, there is a mood of modest success here, amongst the archives we spoke with.

There is some anecdotal evidence to show that this audience is younger than the existing user base, but only slightly. People in their 40s and 50s have engaged with archives as a result of EYA, as opposed to the existing users, who tend to be retired.

“It attracted new visitors to the archive and we were able to join up our events with other projects.”

“New visitors came to the archive, even some who lived very close by, but who had never visited before.”

People who were new to archives tended to have a generally lower level of knowledge of how archives worked before their visit, which is to be expected. And we didn’t find any evidence of people having much deeper understanding or appreciation of archives as a result of their visit.

Overall, there is uncertainty about whether EYA has had a meaningful impact on new audiences.

“We had an increase in the number of followers on Twitter and Instagram during that period, but we can’t calculate whether there has been a change in the type of visitor as a result of the campaign.”

Attracting a new audience to come into the archive space is difficult, and TNA and the ARA know this. That’s part of the reason why EYA exists. It is perhaps most useful at this stage to look at those few examples where archives have managed to engage a new audience and to look at what they were doing. This is explained further in the Hallmarks of success for engaging new audiences at the end of this report.

A key feature of positive engagement is the proactive taking of archive (or facsimile) material out to a new audience. The story box concept was a great success and we heard evidence suggesting that a new audience really did engage with archive content in a new way and take a deeper understanding from the experience.

“The box went on our tour without us with facsimiles. Sometimes we accompanied it - to public health meetings.”

“Story boxes are great. People can engage with documents in a different way. They are rummaging for themselves, rather than being delivered documents.” The most successful projects which attracted a new audience were when archives had thought about their audience in advance of the sessions and what they were most likely to be interested in.

“The events that worked best were the ones which had a local appeal and relevance. They got people who had never been into the archive before to come in for the first time. We chose famous local historical moments and planned our events around those. The public seemed to like it and engage with them.”

But it does need to be remembered that these are isolated examples, not the general trend and that further than their own experiences, people are also unsure about whether the EYA reaches a new audience in general.

The idea of audience development in general caused a small amount of negative conversation with some people questioning whether it was realistic to grow user numbers through a campaign like EYA.

“We already have a lot of visitors coming in, so increasing visitor numbers isn’t a priority.”

### Objective AE3

**EYA engages with a new audience online**

Across the evaluation methodologies we saw that between 80% and 90% of the archives surveyed undertook some online or social media activity, proving that this is a part of the campaign that archives are very happy to get involved with.

The twitter campaign and the use of general and also daily hashtags received much praise. Facebook and flickr were used, but to a lesser extent.

In terms of a web presence, there wasn’t much content placed online by archives other than the occasional listing. I got a sense from local authority archives in particular that they are not in control of their own webpages, so cannot update them easily. But many of them run their own social media channels, which means they can add content there quickly and easily.

It seems as if participation in this area is developing, with year-on-year growth in the numbers of archives getting involved online.

“We gained more twitter/facebook followers and our insights showed greater reach of our posts when using the hashtag.”

“In part, we reached a larger audience via our social media and attained new followers.”

“The #explorearchives hashtag started trending at one point here.” While archives might struggle to inspire new audiences to attend real-life, in-person activities, the online environment is seen as a much more straightforward space to work in. And it’s a lot easier, faster and cheaper to take part in as well.

There were some questions raised about the level of impact that online engagement had on the public though.

“On twitter there was a lot of coverage and activity. But I wonder if that was just people within archives tweeting and retweeting other people in archives.”

“We got some retweets from TNA and the EYA group itself, which was nice. Whether that actually raised our profile short or long term, I don’t know. But it was fun.”

There was also some concern expressed that the social media campaign has not been analysed in a way that lets the sector know what its impact was. Previous research into EYA activity revealed this as a weakness of the campaign and this is something which could be improved on in the future.

The EYA website is generally seen as a good thing, but as the campaign has grown and more and more people are using it, some issues around functionality have arisen.

“The public website wasn't very easy use - it was quite difficult to find an event. All the events in the country were on there but it was hard to find them. It didn't grab people's attention.

The Thunderclap campaign received a mixed response. Some people thought it was easy to use and quick to implement, whereas other people were frustrated about its functionality. There is very little evidence about the success of this part of the campaign, however. Publicity

Objective P1

Create and communicate strategy for a multi-channel campaign that can be delivered by local archives

We have already seen that archives like the EYA branding and the general campaign. They are happy to associate themselves with it and seem to understand its role and significance.

The people working in archives I spoke with were aware of the strategy driving EYA, but did not feel directly connected to a communications strategy for the campaign.

But that is not necessarily a problem. Part of what archives like about EYA is the very fact that they can pick and choose how to get involved, so having a campaign which allows them to select their own level of interaction seems to be working, so far.

Some of the less confident archives suggested said they could do with some training and support in this area, but people are generally pleased with the level of support they receive from the ARA and TNA in relation to communications.

Objective P2

Create collateral for a multi-channel campaign that could be delivered by all kinds of archives, allowing them to create their own events and activities. This toolkit includes advice, guidelines and ready-to-use artwork.

We have already seen that people like the brand and branded materials and that they are available as part of an online toolkit. They also appreciate that there is planning guidance available online. People told us that the resources are generally suitable for their needs.

There are however, some areas which archives think could be addressed in future delivery of the EYA programme. Many people I spoke with had difficulty locating material online. It seems there is a lot of information in the online resources and it’s not that straightforward to find.

“We find it overcomplicated. You often get an email saying there’s an update to the toolkit, but then it’s quite hard to actually find those new things on the website. We think it looks more intimidating than reassuring - lots of folders to look through and it's all a bit much.”

“The website is quite frustrating.” “Some of the resources are difficult to find. There's a page on TNA website which has the actual resources. If you google ‘EYA toolkit’ you get to a pdf that doesn't actually have any of the resources.”

“The EYA website seems to be for the public, which makes sense. And then the resources are on the TNA website. It's really buried and you really have to look for it. For the average archive who's interested, they’re not going to spend ages looking for it. It needs to be easy to find.”

Some archives reported design and software usability issues.

“There were some difficulties getting the logo to use in our artwork.”

“There’s a poster template. We don’t find that too easy to grapple with.”

“There aren’t poster templates you can adapt - they are fixed with images in them. We want to adapt them.”

“Having to pay £30 for a specialist font seems a bit ridiculous. There are plenty of fonts out there you could use.”

Some people ran into a few practical issues.

“The pencils are quite poor quality which is a shame to give something out where the lead keeps breaking.”

“The badges, pencils etc all got delivered far too late.”

**Objective P3**

Develop relationships with a national pool of advocates for archives who act as high-profile spokespeople during the campaign

This objective has been achieved to a certain extent.

The majority of people involved in the research recalled some high-profile people who had been involved in the campaign, but not all of them could remember who was involved and when.

Advocates who were associated with the campaign were Jamie Owens, Sarah Edwards, Paddy Ashdown and “one of the McGanns”.

The most memorable place people working in archives had seen these spokespeople was on BBC Breakfast News. Nobody spoke about the impact any of these people had on their own work or on the campaign in general. It did not seem to be a big part of EYA for most people, more of an aside. This is perhaps because most of them didn’t have any direct contact with the advocates.

**Objective P4**

Create an integrated media campaign that promotes EYA across the country

We can only measure the level of recognition that the media campaign had on the archives we spoke with, meaning that we are not getting a full picture of what the general public think about EYA. But from within the sector, it seems as if people are at least aware of the campaign’s media presence.

Around half of the people surveyed had seen some press coverage relating to EYA, compared to around 90% who have seen something on social media. In ratings exercises, the social media campaign was viewed as more successful than the general EYA press coverage or the campaign website.

From the research carried out by the ARA and TNA it has been suggested people working in archives are neither impressed nor unimpressed by the kind of media coverage that EYA receives. There doesn’t appear to be a consensus about whether the media coverage created a positive impact or not.

There is some concern that the media push only happens in the immediate run up to the EYA launch in November, not earlier than that. It was been suggested that the media be encouraged to cover EYA in advance of it actually happening.

“If media launches were brought forward a bit it would give time for journalists to respond to them in advance of the events starting. As it is, there’s something put out in the media and our events are happening on the same day. People can’t make that. They need a bit of time to plan.”

“Blink and you miss the media coverage. And you miss the campaign too.” Public understanding

The following four objectives are all to do with the impact that EYA has had on people. That includes, in part, existing users of archives. But most importantly it involves the general public. EYA seeks to raise their awareness of archives and to excite them about the role and function archives play in society.

This report has already underlined the lack of any quantifiable public-facing evaluation so far as part of the EYA campaign, and this is reflected in the comments made by archives as part of these interviews.

When we asked archives what they thought the public thought of the campaign they said ...

“It’s hard to say whether the public are more aware or engaged because this campaign doesn’t really talk to them. It talks to existing users and people who are already switched on to or aware of archives, rather than those people who have the potential to be switched on to us, but aren’t.”

“Any publicity has to be positive, although it’s hard to quantify the direct impact.”

“Not sure about this, but it certainly didn’t hurt.”

“What is the real impact? It seems rather internal. Is it really getting more people involved?”

“We have no way of knowing whether there has been any increased use on the back of it.”

“We have not seen the actual stats of how the #explorearchives tweets were interacted with.”

There are some who told us that EYA definitely had a positive impact.

“We got some new people in and people that don’t use archives. We got a lot of comments on social media - people enjoy seeing the collection online.”

“I’ve had random conversations with people because of the badge I wear.”

“We didn’t really talk to the public that much. I think awareness of archives is on the increase.”

“People who had never been before have come back to us again since the events.”

There are others who, although they might not have evidence to describe impact on public understanding, are at least talking in aspirational language about it.

“I hope the social media had an impact. I’d hope. But I don’t know about awareness.”

“I’d like to think there was positive progression there.”

“Being involved in the campaign has hopefully made a meaningful difference.” It also ought to be noted that for some archives, business archives in particular, changing public perception about the archive sector is not a priority, which is understandable. But they still see participation in EYA as a positive thing.

“It's an internal communications exercise for us, so the public perception of archives in general isn't really relevant.”

**Objective PU1**

Increase public awareness and raise profile of local, university of and private archives

Archives say they think the campaign has helped improve awareness of their existence. Around 70% of archives agreed with the statement that the first three years of EYA have made (or are in the process of making) a meaningful difference to public awareness of their particular archive or records service. But they told us the evidence for this is not yet in place.

“Our numbers have increased slightly, and we have had a marked increase in volunteer applications.”

“We still get local people coming in who say they never knew that we were here and when we hold events for EYA we always get people who appreciate seeing the range of what we hold and are encouraged to find out more about their local or family history.”

“I have yet to see a direct link between the campaign and the number of visitors to the reading room but it certainly has helped with creating awareness of archives and archivists.”

Others are less sure that EYA has had a direct impact.

“Mostly the campaign appears to be preaching to the converted.”

“I can't say that any member of the public has mentioned it to me.”

“We will wait and see whether there's any long term pay-off.”

“I still don't think the campaign does very much in reality. The use of social media is definitely an improvement, however, so we have hopes for the future.” Objective PU2

Excite the public and give them a sense of pride about their local archive and record services

While we may not know what the shift in public perception may have been in terms of pride in archives, according to the online surveys between 80% and 90% of archives said that being part of the campaign made them and their colleagues proud about the work they do.

Objective PU3

Raise value of the essential role archives play in society

This research has not found any evidence to suggest that this objective has been met.

When we asked archivists ‘Do you think the public understand the role archives play in society as a result of EYA?’ and ‘Do you think the public would describe archives as an essential service as a result of EYA?’ ten out of 11 respondents told us they were either not sure or didn’t know.

Objective PU4

Help the public to realise the skill and expertise of people who work in archives

As above, the people we spoke with couldn’t answer a question about their understanding of the public perception of people who work in archives.

But it is pleasing to note that when the online surveys asked about internal skills development, between 30% and 40% of participants said EYA had resulted in them developing some new skills.

We don’t know what these skills are, but future research could potentially look at these in more detail. Future development

Objective FD1

EYA continues to exist in the future, with the support of the archive sector

Across the evaluation methodologies used as part of this research there is a very strong message from the archive community that they want **EYA to continue to exist in the future**.

Archives see EYA as having value. The key benefits it delivers for them (which are also listed in section SP1) are:

- a strong brand with which they are pleased to associate themselves;
- an opportunity to focus their attention and publicity;
- a campaign with aims and objectives they believe in; and
- a chance to speak as one voice as a sector and to be linked to other archives.

“It reinforces the notion that we are part of a larger network of archive services helping to preserve our historical resources.”

“It suits us. The product is great and the time of year is perfect. An element of celebration.”

“It’s only by having concerted efforts that we’ll make any real impact. Individually we don’t really have sufficient clout.”

Objective FD2

Archives are given the chance to contribute to the development of EYA in the future

Archives gave some feedback about how they would like the EYA campaign to develop in the future. They suggested a range of improvements to the campaign. Some of these are constructive criticism of the campaign as it currently stands, while others are original ideas or trends.

The main areas of focus for the development of the programme are as follows (in order of popularity). Maintain the brand Keep the focus of attention on a brand that archives recognise and like using.

“I think the purpose of a brand is to associate with it and make it recognisable, so the longer you keep it the stronger the identity becomes and the more recognisable it becomes. I do like how they add new verbs to it, e.g. remembered for the First World War. We should keep it as it is for a while.”

“It could be the kind of thing that becomes ingrained in the public consciousness. So if we repeat it enough times people will think about archives annually. Like heritage open days.”

Maintain the social media presence This was seen as a real positive of the campaign. It was suggested that this needs to be both sustained and developed.

“I do hope the campaign continues - perhaps with a dedicated digital archives element?”

“Social media was really successful in 2015. It's a great way of getting people's attention and getting them involved. So let's build on that. How can we engage non-traditional users through social media?”

Revisit the aims and objectives While people like the main objective of EYA, they had some suggestions to make about what else the campaign could deliver in terms of public understanding.

“It's time to move on from preaching to the converted and to take it to the public more.”

“Should be more focussed on people who don’t use archives or who don’t know what an archive is. So people like artists or in education.”

“It's more about raising the profile rather than getting new users. In the age of dwindling numbers of physical users, we can't compete with that. So we should continue to raise profile and to signpost people to digital resources.”

“I'd like some guidance about how to reach out to new or less experienced audiences.”

“How can we engage non-traditional users in to archives? It's probably not talks, as they're for regulars. What else will get them involved?”

More notice/lead-in time The consensus that 8-9 months in advance would be a good amount of time to know about the themed hashtags.

“The communication from the centre needs to be a bit clearer and earlier. One of the conservators told us that there's going to be a conservation theme this year. We've heard none of this so far, but then I spotted it in a newsletter last year, so not everyone knows about it.”

“We need more notice to create this kind of content.”

“We don't always have enough time to plan - if there's a theme it's often communicated quite late in the day so we can't respond as quick as they might like. 8 months would be a good lead-in time.”

**Simplify the online resources**

Tidy up the back of house and make it easier to find and easier to use.

“I wonder if the webpage could be made a bit clearer to find things. I had to search through a few docs to find things like the logos, the hashtags to use, who to send my content to.”

“Make templates easier to use.”

“Make it obvious where to get the digital resources from.”

“Make the posters easier to customise.”

**Extended media coverage**

Archives would like more media focus with more coverage that goes beyond the archive sector.

“A disinterested local press is the main issue. Suggestions on alternative marketing would be useful.”

“More national press engagement would be good.”

**Improving the web presence**

It was suggested EYA place effort into enhancing the campaign’s public-facing website.

“Would there be a way to encourage more archives to register their events?”

“I couldn't easily see how to add my event to the EYA website or find the appropriate contact person to do it for me. Could we add our own events as per Culture 24 website?”

“The public website wasn't very easy to use - it was quite difficult to find an event. All the events in the country were on there but it was hard to find them. It didn't grab people's attention.”

“EYA website stays up through the year - needs to be updated. For example now [May 2016] it says come to our archive in Nov 2015. Doesn’t look good. Could it just have a holding page for the rest of the year? It looks a bit neglected.” An earlier launch date

“More promotion before the event would be good to raise awareness for archives to get prepared and public to recognise the scheme.”

“It could do with some more publicity from September onwards to get ready for it.”

“Get a high profile person to launch it before the campaign starts.”

Connecting partners Allowing archives to be connected to one another so that they are able to share ideas.

“It would be great to know what other people are doing.” Hallmarks of success for engaging new audiences

In addition to the conclusions and recommendations (listed on the following pages) there are some trends that have been observed as part of this research process that TNA and the ARA should be aware of. These trends are what seems to have worked best in terms of archives engaging new and existing audiences.

I have described these as the hallmarks of success – the elements of the very best kinds of projects and activity I heard about when speaking with archives during the course of this research.

These hallmarks have been observed previously by the researcher in other examples of engagement across the heritage sector – museums, libraries, archives and heritage properties – but were also evidenced in those few projects examined here, which managed to engage new audiences.

Projects which successfully engage new audiences have at least three or more of these hallmarks in common. Projects which are successful ...

... are based on a topic that’s relevant to their audience They have something people can relate to and that they will be interested in.

... are visible They have a presence and make themselves known.

... communicate clearly They communicate something meaningful to an audience, rather than simply ‘we are here’.

... make it easy to get involved They break down barriers and actively invite people in.

... make it meaningful They let people see archives as special. This doesn’t need to be expensive or complicated.

... incentivise users to return They give people a reason to come back. Conclusions and recommendations

This report draws ten main conclusions and offers ideas and recommendations for what TNA and the ARA might consider for the future of EYA.

1

EYA is in a strong position. The campaign has gathered momentum and is now in a great place, with support across the sector, especially when compared with the previous awareness campaign.

*Maintain the EYA campaign into the future.*

2

Archives like the EYA brand and want to be associated with it. They say there is life in it yet and it doesn’t need to be refreshed. There are requests that the media campaigning around EYA could be improved at a local and national level, but the visual identity and language are working just fine.

*Maintain the EYA brand for at least two years before it has a refresh, but look again at the communications plan for the campaign, to see what more can be done in this area.*

3

Archives like that they can pick and choose to what extent they get involved with EYA. They like the lack of a major theme, but they also like being able to get involved with themed daily hashtags on social media. Sometimes they show evidence of needing some support when it comes to inspiration and creativity.

*Keep the optional, modular structure of EYA as it currently stands. But recognise that as the campaign leaders you can be proactive in suggesting what archives might want to do. Offer up inspiring case studies of great EYA activity so that those less confidence archives, or those with very limited resources, might be inspired to go a little further.*

4

The annual lead in time for EYA launch in November is an issue for some archives. Local authorities require a longer lead-in time (sometimes up to a year) while business or independent archives who are naturally more agile and can plan in just a few months.

*Create a planning structure that allows people to start accessing online materials as early as possible – ideally around 8-9 months before the annual campaign launch in November.* While the EYA toolkit is generally welcomed there is some frustration about how well some of the elements are accessed and organised online. Some find it confusing to use – the templates aren’t as user-friendly as they could be and in some cases there is almost too much information. It seems the resources have been added to each year, but not necessarily clarified or sense checked.

Revisit the online resources and check them from an archive’s/member’s point of view, rather than from the campaign point of view. Consider creating a bespoke place for all the resources to sit in an easily accessible location, perhaps in the back-end of the main public EYA site.

There is evidence to suggest that some archivists have increased their skillset as part of being involved in EYA. But we don’t yet know what those skills are.

Put plans in place to evaluate the skills that people have developed over time and see if this could be a more targeted element of future campaigns.

There is only a small amount of anecdotal evidence that EYA engages with audiences beyond existing archive users or those who are aware of them already. For some archives that’s not a problem as they only want to communicate internally to their colleagues and stakeholders. While EYA is great for existing audiences it does not yet entice many new audiences in huge numbers.

With such strong backing from the sector and a robust campaign infrastructure, EYA is in a good place to take on the challenge of engaging new users and the general public further. TNA and the ARA should consider the hallmarks of success and see what works best for attracting in a new audience in those few places that are doing it well. By sharing case studies of successful projects, archives could be inspired to create engaging programmes – if they want to.

Social media is a particular strength of the campaign and has clearly gained momentum over the last three years. But there are concerns that the sector is talking to itself and to existing users, rather than reaching out to new audiences. There is also no real data about online interaction to create a strong evidence base about social media activity.

Maintain the social media campaign and consider what strategies might help take the campaign to the next level, to engage more people beyond the sector and our existing users. Also consider how to evaluate social media activity in the future. No structured evaluation has been undertaken with the public to test the impact EYA has had on them, so we don’t actually know what they think.

Undertake some first-hand evaluation with end users or the general public to find out what they think of the campaign and how they see it developing in the future.

EYA has clearly developed over the three years it has been running. From reading the project paperwork it does not seem as if there was previously a clear vision on how to measure the results and impact of the campaign or how to evaluate success. The summative evaluation process has brought some clarity, but the project could benefit from a set of clear and measurable objectives for the future.

While stopping to take stock of what the campaign has achieved so far, it may also be a good time to revisit the aims and objectives for the next three years of the project and to check the campaign message is exactly what you want it to be. For the next three-year period, decide not only what you will deliver, but what you will evaluate and what your measures of success will be.

This is a positive campaign and it’s fine to have aims and hopes for the future, but be clear about what is an achievable objective of the programme and what is a hope or aspirational outcome. Appendix: 2015 qualitative research data

The Explore Your Archive (EYA) summative evaluation report used a mix of research methodologies and yielded a large amount of both qualitative and quantitative data. The latest piece of research has been in-depth interviews with archives across the country. The findings from these conversations have been amalgamated with the results of online surveys that were issued at various times through the project’s life. Surveys were sent out at the end of years 1 and 2 and two surveys were produced and sent to the sector at the end of year 3.

The raw quantitative results from the year 3 surveys – the most recent data we have – are presented here, for reference.

Some of the questions in these surveys were asked in the same format, allowing us to amalgamate data in places. Other questions were only asked in either Survey 1 (administered by TNA in December 2015) or in Survey 2 (administered by the ARA in January 2016). The relevant survey(s) is shown for each question shown here. The number of responses to each specific question is also shown as N. Surveys 1 & 2 combined

**Have you taken part in Explore Your Archive in 2015-16 or are you planning to?** (This year’s campaign runs from roughly September 2015 to May 2016.)

| Response | Percent | Count | |----------|---------|-------| | Yes | 71.9% | 101 | | No | 28.1% | 20 |

N=121

Survey 2

**Have you taken part in previous Explore Your Archive campaigns?**

| Response | Percent | Count | |----------|---------|-------| | Yes | 65.2% | 15 | | No | 34.8% | 8 |

N=23

Survey 1

**Did you take part in previous Explore Your Archive weeks?**

| Response | Percent | Count | |----------|---------|-------| | Yes, in 2014 | 12.3% | 12 | | Yes, in 2013 | 7.1% | 7 | | Yes, both in 2013 and 2014 | 33.7% | 33 | | No, neither | 46.9% | 46 |

N=98

### Survey 2

**If you have already held an event or series of events, approximately how many people attended on average (a rough estimate is fine)?**

| Response | Open field responses | |----------|----------------------| | c 150 | | | 12 | | | 22 | | | 40-50 | | | c.70 | | | 25 | | | 20 | | | 15-20 | | | 260 | | | 130 in total | | | Exhibition launch and lectures, about 80 Visitor sessions about 15 at a time | | | 0 | | | 1 | | | about 90 | |

N=14 Survey 1

**How did 2015’s Explore Your Archive week compare to those you've participated in previous years?**

| Response | Percent | Count | |----------|---------|-------| | Better | 46.7% | 21 | | The same | 48.9% | 22 | | Worse | 4.4% | 2 |

N=45

Surveys 1 & 2 combined

**Have you used (or do you plan to use) social media as part of your Explore Your Archive activity?**

| Response | Percent | Count | |----------|---------|-------| | Yes | 90.3% | 84 | | No | 9.7% | 9 |

N=93

Survey 2

**Do you think that Explore Your Archive offers cost-benefits, ie in terms of time and money spent?**

| Response | Percent | Count | |----------|---------|-------| | Yes | 62.1% | 18 | | No | 37.9% | 11 |

N=29 Survey 1

**How would you rate the support and communications from The National Archives, ARA and partners for the 2015 Explore Your Archive week?**

| Response | Percent | Count | |-------------------|---------|-------| | Excellent | 29.2% | 21 | | Good | 58.3% | 42 | | Average | 1.5% | 9 | | Not very good | 0.0% | 0 | | Poor | 0.0% | 0 |

N=72

Survey 2

**How would you rate the support and communications from The National Archives, ARA and other partners for the 2015-16 Explore Your Archive campaign?**

| Response | Percent | Count | |-------------------|---------|-------| | Excellent | 4.5% | 1 | | Very good | 36.5% | 8 | | Good | 41.0% | 9 | | Satisfactory | 9.0% | 2 | | Needs improvement | 4.5% | 1 | | Not good at all | 4.5% | 1 |

N=22

### Survey 1

**How would you rate the publicity around the 2015 Explore Your Archive week?**

| | Excellent | Good | Average | Not very good | Poor | Not good at all | N= | |----------------------|-----------|------|---------|---------------|------|-----------------|----| | Press coverage | 3 | 16 | 15 | 5 | 1 | 33 | 73 | | Social media | 18 | 44 | 7 | 1 | 0 | 3 | 73 | | Main campaign website| 6 | 39 | 19 | 2 | 0 | 7 | 73 |

### Survey 2

**How would you rate the publicity around the 2015-16 Explore Your Archive campaign? (Select 'Did not see' if necessary).**

| | Excellent | Very good | Good | Satisfactory | Needs improvement | Not good at all | Did not see | N= | |----------------------|-----------|-----------|------|--------------|-------------------|-----------------|-------------|----| | Press coverage | 0 | 1 | 6 | 3 | 3 | 2 | 7 | 22 | | Social media | 1 | 8 | 6 | 1 | 2 | 2 | 3 | 23 | | Main campaign website| 0 | 2 | 12 | 2 | 2 | 2 | 1 | 22 | Surveys 1 & 2 combined

**If you have taken part, was the 2015-16 Explore Your Archive campaign successful for your service?**

| Response | Percent | Count | |----------|---------|-------| | Yes | 86.8% | 79 | | No | 13.2% | 12 |

N=91

Surveys 1 & 2 combined

**From your perspective, do you think the 2015-16 campaign has been successful for the archive and records sector overall?**

| Response | Percent | Count | |----------|---------|-------| | Yes | 89.4% | 84 | | No | 10.6% | 10 |

N=94

Surveys 1 & 2 combined

**Has taking part in the 2015-16 campaign resulted in you or your colleagues developing any new skills?**

| Response | Percent | Count | |----------|---------|-------| | Yes | 40.9% | 38 | | No | 59.1% | 55 |

N=93 Survey 1

**Did the 2015 Explore Your Archive week help you to improve awareness of your service?**

| Response | Percent | Count | |----------|---------|-------| | Yes | 65.3% | 47 | | No | 2.8% | 2 | | Not sure | 31.9% | 23 |

N=72

Survey 2

**Has participation in the 2015-16 campaign helped you improve awareness of your service?**

| Response | Percent | Count | |----------|---------|-------| | Yes | 85.0% | 17 | | No | 15.0% | 3 |

N=20

Surveys 1 & 2 combined

**Has the 2015-16 Explore Your Archive campaign made you (and colleagues) feel proud about the work you do?**

| Response | Percent | Count | |----------|---------|-------| | Yes | 93.5% | 87 | | No | 6.5% | 6 |

N=93 Surveys 1 & 2 combined

As in 2014-15, there was no prescribed subject or theme for the 2015-16 Explore Your Archive campaign. Was this the right approach for you?

| Response | Percent | Count | |----------|---------|-------| | Yes | 94.6% | 88 | | No | 5.4% | 5 |

N= 93

Survey 2

Do you think that the first three years of Explore Your Archive have made (or are in the process of making) a meaningful difference to public awareness and use of your archive or records service?

| Response | Percent | Count | |----------|---------|-------| | Yes | 71.9% | 23 | | No | 28.1% | 9 |

N=32 Survey 1

| Do you think the Explore Your Archive concept should be continued in future? | Response Percent | Response Count | |---|---|---| | Yes | 95.5% | 85 | | No | 0.0% | 0 | | Not sure | 4.5% | 4 |

N= 89

Survey 2

| Would you like to see Explore Your Archive continue in its current form (an annual campaign with a designated launch week, where press and social media activity is concentrated across the UK and Ireland)? | Response Percent | Response Count | |---|---|---| | Yes | 96.7% | 29 | | No | 3.3% | 1 |

N=30