The National Archives Finding Archives Resources Summary Report

Published: 17th January 2012

Prepared by: Sharon Nichols, Research Director Tel: 01663 732721 Email: snichols@djsresearch.com Website: http://www.djsresearch.co.uk Introduction and Objectives The National Archives has conducted market research to gather feedback on the online resources which it hosts which detail material held by other repositories. This is aligned with the goal for improved online access for archive resources, as identified in the Archives for the 21st Century. This research focuses on understanding stakeholder requirements for these resources in the future. Specifically, the research objectives were as follows:

Objective 1: To understand external stakeholder requirements for The National Archives’ online resources that detail material held by other repositories (a definition of stakeholders is presented in the appendix 1)

Objective 2: To consolidate requirements gathered from external stakeholders with requirements gathered at internal workshops with The National Archives’ staff

Objective 3: To understand the functionality required of these resources by stakeholders

This research focussed on the following online resources hosted by The National Archives:

- National Register of Archives
- Manorial Documents Register
- Hospital Records Database
- Accessions to Repositories
- Access To Archives
- Archon Directory

The research did not focus on The National Archives own catalogue.

Methodology Internal Research: Initially, The National Archives, conducted internal stakeholder workshops to gather information on key issues for internal and external users and contributors as viewed by staff.

Qualitative External Research: 5 focus group discussions with both contributors to, and users of, The National Archives’ online resources were conducted across England, Scotland and Wales. In addition to this, 10 face to face depth interviews were conducted with interest group representatives, other archive networks and 9 telephone depth interviews were conducted with general users.

Quantitative External Research: An open survey was carried out online to generate a prioritised list of requirements; 179 responses were collected. This survey sat on The National Archive’s website between 25th November 2011 and 13th December 2011. The same survey was also sent to 84 stakeholders and the response rate to this was 35. All those who responded to the online survey were screened to check that they were aware of at least one of the resources that the research focussed on and had either used or contributed to at least one of them in the last 12 months.

Throughout the research with external stakeholders, respondents were asked to think of The National Archives online resources which detail material held by other repositories (as outlined earlier). Please note that feedback was not sought on a resource by resource basis. About this document This document summarises key findings from the external research, conducted by DJS Research Ltd. It is worth noting that the research that was conducted internally amongst The National Archives staff did reveal similar themes to those which arose with external stakeholders.

A positive reaction to the research With 214 responses to the online survey and over 50 external stakeholder views received through the qualitative research process, the research includes a wide range of views from different types of repositories and users of The National Archives online resources. Many of those that we spoke to in the qualitative research had to travel a long way to participate in the research. DJS Research and The National Archives wish to thank all of those who participated in the research for their time and comments. These views will help The National Archives to shape these resources moving forward.

The National Archives’ online resources are invaluable The research did reveal a number of issues with these online resources, but it is worth noting that the resources themselves were considered to be valuable by respondents. The following comment is typical:

“I can’t imagine working without any of these resources.”

However, respondents felt that the resources were developed in a less advanced digital age and there is scope to make them better – it is now possible to do much more given technological advances.

Key themes emerging from the qualitative research

Expectations for The National Archives online resources which detail material held by other repositories are shaped by what is possible to achieve with other information discovery websites. Requirements and desires for The National Archives online resources are heavily inspired by respondents’ experiences of using other popular websites such as ebay, BBCnews, google™,amazon.com, expedia (“A lot of the time on the BBC you can get to where you want without having to type in a single letter. You go to the home page and away you go.”). Needs such as a good search function, summary and in depth information and added value extras (links to Google Earth™, alternative interest points, intuitive search) are all taken from such websites.

The process of contributing information to The National Archives is problematic: Contributors also feel that the process of contributing information was frustrating and could be made easier by having fewer resources to update, better communications with The National Archives, clearer guidance and templates for providing information, greater feedback on information provided and more autonomy to update information remotely. The issue of cataloguing was a common theme amongst contributors discussing providing information: they recognise this as a sector-wide issue and look to The National Archives to provide guidance. Contributors were, however, quick to recognise the difficulty of providing a system to suit all institutions’ differing needs for providing information. The challenge, they said, arose from the differences in types of information held and also differences in available resource between different types of organisation.

The search function is extremely important: The ease and success of a search determines a users’ assessment of a website or resource, and respondents’ benchmark search function is definitely Google™. Although there seems to be no standard search profile (i.e. users search The National Archives resources in all sorts of different ways), there are some key requirements which emerge: to be able to perform an advanced search is the most important followed by being able to filter searches. Respondents also wanted to be able to search a specific resource.

**Cataloguing standards are a sector-wide issue:** Throughout the discussions, respondents raised the issue of cataloguing and standards. Users are frustrated by and wary of information returned from archival searches that are so varied in content and format, using language that is very technical in nature. In the qualitative research, respondents were all asked to search for the same piece of catalogue information. The results revealed that people started in all sorts of different places, and obtained different search results depending on how they conducted their search: respondents experienced the impact of non-standardised cataloguing first-hand.

**Awareness is an issue** Some Respondents felt that the inexperienced user may not know that these resources exist, and may struggle to find them as a result (some experienced stakeholders struggled to find these resources on The National Archives website). Some felt that less knowledgeable users may be more likely to expect to find catalogue information via Google™.

**Key insights from the quantitative research**

The online survey was based around a list of attributes (or requirements for an information discovery tool which would host The National Archives online resources detailing information held by other repositories). These attributes were generated from findings from the qualitative research. Respondents were asked to assign from 0 – 100 points per attribute, with 100 being extremely important and 0 being not important at all.

The attributes focussed on a range of areas identified as being important from the qualitative research:

- The process of contributing information (asked of contributors only)
- Entering the site and navigation
- Undertaking a search
- Refining the search
- Context and information provided as a result of the search
- What you can do with the search results (i.e. added value features)
- Access information for the records themselves

There were 47 attributes in total, and these attributes have then been ranked according to the mean score (average score) out of 100. The top 10 most important attributes are as follows (Note that there is an appendix which shows the full list of prioritised attributes):

| Rank | Attribute | Mean Score | Process | |------|---------------------------------------------------------------------------|------------|----------------------------------------------| | 1 | Ability to perform an advanced search (210) | 88.74 | Undertaking a search | | 2 | Ability to filter search results once they have been returned (213) | 86.37 | Refining a search | | 3 | Your own collection reference numbers prioritised over other unique identifier (97) | 85.68 | Updating information (contributor only) | | 4 | Presentation of results in a context which makes clear which collection each result relates to (211) | 85.26 | Context and information | | 5 | A clearly signposted search box for these resources on The National Archives website (211) | 83.67 | Entering the site and navigation | | 6 | Information about whether the document can be viewed online (213) | 82.21 | Access information | | 7 | Search results that include variations on the search term (210) | 81.41 | Undertaking a search | | 8 | Information about requirements for permission to access the original document (213) | 80.57 | Access information | | 9 | Option to choose a specific National Archives resource to search (212) | 80.08 | Undertaking a search | | 10 | Single resource to update not several (96) | 79.26 | Updating information (contributor only) |

*Figure in brackets=base size for each attribute.*

The National Archives – Finding Archives Project – Summary Report It is clear looking at the top 10 attributes that effective search functions are crucially important to successful online resources. Respondents want a clearly signposted search box and the option to search specific resources. They want mechanisms to perform an advanced search, to filter search results, which would include variations on the search term.

There was a lot of detail contained within these 47 attributes. To summarise an additional question was included in the research to understand where the balance of opinion lay. For example, did people want:

- Volume of data displayed (i.e. high number of results) vs quality of data displayed (most relevant results returned)
- Breadth (high level information about collections of data) vs Depth (detailed information about catalogues)
- Structured or filtered search vs a free text search
- Content of the collection and/or item vs Context of the collection and/or item
- Links to data held externally to The National Archives vs Data held internally by The National Archives

Respondents were asked to move a slide scale for each towards the option they most preferred. The results are shown in Appendix 4. It was clear from this exercise that respondents wanted the results to be of high quality with lots of detailed content.

**Conclusions**

The findings showed that effective search functions are crucially important to successful online resources. Respondents want a clearly signposted search box and the option to search specific resources. They want mechanisms to perform an advanced search, to filter search results, which would include variations on the search term.

Improving the processes of contributing information to the online resources is also revealed to be a priority. Contributors felt strongly in favour of having a single resource to update rather than several and want to feel that the systems are built around their needs i.e. having their own collection reference number prioritised over any others.

However, this isn’t to say that the other factors that were included in the research are not important. They are simply less important in the context of this research than other factors, and will be more important for some people. It is therefore important to not lose sight of other issues which came lower down the list. We term these as ‘enhancing factors’. Users will expect good levels of service or functionality from the most important factors as a matter of course – if this isn’t possible a supplier will find it very difficult to satisfy customers or stop them going elsewhere. Conversely, most of the added value attributes included in this research (for example, links to Google Earth™, news alerts, links to other sites, user profiles) fall towards the end of the list of prioritised attributes and can therefore be considered long term goals or *enhancers*. These are added extras, which can also positively influence the user experience and ‘delight’ users.

**Next Steps**

The next steps for The National Archives will be to consolidate these results with the findings of the internal workshops (which revealed similar themes), and identify key actions. Appendix 1: Definitions of stakeholders

Definitions of stakeholders include:

- **Contributors to resources** – those who provide information to The National Archives online resources which detail information held by other repositories

- **Users of resources** – those who use these resources, including archive professionals, academics and those using the resources to research personal interests, whose details were collected through the quantitative research. Note that in the qualitative research we did not include those who use the resources to forward personal interests

- **Organisations** which represent the archive sector Appendix 2

- Ability to perform an advanced search (210) 88.74

- Ability to filter search results once they have been returned (213) 86.37

- Your own collection reference numbers prioritised over other unique identifier (97) 85.68

- Presentation of results in a context which makes clear which collection each result relates to (211) 85.26

- A clearly signposted search box for these resources on The National Archives website (211) 83.67

- Information about whether the document can be viewed online (213) 82.21

- Search results that include variations on the search term (210) 81.41

- Information about requirements for permission to access the original document (213) 80.57

- Option to choose a specific National Archives resource to search (212) 80.08

- Single resource to update not several (96) 79.26

- Ability for repository to update information directly on The National Archives resources (98) 79.06

- High visibility of resources on the National Archives website (209) 78.52

- Inclusion of data from all main online sources (i.e. AIM 25) (201) 77.62

- Repository able to upload data directly into system, using templates provided by The National Archives (96) 76.83

- Simple and easy to use layout similar to Google® (211) 76.57

- Statistics on usage of my data (97) 75.74

- Ability to forward or download search results (209) 75.50

- Ability to perform a simple search of all the NATA archives online resources (211) 75.04

- Description of the whole collection and individual items within the collections (200) 74.26

- Information about the archival institution holding the original document (i.e. opening hours) (212) 73.80

- Results displayed in order of relevance (210) 73.76

- Ability to download information about individual catalogues, collections and repositories (210) 73.30

- Automatic harvesting of info but with an alternative for repositories that don’t hold data electronically (96) 72.24 Appendix 3

- All search results listed no matter how many (206) - 69.27

- Regular import to The National Archives system (89) - 69.18

- User guides (212) - 68.89

- Ability to bookmark individual results (209) - 68.68

- Ability to save searches (209) - 68.13

- Name authorities can be created by contributor and uploaded (97) - 67.59

- Suggestions for alternative search terms that may be of interest (209) - 65.41

- Contextual information about the types of records (209) - 64.42

- Contextual information about the people, organisations and places referred to in the collection (207) - 63.84

- Ability to search resources directly from Google® (208) - 62.42

- Name authorities can be contributed by professional archivists only (94) - 62.34

- Usage of non technical terms (205) - 61.50

- Contextual information about the archival institution holding the records (211) - 60.83

- Links to other sites which may be of interest (211) - 59.49

- News alerts when something new is added to a catalogue of interest (207) - 57.81

- Glossary of key terms (210) - 57.73

- Ability to add or annotate the information once retrieved (207) - 51.40

- Summary information about the whole collection only (204) - 51.30

- Descriptive content available as open data (91) - 49.57

- Being able to create a user profile (204) - 44.42

- Entry point that allows you to distinguish what type of user you are (204) - 43.67

- Detailed descriptions of individual items within collection without summary info about collection as a whole (205) - 41.66

- Links to Google® Earth to allow you to see where the repository is (208) - 35.57

- Name authorities that can be edited by other users (93) - 31.81

Mean score out of 100 Appendix 4

Key Themes – Where Does The Balance Of Opinion Lie?

Q. For each of the following pairs of attributes, please move the sliding scale towards the option that most interests you. If both are equally important, leave it in the middle.

| Attribute | 0% | 20% | 40% | 60% | 80% | |----------------------------|----|-----|-----|-----|-----| | Volume | 13 | 14 | 15 | 16 | 18 | | Breadth | 22 | 23 | 24 | 25 | 26 | | Structured Search | 14 | 15 | 16 | 17 | 18 | | Content of collection/item | 53 | 54 | 55 | 56 | 57 | | Links to data held externally to TNA | 38 | 39 | 40 | 41 | 42 |

Base: All Respondents (214)