Retaining and Preserving Federal Records in a Digital Environment: Background and Issues for Congress
Released: 2013-07-26T00:00:00

Introduction

In the course of executing their missions, all federal departments and agencies create federal records. Federal records are defined in the Federal Records Act (FRA) to include "all books, papers, maps, photographs, machine readable materials, or other documentary materials, regardless of physical form or characteristics … in connection with the transaction of public business." The FRA requires federal departments and agencies to collect, retain, and preserve their records—thus providing Congress, the executive branch, and the public with a history of public-policy execution and its results.

The FRA, as amended, requires agencies to "make and preserve" records that document the "organization, functions, policies, decisions, procedures, operations, or other activities of the Government." Agencies are to use "standards, procedures, and techniques designed to improve the management of records, promote the maintenance and security of records deemed appropriate for preservation, and facilitate the segregation and disposal of records of temporary value." Pursuant to the FRA, agencies are to work with the National Archives and Records Administration (NARA) to create records schedules that permit agencies to dispose of records of temporary value properly and to preserve those with permanent value to the government and the public.

In August 2012, NARA and the Office of Management and Budget (OMB) jointly released a directive to federal departments and agencies. Among other requirements, the "Managing Government Records Directive" instructed each department and agency to designate a Senior Agency Official (SAO) "to oversee a review of their records management program." The memorandum addressed the management of federal records in all formats, but specifically acknowledged the challenges presented by the use of emerging technologies to create and maintain records.

Congress may have an interest in overseeing whether agencies are appropriately capturing and maintaining all federal records. The proliferation of electronic records produced using numerous digital technologies and platforms may make it costly, time consuming, and technically difficult for agencies to comply with the FRA. With the increase in the creation and use of electronic records, Congress may have an interest in examining whether agencies are taking appropriate steps to ensure the authenticity and trustworthiness of the documents they create and preserve.

Congress may seek to ensure that agencies are collecting and maintaining records in all formats using methods that will ensure they are accessible to the public in perpetuity. Broadly, the management of federal records in a digital environment may prompt a series of policy questions for Congress, including the following:

Are agencies appropriately collecting and retaining records in compliance with federal law? Will digital and electronic records created today be accessible in the future? What policies or actions might be needed to ensure that digital federal records are authentic, authoritative expressions of government deliberation or action? What are the costs to the federal government if agencies are not appropriately preserving records?

This report provides background and information on the increasing volume of electronic records and the variety of platforms used to create them. It then presents data on agencies' increasing use of electronic media to create federal records and examines the potential implications of increasing creation and use of electronic mediums, including the complications of mixed platforms, the difficulties of ensuring a record's trustworthiness, and understanding and preparing for the risks of using electronic platforms. The report also details and analyzes actions by the Administration of President Barack Obama to streamline and clarify agencies' recordkeeping responsibilities—with a focus on electronic records.

This report does not address all government records issues, such as the collection, preservation of, and access to Presidential records. This report also does not provide details about the collection and retention of federal records, generally. Related topics, such as the laws governing access to federal records and meetings, as well as background on federal information classification policies, are beyond the scope of this report, but are addressed in other Congressional Research Service products.

Contemporary Federal Records: The Challenges of Multiple Platforms

The contemporary challenges government agencies face in creating and retaining records resemble those discussed and debated prior to the enactment of the FRA in 1950. The act was prompted in part by the recommendations of the Commission on the Organization of the Executive Branch of the Government. The commission researched and provided recommendations on a wide variety of public policies and found that "revolutionary mechanization, specialization, and duplication in recordmaking and recordkeeping" had prompted records to "accumulate in admittedly fantastic quantities" that could be "maintained only at excessive costs." The FRA addressed many of the concerns expressed by the commission and provided the Administrator of the GSA the authority to "make provisions for the economical and efficient management of records of Federal agencies." New and emerging technologies have continuously allowed federal agencies to create and accumulate more records, complicating and adding costs to record collection, retention, and preservation.

Multiple Platforms

Currently, most of the records created by federal agencies are "born digital," meaning they are not produced in a tangible, paper-based format. These records are machine-readable, electronic records—whether produced via e-mail, word processing, social media, websites, databases, or other applications. Digital creation has allowed agencies to create records and provide information in a variety of new formats, but has also presented challenges to recordkeeping processes. The variety of applications and platforms used to create, transmit, and store records may complicate how agencies manage, retain, and retrieve their records. It is unclear, for example whether the devices and applications agencies currently use to create and retain digital records will be viable over long periods of time and ensure enduring access to government information. NARA has stated that it is "evolving its information management strategy" to make its holdings "available in perpetuity." Existing scholarship, however, suggests that perpetual access to electronic records is difficult and inherently challenging.

Current records management approaches—which involve updating digital government information as formats and platforms change, taking "snapshots" of websites or printing e-mails, and using RSS feeds or other aggregating technologies to capture electronic content —suggest that no long-term solutions comparable to the retention of paper records have been identified. New recordkeeping technologies may emerge and existing technologies may become less expensive, which could make electronic recordkeeping less expensive overtime. Conversely, enduring access to the various formats of digital federal records might, over time, become increasingly complicated, costly, and problematic.

Platforms for Records Creation

In recent years, the number of platforms—specifically online and electronic platforms—that agencies employ to create records has rapidly increased. On October 20, 2010, NARA released a bulletin that defined social media and other electronic and online platforms that federal agencies employ. The bulletin said social media and other electronic platforms are created to "connect people to government and to share information (e.g., providing information or promoting discussion about the agency, soliciting responses from the public, recruiting personnel, and providing collaborative space to work in new ways)." The bulletin organized the platforms as follows:

Web Publishing : Platforms used to create, publish, and reuse content.

Microblogging (Twitter, Plurk) Blogs (WordPress, Blogger) Wikis (Wikispaces, PBWiki) Mashups (Google Maps, popurls)

Social Networking : Platforms used to provide interactions and collaboration among users.

Social networking tools (Facebook, LinkedIn) Social bookmarks (Delicious, Digg) Virtual worlds (Second Life, OpenSim) Crowdsourcing/Social voting (IdeaScale, Chaordix)

File Sharing/Storage : Platforms used to share files and host content storage.

Photo libraries (Flickr, Picasa) Video sharing (YouTube, Vimeo) Storage (Google Docs, Drop.io) Content management (SharePoint, Drupal).

Not all content created using these platforms necessarily qualifies as federal records. NARA provides agencies with a "non-exhaustive list" of five questions to help determine whether particular content is a federal record:

1. Is the information unique and not available anywhere else? 2. Does it contain evidence of an agency's policies, business, mission, etc.? 3. Is this tool being used in relation to the agency's work? 4. Is use of the tool authorized by the agency? 5. Is there a business need for the information?

Answering "yes" to any one of the questions above, according to NARA, likely means the content qualifies as a federal record. In many cases, social media and other electronic platforms duplicate or "re-post" content that can be found elsewhere in agency records. Such duplicative content, therefore, may not qualify as a federal record.

Complications of Records Management with Multiple Platforms

Some challenges related to the federal government's management of electronic records have been identified. In 2008, for example, the Government Accountability Office (GAO) determined that "e-mail records were not being appropriately identified and preserved" at certain federal agencies. In June 2010, according to testimony from GAO officials, federal records management "has received low priority within the federal government," and that the creation of "[h]uge volumes of electronic information" posed a "major challenge" in agency record management. GAO noted that poor federal records management could leave the government "exposed to legal liabilities, and historical records of vital interest could be lost forever." GAO added that "poorly managed records risk increased costs" for agencies when they search for records to respond to Freedom of Information Act (FOIA) requests or "litigation-related discovery actions."

In May 2011, NARA published a report on agencies' self-assessments of their recordkeeping that found 90% of agencies had a moderate to high risk of records mismanagement. More specifically, the report found that 45% of agencies had records management programs with "moderate risk" and another 45% had records management programs with "high risk" of records mismanagement.

In June 2011, GAO found that many agencies lacked a formal policy on how to capture and maintain federal records created on social media.

The Obama Administration's Recordkeeping Initiative

The "Managing Government Records" Memorandum

On November 28, 2011, President Barack Obama issued a memorandum on "Managing Government Records." Within it, the President stated that "well managed" federal records could help agencies "to assess the impact of programs, to reduce redundant efforts, to save money, and to share knowledge within and across their organizations." The memorandum pointed to technology as complicating records management:

[d]ecades of technological advances have transformed agency operations, creating challenges and opportunities for agency records management. Greater reliance on electronic communication and systems has radically increased the volume and diversity of information that agencies must manage.... [I]f records management policies and practices are not updated for a digital age, the surge in information could overwhelm agency systems, leading to higher costs and lost records."

Pursuant to the memorandum, agencies were given 30 days to designate "a senior agency official to supervise" the creation and submission of a report on agency plans for electronic recordkeeping.

Additionally agencies were given 120 days to create and submit a report to NARA and the Office of Management and Budget (OMB) on agency plans for electronic recordkeeping. The agency reports were required to

describe the agency's plans to improve or maintain its records management program, with particular focus on electronic records; identify any "provisions or omissions" in statues, regulations, or guidance that "pose an obstacle to the agency's adoption of sound, cost-effective records management policies and practices"; and identify policies or programs that could assist the agency's efforts "to improve records management."

The memorandum directed NARA officials, within 120 days after collection of the agency reports, to coordinate with the Director of OMB to "issue a Records Management Directive that directs agency heads to take specific steps to reform and improve records management policies and practices within their agency."

OMB's and NARA's Records Management Directive

On August 24, 2012, the Director of OMB and the Archivist jointly released the Managing Government Records Directive . The directive's introduction lists three expected benefits of the initiative:

improved performance and promotion of openness and accountability by better documenting agency actions and decisions; improved identification of records that have permanent historical value as well as improved transfer of those records to NARA; and assistance to executive departments and agencies in minimizing costs and operating more efficiently.

The directive requires agencies to focus on two goals:

requiring electronic recordkeeping to ensure transparency, efficiency, and accountability; and demonstrating compliance with federal records management statutes and regulations.

To execute the first goal, by December 31, 2019, agencies were directed to manage and retain electronic records in electronic formats—as opposed to paper formats—"to the fullest extent possible." In addition, agencies were directed to manage both permanent and temporary email records in an accessible electronic format by December 31, 2016.

The directive reinforced the President's 2011 memorandum by instructing agencies to designate a senior agency official (SAO) charged with oversight of recordkeeping and disposal. The directive requires that agency records officers receive NARA certification in records training and to ensure that all records of permanent utility to the federal government are "identified for transfer and reported to NARA." Agencies were also instructed to "establish a method to inform all employees of their records management responsibilities in law and policy, and develop suitable records management training for appropriate staff."

Pursuant to the directive, by December 31, 2013, NARA is to revise its guidance to agencies on how to transfer permanent electronic records to NARA. NARA is also required to create new federal agency guidance "for managing, disposing, and transferring email." Additionally, the directive required NARA and agency officials to investigate methods to collaborate with the private sector to find ways to automate record collection and management. The directive required NARA, by December 31, 2013, to describe methods to automate the records management of email, social media, and other electronic platforms. NARA is also required to improve the current methods to streamline the process for the disposition of records to NARA as well as the management of temporary records.

Pursuant to the guidance, the Archivist is to hold periodic meetings with the agency-appointed SAOs to "discuss progress in the implementation" of the directive, and to establish a community of information technology scholars and legal counsel to propose additional guidance, create training, and find new electronic records management tools.

The guidance directs the Office of Personnel Management to "establish a formal records management occupational series," which would create specific federal job titles for records management purposes.

Measuring the Increasing Number of Federal Records

A single record can take many forms, such as a sheet of paper, several linear feet of paper, a map, a digital document, a vast database, or a VHS videocassette. Each type of record requires particular archival responsibilities and presents unique challenges to ensure its perpetual retrieval.

As noted above, agencies are required to collect, retain, and schedule for disposal or permanent retention all series of records regardless of format. Agencies may not destroy "or permanently transfer any record to NARA unless it has been scheduled." The scheduling requirement provides NARA with an estimate of the records series maintained by each federal agency.

Incomplete Data on Federal Electronic Records

In its Electronic Records Project Summary Report for FY2005–FY2009 , NARA wrote that "[w]ith the volume and complexity of e-records increasing each year, it continues to be a challenge for both NARA and [f]ederal agencies to keep pace with the requirements to identify, schedule, and transfer to NARA all existing e-records."

Every year, NARA requests that agencies complete a records management self-assessment. In NARA's FY2011 summary of agencies' self-assessments (the most recent self-assessment available), NARA found that 112 executive- and legislative-branch agencies (of 247 respondent agencies) operated records management programs at "high risk" of mismanaging their records. Additionally, NARA's summary of the self-assessments found, among other things,

most agencies do not have adequate controls for major activities of their records management programs; and many records management staff have insufficient knowledge and understanding of electronic records, which leads to the continued implementation of poor recordkeeping practices.

NARA, therefore, may not know the true volume of the universe of federal records. Nonetheless, the data NARA provides on federal agency records demonstrate an increase in agencies' scheduling of electronic records series.

NARA's data on federal records demonstrate a trending increase in agency records scheduling and records transfers, but the information may not be the most precise tool to help NARA predict the volume of records and the variety of platforms it should anticipate. Some records schedules provide only information on the records series that agencies maintain, and not the precise volume of records contained in those series. Additionally, records series may not identify the precise platform used to create the records in that series. Moreover, the electronic records transferred to NARA from FY2005 through FY2012 may be as many as 30 years old—created prior to the ubiquitous use of electronic platforms. These data, therefore, may not reflect the volumes of records created and platforms used today.

In June 2010, NARA reported that between FY2005 and FY2009 it approved records schedules for 2,404 series or systems of electronic records, with a general trend toward more approvals over time. For example, in FY2005, NARA reported 80 approvals of electronic records series and systems and 794 in FY2009 (see Table 1 ). In 2011, NARA approved schedules for 1,031 electronic records series or systems, the most records schedules NARA has approved in a single fiscal year. Approvals, however, did not always increase over time. For example, in FY2012, NARA approved 418 records series and systems, which was 613 (59.5%) fewer than in FY2011. According to NARA, its FY2012 goal of approving 1,134 records series and systems was not achieved because it used staff to reduce an existing records scheduling backlog.

Table 2 provides data on the number of records transferred to NARA from FY2005 to FY2012. Like the approval of records schedules, the transfer of electronic records to NARA did not consistently increase over time. NARA received the greatest number of permanent electronic records transfers in FY2011, with 257. FY2005 had the fewest transfers to NARA with 124.

The Presidential Records Proxy

Presidential records may be the best indicator of the actual volume of federal records created by federal agencies. Presidential records, pursuant to the Presidential Records Act (44 U.S.C. §§2201-2207), are provided to NARA at the end of each presidential administration (usually every four or eight years). As a result, NARA has tracked the increasing volumes and varied electronic formats employed by each administration.

Although presidential records are collected and retained pursuant to a different law than federal agency records, both categories of records are affected by the same proliferation in platforms and subsequent increases in volume. While the collection and retention of federal agency records has been inconsistent and challenging, White House officials have testified that many electronic presidential records are automatically captured and stored by electronic archiving technologies—a feature many federal agencies lack. Automatic capture should generate a more robust and complete collection of presidential records than one generated by relying on individuals to print and retain copies of e-mails, tweets, or other electronic records. According to NARA, in FY2011, 80% of agencies captured e-mail records by printing them out and filing them. Such a process may create circumstances that lead to either under- or over-capture of federal records. Presidential records, therefore, may provide a more accurate measure of the scope and scale of federal electronic records currently being created as well as what NARA might anticipate for dispensation in future years.

According to NARA's 2009 Report on Alternative Models for Presidential Libraries , "Presidential Libraries … experienced an explosive growth in the volume of electronic records, especially White House email." The report continued:

Presidential Library holdings in electronic form are now much larger than the paper holdings. Indeed, the email system for the George W. Bush Administration alone is many times larger than the entire textual holdings of any other Presidential Library. These electronic holdings bring new challenges to processing and making available Presidential records. The sheer volume exponentially increases what archivists have to search and isolate as relevant to a request, a lengthy process in and of itself before the review begins. Once review begins, the more informal communication style embodied in Presidential record emails often blends personal and record information in the same email necessitating more redactions.

In that same report, NARA noted that the Administration of William J. Clinton provided NARA 20 million presidential record e-mails at the conclusion of the President's eight-year tenure. In contrast, the George W. Bush Administration provided 150 million e-mail records after his eight-year tenure.

In June 2010, GAO submitted testimony to the House Committee on Oversight and Government Reform's Subcommittee on Information Policy, Census, and National Archives on "The Challenges of Managing Electronic Records." GAO used the growth in electronic presidential records to demonstrate that "[h]uge volumes of electronic information" were a "major challenge" in agency record management. GAO also stated that

Electronic information is increasingly being created in volumes that pose a significant technical challenge to our ability to organize it and make it accessible. An example of this growth is provided by the difference between the digital records of the George W. Bush administration and that of the Clinton administration: NARA has reported that the Bush administration transferred 77 terabytes of data to [NARA] on leaving office, which was about 35 times the amount of data transferred by the Clinton administration.

On April 25, 2013, NARA's blog post provided additional details on the records being transferred to the George W. Bush Library and Museum in Dallas, TX—"more than 70 million pages of textual records, 43,000 artifacts, 200 million emails (totaling roughly 1 billion pages), and 4 million digital photographs (the largest holding of electronic records of any of our libraries)." This amounts to a 3,500% increase in the volume of electronic records created when comparing one two-term administration to the next—an eight-year period.

Electronic Records: Policy Concerns and Potential Policy Options

With the increase in the creation and use of electronic records, and concern about the durability of those records, the 113 th Congress may have an interest in overseeing whether agencies are appropriately capturing and maintaining their federal records. Additionally, Congress may choose to revisit the laws that govern federal recordkeeping and disposal to ensure that they include federal records created on different platforms using diverse technologies. Finally, Congress might examine whether agencies are taking appropriate steps to ensure the authenticity and trustworthiness of the electronic documents they create and preserve.

Agency Activities

The new guidelines and requirements for agencies released by the Obama Administration may make oversight of the implementation of the FRA easier for Congress. For example, Congress will have a single point of contact for information on an agency's recordkeeping plans and records management processes. Each agency, however, may provide the designated senior agency official (SAO) with different resources, different staffing levels, and different levels of authority and autonomy. In certain cases, the SAO may not have the resources to appropriately administer the agency's records management responsibilities. Some officials, for example, may not be aware of certain recordkeeping difficulties that exist within components of the agency. Moreover, the SAO may not have access to all agency federal records, particularly those records with sensitive or classified information. Congress may choose to oversee whether SAOs are provided authority, resources, and access within their agencies to make effective changes to their records management processes—including the authority to require certain agency components to schedule records systems or the authority to impose particular records management training or responsibilities for employees.

Congress may choose to enact into law the position of the SAO, thereby ensuring that these positions will exist beyond a single Administration. On the other hand, Congress may decide that creation of a senior records management official is not the most effective method of ensuring compliance with the FRA. For example, GAO has found that agencies have seen mixed results with the creation of other agency officials, such as chief information officers.

Congress may choose to continue its oversight of the implementation of the Administration's guidance on federal records management. Congress, additionally, could choose to require all SAOs to meet to discuss best practices and share methods of working through records management difficulties. Congress could also require NARA, or another agency, to maintain a website that provides best practices and lessons learned.

The Collection and Retention of Mixed Platform Records

Congress may also need to address the gap between the statutory definition of records and the practical use of digital content that amalgamates multiple technologies. It does not appear that the FRA addresses all potential complexities that emerge when agencies create digital content that provides information using multiple technologies at once.

For example, a June 2011 GAO report found that 23 of 24 major federal agencies used Facebook , Twitter , and YouTube . Social media platforms allow agencies to embed and combine content on agencies' websites. The Department of Transportation, for instance, could embed a YouTube video of a recent advisory committee meeting on its website. The website could also include text that provides context to the meeting and identifies those in attendance. Agencies are not required by the FRA to retain duplicates of federal records in most cases. When considering the integration of various platforms, however, it is unclear whether placing a record in a new, unique context would constitute a discrete federal record. A record that layers various electronic platforms may contain redundant information, but present that information in a different way for a specific purpose that adds value that could constitute consideration as a separate federal record. Current laws and regulations may not make clear to records management employees when combining or repurposing information constitutes a new record. More importantly, it is unclear whether current recordkeeping software is capable of capturing and retaining all electronic content, including all website updates.

Congress may choose to examine whether existing guidance addresses issues that may arise from integrating multiple platforms to create federal records. If existing guidance is silent or unclear, Congress may choose to consider legislative options that would address these concerns, or may choose to task NARA with addressing them.

Ensuring the Trustworthiness of Electronic Records

The increasing use of electronic records also requires agencies to ensure the trustworthiness and authenticity of the information they create. Agencies need to ensure that their records are appropriately protected from corruption or destruction. Additionally, agencies need to make certain that records accessed by the public are accurate and usable. The networks that create and store federal electronic records must be protected from a variety of risks, including "an inability to document or validate transactions that occur via an agency web site." NARA provides agencies with guidance on identifying and maintaining trustworthy websites. According to NARA, trustworthy records have the following characteristics:

reliability: content is trusted as a full and accurate representation of ... transactions, activities, or facts;

authenticity: proven to be what it purports to be;

integrity: complete and unaltered; and

usability: can be located, retrieved, presented, and interpreted.

NARA's guidance suggests that agencies "maintain the content, context, and sometimes the structure of" their websites to ensure their records are trustworthy.

Congress may have an ongoing interest in ensuring that agencies are appropriately providing for the trustworthiness of their records. Congress may choose to hold hearings or pursue informal oversight mechanisms to determine how agencies are verifying that their e-government initiatives are protected and that publicly available electronic information is reliable, authentic, has integrity, and is usable.

Understanding and Preparing for the Risks of Using Electronic Platforms

Congress may have interest in ensuring that agencies understand and prepare for the potential risks of creating and retaining on electronic records. NARA's guidance provides examples of potential risks associated with the use of electronic platforms. Among these risks are "an inability to document or validate transactions that occur via an agency web site," "an inability to reconstruct views of web content," and "financial losses due to compromising the citizens' or government's rights." NARA's guidance states that each agency should conduct a risk assessment that addresses "the possible consequences of untrustworthy, lost, or unrecoverable records, including the legal risk and financial costs of losses, the likelihood that a damaging event will occur, and the costs of taking corrective actions."

Congress may choose to ensure that agencies conduct the risk assessments recommended by NARA. Congress may also have an interest in receiving agencies' risk assessments to learn of the potential consequences of mismanaged, damaged, or otherwise lost records. Congress could choose to have NARA collect these risk assessments and submit a summary to Congress.

Financial Resources

To the extent that Congress believes that a lack of resources leads to poor records management, it could address the problem through appropriations. According to NARA, however, agencies are not required to report their recordkeeping costs. It is not clear, therefore, what resources agencies currently use to administer various recordkeeping duties. Nor is it possible to determine whether these resources enable agencies to appropriately administer their records. It is also not clear what it might cost to collect data on recordkeeping costs, nor would it be easy to determine what might constitute a recordkeeping cost (e.g., storage costs, training costs, staffing costs, oversight costs).

Congress may choose to require NARA to define what would constitute a recordkeeping cost. Congress could also choose to require the senior agency officials tasked with records management duties to report these recordkeeping costs to NARA, which could then aggregate the data and provide Congress and the public a summary document on agencies' recordkeeping costs and how they compare to one another. Such a summary document could inform agencies, NARA, Congress, and the public as to the resources currently spent on federal recordkeeping—and whether those resources are enough to appropriately comply with existing statutes, regulations, and policies. On the other hand, Congress may decide that such data collection and analysis would be too time consuming and costly and may not yield information on more effective records management. An agency's recordkeeping costs, for instance, may not capture the efficient or creative ways that agencies may be complying with the FRA.

Congress could also consider requiring SAOs to report additional records management data points—such as total records schedules submitted to NARA, total number of electronic records submitted to NARA, number of records series that the agency has yet to schedule with NARA, or creative and cost-saving methods of capturing and retaining electronic records. A more thorough records management report could provide Congress with more information about the status of the federal government's records management. Such data collection, however, would not be without cost.