Climate Monitoring: NOAA Can Improve Management of the U.S. Historical Climatology Network
Released: 2011-09-30T00:00:00

Letter

Background

NOAA’s National Weather Service (NWS) manages the approximately 11,000 weather-monitoring stations across the country that are part of the Cooperative Observer Program. Volunteer observers at the stations generally record daily maximum and minimum temperatures and 24-hour precipitation totals and submit the data to NWS over the telephone, by Internet, or by mail. The records for stations in the Cooperative Observer Program can stretch back well over a century, with some records predating the establishment of NWS in 1890. NWS uses data from these Cooperative Observer Program stations to support weather forecasts and warnings and other public service programs. The data are also used by others, including state climatologists, farmers, and resource planners such as energy providers that use weather information to anticipate and plan for varying levels of energy consumption.

NOAA’s National Climatic Data Center (NCDC) established the USHCN in 1987 by selecting a subset of weather-monitoring stations from the existing Cooperative Observer Program network of stations. The USHCN currently consists of 1,218 stations. NCDC has twice revised the makeup of stations that compose the USHCN—in 1996 and 2009—primarily to extend the weather records of stations that have closed over time as volunteer observers have discontinued their service. To address this issue, NCDC added data from nearby stations with similar temperature trends that are continuing to gather and report data. In all, NCDC has added over 100 stations as of the latest revision in 2009. NCDC does not have a direct role in managing USHCN stations but relies on NWS’s weather forecast offices throughout the contiguous United States to continue to manage the stations as part of the larger group of weather- monitoring stations in the Cooperative Observer Program. For example, NCDC relies on weather forecast offices to maintain records on the location of the stations and other conditions that can affect weather observations, including the types of equipment used to measure temperature and precipitation and the time of observation.

NCDC uses USHCN data to assess and monitor climate variation and change, including to quantify national- and regional-scale temperature trends within the contiguous 48 states. On the basis of its analysis of USHCN data, NCDC estimates that the average surface temperature across the contiguous states has warmed by about 1.4 degrees Fahrenheit since 1895. NCDC’s analysis has also identified areas of the country where temperatures have cooled or remained relatively stable. NCDC combines temperature records from the USHCN with temperature records from weather-monitoring stations around the world to analyze global temperature trends. This analysis has in turn been summarized in the assessment reports of the Intergovernmental Panel on Climate Change, an international body that reviews and assesses the most recent scientific and technical information produced worldwide relevant to the understanding of climate change.

NWS headquarters establishes the policies, standards, and requirements for managing the Cooperative Observer Program, and weather forecast offices in six NWS regions (central, eastern, southern, western, Alaska, and Pacific) have responsibility for recruiting and training observers and installing and maintaining temperature-measuring equipment and rain gauges on observers’ properties. NWS applies the same standards and requirements to all stations in its Cooperative Observer Program, including those in the USHCN. In particular, NWS has established siting standards for measuring air temperature to ensure uniformity in meeting national and international requirements for climate observation. The standards, which cover conditions in the immediate vicinity of the stations, specify that temperature-measuring instruments should  not be sited on rooftops;  be installed over level terrain;  be installed at least 100 feet from any extensive concrete or paved surface;  be mounted 4 to 6 feet above the surface; and  be no closer than four times the height of any nearby building, tree, fence, or similar obstruction.

NWS guidelines state that implementation of these standards should be flexible and balanced with other factors, such as the availability of space. According to NWS, these siting standards are based in part on recommendations of the World Meteorological Organization, an agency of the United Nations that, among other things, coordinates the activities of member states to generate data and information on weather and climate in accordance with international standards. For example, according to World Meteorological Organization guidelines, the best sites for measuring air temperature are over level ground; freely exposed to sunshine and wind; and not shielded by or close to trees, buildings, and other obstructions.

NWS has also established management requirements for weather- monitoring stations that call for inspections of stations and updates of station records to reflect any changes. The requirements for inspections call for a minimum of one inspection by weather forecast office officials per year and specify that during these inspections, the officials are to review observers’ practices for taking weather measurements, check equipment and perform any needed repairs, and assess the conditions surrounding the station, among other things. The management requirements state that, even if there are no changes at a station, officials from weather forecast offices should update each station record at least once every 5 years. To provide a complete and permanent record of a station, NWS has designed an information system that weather forecast offices are to use to record the dates of inspections and update station records. Such records are used by NCDC and other researchers to help interpret weather records from a station and determine how factors such as station location and measurement instruments affect the weather records. Figure 1 depicts the roles of NWS and NCDC in managing the USHCN.

USHCN Stations Were Selected on the Basis of Several Factors, but Siting Conditions Played a Limited Role

According to NCDC officials, achieving a relatively uniform geographic distribution across the contiguous 48 states was a high priority when selecting USHCN stations and was balanced with other factors, including how long stations had collected temperature records, limited periods of missing temperature data, and the stability of measurement conditions. According to NCDC officials, consideration of siting conditions in the immediate vicinity of stations played a limited role in both the initial selections in 1987 and when stations were added in 1996 and 2009 because they considered other factors, such as geographic distribution, to be more important to the analysis of long-term temperature trends.

NCDC Selected USHCN Stations by Balancing Geographic Distribution with Other Factors

NCDC officials told us that in selecting stations for the USHCN, the agency placed a high priority on achieving geographic distribution across the contiguous 48 states, so that the network could help identify both national and regional warming and cooling trends. To achieve the geographic distribution needed to identify regional trends, according to agency officials, NCDC aimed to select a minimum of two stations from each of the 344 climate divisions across the country. NCDC officials acknowledged that they encountered difficulties achieving the desired geographic distribution in certain areas of western states—such as Nevada—that have a relatively low population density and thus fewer stations to choose from because of a lack of volunteers to serve as observers. As a result, according to NCDC officials, station density is slightly higher across the eastern states than in the western states. Our analysis of all 1,218 USHCN stations (including active stations and those that were inactive or closed) found that while NCDC generally met its aim of two stations per climate division, 14 percent of climate divisions had fewer than two stations. As of April 2011, 20 percent of climate divisions had fewer than two active stations (see fig. 2). According to NCDC officials, the existing climate divisions are only one way to partition the nation’s climate, and if divisions were being developed today, the climate divisions would differ in number and in the areas they cover.

According to NCDC officials and documents describing the process used to select USHCN stations in 1987 and to amend the list of stations in 1996 and 2009, the agency also sought stations that had temperature records dating back to the early 20th century, had limited periods of missing data, and had a limited number of station changes, but sometimes made exceptions to these factors.  Number of years of temperature records. In order to detect long-term temperature trends, NCDC aimed to select stations that had a long history of temperature records, ideally dating back to the early 20th century. In some cases, however, NCDC selected stations with a shorter history of temperature records than was ideal to ensure geographic distribution of stations across the contiguous 48 states, according to officials. NCDC officials also told us that they created composite stations to achieve a minimum record length when no stations in a particular geographic area had been collecting temperature records as long as they sought. According to NCDC officials, they create a composite station by combining data from one or two stations that have closed with data from an active station in the same area whose temperature records overlap in time with records from the closed station or stations and continue to the present. NCDC officials told us that they compare the stations’ overlapping temperature trends before creating a composite to help ensure that the climates at the stations are similar. According to NCDC documents, the initial selection of USHCN stations in 1987 included 84 composite stations, and, as of the latest revision to the network in 2009, the number of composite stations had increased to 208, largely in response to station closures. Our analysis of the 1,218 stations that make up the USHCN as of the latest revision in 2009, including composite stations, found that NCDC has largely achieved its desired record length. Specifically, as of 2010, over 85 percent of the stations had a record length dating back more than 100 years, and another 14 percent had temperature record lengths of 76 to 100 years. Less than 1 percent of stations had record lengths of 75 or fewer years.  Extent of missing data. NCDC officials told us that they also attempted to select USHCN stations with limited periods of missing data but that they often had to select stations with incomplete temperature records, including stations that were missing data for multiple years, because few stations have complete records. For example, about half the data from the Little Falls Mill Street station, located in upstate New York, are missing. The station’s record has data for a few years in the 19th century, but data in the intervening years are sparse, with frequent gaps in the middle of the 20th century, according to a 1990 NCDC report on the USHCN. According to NCDC, various factors result in missing data, such as periods when a volunteer observer is not available or when instruments malfunction and need to be repaired. Our analysis of temperature records shows that only 24 of the 1,218 USHCN stations (about 2 percent) have complete temperature data from the time they were established through 2010; the remaining 98 percent of stations are missing an average of 5 percent of temperature data. To generate uninterrupted temperature records, NCDC uses estimates for the missing data based on records from nearby stations in the larger set of Cooperative Observer Program weather-monitoring stations. For example, according to agency officials, NCDC used this process to fill in missing data for the Little Falls Mill Street station. According to NCDC officials, filling in missing data ensures that temperature records from all areas of the contiguous 48 states are represented when the agency uses the USHCN to identify national temperature trends.  Stability of measurement conditions. A final consideration in selecting USHCN stations was NCDC’s desire to maximize the stability of measurement conditions—such as station location, type of temperature-measuring instrument, and time of day when observations were recorded—because such stability makes it easier to discern actual temperature trends at a station. NCDC officials told us that, like stability in other measurement conditions at USHCN stations, stability in siting conditions facilitates officials’ ability to use temperature data to accurately identify long-term warming and cooling trends, even if those conditions do not meet NWS siting standards. Most stations with long temperature records, however, are likely to have undergone multiple changes in measurement conditions. For example, according to NCDC’s records, the Reno, Nevada, USHCN station was originally located at an NWS weather forecast office before being moved in the mid-1930s to an airport and then again in the 1990s to another location at the same airport. According to NCDC, such changes in measurement conditions may cause a rise or drop in the temperatures recorded at stations, which could affect the temperature trends identified using the USHCN. For example, NCDC has studied the impact of a gradual change in the time that observers record temperature measurements from afternoon to morning observation times and concluded that the change has obscured the warming trend across the contiguous 48 states, which would otherwise have appeared more pronounced. NCDC officials told us that they use statistical methods to identify significant shifts in temperature data unrelated to actual trends in temperature and to adjust the data to remove such shifts. According to NCDC officials, all 1,218 USHCN stations have undergone at least one change in measurement conditions requiring such an adjustment, with an average per station of four to five changes.

NCDC officials acknowledged that a greater degree of stability of measurement conditions than typically found at USHCN stations would be preferable. As a result, NCDC has established a new network of surface weather-monitoring stations specifically to monitor the nation’s climate— the U.S Climate Reference Network—and is establishing a second one— the U.S. Regional Climate Reference Network (see app. II). According to agency officials, they have developed criteria for selecting locations for stations in the new networks to help ensure a greater degree of station stability in comparison with the USHCN and reduce the need to identify and remove shifts in temperature records that are unrelated to actual warming or cooling. NCDC officials told us these new networks can be used to construct a continuous temperature record with the USHCN once the new networks have a sufficient period of overlap with the USHCN to allow for a comparison of temperature trends.

Siting Conditions at Weather-Monitoring Stations Played a Limited Role in Selections for the USHCN

The extent to which stations met specific NWS siting standards played only a limited role in the initial selection of stations for the USHCN in 1987 and when the makeup of the USHCN was revised in 1996 and 2009, according to NCDC officials. NCDC officials told us they considered other factors, such as geographic distribution and a long history of temperature records, to be more important to their ability to analyze long-term temperature trends than strict adherence to NWS’s siting standards. For example, NCDC has included in the USHCN the Central Park station in New York City, which has a temperature record dating to 1876 and has had limited moves, even though current information on the station shows it is encircled by trees. NCDC officials said that, in an effort to consider some information on siting as part of the process of selecting stations, they obtained recommendations from state climatologists and others with detailed knowledge of the siting conditions at stations in their states.

NCDC officials told us that another reason siting conditions played a limited role in their initial selection of USHCN stations in 1987 was that NCDC had limited information about siting conditions at the time. NCDC officials said they generally did not visit stations to examine siting conditions, except for a few stations near their headquarters in Asheville, North Carolina, because it was not feasible to do so with so many stations distributed nationwide. In addition, according to NCDC officials, when they first considered stations for inclusion in the USHCN, they had more limited electronic access to station histories and information about siting conditions than they do today. NCDC officials also said that the station histories they did have may not have included all relevant siting information, such as proximity to obstructions.

According to NCDC officials, they may have kept some weather-monitoring stations that do not meet specific NWS siting standards out of the USHCN by generally excluding many sites in large urban areas. For example, weather-monitoring stations located in large urban areas may be too close to extensive paved surfaces or obstructions to meet specific NWS siting standards. Nevertheless, individual stations were excluded because they were located in a large urban area, not because they did or did not meet a specific NWS siting standard. Similarly, NCDC officials told us that many stations with the longest records were not selected because NCDC considered the temperature records for these stations to have been affected by the stations’ location in or adjacent to large urban areas. Nevertheless, the officials told us, NCDC made exceptions and selected some stations in or near large urban areas. According to NCDC’s 1987 report on its initial designation of the USHCN, 70 percent of the selected stations were located in areas with populations of less than 10,000 in the 1980 census, and 90 percent were located in areas with populations of less than 50,000.

USHCN Stations Meet Siting Standards and Management Requirements to Varying Degrees

According to our survey of NWS weather forecast offices, close to half of USHCN stations do not adhere to one or more siting standards. Weather forecast offices cited a variety of factors that contributed to stations not adhering to siting standards, such as the use of temperature-measuring equipment that limits NWS’s ability to locate stations so that they adhere to the standards. With regard to management requirements for USHCN stations, we found that the weather forecast offices generally but not always met requirements to conduct annual inspections and update station records.

Close to Half of USHCN Stations Do Not Meet NWS Siting Standards

The survey responses we received from weather forecast offices that manage stations included in the USHCN indicate that about 42 percent of the active stations in 2010 did not adhere to one or more of the NWS siting standards for air temperature measurement. This percentage is slightly higher than the percentage not meeting the standards in the larger set of Cooperative Observer Program stations in the contiguous 48 states, of which the USHCN is a part. Specifically, according to our survey responses, about 37 percent of the active Cooperative Observer Program stations in 2010 did not adhere to one or more of the standards.

The two standards most commonly cited by weather forecast offices as unmet by USHCN stations were distance to obstructions, such as buildings and trees, and distance to extensive concrete or paved surfaces (see fig. 3). According to weather forecast offices’ survey responses, only a small fraction of the stations did not adhere to the other siting standards, including that temperature-measuring instruments be mounted 4 to 6 feet off the ground. In particular, according to our survey responses, only five active USHCN stations (less than 1 percent) were located on a rooftop.

We also visited a nonprobability sample of 8 weather forecast offices and 19 stations in the USHCN that are managed by these offices. During these visits, we observed stations that were located closer to obstructions or to extensive concrete or paved surfaces than specified in the siting standards, although the degree to which the stations did not adhere to the standards varied. For example, figure 4 shows 2 stations that did not meet the siting standards. One station was located too close to a building and trees at a wildlife preserve in an otherwise relatively undeveloped area, but the other station was located in a relatively urban area and surrounded by a parking lot, building, and street.

The two factors most commonly cited by NWS weather forecast offices responding to our survey as contributing to USHCN stations not adhering to one or more of the siting standards were (1) NWS’s preference for locating stations at sites that provide a high degree of station stability and data continuity, even if these sites do not adhere to standards, and (2) the use of temperature-measuring equipment that limits NWS’s ability to locate stations so that they adhere to the standards (see fig. 5):  Preference for station stability and data continuity. In our survey of weather forecast offices, the most commonly cited factor contributing to USHCN stations not meeting the siting standards was a preference for locating stations at sites that provide stability and continuity of data. For example, officials in the Tampa weather forecast office told us that one USHCN station that was located in a downtown area and did not meet siting standards has a temperature record that begins before 1895, the first year of data used in the USHCN. They said they could either keep this station open or close it, since there were no other options that met the standards either on the current observer’s property or in the surrounding area. They chose to keep the station open because of its long temperature record.  Limitations due to temperature-measuring equipment. The use of temperature-measuring equipment that is connected by a cable to an indoor readout device can require installing equipment closer to buildings than specified in the standards, according to our survey. Weather forecast office staff must dig trenches for the cables, and paved surfaces such as sidewalks and driveways, as well as the cost of cable for trenching, can limit the length of trenches and consequently the ability to locate stations so that they adhere to the siting standards. According to data from NCDC, about three-quarters of stations in the USHCN use such equipment.

NWS headquarters officials told us they hope to replace cabled temperature-measuring equipment with new wireless equipment that can more easily be located in accordance with siting standards. Specifically, the NWS headquarters office with overall responsibility for the Cooperative Observer Program has developed a draft plan for the program that envisions replacing current equipment with wireless equipment. The draft plan does not specify the number of stations where equipment will be replaced but rather calls for evaluating weather- monitoring stations to determine if they meet the siting standards, identifying candidate stations for installing wireless equipment or relocating them to meet siting standards, and identifying stations that are candidates for being closed. NWS officials said, however, that the agency has not yet approved the plan for implementation. We did not specifically ask about wireless equipment in our survey, but 35 weather forecast offices entered comments expressing support for replacing the temperature-measuring equipment currently used at weather-monitoring stations with wireless equipment. Twenty of the offices specifically cited the ability to improve station siting as the reason for making this change. Comments entered by weather forecast offices on our survey, as well as the draft plan, also cited greater ease of installation and maintenance as additional benefits of wireless equipment. For example, installing wireless equipment would not require digging a trench for a cable.

Even if NWS approves the draft plan, the use of wireless equipment may not address all siting issues. First, according to NWS officials, commercially available wireless equipment has not yet been developed that meets NWS standards for temperature observations at a cost that is feasible for use at weather-monitoring stations nationwide. The NWS official in charge of monitoring the development of wireless equipment said, however, that such equipment would most likely be available within 5 years. Second, the use of wireless equipment, when available, would not allow NWS to improve siting at all stations that do not currently meet the standards. For example, the observers’ properties at some of the stations we visited were too small to allow any temperature-measuring equipment to be placed far enough from buildings or other obstructions to meet the siting standards, regardless of whether the equipment was wireless or cabled. NWS officials acknowledged that the use of wireless equipment would improve station siting but not eliminate all stations that currently do not meeting siting standards.

NWS officials also described a wide range of other factors contributing to stations not adhering to siting standards. These include the difficulty of recruiting new volunteer observers at sites that meet standards, particularly as the nation’s population has become more mobile and thus less apt to serve as long-term observers;  properties that may be too small or have trees or other features that make it difficult to locate instruments as far from obstructions as the standards specify; the reluctance of observers to allow equipment sited in a location on their property that would meet the standards; changes to the observer’s property (e.g., growth of trees) or urbanization of the surrounding area that can cause the stations to not meet standards; and  natural geographic features in certain areas, such as heavily forested or mountainous terrain, that can hamper the ability to meet the standards.

NWS Weather Forecast Offices Have Generally but Not Always Met the Requirement to Annually Inspect Stations and Periodically Update USHCN Station Records

Our review of files for USHCN stations at 8 weather forecast offices, as well as our survey results, show that the offices have generally but not always met the requirement to annually inspect stations to maintain temperature-measuring equipment and determine if changes have occurred requiring station records to be updated, such as changes to siting conditions. Our file reviews also show that the offices generally but not always met the requirement to periodically update station records, even if no changes had taken place at a station. According to NCDC and NWS officials, it is important to annually visit stations and keep station records up to date so that users of the stations’ temperature records, such as NCDC, know the conditions under which the observations were recorded. Any information NCDC has about these conditions, according to agency officials, can be used in conjunction with its statistical methods to identify significant shifts in a station’s temperature data that are unrelated to actual warming or cooling trends and to adjust the data to remove such shifts.  Annual inspections. The results of our survey indicate that in 2010, 102 of 114 weather forecast offices met the annual inspection requirement for stations in the USHCN. According to our survey results, 12 offices did not meet the requirement at a total of 35 stations. In reviewing files at the 8 weather forecast offices we visited, we also found instances where the annual inspection requirement was not met in 2008 and 2009. Specifically, the results of our file reviews show that 3 of the 8 offices did not meet the annual inspection requirement for five stations in 2008, and 1 office did not meet the requirement for one station in 2009. In contrast, for the stations where the requirement had been met, the weather forecast offices had frequently conducted multiple inspections during a year. For example, office staff may have visited a station multiple times to repair equipment, to temporarily relocate temperature-measuring instruments to allow for construction at the observer’s property, or to meet the requirement for semiannual inspections of stations that also record precipitation.  Station record updates. Until 2005, NWS required that station records be updated at least once every 10 years. At that time, NWS changed the requirement to once every 5 years. In reviewing files at the 8 weather forecast offices we visited, we found that two of the 8 offices had consistently met the requirement to update station records within 5 years. In contrast, at two of the other offices, the time between updates for four stations was over 10 years. At the remaining four offices, the time between updates for one or more stations was over 5 years but less than 10 years. For example, one office did not update a record dated February 2002 until January 2011—almost 9 years after the previous update. When the weather forecast offices updated records, the types of changes they documented included those that can cause shifts in temperature data unrelated to any actual temperature change, including replacement or relocation of temperature-measuring equipment, changes in time of observation, and descriptions of obstructions.

Through our survey and visits to 8 weather forecast offices, weather forecast office officials identified a number of challenges to their ability to ensure that station records are updated and to carrying out other responsibilities for managing stations in the Cooperative Observer Program, including those in the USHCN. In our survey, the most frequently cited challenge was that weather forecast offices rely on staff assigned to manage the stations to also assist with other office responsibilities. Competing mission requirements at the offices was a closely related and often-cited challenge. For example, weather forecast offices operate 24 hours a day, and office officials explained that staff assigned to manage the stations may also be expected to work shifts, which limits the time they can visit the stations. Some weather forecast offices we visited told us that turnover and reductions in the number of staff assigned to the Cooperative Observer Program results in the loss of institutional knowledge needed to manage weather-monitoring stations. Weather forecast offices, particularly those with large areas to cover, also identified long driving distances to stations as a challenge. For example, officials at one office we visited told us that completing the required annual station visits requires driving 17,000 miles per year; that the round-trip drive to some stations takes longer than 10 hours, leaving limited time to maintain equipment or install equipment at new stations; and that during the winter, some stations are inaccessible.

NWS Does Not Centrally Track Whether USHCN Stations Meet Siting Standards or Have a Policy for Addressing Stations That Do Not Meet the Standards

NWS does not use its information systems to centrally track whether USHCN stations adhere to siting standards or if weather forecast offices are meeting the requirement to update station records at least once every 5 years. NWS also does not have an agencywide policy on what actions to take at stations that do not adhere to siting standards, which creates the potential for inconsistency in how weather forecast offices address such stations. The lack of centralized electronic tracking of performance information for the USHCN and the lack of an agencywide policy on the actions to take at stations that do not meet siting standards limit NWS’s ability to manage the USHCN in accordance with performance management guidelines and federal internal control standards.

NWS Does Not Have Centrally Available Information to Assess USHCN Performance

NWS’s siting standards for weather-monitoring stations and the requirement that station records be updated at least once every 5 years in effect establish goals for each weather forecast office to meet. NWS does not, however, centrally capture data on the extent to which stations in the Cooperative Observer Program and the USHCN meet its siting standards and station record update requirement. NWS has an information system it uses to help manage weather-monitoring stations, but the system has several limitations. The information system allows the agency to record basic identifying information about the stations and the conditions under which volunteer observers record weather observations, including some information about siting conditions and the dates of station record updates. According to NWS officials, however, the agency did not design the system to centrally track adherence to siting standards or the requirement to update station records at least once every 5 years. As a result, NWS is limited in its ability to use its information system to track USHCN performance information.

The information system NWS uses to track information related to Cooperative Observer Program stations, including those selected for the USHCN, has the following specific limitations:  Limited information on adherence to station siting standards.

According to NWS headquarters officials, they cannot query their information system to identify the specific weather-monitoring stations that do not meet siting standards or the total number of stations that do not meet standards. For example, in 2009, NCDC requested that NWS verify siting conditions at USHCN stations for a study evaluating the effects of siting conditions on temperature trends. NWS headquarters did not have a way to easily gather this information and instead had to direct its regional offices to have weather forecast office staff verify siting conditions at the stations. According to NWS officials, weather forecast office staff did not make special visits to the stations to gather the information requested by NCDC, as these stations might not have been on their immediate visit schedule, but reviewed files instead.

Incomplete information on siting conditions. NWS guidance for using the information system directs weather forecast offices to enter descriptions of obstructions at weather-monitoring stations. We found that in some cases, however, such descriptions may not accurately reflect whether temperature-measuring equipment meets the siting standard related to obstructions. This discrepancy can arise because NWS guidance directs weather forecast offices to describe obstructions in relation to a station’s rain gauge. We found that obstructions to temperature measuring-equipment can differ from those to the rain gauge, depending where the two instruments are located. For example, the record for one USHCN station we visited did not include any obstructions because the rain gauge was on a rooftop where there were no obstructions. The station’s temperature- measuring equipment, however, was at ground level and surrounded by buildings on three sides and, as a result, did not meet NWS’s standards for siting temperature sensors. (The station’s temperature- measuring equipment can be seen in fig. 6.) In addition, NWS guidance for using the information system does not direct weather forecast offices to enter descriptions of other conditions that might indicate whether specific siting standards are being met, such as proximity to extensive concrete or paved surfaces.  Limited ability to track whether the requirement to update station records is being met. The NWS information system is designed to allow weather forecast office staff to enter the date each station record is updated and to store previous versions of station records. NWS officials stated, however, that the information system is not set up to centrally track the performance of weather forecast offices in updating station records within the required 5-year time frame. In addition, the system is not set up to allow NWS to notify weather forecast offices when station records are nearing the 5-year mark and need to be updated. NWS regional offices and weather forecast offices are instead responsible for tracking the status of updates. For example, an official at one office we visited told us that he sometimes forgets to update station records within the 5-year time frame but that the NWS regional headquarters keeps track of the requirement.

Inconsistent identification of stations included in the USHCN. The NWS information system includes information about all stations in the Cooperative Observer Program, including those designated by NCDC as part of USHCN, but NWS does not consistently use the system to identify USHCN stations. Specifically, station records allow weather forecast offices to indicate whether stations are part of the USHCN, but offices had done so in only some of the station records we reviewed. As a result, NWS headquarters officials cannot use their information system to determine which stations NCDC has designated as part of the USHCN. Officials at some of the weather forecast offices we visited were also unsure which of the stations they manage had been designated as part of the USHCN. According to NCDC officials, it is important that weather forecast offices have the ability to determine which stations belong to the USHCN so that they can set appropriate priorities for the maintenance, repair, and replacement of temperature-measuring equipment at these stations.

Our work related to the Government Performance and Results Act of 1993 and the experience of leading organizations have shown the importance of developing program performance goals that identify desired results of program activities and reliable information that can be used to assess results. NWS headquarters officials we spoke with acknowledged the need to centrally track performance information related to the management of Cooperative Observer Program stations, which includes those selected for the USHCN. The officials said they only recently began tracking the requirement that weather forecast offices inspect stations at least once annually. According to these officials, they selected annual station inspections as a performance indicator because the inspection requirement is easy to track using the current information system for managing the stations. NWS provided us with summary data on inspections for all stations in the Cooperative Observer Program, including USHCN stations. The data show that the percentage of stations for which weather forecast offices met the annual inspection requirement increased from 70 percent in 2005 to 80 percent in 2010.

NWS headquarters officials also told us that the agency has begun upgrading its current information system and that they are considering options to notify weather forecast offices when updates of station records are overdue and to better track adherence to siting conditions. The officials said they hope to complete the upgrade by the end of fiscal year 2013, depending on the availability of funding. Options being considered for tracking siting conditions include photographs of stations and the use of a rating scale to summarize the extent to which stations adhere to siting standards, similar to the rating scale created for the newer networks developed specifically for climate monitoring (see app. II for further details). According to our survey results, 63 percent of weather forecast offices believe that the use of photographs in the NWS information system would be either very helpful or extremely helpful in evaluating stations’ adherence to siting standards. In addition, 52 percent of the offices responded that the option to check a box in a station’s electronic record to indicate that it does not adhere to the standards would be either very helpful or extremely helpful. Some offices also suggested other tools they would consider helpful in evaluating siting conditions at stations, such as the use of commercially available satellite imagery and maps.

NWS Does Not Have an Agencywide Policy to Address Stations That Do Not Adhere to Siting Standards

NWS does not have an agencywide policy for stations not adhering to siting standards that clarifies for staff in weather forecast offices whether the stations should be closed, relocated, or maintained in their present condition to preserve the continuity of their temperature records. Standards for internal control in the federal government call for federal agencies to document their policies and procedures to help managers achieve desired results. Without an agencywide policy, weather forecast offices do not have a basis for making consistent decisions about what actions to take at USHCN stations that do not adhere to siting standards.

NWS headquarters officials we spoke with acknowledged that they had not developed an agencywide policy on the actions, if any, that weather forecast offices should take to address stations that do not adhere to siting standards. They said they recognized the need to develop an agencywide policy and that, in the absence of such a policy, decisions on how to address stations that do not meet siting standards are up to individual weather forecast offices. For example, they said weather forecast offices might consult with NCDC or state climatologists when deciding whether to close stations that do not meet siting standards, but that such outreach is not required.

In the absence of an agencywide policy, the NWS western regional office directed weather forecast offices in the region to stop submitting data to NCDC from stations with “egregious” siting conditions in a format that would allow NCDC to use these data when analyzing long-term temperature trends. The meaning of “egregious” was not defined and was left to each weather forecast office to interpret. According to NWS officials, none of the other three regional offices in the contiguous United States have developed a similar policy. According to the western regional manager responsible for the Cooperative Observer Program, the region’s policy affected about a dozen stations, which had not been designated as part of the USHCN. Through our visits to weather forecast offices, however, we found that, even in the western region, the offices did not consistently implement the region’s policy. In particular, an official at one of the two offices we visited in the region told us he did not follow the policy because doing so would have affected the majority of stations in his state.

NCDC officials told us that, as NWS develops a policy regarding how to address stations in the Cooperative Observer Program (including those designated as part of the USHCN) that do not meet siting standards, it should consider how NCDC uses the temperature data from the stations. Because the data from USHCN stations are used to identify long-term climate trends and the stations were thus selected in part on the basis of the stations’ stability of measurement conditions and continuity of data, NCDC officials said they would caution NWS against relocating or closing stations that do not meet siting standards. NCDC officials said they would consider closing a station only in certain situations, such as an observer not following NWS guidelines when recording weather observations.

Conclusions

Given the importance of data from the USHCN in monitoring and in formulating public policy related to climate change, it is important that the public and policymakers have confidence that the network is being managed effectively. NWS has developed station siting standards and management requirements for the USHCN, and performance management guidelines dictate that NWS should gather data on the extent to which these standards are being met. But NWS’s information system does not centrally capture such information. As a result, the agency cannot easily measure the USHCN’s performance against its siting standards and management requirements. Without more complete data on siting conditions, including when siting conditions change, it is difficult for agency management to assess the extent to which the stations meet its siting standards. Similarly, NWS does not have easily accessible data on when station records were last updated for monitoring whether the records are being updated at least once every 5 years as the agency requires. In addition, although federal internal control standards call for agencies to develop policies to maintain control over program activities, NWS has not established agencywide policy for what to do when, over years and decades, stations no longer adhere to its siting standards because conditions have changed. In the absence of such policy, it is not clear to weather forecast office officials whether stations that do not adhere to siting standards should remain open because data continuity is important for analyzing long-term climate trends, or whether the stations should be moved or closed. As a result, without a policy with actions for all offices to follow, weather forecast offices may be taking different approaches to address stations that do not meet siting standards.

Recommendations for Executive Action

To improve NWS’s ability to manage the USHCN in accordance with performance management guidelines and federal internal control standards, as well as to strengthen congressional and public confidence in the data the network provides, we recommend that the Acting Secretary of Commerce direct the Administrator of NOAA to take the following two actions:  Enhance NWS’s information system to centrally capture information that would be useful in managing stations in the USHCN, including (1) more complete data on siting conditions (including when siting conditions change), which would allow the agency to assess the extent to which the stations meet its siting standards, and (2) existing data on when station records were last updated to monitor whether the records are being updated at least once every 5 years as NWS requires.  Develop an NWS agencywide policy, in consultation with NCDC, on the actions weather forecast offices should take to address stations that do not meet siting standards.

Agency Comments and Our Evaluation

We provided a copy of our draft report to the Department of Commerce for review and comment. In written comments from the department, NOAA agreed that it can improve its ability to manage the USHCN in accordance with performance management guidelines and federal internal control standards. NOAA also agreed with our two recommendations. Regarding our first recommendation, NOAA stated that NWS has begun the planning process to upgrade the existing information system that captures data for managing Cooperative Observer Program stations, including those that are a part of the USHCN. According to NOAA, the upgrade will include the ability to capture more complete data on siting conditions and to determine if a station’s record has been updated in the last 5 years. Regarding the second recommendation, NOAA said that NWS will work with NCDC to develop a policy to assist weather forecast offices in taking action on stations that do not meet siting standards.

NOAA also stated that it understood that, given the scope of our review, we did not assess the effect of stations not meeting siting standards on the reliability of the agency’s analysis of temperature trends. Nevertheless, NOAA added that it was important for our findings to include a discussion of the published peer-reviewed studies that have explicitly examined the USHCN’s data quality and its effects on the reliability of NOAA’s temperature trend data. We did not include such a discussion in our report because this issue was outside the scope of our work. We did, however, reproduce NOAA’s list of relevant studies on this topic together with its comments. NOAA also provided technical comments, which we incorporated into the report as appropriate. NOAA’s comments are reproduced in appendix III.

As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the appropriate congressional committees, the Acting Secretary of Commerce, and other interested parties. In addition, this report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff members have any questions about this report, please contact me at (202) 512-3841 or mittala@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix IV.

Appendix I: Scope and Methodology

To determine how the National Oceanic and Atmospheric Administration (NOAA) selected stations for the U.S. Historical Climatology Network (USHCN) from the larger set of existing stations in the Cooperative Observer Program network, we reviewed documents from NOAA’s National Climatic Data Center (NCDC) describing the selection process, interviewed NCDC officials, and analyzed NCDC data on stations in the USHCN. Specifically, we reviewed documents written by NCDC officials to identify the factors the agency considered important for monitoring long-term temperature trends, and we interviewed NCDC officials regarding these factors, as well as how they applied them in the selection process. We also obtained data from NCDC on the geographic distribution of USHCN stations across the contiguous 48 states and the length and completeness of the stations’ temperature records. The data we obtained and analyzed came from NCDC’s version 2 of the USHCN. We assessed the reliability of the data by electronically testing them and comparing selected samples to data from other NOAA sources to check for obvious errors in accuracy and completeness. We also reviewed information about the data and the systems used by NCDC to produce the data, interviewed NCDC officials knowledgeable about the data, and worked with officials to clarify inconsistencies before using the data in our analyses. We determined that the data were sufficiently reliable for reporting on the extent to which USHCN stations met the factors NCDC considered important in selecting the stations.

To examine the extent to which USHCN stations meet National Weather Service (NWS) siting standards and management requirements for weather-monitoring stations, we developed and administered a survey of meteorologists-in-charge at the 116 NWS weather forecast offices responsible for managing stations in the network. Our questionnaire included questions about adherence to siting standards, reasons for stations not adhering to the standards, and general management challenges. The survey was Web based and accessible through a secure server. On February 7, 2011, we sent an e-mail notification to the 116 meteorologists-in-charge describing the survey and notifying them that it would be activated on the Internet shortly. On February 9, 2011, we formally activated the survey and sent another e-mail containing a link to the survey along with each respondent’s unique username and password. We sent follow-up e-mail messages on February 16, 2011, and February 24, 2011, to those who had not yet responded. Then, starting on March 11, 2011, we contacted the remaining nonrespondents by telephone or e-mail. The questionnaire was available online until March 23, 2011. By that date, the surveys were completed by all 116 weather forecast offices, for a response rate of 100 percent.

Because our survey covered all weather forecast offices in the contiguous 48 states, not a sample of them, it was not subject to sampling error. Surveys are, however, subject to nonsampling errors. For example, how a particular question is interpreted, sources of information available to respondents, and how the data are entered in a database or are analyzed can introduce unwanted variability into survey results. We took steps in developing the survey, collecting the data, and analyzing them to minimize such nonsampling errors. For example, GAO survey specialists designed the questionnaire in collaboration with GAO staff who had subject-matter expertise. In addition, we conducted four pretests of the draft questionnaire to ensure that the questions were clear and unambiguous, terminology was used correctly, the questionnaire did not place an undue burden on agency officials, the information could feasibly be obtained, and the survey was comprehensive and unbiased. We conducted each of the four pretests over the telephone with one or more NWS officials from each of the four NWS regions in the contiguous United States. On the basis of the feedback we received, we made changes to the content and format of the survey after each of the four pretests. When we analyzed the data, an independent GAO analyst checked all computer programs. Since this was a Web-based survey, respondents entered their answers directly into the electronic survey, eliminating the need to key data into a database, thus minimizing data entry errors.

Our survey results are also subject to errors made by weather forecast office staff regarding the number of stations they reported as not adhering to siting standards. For example, one weather forecast office indicated in its survey response that one USHCN station in its area was on a rooftop, but office staff later told us that the survey response was wrong and that none of its USHCN stations is located on a rooftop. In addition, the number of stations reported by weather forecast offices as not adhering to siting standards is subject to the staff’s interpretation of NWS’s siting standards. For example, it is a matter of interpretation and judgment by NWS staff whether objects surrounding a station, such as trees or structures, are considered to be obstructions and thus whether a station is considered to meet or not meet the siting standards. The response to our survey from one weather forecast office we visited indicated that the temperature-measuring equipment at only one of its USHCN stations was closer to an obstruction than specified in the siting standards. We observed obstructions at all three stations we visited, however, and the records for these stations also listed obstructions.

In calculating the percentages of the total number of active stations that met the specific criteria we asked about in our survey questions, we did not use responses from weather forecast offices that provided incomplete or inconsistent information. Depending on the percentage being calculated, we did not use responses from at most 6 of the 116 offices. For example, we did not use responses from 3 offices when calculating the percentage of active USHCN stations that did not adhere to one or more of the siting standards, and we did not use responses from 5 offices when calculating the percentage of active Cooperative Observer Program stations not adhering to one or more of the standards.

To examine in greater depth the extent to which USHCN stations meet siting standards and management requirements for weather-monitoring stations, we visited a nonprobability sample of 8 NWS weather forecast offices. To ensure geographic distribution in the weather forecast offices we visited, we selected two offices in each of the four NWS regions in the contiguous United States, and we selected the specific offices we visited to ensure a range of sizes in terms of the offices’ forecast areas. We also selected offices with differing structures for supervising staff assigned to manage the Cooperative Observer Program and with a range of other programs the offices are responsible for, such as marine forecasts. To examine adherence to siting standards for weather-monitoring stations, we observed siting conditions at a nonprobability sample of 19 USHCN stations. We selected stations to visit to ensure variety in the type of temperature-measuring equipment used at the stations and other factors that could affect siting conditions. During the station visits, we developed and used a checklist that tracked how well the conditions of the site met what was recorded in the stations’ records. In addition, we reviewed files on USHCN stations at each of the weather forecast offices we visited. We reviewed records and annual inspection reports from a total of 81 USHCN stations. We entered information from the records and inspection reports into a database to capture information on the extent to which weather forecast offices adhered to the requirements to update station records and conduct annual inspections, among other things. Because we used a nonprobability sample to select weather forecast offices and USHCN stations to visit, the information we obtained from these visits cannot be generalized to other weather forecast offices or USHCN stations. The visits instead provided us with information on the perspectives of various participants in the weather forecast offices about managing weather- monitoring stations and examples of station siting conditions.

To gather additional information on the extent to which USHCN stations meet NWS siting standards for weather-monitoring stations, we also reviewed academic literature that addressed issues and concerns related to siting of weather-monitoring stations, including those in the USHCN. We also reviewed NWS’s policy directives related to station siting and interviewed officials in NWS headquarters, regional offices, and weather forecast offices who were responsible for managing the Cooperative Observer Program. In addition, we interviewed the person at the NWS training center responsible for training NWS staff on management and operation of the Cooperative Observer Program and individuals who have raised concerns about the extent to which stations in the USHCN are meeting siting standards.

To evaluate the extent to which NWS tracks USHCN stations’ adherence to siting standards and management requirements and has established a policy for addressing stations that do not adhere to siting standards, we took several actions. In particular, we evaluated the types of data that are captured in NWS’s information system for managing weather-monitoring stations—the Cooperative Station Service Accountability system. We also interviewed NWS officials responsible for managing the USHCN, as well as NCDC officials; reviewed NWS policy directives, briefings, and memorandums related to managing the network; and examined data on the extent to which NWS conducted the required annual inspections of weather-monitoring stations. To determine the extent to which NWS has established a policy to address stations that do not adhere to siting standards, we reviewed NWS documents, such as agency directives, memos, briefings on the future of the Cooperative Observer Program, and an executive summary for a draft strategic plan for the program. In addition, we interviewed NCDC officials and NWS officials from regional, headquarters, and weather forecast offices and from the Cooperative Observer Program training center.

Appendix II: Other National Climate Networks

Since 2001, NOAA has supported establishment of two new networks of climate monitoring stations. The first to be established, the U.S. Climate Reference Network, is intended to detect indications of climate change at a national scale. This network consists of 114 stations in the contiguous 48 states, and NOAA has plans to expand the network to include stations in Alaska. The purpose of the second new network, the U.S. Regional Climate Reference Network, is to detect indications of climate change at a regional rather than national scale. As of July 2011, this network consisted of 63 stations in the southwestern United States, and NOAA hopes to complete the installation of stations for the network across the contiguous 48 states by about 2020, depending on the availability of funding. According to NOAA, once both networks are fully established, about 538 locations in the contiguous United States will have either a U.S. Climate Reference Network station or a U.S. Regional Climate Reference Network station.

The station-siting standards for the two new networks have similarities to the siting standards established for the Cooperative Observer Program and that also apply to the USHCN, such as installation over level terrain and not on rooftops. The new networks also have differences, including instances where the siting standards for the new networks are more stringent. For example, the standards for the U.S. Climate Reference Network call for stations to be located farther from concrete or paved surfaces than specified in the standards for stations in the USHCN. The differences reflect the fact that, whereas NCDC designated stations for the USHCN from an existing network of NWS weather-monitoring stations, NOAA has specifically located and designed stations in the newer networks for monitoring the nation’s climate. For example, the use of automated equipment, as well as solar power at many stations, allows for greater flexibility in locating stations in comparison with NWS weather- monitoring stations that rely on volunteer observers and equipment connected by a cable to an indoor readout device. Similarly, to the extent possible, NOAA has placed a priority on locating stations for the new networks on public lands, such as national and state parks. According to NOAA, in comparison with the properties of volunteer observers, such locations have a higher probability of continuing in their present condition without major changes for long periods of time (50 to 100 years). Figure 7 depicts a station in the U.S. Climate Reference Network.

NOAA has also established a process to evaluate and select potential sites for stations in the new networks. For example, selecting stations for the U.S. Regional Climate Reference Network includes field and desk surveys, which involve collecting information about the site’s condition from photographs and other sources; evaluation of the surveys by a site selection panel; a vote among panel members to decide among candidate locations; and final approval or disapproval of panel recommendations by a site selection lead. See table 1 for differences in the siting standards applied to the USHCN and the new climate networks.

Appendix III: Comments from the Department of Commerce

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Stephen D. Secrist (Assistant Director), Richard Bulman, William Carrigg, Joanna Chan, Ellen W. Chu, Joseph Cook, Alysia Davis, N’Kenge Gibson, Stuart Kaufman, Cheryl Peterson, Anne Rhodes-Kline, and Jerome Sandau made key contributions to this report.