Drinking Water: Additional Data and Statistical Analysis May Enhance EPA's Oversight of the Lead and Copper Rule
Released: 2017-10-02T00:00:00

Letter

Background

This section provides information on (1) the known health effects of lead in drinking water; (2) how water systems deliver drinking water to the public and where lead may be present; (3) the requirements of the LCR; (4) LCR data that states report to EPA; and (5) the roles of federal, state, and local entities in implementing the LCR.

Health Effects of Lead in Drinking Water

EPA, the Centers for Disease Control and Prevention (CDC), and others have indicated that the rates of lead contamination in the U.S. population have decreased over the years. However, lead remains a significant concern to public health because lead is persistent and can accumulate in the body over time with long-lasting effects, particularly for children and pregnant women. According to EPA documents, low levels of lead exposure in children are linked to hyperactivity, anemia, lower intelligence quotient (IQ), physical and learning disabilities, and slowed growth. In pregnant women, lead can store in bones and be released as maternal calcium used to form the bones of the fetus, reduce fetal growth, and increase risk of miscarriage and stillbirth. For adults, lead can have detrimental effects on cardiovascular, renal, and reproductive systems; and, it can prompt memory loss. The presence of lead in the bloodstream can disappear relatively quickly, but bones can retain the toxin for decades.

According to the National Institutes of Health and CDC documents, medications can remove some lead from the body but cannot undo the damage lead causes, although additional services may mitigate some of the damage. Recognizing that vigilance and collaboration are necessary to ensure that children negatively affected by lead exposure receive services designed to compensate for lead’s effect on the brain, and behavior of children, some medical experts promote early-childhood intervention, education, and other programs. According to CDC documents, early intervention for children can help improve IQ scores, academic readiness, and language development as well as decrease placement in special education classes. For these reasons, EPA and others recommend the prevention of lead exposure before it occurs.

How Water Systems Deliver Drinking Water

Water systems depend on distribution systems, both simple and complex, composed of interconnected components to deliver drinking water from a source to their customers. Source water can be either surface (streams, rivers, and lakes) or ground (aquifers). As figure 1 illustrates, the distribution system used to deliver water from the source can include a network of pipes and other components. A distribution system comprises water towers, pipes, pumps, and other components to deliver treated water from treatment systems to consumers. Particularly among larger water systems, distribution systems may contain thousands of miles of pipes, including water mains.

There are 1 million miles of drinking water mains in the country, according to a 2017 American Society of Civil Engineers study. Service lines are the smaller pipes that connect the water mains to homes and buildings and can also include smaller pipes used for connecting a service line to the water mains (e.g., called pigtail and gooseneck pipes). In contrast to most other drinking water contaminants, lead is rarely found in the source water. More commonly, lead enters drinking water after the water comes into contact with water mains; service lines; smaller pipes that connect the two; and other plumbing materials that contain lead, such as faucets and water coolers. Schools and day care centers with their own water supplies generally rely on well-water systems using groundwater to deliver drinking water.

According to the 2017 American Society of Civil Engineers study and EPA documents, communities, both urban and rural, have aging and deteriorating drinking water infrastructure, which, according to EPA documents, can contribute to lead hazards in drinking water. Since the early 1970s, when several medical studies confirmed that lead exposure negatively impacts health, measures have been taken to reduce the public’s exposure to lead in drinking water, including the enactment of amendments to the SDWA in 1986 and 1996, the enactment of the Lead Contamination Control Act in 1988, the issuance of the LCR in 1991, and amendments to state building codes prohibiting the use of lead pipes.

The Lead and Copper Rule

The LCR generally requires water systems to minimize lead in drinking water by controlling the corrosion of metals in the infrastructure they use to deliver water and in household plumbing. EPA has stated that the LCR is one of the most complicated drinking water regulations for states to implement because of the need to control the corrosion of pipes and plumbing fixtures as water is delivered to consumers. The corrosion of pipes results from a chemical interaction between water and pipes that wears the metal away and allows particles of metal to flake away over time. All large water systems (serving populations larger than 50,000) are generally required to install corrosion control treatment. While the majority of the U.S. population receives its drinking water from medium and large water systems, most water systems are small.

Characteristics of water can affect the occurrence and rate of corrosion. For example, corrosion occurs more frequently in soft water—water with low concentrations of calcium and magnesium—and also in acidic water, or water with low pH. Water systems control corrosion by adjusting the pH and alkalinity of water or by adding corrosion inhibitors. The LCR establishes corrosion control as the required treatment technique for large water systems and, for medium and small systems, the required treatment technique when the federal lead action level is exceeded (also known as an action level exceedance). Lead concentrations exceeding an action level of 15 parts per billion, or 0.015 milligrams per liter (mg/L), in over 10 percent of tap water samples (i.e., the 90th percentile level) are an indicator that corrosion control is needed or is not working correctly. A water system’s 90th percentile sample result does not exceed the lead action level if it is equal to or less than 15 parts per billion. As figure 2 illustrates, the LCR also requires water systems to identify locations where lead may be present and periodically obtain tap water samples from those locations (of which single-family homes are the highest priority).

Under the LCR, an action level exceedance requires the water system and state to take a number of additional steps. Those additional steps require that small and medium water systems install or modify corrosion control treatment, and water systems of all sizes provide information (known as public education) about the harmful effects of lead to consumers and vulnerable populations (e.g., schools, if the water system serves a school, and public health departments). Water systems are also required to test and, if necessary, treat the source water. If, after installing corrosion control and treating source water, a system continues to have 90th percentile sample results that exceed the lead action level, the LCR requires the water system to begin replacing lead service lines, if they exist. In most communities, lead service lines are partially owned by the water system and partially owned by the homeowner. The LCR allows for a partial replacement when an owner of a home or building is unable or unwilling to pay for replacement of the portion of the service line not owned by the water system.

In an October 2016 study, EPA noted that sample requirements under the LCR are complex for many reasons, one reason being that it is the only drinking water regulation in which homeowners or consumers collect the drinking water samples. Water systems are in compliance with the LCR when they follow the various federal requirements for collecting samples, reporting, installing treatments, providing public education, and replacing lead service lines; as well as when they follow any state requirements that are more stringent than the federal requirements. States and EPA can take several different types of enforcement actions when water systems fail to complete requirements in these areas. Sample results that exceed the lead action level do not by themselves constitute violations of the LCR.

The SDWA, as amended in 1996, requires EPA to review and revise, as appropriate, each national primary drinking water regulation, including the LCR, at least once every 6 years. The 1991 LCR was revised in 2000 and 2007. EPA initiated an extensive review of the LCR in 2004 after widespread increases in lead levels were detected in the District of Columbia’s water following a water treatment change. EPA promulgated short-term revisions and clarifications in 2007 and has continued working on comprehensive revisions. In 2016, the agency announced that it would revise the LCR and issue proposed revisions in 2017 and a final revised rule in 2019. EPA also released a Lead and Copper Rule Revisions White Paper in 2016 that outlined potential elements of the rule under consideration for revision such as use of corrosion control practices, requirements for collecting samples, and lead service line replacement.

LCR Data That States Report to EPA

The LCR generally requires that water systems submit data to states to demonstrate their compliance with the treatment technique required by the rule. The LCR also requires states to submit some of these data to EPA’s SDWIS/Fed database on a quarterly basis. Specifically, states are required to submit the following data to EPA: for large and medium water systems, all 90th percentile sample results (i.e., sample results that meet, fall below, and exceed the lead action level); for small water systems, 90th percentile sample results that exceed the lead action level; on water systems that have been designated as having achieved corrosion control because the state has determined that the source water is minimally corrosive; on water systems that were required to install corrosion control treatment, source water treatment, and lead service line replacement and have completed the applicable requirements as a result of having sample results exceed the lead action level; on water systems that have begun the process of replacing lead on water systems that have new violations of the LCR; and on enforcement actions taken in response to violations of the LCR.

For corrosion control, the LCR requires the states to report what EPA refers to as “milestone” data to the SDWIS/Fed database: data on the status of required actions, such as installing corrosion control treatment, as required, after reporting sample results that exceed the lead action level; and data on those water systems deemed to have corrosion already under control, such as when the water is minimally corrosive.

The states collect and manage relevant data (including violations and enforcement information) in either a database provided by EPA—known as the Safe Drinking Water Information System/State—or in a data system of their own design. States must then transfer the data from one of those databases into SDWIS/Fed. In 2010, EPA announced that it would redesign SDWIS/Fed. We reported in June 2011 that EPA officials expected this redesign of SDWIS/Fed to expand the amount of data that EPA receives electronically from states. The name of the redesigned database is SDWIS Prime, which according to EPA officials, is expected to be complete by 2018.

Roles of Federal, State, and Local Entities in Implementing the LCR

Generally, the responsibility for reducing lead in drinking water and ensuring safe drinking water overall, is shared by EPA, states, and, local water systems. As shown in figure 3, EPA is responsible for national implementation of the LCR and setting standards; overseeing states’ implementation of the LCR; providing infrastructure funding, training, and technical assistance to states and water systems; and conducting some enforcement activities. However, the primary responsibility for ensuring that drinking water is free of lead resides with states and local water systems.

Generally, states with primary enforcement responsibility initiate enforcement actions against water systems that do not comply with the LCR and other drinking water regulations. However, EPA can also issue orders necessary to protect human health where a contaminant in a public water system presents an imminent and substantial endangerment. According to a 2013 EPA drinking water compliance report, states generally implement and enforce the LCR, and other drinking water regulations, in the following ways: provide technical assistance through such actions as offering training, holding public information meetings, and lending monitoring equipment; take informal actions such as field visits, reminder letters, telephone calls, and notices of violation; and take formal actions such as issuing citations, administrative orders with or without penalties, civil and criminal cases, and emergency orders.

Since 2009, according to an EPA document, the agency’s enforcement strategy, in collaboration with states, has focused on identifying water systems with a history of violations across multiple drinking water rules for enforcement actions in states, territories, and tribal regions. To facilitate this strategy, EPA’s headquarters staff are to review data on violations in EPA’s SDWIS/Fed using an Enforcement Targeting Tool to identify systems that merit action by states based on the seriousness of their violations. EPA staff also are to use these data to determine whether water systems are achieving the agency’s national targets for compliance. According to the EPA FY 2014-2018 Strategic Plan, the agency’s goal is for 92 percent of water systems that provide drinking water year-round to meet all applicable health-based drinking water standards by 2018.

Available EPA Data Show Sample Results, Use of Corrosion Control, Violations, and Enforcement Actions Taken, but Data Are Not Complete

The available EPA data show sample results, use of corrosion control, violations, and enforcement actions taken for the 68,000 water systems from July 2011 to December 2016, but data are not complete. The available data reported by states in EPA’s SDWIS/Fed database show at least 2 percent of drinking water systems with sample results exceeding the lead action level (from 2014 to 2016), and at least 10 percent of water systems being out of compliance with the LCR (i.e., having at least one reported violation) as of December 31, 2016. In addition, the state- reported data in SDWIS/Fed show 99 percent of enforcement actions were taken by states, as expected because states generally have primary responsibility for monitoring and enforcement of the SDWA requirements, including the LCR. According to recent EPA assessments, the EPA OIG report, and our January 2006 and June 2011 reports, some of the data in the SDWIS/Fed database are not complete. Specifically, the data are underreported, and therefore, data available in SDWIS/Fed likely understate the number of sample results, violations, and enforcement actions that actually occurred. In addition, the available EPA data on water systems’ use of corrosion control are not complete. We also found that because the LCR does not require states to submit certain data to EPA, EPA’s SDWIS/Fed database does not contain data on key parts of the rule, such as the presence or location of lead pipes—information that water systems use to identify the locations from which they will draw tap samples—or complete sample results for small water systems.

Available EPA Data Show Sample Results, Violations, and Enforcement Actions Taken, but Data on Water Systems’ Use of Corrosion Control Are Not Complete

EPA’s SDWIS/Fed database contains descriptive data on, for example, drinking water sample results, corrosion control, violations, and enforcement actions, as required by the LCR.

Sample Results

The available state-reported data in EPA’s SDWIS/Fed database show that of the approximately 68,000 drinking water systems subject to the LCR, at least 1,430 water systems (2 percent) had 90th percentile sample results that exceeded the lead action level of 15 parts per billion from 2014 to 2016. EPA officials told us that they analyze these sample data over a 3-year period rather than yearly to ensure that the majority of water systems will have submitted sample results. These 1,430 systems serve a population of approximately 3 million people. Of the 1,430 systems with sample results exceeding the lead action level, 258 (18 percent) were schools and day care centers with their own water supplies. As we reported in January 2006, the LCR sample data in SDWIS/Fed were underreported; recent EPA file reviews in selected states found that sample data were not always reported to SDWIS/Fed; and a 2017 EPA Office of Inspector General report indicated that sample data, specifically, are potentially underreported. Appendix II provides additional information about the available EPA data on sample results, reported violations, and enforcement. In addition, some state regulators with whom we interviewed in 2016 told us that homeowners and water systems may take LCR samples improperly as we discuss later in this report. See appendix III for these state regulators’ views on challenges associated with waters systems’ implementation of the sample requirements under the LCR.

Use of Corrosion Control

The 2015 Report of the Lead and Copper Working Group to the National Drinking Water Advisory Council noted the importance of corrosion control because it is intended to achieve a water quality that minimizes lead in water. Our analysis of the available state-reported data in EPA’s SDWIS/Fed database on corrosion control from July 2011 to December 2016, shows that the database contained milestone data for 904 water systems on the status of required actions about corrosion control treatment after a sample result exceeded the lead action level, and 1,479 water systems were deemed to have corrosion already under control. In addition, 34 water systems had milestone data in SDWIS/Fed for lead service line replacement. For the approximately 68,000 water systems subject to the LCR, 1,665 systems, had milestone data in SDWIS/Fed, or about 2 percent of all water systems from July 2011 to December 2016. According to EPA officials, when including milestone data available prior to July 1, 2011, almost half of these systems have submitted the required information regarding corrosion control milestones. Each water system can have up to three types of milestone data (i.e., status of required actions about corrosion control, systems deemed to have corrosion already under control, and lead service line replacement) in SDWIS/Fed. In June 2017, EPA officials said that all water systems subject to the LCR are expected to have data on corrosion control in SDWIS/Fed. However, these officials also said that states may not report the data to SDWIS/Fed because of technical limitations with some state databases and confusion among some state officials about how to report the data to SDWIS/Fed.

Of the 983 large systems in the SDWIS/Fed database, milestone data were available for 13 from July 2011 to December 2016, and 5 of those systems had sample results exceeding the lead action level at some point over that time period. Of the small and medium water systems for this period that installed corrosion control treatment because their sample results exceeded the lead action level, milestone data were available for 884 water systems.

We reported in January 2006 that EPA did not have complete milestone data, including data on corrosion control. Specifically, we reported that EPA had, at that time, collected milestone data for about 28 percent of water systems. At the time of our 2006 report, EPA officials told us that in most instances water systems should have data on corrosion control treatment and that it was more likely the case that states were not reporting the data rather than a case of noncompliance by water systems. We recommended that EPA ensure that data on water systems’ test results, corrective action milestones, and violations were current, accurate, and complete. EPA generally agreed with our recommendation, but has not fully implemented it. In 2016, EPA highlighted its response to our January 2006 recommendation through such efforts as having staff review SDWIS/Fed data for accuracy and timeliness and promoting electronic reporting of the drinking water data states submit to SDWIS/Fed.

In addition, EPA headquarters officials said in June 2017 that the agency also worked with the states on reporting corrosion control data by conducting webinars and in-person training that included information about reporting data to SDWIS/Fed. For example, EPA conducted a three-part series of LCR 101 webinars. EPA officials said that the webinars in this series reached over 1,600 attendees with individual webinars ranging from 227 to 551 viewers. EPA’s efforts regarding training sound promising, but it may be too early to see the impact of these efforts to work with states on reporting milestone data on corrosion control to SDWIS/Fed. We continue to believe that EPA should take steps to ensure that data, including those on milestones, are current, accurate, and complete.

Violations

The available data reported by states in EPA’s SDWIS/Fed database show that of the approximately 68,000 drinking water systems subject to the LCR, states reported that at least 6,567 water systems (about 10 percent) had at least one reported open violation of the LCR as of December 2016. In total, these 6,567 water systems had a total of at least 12,884 open violations as of December 2016. As we reported in January 2006 and June 2011, the violations data in SDWIS/Fed were underreported. Recent EPA file reviews in selected states found that some violations data were not reported to SDWIS/Fed.

LCR violations fall into two categories: (1) monitoring and reporting and (2) treatment technique. Monitoring and reporting violations generally refer to a water system failing to collect samples of drinking water from the tap, within the distribution system, and from source water and failing to report sample results to the states.

Treatment technique violations, which EPA considers to be health-based violations, generally refer to a water system failing to take actions as required after water samples exceed the federal lead action level.

The two most frequent violations were for not following requirements for (1) monitoring and reporting routine follow-up and (2) initial tap sampling. Taking samples from homes is the only way that water systems, states, and ultimately EPA can obtain the indicators needed to determine whether corrosion control treatment is needed or if corrosion control treatments already installed are working, in addition to other treatment technique requirements. The third most frequent violation was lead consumer notification, which states or water systems are to do in writing, about the results of the samples taken from homes or buildings they occupy regardless of the presence of lead in the samples taken, known as lead consumer notice violations. These notifications are to provide consumers with information about their drinking water sample results so that they can determine what actions to take to reduce their exposure to lead if lead is present.

Of the approximately 68,000 water systems subject to the LCR, approximately 7,000 schools and daycare centers make up about 10 percent. As their missions would indicate, these schools and daycare centers provide drinking water to children, one of the populations most at- risk for adverse health effects from even small amounts of lead. Most of the schools and daycare centers in the EPA data we analyzed were classified as small water systems. EPA data show that schools and daycare centers comprise about 10 percent (664 water systems) of the 6,567 water systems with at least one open violation of the LCR as of December 31, 2016. Much like the overall group of water systems, schools and daycare centers were most frequently violating the LCR requirements for not (1) monitoring and reporting routine follow-up (2) initial tap sampling, and (3) lead consumer notification.

Enforcement

The available data reported by states in the SDWIS/Fed database show reported information on the enforcement actions taken by states and EPA against water systems that have violated requirements of the LCR. States reported taking 98 percent of the enforcement actions from July 1, 2011, to December 31, 2016, as would be expected given that states generally have primary responsibility for enforcement of the LCR. In our January 2006 report, we found that because sample results, milestones, and violations data for the LCR in SDWIS/Fed were underreported, it was difficult to assess the adequacy of enforcement. We then found in June 2011 that the enforcement data, generally, in SDWIS/Fed were incomplete. States and EPA can take a range of enforcement actions both formal and informal. Formal enforcement actions include issuing state administrative orders with or without penalties, filing state or federal civil and criminal cases, and issuing emergency orders. Informal enforcement actions include reminder notices of a violation, formal notices of violation, public notification requests, and state referrals of cases to EPA. According to a 2013 EPA compliance report, the number of enforcement actions in a year does not necessarily correlate with the number of violations that are reported in the same year. The two most frequently reported enforcement actions taken were informalstate violation/reminder notice, which inform water systems that the system has open violations, and state public notification requested, in which the state requests a copy of the information water systems sent to homeowners. Most of the EPA officials we interviewed in the 10 regional offices told us that states primarily rely on informal actions and technical assistance and training because they are the most effective means of getting water systems to comply with regulations.

EPA’s SDWIS/Fed Database Does Not Contain Data on the Presence of Lead Pipes and Complete Sample Results for Small Systems

The LCR does not require states to submit data to EPA’s SDWIS/Fed database on (1) location of lead pipes or (2) all sample results for small water systems. As a result, EPA does not have available data on either the location of lead pipes or complete sample results for small water systems. Water systems were required to collect information on the presence of lead pipes when the LCR was promulgated in 1991, but there is currently no requirement that this information be reported to EPA. States are to submit to SDWIS/Fed on a quarterly basis all 90th percentile sample results for large and medium water systems (including those that exceed the lead action level). However, for small water systems, states are required to submit data to SDWIS/Fed only for those 90th percentile sample results that exceed the lead action level. As a result, sample results for small water systems are not complete in SDWIS/Fed.

Collection of Data on the Presence of Lead Pipes Was a 1991 LCR Requirement

When the LCR was promulgated in 1991, all drinking water systems were required to collect information about the infrastructure that delivered water to customers, including any known lead pipes and lead service lines. The purpose of this effort, referred to as a materials evaluation, was to identify locations that may have been particularly susceptible to high lead or copper concentrations, which would become the pool of targeted sample sites. Water systems that must replace their lead service lines under the LCR also must report their materials evaluations to their respective states. In addition, a 1980 EPA regulation required community water systems to identify, among other things, whether lead from piping, solder, caulking, interior lining of distribution mains, alloys, and home plumbing was present in their distribution system and report this information to the state. However, the LCR does not require states to report information on known lead pipes and service lines to EPA’s SDWIS/Fed database. As a result, the agency may not have information at the national level about the lead infrastructure in the country.

In February 2016, in light of the events in Flint, Michigan, and other U.S. cities, EPA asked states to collect information about the locations of lead service lines and publish the information on local or state websites to better inform the public. In a July 2016 letter to the Environmental Council of States and the Association of State and Territorial Health Officials, EPA noted that some states had successfully taken action to fulfill the request, citing (1) water systems with online searchable databases that provide information on lead service lines and (2) several states that were requiring water systems to update their inventories of lead service lines. In the letter, EPA also noted that many states identified challenges in identifying lead service lines but that improving knowledge of lead service lines is important to ensure that water systems are (1) collecting drinking water samples from valid high-risk locations, as required under the LCR, (2) managing the risks associated with disruption of lead service lines, and (3) providing information to customers on how to assess and mitigate risks posed by lead.

In written responses to EPA’s letter, most (37) of the 50 states (or primacy agencies) indicated that they had fulfilled or intended to fulfill EPA’s request to work with water systems to collect and make public information about lead pipes. Four states indicated that they were considering EPA’s request. However, 9 states indicated that they would not or did not intend to fulfill EPA’s request because of challenges in finding the historical documentation about lead pipes used to create original sample plans or dedicating staff resources to do so. In addition, in their responses to EPA’s letter, 13 states noted that the LCR does not require states to maintain information about water systems’ lead pipes or to provide the information to the public. EPA stated in its 2016 Lead and Copper Rule Revisions White Paper that it was considering a proposal in the upcoming revision to the LCR for water systems to update their information on lead service lines and share the results of their “materials evaluation.” In June 2017, EPA headquarters officials said that the agency was evaluating all options outlined in its 2016 white paper as well as recommendations related to lead pipes by other stakeholders.

According to EPA technical guidance on corrosion control, knowledge about lead service lines is needed for studies of corrosion control treatments. In addition, the National Drinking Water Advisory Council stated in its 2015 final report that knowledge about the location of lead service lines is essential to ensuring replacement and outreach to customers who are most likely to have a lead service line. We reported in March 2013 that, as the nation faces limited budgets and funding for federal programs, the importance of targeting federal funds to communities with the greatest need and spending funds efficiently increases. For example, the Water Infrastructure Improvements for the Nation Act, enacted in December 2016, directs EPA to establish a grant program for reducing the lead in drinking water by, among other things, replacing publicly owned lead service lines and assisting homeowners with replacing the lead service lines on their property. In addition, EPA’s 2016 action plan identifies the reduction of lead risks as a priority area. By requiring, in the upcoming revision of the LCR, that states report the available information about lead pipes in its SDWIS/Fed (or in future redesigns, such as SDWIS Prime) database, EPA and congressional decision makers would have important information at the national level on what is known about lead infrastructure in the country, thereby facilitating the agency in its oversight role.

Data on Drinking Water Sample Results for Small Systems

In a 2016 report on how science and technology can address drinking water challenges, the President’s Council of Advisors on Science and Technology stated that, sample data are essential for evaluating the performance of a drinking water system. While the LCR requires small water systems to report all 90th percentile sample results (i.e., results that meet, fall below, and exceed the lead action level) to the states, it does not require the states to report all of this information to EPA through the SDWIS/Fed database. EPA headquarters officials said that the agency had not required states to submit the results for all small systems due to the reporting burden on states. According to EPA’s reporting guidance for states, however, reporting all sample results to the SDWIS/Fed database for small water systems that do not exceed the lead action level is encouraged and will be accepted. EPA officials told us that SDWIS/Fed contained complete sample results for about 20,000 of the approximately 58,000 small water systems, or about 30 percent, of the 68,000 water systems.

Officials we interviewed in 1 of EPA’s 10 regional offices said that the lack of all 90th percentile sample results for small systems prevents the agency from observing such systems in SDWIS/Fed. In June 2017, EPA headquarters officials said that having all 90th percentile sample results for small systems would give the agency a more complete national picture of lead in drinking water. According to information on EPA’s website, small water systems can face unique managerial, financial and operational challenges in consistently providing drinking water that meets EPA standards and requirements. In 2016, EPA’s Office of Inspector General reported that small water systems are less likely to have the technical, managerial, and financial capacity to conduct actions that would ensure safe drinking water. The SDWA requires that EPA assist states in ensuring that water systems acquire and maintain technical, managerial, and financial capacity. In addition, the SDWA also authorizes EPA to provide technical assistance to small public water systems to enable such systems to achieve and maintain compliance with applicable national primary drinking water regulations, including the LCR.

Because it does not have complete 90th percentile sample results on small water systems, EPA does not have information on how such systems are managing the reduction of lead in their drinking water. Small systems represent the majority of water systems reporting samples that have exceeded the lead action level, but states are not required to submit all 90th percentile sample results for small systems in the SDWIS/Fed database; this would require a revision to EPA’s regulations. By requiring, in the upcoming LCR revision, that states report all 90th percentile sample results for small systems in the SDWIS/Fed database, EPA would have data to track the changes in lead levels over time among small systems and would be better positioned to assist states in early intervention for small water systems that are near the lead action level where appropriate. In June 2017, EPA officials said that as states move toward more modernized data flows using electronic reporting and SDWIS Prime, the burden for reporting should be significantly lowered.

EPA Uses Available Data to Monitor Compliance with the LCR, and Officials Said That They Have Increased the Use of Data in the Last Year

EPA officials said that they analyze data in their SDWIS/Fed database and meet quarterly with state regulators to monitor compliance across all drinking water rules and that, in the last year, in response to the events in Flint, Michigan, they have increased their use of these data to monitor compliance and address implementation of the LCR.

EPA applies its Enforcement Targeting Tool to the violations data associated with the more than 90 drinking water contaminants regulated under SDWA for the purpose of identifying systems that merit action by states based on the seriousness of their violations. Specifically, the Enforcement Targeting Tool assigns a score to each water system based on, among other criteria, the types of violations and number of unresolved violations over the previous 5-year period. The Enforcement Targeting Tool assigns higher scores to health-based violations, such as treatment technique violations. Water systems whose scores meet or exceed a certain threshold are given higher enforcement priority for states (and EPA, if necessary). EPA officials we interviewed in all 10 of the regional offices said that they meet quarterly with state regulators to discuss the results generated by the Enforcement Targeting Tool and generally considered it to be a success. EPA headquarters officials agreed that the Enforcement Targeting Tool was a success, even with the agency’s challenges with the SDWIS/Fed data, including using data that are not always complete and accurate. However, these officials also told us that the Enforcement Targeting Tool was not designed for and therefore would not be appropriate for monitoring compliance with any single regulation, including the LCR. In April 2017, EPA headquarters officials told us that as of January 2017, the Enforcement Targeting Tool includes information on water systems’ most recent 90th percentile sample result and the number of 90th percentile sample results exceeding the lead action level over the previous 5-year time period.

EPA officials told us that they also conduct on-site file reviews of one to two states each year. File reviews involve regional staff comparing information on a sample of water systems in states’ databases with that in SDWIS/Fed to identify any discrepancies and to assess states’ compliance decisions. EPA headquarters officials told us that the agency developed a protocol for conducting file reviews and provided training on this protocol for the regions. Staff have discretion on how to prioritize the states in their regions. These file reviews cover all of the drinking water regulations, which allows them to also periodically assess how well states were implementing the LCR. According to EPA officials, in 2011, these file reviews replaced the data verification audits, which were discontinued in 2010; were designed to be generalizable to all water systems; and involved contractors comprehensively reviewing states’ water system inventories and violations and enforcement data and comparing them against the information in SDWIS/Fed. Agency officials said that the agency can no longer conduct these audits due to a lack of resources.

EPA headquarters officials told us that the agency had begun using SDWIS/Fed data, in the last year, in response to the discovery of drinking water contaminated with elevated levels of lead in Flint, Michigan, as part of a two-pronged approach for reviewing states’ and water systems’ implementation of the LCR.

The first part of EPA’s approach was to identify all of the water systems that reported sample results exceeding the federal action level from 2013 to 2016. EPA officials said that they requested that state officials provide updates on the status of each of the approximately 2,400 water systems identified as reporting such results. The purpose of this approach, according to EPA officials, was to determine whether the states and water systems were properly following the LCR’s requirements after a water system’s sample results exceeded the federal lead action level. In addition, the approach would allow, if necessary, states and EPA to have an opportunity for early intervention. EPA officials said that previously they had not systematically and uniformly analyzed all of the water systems in their database with sample results that exceed the federal lead action level or asked states, at any one time, to provide updates on all of the water systems with sample results exceeding the action level. Instead, EPA headquarters officials said that staff in the regional offices generally had worked with individual states on individual cases of water systems with sample results exceeding the action level as a part of the agency’s routine oversight efforts. EPA headquarters officials said that one outcome of their effort since the discovery in Flint, Michigan, was “lessons learned” about the importance of knowing where lead service lines are located and the need for states to focus more attention on small water systems and schools with their own water supplies. EPA officials we interviewed in some of the 10 regional offices said that meetings with state officials to discuss the water systems that had exceeded the lead action level had been beneficial because agency officials gained a better understanding of how states understood and implemented the requirements of the LCR. However, officials in 3 of the 10 regional offices said that they would ask states to provide these updates less frequently because of limited staff resources. EPA headquarters officials told us that an additional outcome of this approach was insight, for EPA staff, into the types of training state regulators may need about the implementation of LCR requirements.

The second part of EPA’s approach, according to headquarters officials, was to review state protocols and practices against all of the requirements of the LCR to ensure that states were implementing the rule, including protocols and procedures for using corrosion control treatments. After reviewing state protocols and practices, EPA requested that states take such actions as providing information on their websites and documenting protocols and practices for greater transparency. In addition, EPA staff in the 10 regional offices conducted meetings with the state officials in their regions. Some of these EPA officials also told us that the agency determined that generally states were implementing the LCR appropriately. However, EPA identified weaknesses among states and water systems with identifying lead pipes and understanding the requirements for installing and maintaining corrosion control. In response, EPA officials told us that they updated guidance to states and water systems and offered training and written technical guidance on implementing corrosion control. Specifically, EPA officials said, they offered in- person training for state regulators in each of the 10 EPA regions on implementing the corrosion control requirements of the LCR.

Certain Factors May Contribute to Water Systems’ Noncompliance with the LCR

Through discussions with state regulators, we identified multiple factors that may contribute to water systems’ noncompliance with the LCR. To determine whether such factors were associated with a higher likelihood of having a reported violation of the LCR, we conducted a statistical analysis that calculated a system’s likelihood of a violation using selected factors, such as the size of the population served and source water, and currently available EPA data and found that incorporating multiple factors in the analysis may help identify water systems at a higher likelihood of violating the LCR.

Certain Factors May Contribute to Water Systems’ Noncompliance with the LCR

Based on our analysis of transcripts of discussion groups, state regulators representing 41 states and 1 territory identified 29 factors that may contribute to water systems’ noncompliance with the LCR. We also reviewed 31 studies and summarized the factors the authors identified. Table 1 identifies the 10 factors state regulators most frequently identified.

During our discussion groups, state regulators provided examples of how these factors contributed to noncompliance with the rule. For example, regulators in 37 states said that the size of the population served by water systems may influence noncompliance with the LCR. Regulators in 28 of the 37 states said that small systems are more likely to have drinking water sample results that exceed the federal action level, to be in noncompliance, or to face challenges that may contribute to noncompliance. Regulators in 5 states explained that this may be because small systems are generally less likely to have operators with the knowledge to properly collect samples or manage corrosion control treatment. Regulators in 28 states said that the required LCR process for collecting drinking water samples to test for lead levels may contribute to noncompliance. Regulators in 19 of these 28 states said that collecting the required number of samples is a challenge for water systems that can lead to noncompliance, because homeowners are frequently not willing to collect samples or, if they agree to collect samples, often collect them improperly. For example, homeowners may sample from an infrequently used faucet (e.g., outside spigot) instead of the required drinking water tap. Regulators in 20 states also described how the type of water system can lead to noncompliance. For example, they said that water systems for which water management and treatment are not the primary missions, such as schools, mobile home parks, and other entities, have challenges complying with the LCR.

These regulators also told us that the presence of multiple factors could, together, contribute to violations of the LCR. For example, a regulator in one state said that the presence of lead in the pipes, combined with corrosive water, could lead to sample results that exceed the federal lead action level for a water system. A 90th percentile sample result that exceeds the lead action level is not by itself a violation. However, if the same water system did not conduct the required corrosion control treatment study for any reason, including because it lacked the financial capacity to pay for the study, the system would be in violation of the LCR. Appendix III provides information on all of the factors that state regulators in the 41 states and 1 territory identified in our discussions as well as examples of how those factors, individually and together, may contribute to violations of the LCR.

The 31 academic studies we reviewed associated certain factors with elevated concentrations of lead in public drinking water, human exposure to lead in drinking water, or violations of drinking water laws and regulations. These studies identified the potential effects of, among other factors, the presence of lead in pipes or lead solder, within the water system’s pipes; natural disturbances within drinking water pipes, such as stagnant or soft water; operator actions to address lead in drinking water, such as the use of corrosion control to decrease the presence of lead and the use of chemical treatments to decrease the presence of other contaminants that may increase the presence of lead; a water system’s capacity to address existing lead challenges, such as the size of the population it serves and whether the system is publically or privately owned; and state and local policies designed to reduce drinking water violations or human exposure to lead in water.

Use of a Statistical Analysis Could Help Identify Systems at Higher Likelihood of LCR Violations

Our interviews with state regulators and review of academic studies suggest that certain factors could indicate whether water systems are at a higher likelihood for having a reported violation of the LCR. We selected four system characteristics that were consistent with the factors reported by state regulators in discussion groups and were available in SDWIS/Fed to conduct a statistical analysis: the population served by (or size of) the drinking water system, whether the drinking water system was publicly- or privately-owned, whether the drinking water system used groundwater or surface water whether the drinking water system was classified as a community water system or a non-transient non-community water system.

We also included the factor of whether a system had sample results that exceeded the lead action level. SDWIS/Fed does not include data on such factors as the presence of lead service lines or technical, managerial, and financial capacity.

We were unable to develop a nationwide statistical model referred to as a logistic regression analysis. A logistic regression analysis can identify factors that are associated with a violation and can estimate a drinking water system’s likelihood of a violation based on these factors. We have previously found that regression analysis can identify entities, regulated by a federal program, that pose a higher likelihood for a particular outcome. However, during our review of the reliability of EPA’s data on violations, we could not verify that the limitations in the completeness of the data identified in our June 2011 report had been sufficiently addressed, nationwide. Specifically, in June 2011, we found that EPA had not been able, among other things, to resume the comprehensive and routine data verification audits that would provide it with current information on the completeness of the data states provide to SDWIS/Fed. As a result, in June 2011, we recommended that EPA resume data verification audits to routinely evaluate the quality of selected drinking water data on health-based and monitoring violations that the states provide to EPA. These audits should also evaluate the quality of data on the enforcement actions that states and other primacy agencies have taken to correct violations. EPA partially agreed with our recommendation and stated that it has found that data verification audits provide valuable information on data completeness but did not commit to conducting such audits beyond 2011. Instead, EPA said that until the next generation of SDWIS (SDWIS Prime) is deployed, thus enabling the agency to view compliance monitoring data and compliance determinations directly, it will consider using data verification audits to evaluate data quality. As of October 2016, EPA reported that it has not conducted another data verification audit.

Because of the limitations of using SDWIS/Fed data to conduct a nationwide analysis, we sought to use such data to conduct an analysis for individual states to determine whether factors could predict the likelihood that a water system would violate the LCR. As such, we used data from Ohio and Texas to examine the potential for developing a statistical analysis to identify drinking water systems at higher likelihood of having a reported violation. EPA found few or no discrepancies between the LCR data in these state systems and in SDWIS/Fed for the time period of our statistical analysis, 2013 to 2016. The results of our analysis are not generalizable to other states.

To conduct an analysis for the two states, we developed a series of logistic regression models for these states using (1) LCR violations data for Ohio and Texas in SDWIS/Fed for 2013 and 2014 and (2) the four factors for which data were available in SDWIS/Fed (size of the population served, ownership, source water, and water system type). Our models estimated the likelihood that a water system in those two states would have a reported violation of the LCR based on these factors. We found that water systems with certain factors had a higher likelihood of having a reported violation of the LCR than water systems without those factors. For example, in both states, a water system serving 100 people was more likely to have a reported violation of the LCR than a water system serving 1,000 people. In addition, systems with a previous sample result that exceeded the lead action level were more likely to have a reported violation of the LCR than systems without a previous sample result that exceeded the lead action level.

We then tested the ability of our models to predict subsequent rates of having reported violations. Specifically, we compared the estimates from our models to violations that were actually reported in SDWIS/Fed in 2015 and 2016. We found that water systems that we identified as having higher likelihoods of having a reported violation, based on our models, had significantly higher rates of reported violations in 2015 and 2016. The results of our analysis indicate that multiple factors, in addition to whether a system had sample results that exceeded the lead action level, could be used to predict water systems with a higher likelihood of having a reported violation of the LCR.

Our analysis suggests that a statistical analysis of EPA data could be used to identify water systems with a higher likelihood of having a reported violation of the LCR. However, we identified two key limitations, among others, based on the state of the data in SDWIS/Fed as of December 2016. The first was the quality of the data for the purposes of conducting an analysis. We could not be confident in the specific results of a nationwide or, for some states, a state-specific analysis, because we did not have the necessary assurances of the accuracy and completeness of the SDWIS/Fed data, issues about which we previously reported in January 2006 and June 2011. EPA headquarters officials told us in June 2011 and April 2016 that their upcoming SDWIS/Fed upgrade, SDWIS Prime, could give the agency direct access to state data. Having complete and accurate data for all states or a nationally representative sample of states would allow for a nationwide analysis.

The second limitation was that data are not available for many of the factors identified by state regulators that may contribute to water systems’ noncompliance with the LCR. Our analysis was limited to those four factors for which states submit data to EPA’s SDWIS/Fed. Because data were unavailable for all potentially relevant factors, we were unable to include information on the presence of lead pipes, lack of financial capacity, and lack of technical capacity. EPA headquarters officials told us that they were considering the development of indicators of capacity. For example, these officials said that potential indicators suggesting a drinking water system is challenged by capacity are the drinking water system (1) not having raised rates in 20 years; (2) not having recently used asset management; or (3) having experienced difficulty in retaining trained operators. Data on the presence of lead pipes, financial capacity, technical capacity, and other factors may allow for stronger logistic regression models that more accurately identify water systems with a higher likelihood for violations. Appendix V provides a technical description of the statistical analysis we conducted.

According to EPA, the agency promulgated the LCR to protect public health by minimizing the levels of lead in the drinking water supply. EPA’s current approach for oversight of the LCR targets water systems with sample results that exceed the lead action level. This approach is reasonable because water systems that exceed the action level have a known and documented lead exposure risk and are required under the LCR to take actions that are considered health-based. This approach, however, primarily incorporates one factorsample results that exceed the lead action leveland does not include the potential of having reported violations across all of the requirements of the LCR. In addition, EPA officials we interviewed in 3 of the 10 regional offices said that they do not have the resources to sustain the agency’s current approach. Under federal standards for internal control, management should identify, analyze, and respond to risks related to achieving the defined objectives. Although EPA may not have the resources to continue the use of its current approach of following up on all sample results that exceed the lead action level, our analysis illustrates that EPA collects data that, where complete and accurate, could be incorporated into a risk-based analysis. For example, such an analysis could be used in individual states or geographical areas, while EPA is taking steps to improve its data and implement SDWIS Prime. A statistical, risk-based analysis, whether it is used for individual states or nationwide, may provide EPA with an additional tool by which it may be able to efficiently target its limited resources for oversight of water systems and meet its goal of reducing the risk of lead exposure. By developing a statistical analysis that incorporates multiple factors—including those currently in SDWIS/Fed and others such as the presence of lead pipes and the use of corrosion control—to identify water systems that might pose a higher likelihood for violating the LCR once complete violations data are obtained such as through SDWIS Prime, EPA could supplement its current efforts to better target its oversight to the water systems that present a higher risk of violating the LCR.

Conclusions

EPA has taken several actions to increase transparency about lead hazards, focus on water systems’ sample results over the federal lead action level, and ensure a better understanding of how states and water systems interpret and implement the LCR. However, most states are not submitting data to the SDWIS/Fed database on water systems’ use of corrosion control as required by the LCR. We continue to believe that EPA should take actions to address our 2006 recommendation. Further, by requiring that states report the available information about lead pipes in EPA’s SDWIS/Fed database nationally, EPA and congressional decision makers would have important information at the national level about lead infrastructure, thereby facilitating the agency in its oversight role.

The LCR does not require states to submit data to EPA’s SDWIS/Fed database on all 90th percentile sample results for small water systems, only to provide sample results that exceed the lead action level. EPA has long acknowledged the challenges experienced by small water systems, as evidenced in the data for samples that exceed the lead action level and violations for taking samples as required and reporting sample results. The upcoming revision of the LCR provides an opportunity for EPA to require states to report all 90th percentile sample results for small systems. By doing so, EPA would have data to track the changes in lead levels over time among small systems and would be better positioned to assist states in early intervention for small water systems that are near the lead action level where appropriate. EPA also has an opportunity to enhance its oversight of the LCR by using statistical analyses to analyze those data that it currently collects and has determined to be complete. With the LCR applying to about 68,000 water systems across the country (or approximately 45 percent of all drinking water systems), it is important to target limited resources to those water systems that pose the highest likelihood of a violation. By developing a statistical analysis that incorporates multiple factors—including those currently in SDWIS/Fed and others such as the presence of lead pipes and the use of corrosion control—to identify water systems that might pose a higher likelihood for violating the LCR, EPA could supplement its current efforts and better target its oversight to the water systems that present a higher likelihood of violating the LCR, particularly when complete violations data are more readily available through upgrades, such as SDWIS Prime.

Recommendations for Executive Action

We are making the following three recommendations to EPA:

The Assistant Administrator for Water of EPA’s Office of Water should require states to report available information about lead pipes to EPA’s SDWIS/Fed (or a future redesign such as SDWIS Prime) database, in its upcoming revision of the LCR; (Recommendation 1)

The Assistant Administrator for Water of EPA’s Office of Water should require states to report all 90th percentile sample results for small water systems to EPA’s SDWIS/Fed (or a future redesign such as SDWIS Prime) database, in its upcoming revision of the LCR; (Recommendation 2) and

The Assistant Administrator for Water of EPA’s Office of Water and the Assistant Administrator of EPA’s Office of Enforcement and Compliance Assurance should develop a statistical analysis that incorporates multiple factors—including those currently in SDWIS/Fed and others such as the presence of lead pipes and the use of corrosion control—to identify water systems that might pose a higher likelihood for violating the LCR once complete violations data are obtained, such as through SDWIS Prime. (Recommendation 3)

Agency Comments

We provided a draft of this report to EPA for review and comment. In its written comments, reproduced in appendix VII, EPA stated that it generally agreed with all three of our recommendations and the importance of ensuring that the agency has the information needed to ensure effective oversight of the drinking water programs. EPA also provided technical comments which we incorporated, as appropriate.

EPA stated that our first two recommendations relate to the LCR revisions: (1) report available information about lead pipes to EPA’s database and (2) report all 90th percentile sample results for small water systems to EPA’s SDWIS/Fed (or a future design such as SDWIS Prime). As a result, EPA said that it would consider our recommendations along with those of other stakeholders as the agency continues to support the development of the proposed LCR for publication in the Federal Register and follows the public review and comment process in 2018. In addition, EPA said that the agency would continue to work with states to develop SDWIS Prime and another electronic reporting tool, which will facilitate electronic reporting, which in turn will increase data accuracy and completeness.

In response to our third recommendation, EPA stated that it agrees with the concept of our third recommendation to develop a national statistical analysis that could identify water systems with a higher likelihood of violating the LCR and that the agency previously tried to build a similar tool but faced challenges due to variations between selected factors and violations between states. EPA also said that while developing a national tool would be a challenge, it would be beneficial to both the agency and state primacy agencies.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to the appropriate congressional committees; the Administrator of the Environmental Protection Agency, and other interested parties. In addition, this report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff members have any questions about this report, please contact me at (202) 512-3841 or gomezj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix VIII.

Appendix I: Objectives, Scope, and Methodology

This report examines the issue of elevated lead in drinking water and the Environmental Protection Agency’s (EPA) use of compliance data for oversight of the Lead and Copper Rule (LCR). Our objectives were to examine (1) what the available EPA data show about compliance with and enforcement of the LCR among water systems, including schools; (2) how EPA uses these data to monitor compliance; and (3) factors, if any, that may contribute to water systems’ noncompliance with the LCR. We compared our evidence on EPA’s a use of these data for oversight of the LCR to Standards for Internal Control in the Federal Government. According to these standards, internal control is a process by an entity’s oversight body, management, and other personnel that provides reasonable assurance that the objectives of an entity will be achieved. An effective internal control system increases the likelihood that an entity will achieve its objectives. For this review, we used the standard for one of the five components of internal controlrisk assessmentas criteria.

To examine what the EPA data show about reported compliance and enforcement, we reviewed LCR data in EPA’s Safe Drinking Water Information System (SDWIS/Fed) for the time period July 1, 2011, to December 31, 2016. We chose this time period because it provided the most recent history of available compliance data without a change in the regulations at the time of our analysis. The LCR data contained information on 67,581 active community water systems and non-transient non-community water systems, including those that were schools or daycare centers with their own water supply. Table 2 provides the water systems, by type and size, included in our analysis. The LCR divides water systems into three broad categories: small, medium, and large. Size is a factor in determining the number of samples that must be collected as well as the applicability and timing of some of the LCR requirements.

We reviewed the available data on corrosion control, drinking water sample results, violations of the requirements of the LCR, and state and EPA enforcement actions. We also described the data by population served/size, whether the water system was a school or daycare center, and whether the water system was a community water system or a non- transient non-community water system, among other factors. We analyzed data on sample results for a 3-year time period (from January 2014 to December 2016) and for a 5 1/2-year period (from July 2011 to December 2016). EPA officials told us that they analyze sample data over a 3-year period rather than yearly to ensure that the majority of water systems will have submitted sample results. When presenting a comparison of the sample data and the milestone data on corrosion control, we used the 5 1/2-year period for both sets of data. For violations data, we presented open violations as of December 2016. Violations are considered open when the state has not determined that a water system is in compliance with the specific requirement for which it received the violation. Finally, we presented data on enforcement actions for a 5 1/2- year period (July 2011 through December 2016) to ensure that we provided the most complete picture of the range of state and federal actions taken and to avoid comparisons with the violations data. According to a 2013 EPA compliance report, enforcement data, in any one year, do not necessarily correlate with violations data. In addition, the compliance report states that enforcement actions can be initiated against violations that occurred in a previous year, one enforcement action may address numerous violations at the same system; and it can take several years for a system to return to compliance.

We reviewed the data available in the SDWIS/Fed database and the compliance requirements in the LCR to evaluate those aspects of the LCR for which implementation data were available. We interviewed officials from EPA’s Office of Water and Office of Enforcement and Compliance Assurance on the reliability, completeness and accuracy of LCR data in SDWIS/Fed. In addition, we reviewed EPA data reliability assessments, recent file reviews for selected states, a 2017 EPA OIG report on the reliability of SDWIS/Fed sample data, data verification reports and past GAO reports on the reliability of the data in SDWIS/Fed. For example, EPA’s file reviews in some states found that not all violations data were reported to SDWIS/Fed, which could lead to undercounting. In addition, some state regulators told us that samples may be collected incorrectly by some homeowners, which could lead to inaccurate sample results. EPA has stated on its website that the agency acknowledges challenges related to the data in SDWIS/Fed, specifically underreporting of some data by states. GAO has also reported on EPA’s challenges with SDWIS/Fed. Based on this, the compliance data in SDWIS/Fed likely underreport the actual number of sample results that exceed the lead action level, milestones, violations, and enforcement actions, which we note in this report. Because of the incompleteness of reported data on sample results, violations, and enforcement actions, and because of concerns raised by state officials about sample data, we found the data to be of undetermined reliability. For this review, we describe the data about water systems’ compliance with the LCR compliance and EPA’s enforcement actions as they are reported in SDWIS/Fed for the purpose of providing a current assessment of EPA’s use of the data.

To examine how EPA uses LCR data to monitor compliance we conducted semistructured interviews with EPA officials. We used a standard set of questions to interview officials in EPA’s headquarters and in each of the 10 regional offices. Our standard set of open-ended questions for EPA’s 10 regional offices asked about state actions responding to EPA’s requests about, among other things, implementation of the LCR, the use of SDWIS/Fed data, enforcement tools, and compliance with the LCR among water systems and schools. We conducted in-person interviews with officials responsible for monitoring compliance in states within EPA regions 1, 2, 3, 4, 5, and 7. We identified these regions based on a 2016 survey that estimated that these regions have the highest number of lead service lines. We spoke with officials in EPA regions 6, 8, 9, and 10 on the telephone. Table 3 provides a list of the EPA regions and the states under the regulatory jurisdiction of those regions. Our in-person interviews with officials in EPA regions 1through 5 and 7 were in offices located in Boston, Massachusetts; New York, New York; Philadelphia, Pennsylvania; Atlanta, Georgia; Chicago, Illinois; and Lenexa, Kansas, respectively. In these cities, we also met with state primacy agencies and local water systems and other local officials, when possible, to obtain examples of compliance and enforcement practices and implementation challenges. Specifically, we met with state drinking water officials in Massachusetts and Georgia. We met with officials representing local water systems in Atlanta, Boston, New York, Chicago, and Kansas City, Missouri. In total, we held 10 interviews with EPA staff in the regional offices and seven interviews with state and local officials in the cities we visited. We also reviewed EPA policy documents that outlined the agency’s enforcement approach and documents related to EPA’s request that states take certain actions following the events in Flint, Michigan. Finally, we reviewed federal regulations; EPA guidance to states and water systems on how to implement the LCR; the 2016 action plan; information on what constitutes a violation of the LCR, action plans, and other relevant documents.

To identify the factors that may influence water systems’ risk of noncompliance with the LCR, we conducted a content analysis of information provided by state regulators in discussion groups. To assess whether selected factors available in SDWIS/Fed could be used to predict reported violations, we conducted a statistical analysis of EPA data to develop an illustrative model. We conducted a literature review to identify factors associated with elevated concentrations of lead in public drinking water, human exposure to lead in drinking water, or violations of drinking water laws and regulations.

Discussion groups with state regulators. We conducted discussion groups with a nonprobability sample of state drinking water regulators to contribute to our understanding of the potential factors that may influence noncompliance with the LCR. We invited regulators from all states and territories to participate via email. In total, we conducted eight, 1-hour discussion groups over the telephone in September and October 2016. Regulators representing 41 states and 1 territory participated in these discussion groups. Each discussion group had from 2 to 8 states or territory, and each state or territory had a primary designated spokesperson. During each discussion group, the GAO moderator asked participants to list one or two factors that, in their experience, most strongly influence a water system’s ability to comply with the LCR. Each state provided a list of factors. The moderator then asked participants to elaborate on how the factors reported could influence compliance. When necessary, the moderator asked probing questions to further clarify participants’ comments. Two or three analysts transcribed each session and combined and reconciled notes to develop transcripts for each of the discussion groups. We conducted a content analysis of the transcripts from the eight discussion groups to identify the factors most frequently reported by the participants in the groups. Two GAO analysts independently classified each comment using qualitative analysis software. The findings from these discussion groups may not be generalizable to all state regulators. We provide a narrative description of the results of our discussions with the state regulators in appendix III and a technical description of the content analysis we conducted in appendix IV.

Statistical analysis. We conducted a statistical analysis to illustrate whether predictive modeling could be used to identify water systems with a higher likelihood of a reported violation of the LCR. To conduct our analysis, we used the same data from EPA’s SDWIS/Fed for systems listed as active as of December 31, 2016 as mentioned above. We selected two statesOhio and Texasbecause EPA’s file reviews indicated that there were not significant discrepancies during the scope and time period of our analysis, which focused on 2013 to 2016, in the LCR data reported by these states to SDWIS/Fed. We reviewed EPA’s 2016 file reviews of data the states provide to SDWIS/Fed and interviewed EPA and state officials for Texas and received written responses to our questions from Ohio. For each of the sampled systems, EPA reviewed state records to determine whether the state was correctly identifying violations and reporting those violations to SDWIS/Fed. Although, unlike EPA’s previous reviews, EPA’s 2016 file reviews are not based on generalizable samples, they were conducted for a broad range of drinking water systems in each of the states. Based on the results of EPA’s reviews, we determined that these two states had sufficiently reliable data for our purposes of illustrating a statistical approach. The results of our analysis for these two states are not generalizable to other states. Our analysis included three steps. We first conducted a bivariate analysis to determine whether the following four factors correspond to violations of the LCR for 2013 to 2014: (1) size of the population served; (2) water source (groundwater or surface water); (3) ownership (public or private); and (4) whether the system is a community water system or non- transient, non-community water system. We also included the factor of whether sample results exceeded the lead action level. We then developed a series of multivariate logistic regression models. Specifically, multivariate logistic regression modeling is statistical method for analyzing the potential influence of each individual factor on the likelihood of a binary outcome (e.g., a violation) while simultaneously accounting for the potential influence of the other factors. We selected this type of model because it could account for the factors simultaneously. Lastly, to test whether our models could be used to identify systems with a higher likelihood of a future violation, we compared the values generated by our models to actual violations reported in the SDWIS/Fed data in 2015 to 2016. We provide a technical description of the statistical analysis we conducted, including determinations about the reliability of the data and the limitations of the analysis, in appendix V.

Literature review. We reviewed studies concerning detection of lead in drinking water and violation of drinking water regulations. These studies were identified through searches by GAO research librarians for peer- reviewed materials in such databases as ProQuest, Scopus, Academic One-File, and Web of Science. Librarians conducted searches using such terms and phrases as lead and copper, water supply, drinking water, lead exposure and lead poisoning alone and in combination with one another. We also identified and reviewed relevant publications by trade groups, think tanks, and other nongovernmental organization. We narrowed a preliminary selection of results by reviewing abstracts and introductions, where applicable. Based on that preliminary review, we determined that 31 sources fit within the scope of our engagement objectives. We then reviewed the data and key findings of each of these 31 sources to formulate and refine some hypotheses concerning violations of the LCR and detection of lead in drinking water. The hypotheses were reviewed by a GAO technical expert to ensure that they were sufficiently supported by the cited corresponding research.

Appendix II: Additional Analysis of Available EPA Data on Reported Sample Results, Violations, and Enforcement for the Lead and Copper Rule

Environmental Protection Agency (EPA) data from July 1, 2011, to December 31, 2016, can provide information on compliance by water systems and enforcement by states and EPA regarding the Lead and Copper Rule (LCR). This appendix provides additional information from our analysis of what the EPA data in the agency’s Safe Drinking Water Information System (SDWIS/Fed) show about compliance with and enforcement of the LCR. The LCR requires water systems to monitor drinking water at customer taps, and if lead levels are elevated, take additional actions to control corrosion, inform the public, and in some circumstances replace lead service lines under the systems’ control. States generally have primary responsibility for monitoring and enforcement of Safe Drinking Water Act requirements, including the LCR. In this appendix, we provide additional results of our analysis of the LCR data for (1) sample results, (2) violations, and (3) enforcement. We reported in January 2006, that the LCR data, and in June 2011, the data in SDWIS/Fed generally, were not accurate or complete. According to EPA, some of the violations data are underreported. In addition, a 2017 EPA Office of Inspector General report indicated that sample data, specifically, are potentially underreported. In addition, some state regulators with whom we interviewed in 2016 told us that homeowners and water systems may take LCR samples improperly as we discuss in this report. See, also, appendix III for these state regulators’ views on waters systems’ challenges with implementing the sample requirements under the LCR. We present the data that were available in the SDWIS/Fed database at the time of our review.

Sample Results

The LCR requires that all water systems periodically obtain tap water samples and for sample results that exceed an action level of 15 parts per billion (ppb) to determine if corrosion control treatments are working properly. EPA requires states to report (1) sample results for any water system whose 90th percentile sample results exceed the federal action level of 15 parts per billion; and, (2) sample results for large and medium water systems even if the sample results do not exceed the lead action level. From January 1, 2014, to December 31, 2016, there were approximately 1,430 water systems reporting sample results over the lead action level (see table 4), the majority of which were small water systems. EPA officials told us that they analyze these sample data over a 3-year period rather than yearly to ensure that the majority of water systems will have submitted 90th percentile sample results.

The available EPA data show that almost all of the water systems (1,364, or 95 percent) reporting sample results that exceeded the lead action level from 2014 to 2016 were small and, together, served a population of about 505,000. In contrast, the remaining 66 large and medium water systems (5 percent) reporting sample results that exceeded the lead action level from 2014 to 2016, together, served a population of 2.7 million. In addition, as shown in table 5, states within EPA’s regions 1 and 3 had the highest number of water systems that reported sample results exceeding the lead action level. EPA headquarters officials we interviewed provided possible explanations for why 90th percentile sample results would be higher in these states. They said that there are more lead service lines in the northeastern states, such as those within regions 1 and 3. See table 5.

Table 6 provides information on the EPA data available for those water systems that have results under the federal action level in 2016. As previously mentioned, states are to report all 90th percentile sample results for large and medium water systems to EPAthose that exceed and fall below the federal action level.

Violations

The 6,567 water systems (or 10 percent of all water systems) with reported open violations as of December 2016, had at least one open violation, based on our analysis of the available EPA data. Violations are considered open when the state has not determined that a water system is in compliance with the specific requirement for which it received the violation. Table 7 provides an overview of the number of water systems (including schools and day care centers) with violations of the LCR, by size.

Table 8 provides a summary of the available EPA data on the violations of the LCR among schools and daycare centers.

Enforcement Actions and Outcomes

The available EPA data show that from July 1, 2011 to December 31, 2016, 99 percent of the 589,827enforcement actions and outcomes were taken by states, as would be expected given that states generally have primary responsibility for enforcement of the LCR . Enforcement actions in SDWIS/Fed include actions taken and what we considered in our analysis as outcomes, such as the receipt of information or a water system having achieved compliance. The enforcement codes in SDWIS/Fed that we defined as outcomes were: federal civil case concluded, federal bilateral compliance agreement signed, federal public notification received, federal no longer subject to rule, federal compliance achieved, federal variance/exemption issued, state civil case concluded, state bilateral compliance agreement signed, state public notification received, state no longer subject to rule, state compliance achieved, and state variance/exemption issued. Collectively, outcomes represented 43 percent (256,107) of the enforcement data in the database. Table 9 provides the number of enforcement actions and outcomes reported from July 1, 2011, to December 31, 2016, at the federal and state levels. The data show that states in regions 6 and 4 had the highest numbers of enforcement actions and outcomes.

Table 10 shows the five most frequently reported enforcement actions taken by states for LCR violations as they were reported in SDWIS/Fed as of December 31, 2016.

In a 2009 document outlining its enforcement policy, EPA stated that the policy would focus on “return to compliance.” According to this document, “return to compliance” is intended to show the effectiveness of the agency’s protection of public health. The available EPA data show that from July 1, 2011, to December 31, 2016, 10,702 water systems had at least one violation of some type and were returned to compliance (see table 11). As table 11 illustrates, small systems were most frequently designated as returned to compliance for monitoring and reporting violations.

Appendix III: Key Factors State Regulators Identified That May Contribute to Noncompliance with the Lead and Copper Rule

State drinking water regulators who participated in discussion groups we conducted identified 29 factors that may contribute to noncompliance with the Lead and Copper Rule (LCR). Of these factors, the state regulators most frequently mentioned size, technical capacity, and sample collection, among other factors. State regulators mentioned other factors less frequently, including requirements to comply with multiple drinking water regulations and the number of water samples required to be collected by the LCR as also contributing to noncompliance. Regulators in 12 states identified factors that they thought specifically helped water systems comply with the LCR. We obtained this information from drinking water state regulators representing 41 states and 1 territory through eight discussion groups held in September and October 2016. The purpose of the discussion groups was to develop an understanding of the factors that may influence noncompliance with the LCR. We analyzed the transcripts of those discussion groups using a content analysis software package. For a detailed description of the methodology we used to conduct these groups and analyze the content of these discussions, see appendix I and appendix IV.

State Regulators Identified 29 Factors That May Contribute to Noncompliance with the LCR

State regulators who participated in our discussion groups identified 29 factors that may contribute to water systems’ noncompliance with the LCR. The LCR requires water systems to identify locations where lead may be present and periodically obtain tap water samples from those locations (of which single-family homes are the highest priority). When a water system’s 90th percentile sample result for lead exceeds 15 parts per billion, the system has exceeded the federal action level (also known as an action level exceedance). Sample results that exceed the lead action level do not by themselves constitute violations of the LCR.

Under the LCR, an action level exceedance requires the water system and state to take additional steps. Those additional steps require that small and medium water systems install or modify corrosion control treatment and water systems of all sizes provide information (known as public education) about the harmful effects of lead to consumers and vulnerable populations (such as schools if the water system serves a school and public health departments). Water systems are also required to test and, if necessary, treat the source water. If, after installing corrosion control and treating source water, a system continues to have 90th percentile sample results that exceed the lead action level, the LCR requires the water system to begin replacing its lead service lines, if they exist.

As part of our analysis, we grouped the 29 factors into seven broad groups: (1) water system characteristics, (2) water system operations, (3) characteristics of water, (4) sample procedures required to comply with the LCR, (5) actions that states take to ensure water systems comply with the LCR, (6) actions that the Environmental Protection Agency (EPA) can take to assist with compliance, and (7) features of the LCR regulation. Figure 4 provides these seven groups and the factors that fell into each one.

State Regulators Most Frequently Identified 10 Factors That May Contribute to Noncompliance with the LCR

State regulators we interviewed most frequently identified 10 factors that may contribute to noncompliance of the LCR. Among those were size, technical capacity of operators, and the collection of drinking water samples. The 10 factors fell into the following broad groups: (1) water system characteristics, (2) water system operations, (3) characteristics of water, and (4) sample procedures. To identify the factors most frequently identified by the state regulators as contributing to noncompliance, we focused on those factors that were mentioned by regulators in at least 13 of the 41 states participating in the discussion groups (30 percent). Table 12 provides a description of each factor, the definition we used for our analysis and the number of states in which officials mentioned the factor.

State regulators who participated in the discussion groups explained how each of these factors may contribute to noncompliance. In most instances, regulators also described what they observed as relationships between factors and how, together, multiple factors could contribute to noncompliance.

Size. Regulators in 37 states said that the size of the population served by water systems may influence noncompliance with the LCR. Regulators in 28 of the 37 states said that small systems (serving populations of 3,300 and fewer) are more likely to have drinking water sample results that exceed the federal action level, to be in noncompliance, or face challenges that may lead to noncompliance. Most of these regulators mentioned the size of a system and the technical, managerial, or financial capacity of the system as factors that, together, may influence noncompliance. For example, regulators in 10 states said that small systems are more likely to receive a violation because they are generally less likely to have operators with the knowledge to properly collect samples (sample collection) or manage corrosion control treatment (technical capacity) or have the financial resources to pay for corrosion control treatment or to hire professional help to do so (financial capacity). A regulator from 1 state provided an example of a small water system in noncompliance because it has a part-time operator with little training on the rule and with other professional responsibilities, such as snow removal and animal control, which prevent this operator from providing drinking water test results to homeowners whose water was tested within the required timeframe.

Technical capacity. Regulators in 33 states said that the technical capacity of water systems may influence noncompliance with the LCR. Regulators in 18 of the 33 states said that water systems that do not have personnel with the knowledge to adequately operate a system or who understand the LCR are less likely to have the skill set to interpret and implement the LCR appropriately.

Sample collection. Regulators in 28 states said if systems fail to collect drinking water samples, improperly collect samples, or have other problems with collecting samples they may be out of compliance with the LCR. For example, regulators in 9 of these 28 states said that some water systems struggle to find enough homeowners willing to collect water samples for testing and regulators in 3 states said that this may cause the systems to collect samples from taps that are not used for drinking water, contrary to the LCR. In addition, regulators in 14 states said that even when systems are able to find homeowners willing to collect drinking water samples, the homeowners themselves may collect the samples improperly. A regulator from 1 state provided an example of a homeowner who was out of town for the weekend and upon return collected a water sample from tap water that sat stagnant for 4 days, which is problematic because the sample taken should be representative of everyday use. Sample results that exceed the lead action level do not by themselves constitute violations of the LCR.

Financial capacity. Regulators in 28 states said that water systems that do not have sufficient financial resources will experience challenges complying with the LCR, including paying for chemicals and professional help needed to install corrosion control treatment. For example, a regulator in 1 state said that a system without adequate financial resources may not be able to pay for the required, and often costly, corrosion control study.

Presence of lead. Regulators in 23 states said that the presence of lead in the pipes may influence noncompliance with the LCR. Regulators in 9 of the 23 states said that the presence of lead in pipes increases the likelihood that drinking water samples will exceed the federal action level; and require a system to perform additional actions. Regulators in 11 of the 23 states specifically said that a water system with old infrastructure is more likely have lead service lines. Regulators in 9 states identified the presence of lead service lines and managerial capacity as factors that may work together. These regulators said that water systems that maintain good records of the materials in their distribution systems know about the presence of lead service lines and may be better able to collect drinking water samples from the appropriate locations.

Managerial capacity. Regulators in 24 states said that the managerial capacity of water systems may influence noncompliance with the LCR. Regulators in 16 of the 24 states said that water systems that do not have effective management structures and practices will have problems keeping up with the rule requirements and deadlines. Regulators in 6 states explained that proper data and records management help systems comply with the LCR.

Water chemistry. Regulators in 23 states said that the chemistry of the water may influence noncompliance with the LCR. Regulators in 20 of the 23 states said that having corrosive water increases the likelihood of samples that exceed the action level—which is not a violation—and will require a system to perform additional actions. For example, if these systems do not install corrosion control or manage it properly—for example, because the operator does not understand water chemistry—the system will get a violation, according to regulators in 6 states.

Corrosion control. Regulators in 16 states said that the corrosion control may influence noncompliance with the LCR. Specifically, regulators in 15 of the 16 states said that water systems that have installed corrosion control treatment are more likely to be in compliance with the LCR because corrosion control is the primary method used to prevent lead from entering drinking water. A regulator from 1 state said that despite the corrosive water that exists in that state, water systems are not getting samples that exceed the action level and are staying in compliance because they have corrosion control installed. Regulators also discussed how the size of the system and corrosion control, together, can influence compliance. Regulators in 5 states said that large water systems are generally in compliance with the LCR because the rule requires them to install corrosion control treatment.

Type. Regulators in 18 states said that the type of water system may influence noncompliance with the LCR. Regulators in 14 of the 18 states said schools and daycare facilities with their own water supplies experience challenges in complying with the LCR, and regulators in 6 states explained that it is because their primary mission is not water delivery and management. A regulator in 1 state said that they had a school submit improper samples because school officials collected samples after the summer break during which the faucets had not been used for 6 weeks, thus not being representative of normal drinking water use. Sample results that exceed the lead action level do not by themselves constitute violations of the LCR.

Source. Regulators in 14 states said that the source of drinking water may influence noncompliance and offered a range of opinions as to how corrosive or non-corrosive groundwater may influence actions. State regulators frequently discussed source water, water chemistry, and corrosion control as factors that presented themselves together. State regulators in 4 states said that systems using ground water can more easily comply with the LCR because ground water is non- corrosive compared to surface water. In contrast, regulators in 4 states said that the ground water in other parts of the country is more corrosive. However, regulators in 5 states said that systems with corrosive water sources are still able to comply when they properly install and manage corrosion control treatment.

State Regulators Identified Additional Factors That May Influence Noncompliance

State regulators who participated in our discussion groups identified additional factors that may contribute to water systems’ noncompliance with the LCR, though less frequently. These regulators cited factors such as compliance with multiple drinking water rules, the number of samples that systems are required to collect under the LCR, and the complexity of the LCR. Table 13 describes each factor less frequently mentioned as contributing to noncompliance, the definition of the factor, and the number of states in which officials mentioned the factor. Some of these factors are more specific attributes that may impact of some the 10 factors that were most frequently identified by state regulators. For example, regulators told us about several aspects of water (age, stability, and flow) that may impact water chemistry.

The regulators participating in our discussion groups provided examples of these less frequently mentioned factors, below:

Simultaneous compliance. Regulators in nine states said that systems that have to simultaneously comply with multiple drinking water regulations can lead to noncompliance with the LCR and regulators in three of the nine states explained that this is because changes to water treatment to address one problem can create additional problems. For example, regulators in five states said that systems that have to comply with a rule aimed at reducing drinking water exposure to disinfection byproducts may require a reduction in the pH level of their water, and this, in turn, may affect the effectiveness of their control corrosion treatment. Much like other factors, this causes samples results to exceed the federal action level which subjects the water system to additional rule requirements.

Lead and Copper Rule. Regulators in seven states said that aspects of the LCR may influence noncompliance. Regulators in two of the seven states said that the LCR does not require states to routinely approve material surveys or for systems to update these surveys periodically, which can prevent water systems from knowing if they are collecting samples from high-risk sites. In addition, regulators in four states said that the LCR allows too much time for systems to complete requirements, such as the installation of corrosion control treatment and the issuance of public education notices to consumers. For example, a regulator in one state provided an example of a system that started the process of installing corrosion control. However, the system stopped the treatment installment because, as allowed by the LCR, the system sampled the water again and did not exceed the action level. Regulators in three of the states also said that the LCR does not allow state regulators to invalidate samples that they know were taken using poor practices at the sample site.,

EPA guidance. Regulators in seven states said that EPA’s guidance may contribute to noncompliance, and according to regulators in four states, this is because the guidance may not be clear which may cause states and water systems to incorrectly implement the LCR. For example, regulators in three states said that EPA guidance on sample procedures and public education was confusing for states and water systems because it is not clear about the timeframes that systems should adhere to when repeating the collection of water samples or providing public education to ensure that they conduct these actions properly and in accordance with the LCR.

Regulators also identified several additional factors that could lead to noncompliance or to 90th percentile sample results over the action level and thus additional requirements for water systems to implement, which could increase the chances of a violation. For example, regulators mentioned that the ownership of a water system could be a factor and provided the example of privately-owned, small water systems with less knowledgeable or available operators. Regulators also said that the age of the water can interfere with corrosion control and that some systems buying treated water are not doing any treatment themselves. Finally, regulators in two states said that water systems that are geographically isolated may not be able to access alternative water sources if their existing source water is corrosive or to attract operators with the skills to implement the LCR.

Regulators in 12 States Identified Operator Engagement, State Assistance and Local Support as Factors Helping Water Systems Comply with the LCR

Regulators in 12 states specifically identified factors that they thought helped water systems comply with the LCR (see table 14). Regulators in 7 states said that assistance from the states helps water systems comply with the LCR by providing systems with information about the requirements of the rule, training or technical assistance including using state rural water associations. Regulators in 4 different states said that the engagement of the water system with the state regulatory office—for example, through training—places the system in a better position to implement the LCR because they are gaining an understanding of the requirements. Further, regulators in 1 state said that support from state and local decision makers provides water system managers with the tools they need to implement the rule appropriately.

Appendix IV: Content Analysis of the Transcripts of Discussion Groups on Factors That May Contribute to Noncompliance with the Lead and Copper Rule

We conducted eight discussion groups with drinking water regulators representing 41 states and 1 territory to develop an understanding of the potential factors that may influence noncompliance with the Lead and Copper Rule (LCR). These were hour-long discussions conducted over the telephone. We held these discussion groups in September and October 2016. From two to eight states participated in each discussion group and each state had a primary designated spokesperson. For more information about our overall methodology, see appendix I.

Discussion Groups

In each discussion group, the moderator asked two questions. First, the moderator asked participants to list one or two factors that most influence a water system’s ability to comply with the LCR. Each state provided a list of factors. After all of the states responded, the moderator noted the factors provided by the group participants and asked for consensus on the list of factors reported. Second, the moderator asked participants to elaborate on how the factors reported could influence compliance. When necessary, the moderator asked probing questions to further clarify participants’ comments. Two or three analysts transcribed each session and combined and reconciled notes to develop transcripts for each of the discussion groups.

Identifying and Defining Factors

Using the factors that participants mentioned in each discussion group, we compiled an initial aggregate list of factors. We reviewed the initial list to determine if certain factors were closely related and could be combined. To check for completeness, we reviewed the transcripts and noted factors that participants repeatedly mentioned throughout the discussion groups but that were missing from the current list, and we added them to the list. This allowed us to delete some factors and incorporate them into other factors under which we determined they could reasonably fit. For example, we determined that “water chemistry” and “water corrosivity” were too closely related to be separate factors, so we combined them. Our goal was to develop a list of complete, distinct and mutually exclusive factors based on the information that participants shared in the discussion groups.

We took additional steps to ensure that we identified significant factors by conducting a word frequency count in all of the discussion group transcripts using a content analysis software package. We grouped similar wordsfor example, “system,” “systems and systems’” so that they were counted together. We determined that the top 11 words identified by the frequency count—which were mentioned 100 times or more—represented factors that we already had in our list. We also determined that the top 50 words identified by the frequency count — which were mentioned 32 times or more—represented factors that were already on our list.

To have a clear and consistent understanding of each factor for classification purposes, we defined each factor using information from the LCR, other federal regulations, Environmental Protection Agency guidance to states and water systems, and published GAO reports. Using the factors and their definitions, we developed a guide to use in the classification process.

To identify broad themes when classifying comments in the transcripts, we developed groups under which the factors could reasonably fit. We took steps to make every group distinct and mutually exclusive and to ensure that every factor fell into its associated category. For example, we determined the factors “system size,” “system type,” and “financial capacity of system,” could naturally be grouped under “water system characteristics”. Over the course of several meetings, four analysts reviewed and finalized the factors and their associated groups. We agreed on a final list of 29 factors, which included issues like “system size,” “corrosion control,” and “water source.” The 29 factors were placed under seven groups, including “water system characteristics,” “water system operations,” and “characteristics of water.” For a detailed discussion of these factors, see appendix III.

Analysis

To analyze the content of the discussion groups, using a content analysis software package, two analysts independently classified each comment in the transcripts into the factors we defined, and one analyst analyzed the classification to identify the factors that were most frequently reported. During the classification process, the analysts classified each participant’s individual statements in the transcripts separately. For the purposes of analysis, we defined an individual comment to be a statement made by a single individual. Across the eight discussion groups, there were a total of 225 such comments. Some comments were brief and covered a single issue while others were extensive and covered multiple issues. The analysts applied multiple classifications if statements covered a range of factors. For example, a statement made by a specific drinking water regulator could have been classified as relating to the source of the water and the size of the system. The analysts only coded statements made in response to the moderators’ questions. The analysts did not code statements that did not discuss factors or directly answer the moderators’ questions. After independently coding the transcripts, we used software to run an intercoder reliability report. The two analysts met on three occasions to compare and discuss the coding results. In instances where the analysts applied different codes to the same statement, they discussed their reasoning and reached agreement on which codes were the most appropriate. Each analyst then updated the database to reflect the agreements reached.

We also used software to identify the factors most frequently reported by the participants. We determined these by identifying the number of states that reported each factor (regardless of how many times a factor was mentioned) because this approach presented the number of states that agreed on the validity of each factor as contributing to noncompliance. To do this, we used software to cross-tabulate the factors that were classified with the states that participated in the discussion groups. During the classification process, we classified each participant’s statements as (1) the state the participant represented and (2) the factors that the statement covered. For example, an individual statement could have been classified as “Texas” and “water source.” Thus, for each factor, the cross tabulation showed which states made statements that were classified into that factor. We also ran a cross tabulation of the seven broad groups (under which the factors were grouped) and the states that reported each group. We identified the factors that were most frequently reported by focusing on those that were reported by at least 30 percent of the states, and reported this information in the report.

Appendix V: Statistical Analysis to Identify Water Systems with a Higher Likelihood for LCR Violations

To identify any factors that may contribute to noncompliance with the Lead and Copper Rule (LCR), we conducted discussion groups with a nonprobability sample of state drinking water regulators representing 41 states and 1 territory. We conducted a literature review of 31 academic studies about the detection of lead in drinking water and violations of drinking water regulations to corroborate our findings from the discussion groups. Our discussion groups with state regulators and review of academic studies suggested that certain factors could indicate whether water systems are at a higher likelihood for violating the LCR. To determine whether data on these factors could be used to predict LCR violations, we developed a series of statistical models, specifically multivariate logistic regression models. To conduct our analysis, we used the available data from the Environmental Protection Agency’s (EPA) Safe Drinking Water Information System (SDWIS/Fed) database for community water systems and non-transient non-community water systems active as of December 31, 2016, in 2 states, Ohio and Texas. We selected these states because recent EPA file reviews did not find significant discrepancies in the LCR violations data reported to SDWIS/Fed. In both states, we found that water systems with some factors were significantly more likely to violate the LCR than systems without those factors. Furthermore, we found that our models, which were based on data for 2013 and 2014, could predict systems with a higher likelihood of a violation in 2015 and 2016 significantly better than chance. Our analysis is limited because it is based on 2 states and thus, not generalizable to other states. It is also based on a subset of the relevant factors that might predict LCR violations and therefore is illustrative of the potential for statistical models to predict violations rather than the definitive model of violations.

Analytical Approach

In a review of previous GAO reports and peer-reviewed literature, we found that statistical models have been used to predict the risk of a violation for regulated entities. For example, in October 2016, we reported on the potential for statistical models to identify motor carriers that posed a high risk of a highway crash. In addition, several peer-reviewed studies have developed statistical models to predict the likelihood of drinking water systems violating Safe Drinking Water Act requirements. Based on this prior research, we considered predictive modeling as a potential approach to identify drinking water systems with a higher likelihood of violating the LCR. The specific steps we took to conduct this analysis are described below.

Available Data

To conduct our analysis, we used the available data on community water systems and non-transient non-community water systems active in SDWIS/Fed as of December 31, 2016. We analyzed data for drinking water systems that serve more than 25 people, which is EPA’s size threshold for a public drinking water system. EPA’s SDWIS/Fed database contains descriptive data on water systems (e.g., size, location, and water source), drinking water sample results, violations, and enforcement actions, as required by the LCR. Generally, states with primary enforcement responsibility initiate enforcement actions against water systems that do not comply with the LCR and other drinking water regulations. The LCR requires states to submit certain data to EPA’s SDWIS/Fed database on a quarterly basis.

Selected Factors

In our discussion groups with state regulators and review of academic studies and peer-reviewed literature, we identified 29 factors that may influence a drinking water system’s noncompliance with the LCR (see apps. I and III). We examined the SDWIS/Fed database to identify data elements that might represent these factors. Of the factors that were consistent with findings reported in the literature we reviewed and reported by state regulators in discussion groups, we selected four that were available in SDWIS/Fed to conduct a statistical analysis: the population served by (or size of) the drinking water system, whether the drinking water system was publicly or privately owned, whether the drinking water system used groundwater or surface water as a source, and whether the drinking water system was classified as a community water system or a non-transient non-community water system.

In addition, EPA’s current approach for targeting oversight of the LCR is to identify water systems with sample results that exceed the lead action level. Therefore, we also included the factor of whether the system had sample results exceeding the lead action level.

Data Reliability

We conducted steps to assess the reliability, completeness, and accuracy of the LCR compliance data in SDWIS/Fed for the purpose of conducting this analysis. We determined that data in SDWIS/Fed were not sufficiently reliable to conduct a nationwide statistical model of LCR violations. We could not verify that the limitations in the completeness of the data identified in our June 2011 report had been sufficiently addressed nationwide. Thus, we could not be assured that the LCR violations data submitted to SDWIS/Fed were sufficiently complete, accurate, or comparable across the states. Instead, we used the data in SDWIS/Fed to conduct an illustrative analysis for two statesOhio and Texas. We selected these states because EPA’s recent reviews of the completeness and accuracy of LCR data reported by these states did not find significant discrepancies in LCR violations data. We examined EPA’s reviews and either obtained written responses to questions or interviewed EPA and state officials to determine that these two states had sufficiently reliable data for this purpose. EPA’s reviews were not based on statistically representative samples of drinking water systems in these states. Therefore, we cannot conclude definitively that the agency has addressed problems with the completeness and accuracy of violations data. However, we found that these states had sufficiently reliable data for the purpose of testing the feasibility of statistical modeling to predict drinking water systems with a higher likelihood of violating the LCR.

Bivariate Analysis

Before developing the logistic regression model, we analyzed whether or not each water system in the two states violated the LCR from January 1, 2013, to December 31, 2014, with respect to each of the four selected factors. We conducted this analysis with cross tabulations and graphical analysis. In cross tabulations, each of the factors we examined was significantly associated with LCR violations, although the nature of these relationships varied between the states. In general, in both states, privately owned systems were more likely to violate the LCR than publically-owned systems, community water systems were less likely to violate the LCR than non-community non-transient water systems; and systems that had sample results exceeding the lead action level were more likely to violate the LCR than those that had not. In Ohio, water systems that used groundwater were more likely to violate the LCR than surface water systems, whereas, in Texas, water systems that used purchased groundwater were less likely to violate the LCR.

In graphical analysis, we found that the likelihood of a violation was related to the size of the population served by the water system. For example, in Ohio, grouped data plots displayed a negative, linear relationship between the likelihood of a violation and the number of people served by a system. In Texas, these plots displayed a negative, linear relationship for systems serving 3,300 people or fewer and a positive linear relationship for systems serving larger populations. The threshold of 3,300 people is the threshold that the LCR uses to distinguish small systems.

The results of these cross tabulations were illustrative of factors influencing violations, but they provided only a partial assessment of the relationship between LCR violations and the factors. This is because the cross tabulations compared LCR violations with each factor individually without accounting for the influence of the other factors. For example, we found in our analysis that while systems that had exceeded the lead action level were more likely to have a violation than systems that had not exceeded this level, such systems are also more likely to serve smaller populations. Because these factors are related, bivariate cross tabulations cannot distinguish between their respective influences on the likelihood that a system violated the LCR. To account for multiple factors simultaneously, we developed logistic regression models.

Logistic Regression Models

We developed a series of logistic regression models for each state to determine whether factors collectively could identify the likelihood that a water system would violate the LCR. A logistic regression model is an equation, which is developed through statistical procedures, that estimates the individual association of each factor with the likelihood of a violation, while simultaneously accounting for the association between each of the other factors and the likelihood of a violation. It provides a basis for combining multiple variables to predict outcomes and is more inclusive than the bi-variate analysis described in the previous section. ) for groups of systems ranked by the size of the population served. whether the drinking water system was classified as a community water system or a non-transient non-community water system.

We also included whether the system had a sample result exceeding the lead action level during the monitoring period from January 1, 2012, to December 31, 2014. We specified different logistic regression models for each state because of differences in the distributions of the data between the states. For example, in Ohio, nearly all non-transient, non-community water systems used ground water as their primary source water, which made it difficult to disentangle the unique effects of community water systems from those of ground water systems. Therefore, we collapsed non-community water systems using groundwater and non-community water systems not using ground water into a single group for analysis. In Texas, the relationship between the likelihood of a violation and size of the population served shifted as the number of people served by a system reached 3,300. Therefore, we added a term to our logistic regression equation for Texas that allowed us to account for the difference in the relationship between size and the likelihood of a violation or systems below the 3,300 threshold and systems above that threshold.

To test the adequacy of these models, we verified that our data contained a sufficient number of systems with each combination of characteristics, that it adequately fit the data based on the chi-squared goodness-of-fit tests, and that estimated effects were generally stable across multiple model specifications. We tested for nonlinear relationships between the likelihood of a violation and the size of the population served by a system and we transformed the variable accordingly. We also tested for interaction effects between the categorical system characteristics and the size of the population served by the system. In each state, we tested several model specifications to identify the combinations and transformations of variables that best met these conditions.

Among the models we tested, the best-fitting model for Ohio included explanatory variables for whether the system had a sample result exceeding the lead action level, whether the system was privately owned, the size of the population served by the system, whether the system was a non-community water system using groundwater, and whether the system was a non-community water system not using ground water. This model also included an interaction term between community water systems using groundwater and the size of the population served by the system. The data for this model included 1,849 systems, of which 137 violated the LCR in the compliance periods that began in 2013 and 2014 and 1,712 of which did not. The model had an adequate fit to the data based on chi-squared and Hosmer-Lemeshow goodness-of-fit tests, and it had a good accuracy in predicting LCR violations in 2013 or 2014 based on the area under the Receiver Operator Characteristic (ROC) curve.

The best-fitting model for Texas included explanatory variables for whether the system had a sample result exceeding the lead action level; whether the system was privately owned; whether the system was a community water system; whether the system used groundwater as a source of water; the size of the population served by the system; and a linear spline term, which accounted for a different relationship between system size and the likelihood of a violation for systems served more than 3,300 people. Data for this model included 5,395 systems, of which 2,321 violated the LCR in the compliance periods that began in 2013 and 2014 and 3,074 of which did not. The model had an adequate fit to the data based on the chi-squared and Hosmer-Lemeshow goodness–of-fit tests, and a moderate accuracy in predicting violations in 2013-14 based on the area under the ROC curve.

In each model, we found that certain factors were consistently associated with violations. For example, in both states, water systems that had reported a previous sample result exceeding the lead action level were significantly more likely to violate the LCR. The size of the population served was a statistically significant predictor of a violation in both states but in different ways. In Ohio, water systems were less likely to violate the LCR as their size increased. In Texas, water systems were less likely to violate the LCR as the size of their population increased to 3,300 but were more likely to violate the LCR as the size of the population over 3,300 increased. These patterns persisted in our models even after accounting for whether the system was privately owned, whether the system was a community water system and whether the system used groundwater as a source of water. These three other factors were also associated with the likelihood of a violation in some of our models, but the direction, magnitude and the significance of these associations were not consistent This could be the result of strong associations among the factors, which would make it difficult for the models to precisely estimate their association with violations. Because this imprecision, we do not report the associations between these three factors and the likelihood of a violation. Since the purpose of these models was to identify drinking water systems with a higher likelihood of a violation, rather than to estimate the influence of specific factors on the likelihood of a violation, we focus on the predictive accuracy of these models as described in the next section.

Predictive Accuracy

To test whether our models could be used to predict water systems with a higher likelihood of a future violation, we compared the predicted violation results from our models to actual violations that were reported in SDWIS/Fed for 2015 and 2016. Our models, which were based on data from 2013 and 2014, predicted subsequent violations in 2015 and 2016 significantly better than chance. Systems with higher average predicted probabilities of violations had higher observed rates of violations in the subsequent year than systems with lower predicted probabilities, and this difference was statistically significant. To make this determination, we took three steps. First, we used our models to estimate the likelihood that each system violated the LCR in 2013 or 2014 based on the factors identified in the logistic regression models. Second, we divided the water systems into five equally sized groups, referred to as quintiles, based on their estimated likelihood of a violation. Third, we compared the percentage of systems that violated the LCR in 2015 or 2016 across each of these five groups.

We found that systems in the highest likelihood group, based on our models of 2013 to 2014 data, had significantly higher violation rates in 2015 and 2016 as compared to systems in the lowest likelihood group. This result was true for each of the two states and for each of the models that we tested in those states. The tests of predictive accuracy in 2015 and 2016 for the best-fitting models in each state are shown in table 15. For example, in Ohio, 7.9 percent of systems in the fifth quintilethe group with the highest violation likelihood scoresviolated the LCR in 2015 or 2016 as compared to 2.4 percent of those in the first quintile, the group with the lowest likelihood scores. Similarly in Texas, 43.1 percent of systems with the highest likelihood scores violated the LCR in 2015 or 2016 as compared to 21.4 percent of those in the lowest likelihood group.

Discussion

Based on our illustrative analysis, we found that statistical models could be used to predict water systems with a higher likelihood of violating the LCR. However, our analysis was subject to certain limitations. First, our models used only data for factors available in SDWIS/Fed. They did not include other factors that might be important to predicting violations, such as the treatment technique used by a drinking water system; the presence of lead pipes in a community; or the technical, financial, and managerial capacity of a drinking water system. Second, our models were limited to the two states for which we could obtain reasonable assurances of data reliability, and therefore, the results are not generalizable to other states. While we found some commonalities in the factors that may contribute to violations between the states, we also found several differences between them, suggesting that specific factors may influence violations differently in different states. Finally, while we took several steps to confirm that the data for these states were sufficiently reliable for the purpose of developing illustrative regression models to predict violations, we cannot be confident that the inaccuracies and incompleteness that we and EPA identified in June 2011 have been addressed nationwide. Reliable and sufficient data for additional states would increase the external validity of future analysis. Additionally, data for additional explanatory variables mentioned in the literature and by state regulatorssuch as the presence of lead service lines and the technical, financial, and managerial capacity of a systemwould allow for a more fully specified model with the potential to increase the explanatory power of those models. Taken together, reliable data for a broader sample and a fuller range of explanatory variables could potentially improve the usefulness of models predicting LCR violations.

Appendix VI: EPA Guidance to the Public on Addressing Lead in Drinking Water

The Environmental Protection Agency (EPA) provides information on its website for the public on lead hazards in drinking water. EPA’s website includes, among other documents, a February 2005 fact sheet for the public entitled Is There Lead in My Drinking Water? (see fig. 5).

Appendix VII: Comments from the Environmental Protection Agency

Appendix VIII: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, Diane Raynes (Assistant Director); Jennifer Beddor; David Blanding, Jr.; Mark Braza; Richard P. Johnson; Tahra Nichols; Jerry Sandau, and Karen Villafana made key contributions to this report. In addition, Sarah Gilliland, Lindsay Juarez, Maureen Lackner, Dan Royer, and Kiki Theodoropoulos made important contributions to this report.

Related GAO Products

Water Infrastructure: Information on Selected Midsize and Large Cities with Declining Populations. GAO-16-785. Washington, D.C.: September 15, 2016.

Water Infrastructure: EPA and USDA Are Helping Small Water Utilities with Asset Management; Opportunities Exist to Better Track Results. GAO-16-237. Washington, D.C.: January, 27, 2016.

Environmental Health: EPA Has Made Substantial Progress but Could Improve Processes for Considering Children’s Health. GAO-13-254. Washington, D.C.: August 12, 2013.

Drinking Water: Unreliable State Data Limit EPA’s Ability to Target Enforcement Priorities and Communicate Water Systems’ Performance. GAO-11-381. Washington, D.C.: June 17, 2011.

Environmental Health: High-level Strategy and Leadership Needed to Continue Progress toward Protecting Children from Environmental Threats. GAO-10-205. Washington, D.C.: January 28, 2010.

Drinking Water: The District of Columbia and Communities Nationwide Face Serious Challenges in Their Efforts to Safeguard Water Supplies. GAO-08-687T. Washington, D.C.: April 15, 2008.

Drinking Water: EPA Should Strengthen Ongoing Efforts to Ensure That Consumers Are Protected from Lead Contamination. GAO-06-148. Washington, D.C.: January 4, 2006.

District of Columbia’s Drinking Water: Agencies Have Improved Coordination, but Key Challenges Remain in Protecting the Public from Elevated Lead Levels. GAO-05-344. Washington, D.C.: March 31, 2005.

Drinking Water: Safeguarding the District of Columbia’s Supplies and Applying Lessons Learned to Other Systems. GAO-04-974T. Washington, D.C.: July 22, 2004.