You are given two reports by the government agency.  

Report 1:
According to VA officials and Omaha donor group representatives, two main factors coalesced to become the impetus for the CHIP-IN Act. One factor was an Omaha donor group’s interest in constructing an ambulatory care center that could help address the needs of veterans in the area, given uncertainty about when or whether VA would be able to build a planned replacement medical center. In 2011, VA allocated $56 million for the design of the replacement medical center in Omaha, which had a total estimated cost of $560 million. However, VA officials told us that given the agency’s backlog of construction projects, the replacement medical center was not among its near-term projects. In the meantime, according to VA officials and the Omaha donor group, they discussed a change in the scope of the project— from the original plan of a replacement medical center to a smaller- scope project for a new ambulatory care center—that could potentially be constructed using the existing appropriation of $56 million plus a donation from the Omaha donor group. Another factor was the Congress’s and VA’s broader interest in testing innovative approaches to meeting VA’s infrastructure needs. According to VA officials, the agency was interested in constructing medical facilities in a more expeditious manner and developing legislation that allowed private money to help address VA’s needs. The CHIP-IN Act authorized a total of five pilot projects but did not name any specific project locations. Subsequently, the Omaha donor group applied to participate in the pilot program—with the construction of an ambulatory care center—and VA executed a donation agreement in April 2017. VA may accept up to four more real property donations under the pilot program, which is authorized through 2021. The CHIP-IN Act places certain requirements on donations under the pilot program. VA may accept CHIP-IN donations only if the property: (1) has already received appropriations for a VA facility project, or (2) has been identified as a need as part of VA’s long-range capital planning process and the location is included on the Strategic Capital Investment Planning process priority list provided in VA’s most recent budget submission to Congress. The CHIP-IN Act also requires that a formal agreement between VA and the non-federal entity provide that the entity conduct necessary environmental and historic preservation due diligence, obtain permits, and use construction standards required of VA, though the VA Secretary may permit exceptions. VA entered into an agreement with the Omaha donor group for the design and construction of an ambulatory care center in April 2017—4 months after enactment of the CHIP-IN Act. According to this agreement, which establishes the terms of the donation, the Omaha donor group will complete the design and construction of the facility and consult with VA. The facility will provide approximately 158,000 gross square feet of outpatient clinical functions, including primary care, an eye clinic, general purpose radiology and ambulatory surgery, specialty care, and mental health care. According to VA officials, planning for the facility began in April 2017, after the donation agreement was executed, and the project broke ground in April 2018. This donation agreement includes the mutually agreed- upon design and construction standards, which incorporate both VA’s standards and private sector building standards. The donation agreement also sets the terms of VA’s review of the design and construction documents and establishes escrow operations for the holding and disbursement of federal funds. Upon the Omaha donor group’s completion of the facility (scheduled for summer 2020) and VA’s acceptance, the Omaha donor group will turn the facility over to VA. The total estimated project cost is approximately $86 million. VA is contributing the $56 million that had already been appropriated for the design of the replacement medical facility. The Omaha donor group will donate the remaining approximately $30 million in private sector donations needed to build the facility. As shown in figure 2 and described below, VA officials told us that several offices are involved in various aspects of the CHIP-IN pilot—such as executing the Omaha project, seeking additional partnerships, and establishing the overall pilot program effort. The VA Office of Construction and Facilities Management (CFM) includes its Office of Real Property (ORP) and Office of Operations. ORP has taken a lead role in establishing the pilot program, while CFM Operations has led the execution of the Omaha project. Other VA offices that have been involved at different stages include the Office of General Counsel and the Secretary’s Center for Strategic Partnerships. Within the Veterans Health Administration (VHA), the local medical-center leadership was involved with developing the Omaha project, and the Office of Capital Asset Management, Engineering, and Support (Capital Asset Management Office) has contributed to efforts to identify additional projects. Some of these offices are involved with a steering committee created to implement the CHIP-IN Act (CHIP-IN steering committee). This steering committee met for the first time in September 2018. In 2016, we identified five leading practices for designing a well- developed and documented pilot program: articulating an assessment methodology, developing an evaluation plan, assessing scalability, and ensuring stakeholder communication. (See fig. 3.) These practices enhance the quality, credibility, and usefulness of pilot program evaluations and help ensure that time and resources are used effectively. While each of the five practices serves a purpose on its own, taken together, they form a framework for effective pilot design. VA officials have worked to communicate with relevant stakeholders, but have not yet established objectives, developed an assessment methodology and evaluation plan, or documented how they will make decisions about scalability of the pilot program. In 2016, we reported that clear, measurable objectives can help ensure that appropriate evaluation data are collected from the outset of a pilot program. Measurable objectives should be defined in qualitative or quantitative terms, so that performance toward achieving the objectives can be assessed, according to federal standards for internal control. For example, broad pilot objectives should be translated into specific researchable questions that articulate what will be assessed. Establishing well-defined objectives is critical to effectively implementing the other leading practices for a pilot program’s design. Objectives are needed to develop an assessment methodology to help determine the data and information that will be collected. Objectives also inform the evaluation plan because performance of the pilot should be evaluated against these objectives. In addition, objectives are needed to assess the scalability of the pilot, to help inform decisions on whether and how to implement a new approach in a broader context (i.e., whether the approach could be replicable in other settings). Relevant VA stakeholders have not yet collectively agreed upon and documented overall objectives for the CHIP-IN pilot program, but the stakeholders said they are planning to do so. However, at the time of our review, each of the VA offices we interviewed presented various ideas of what the objectives for the pilot should be, reflecting their varied missions and roles in the CHIP-IN pilot. For example, A senior VHA official said the objectives should include (1) determining whether the CHIP-IN donation partnership approach is an effective use of VA resources and (2) defining general principles for the pilot, including a repeatable process for future CHIP-IN projects. A senior VA official who has been closely involved with the pilot said one objective should be determining how VA can partner with the private sector for future construction projects, whether through donation partnerships or other means. Officials from ORP, who have taken a lead role in establishing the pilot, told us their objectives include identifying the four additional projects authorized by the CHIP-IN Act, developing a process to undertake potential projects, and determining whether a recommendation should be made that Congress extend VA’s CHIP-IN authority beyond the 5-year pilot. ORP officials said they have written some of these objectives in an early draft of plans for the CHIP-IN steering committee, but they have also discussed other objectives that are not yet documented. While the various VA offices involved may have somewhat different interests in the pilot program, developing a set of clear, measureable objectives is an important part of a good pilot design. For example, several VA officials who are involved in the pilot told us that it would be useful for relevant internal stakeholders to collectively agree upon and document overall objectives. ORP officials told us that the newly formed CHIP-IN steering committee will discuss and formalize objectives for the pilot. However, at the time of our review, a draft of these objectives had not been developed and a timeline for developing objectives was not yet established. A discussion of objectives was planned for the steering committee’s first meeting in September but had been rescheduled for the next meeting in October 2018. VA officials told us that they did not immediately move to establish a framework for the pilot program—which would include objectives for the pilot—for various reasons. Some officials said that VA and the Omaha donor group entered into formal discussions shortly after the CHIP-IN Act was enacted, and that their focus at the time was on negotiating and then executing a donation agreement for that particular project. As such, formal efforts to establish the framework for the overall pilot effort were in initial stages at the time of our review. ORP officials also said that the enactment of the CHIP-IN Act was not anticipated at the time CFM was planning and budgeting its resources for fiscal years 2017 and 2018, so work on the pilot had to be managed within available resources, largely as an additional duty for staff. In addition, a senior VHA official said a meeting to agree upon the pilot program’s objectives was needed but had not been held yet, noting that VA has competing priorities and vacancies at the senior executive level. ORP officials said they are now following project management principles in implementing the pilot. As part of this effort, they said that they intend to develop foundational documents for review by the CHIP-IN steering committee—such as a program plan containing objectives—but they have not done so yet. Without clearly defined and agreed-upon objectives, stakeholders within VA may have different understandings of the pilot’s purpose and intended outcomes. As a result, the agency risks pursuing projects that may not contribute to what VA hopes to learn or gain from the pilot. While VA officials are planning to establish objectives as they formalize the CHIP-IN steering committee, at the time of our review these objectives had not been documented and no timeline has been established for when they would be. Without clear, measurable objectives, VA will be unable to implement other leading practices for pilot design, such as determining how to make decisions about scalability. Further, not defining objectives in the near future would ultimately affect VA’s ability to evaluate the pilot and provide information to Congress about its results. We have reported that developing a clearly articulated assessment methodology and a detailed evaluation plan are leading practices for pilot design. The assessment methodology and evaluation plan should be linked to the pilot’s objectives so that evaluation results will show successes and challenges of the pilot, to help the agency draw conclusions about whether the pilot met its objectives. The assessment methodology and evaluation plan are also needed to determine scalability, because evaluation results will show whether and how the pilot can be expanded or incorporated into broader efforts. Given that several VA offices are involved in the pilot’s implementation, it is important for relevant stakeholders to be involved with defining and agreeing upon the assessment methodology and evaluation plan. VA has not yet fully developed and documented either an assessment methodology or evaluation plan for the pilot, but VA officials told us they plan to do so. For example, ORP officials said they intend to collect lessons learned and then evaluate the pilot at its end in 2021 by reviewing this information with relevant stakeholders. However, more specific details for this assessment methodology have not been defined in accordance with this leading practice. For example, we found that ORP has not yet determined which offices will contribute lessons learned, how frequently that information will be collected, or who will collect it. Similarly, details for an evaluation plan have not been defined, including who will participate in the evaluation and how information will be analyzed to evaluate the pilot’s implementation and performance. Now that the CHIP- IN steering committee has met for the first time, this group intends to discuss assessment of the pilot at a future meeting, but it is not clear when that discussion will occur, what leading practices will be considered, and when plans will be defined and documented. According to VA officials, an assessment methodology and evaluation plan have not been developed because, as discussed above, after the CHIP-IN Act was enacted, efforts were focused on negotiating the Omaha donation agreement and then executing that project. As such, formal efforts to establish the pilot through the CHIP-IN steering committee were in initial stages at the time of our review. Further, until VA has agreed- upon and documented objectives for the pilot program, it may be difficult to determine what information is needed for an assessment methodology and how the pilot will be evaluated. Unless VA establishes a clear assessment methodology that articulates responsibilities for contributing and documenting lessons learned, VA may miss opportunities to gather this information from the pilot. For example, while some stakeholders are documenting lessons learned relevant to their roles in the pilot, others are not. Specifically, ORP and CFM Operations are documenting lessons learned, but other VA offices and the Omaha donor group have not, though some told us they would be willing to share lessons learned if asked. Without an assessment methodology, there may also be confusion about who is responsible for documenting lessons learned. For example, a senior CFM official said that the Omaha donor group was compiling lessons learned from the pilot overall and would subsequently share those with VA. However, representatives from the donor group told us they have not been asked to share lessons learned with VA, but they would be willing to do so. When key individuals leave their positions—a situation that has occurred a number of times during implementation of the CHIP-IN pilot—their lessons learned may not be captured. For example, VA officials and donor group representatives told us that two VA officials who were involved in developing the pilot have since left the agency. In addition, stakeholders’ memories of lessons learned may fade unless they record them. Waiting to develop an evaluation plan—which should include details about how lessons learned will be used to measure the pilot’s performance—may ultimately affect VA’s preparedness to evaluate the pilot and provide information to Congress about its results. The purpose of a pilot is to generally inform a decision on whether and how to implement a new approach in a broader context—or in other words, whether the pilot can be scaled up or increased in size to a larger number of projects over the long term. Our prior work has found that it is important to determine how scalability will be assessed and the information needed to inform decisions about scalability. Scalability is connected to other leading practices for pilot design, as discussed above. For example, criteria to measure scalability should provide evidence that the pilot objectives have been met, and the evaluation’s results should inform scalability by showing whether and how the pilot could be expanded or how well lessons learned from the pilot can be incorporated into broader efforts. VA officials have begun to implement this leading practice by considering the pilot as a means of testing the viability of the donation partnership approach; however, plans for assessing scalability have not been fully defined and documented. A senior VA official said scalability is seen as a way to determine if the donation approach or other types of private sector partnerships are a viable way to address VA’s infrastructure needs. Similarly, ORP officials told us they are first considering scalability in terms of whether the CHIP-IN donation approach is an effective or feasible way of delivering VA projects. These officials said scalability will be largely determined by whether all five authorized projects can be executed before authorization for the CHIP-IN pilot program sunsets. For example, if VA can find four additional projects and execute donation agreements before the pilot’s authority expires, then potentially VA could seek congressional reauthorization to extend the program beyond the 5- year pilot. ORP officials are also considering scalability in terms of any changes to the program, such as incentives for donors, that could potentially increase its effectiveness. However, ORP officials explained that scalability may be limited because the types of projects that can be accomplished with the CHIP-IN donation approach may not be the projects that are most needed by VA. Along with other pilot design topics, the CHIP-IN steering committee intends to discuss scalability at a future meeting, but it is not clear when that discussion will occur. Thus, while VA officials have considered what scalability might look like, they have not fully determined and documented how to make decisions about whether the pilot is scalable. Since VA has not defined and documented the pilot’s objectives and its evaluation plans, it may be more difficult to determine how to make decisions about scalability. Considering how the pilot’s objectives and evaluation plans will inform decisions about scalability is critical to providing information about the pilot’s results. For example, at the end of the pilot, VA and Congress will need clear information to make decisions about whether the CHIP-IN donation approach could be extended beyond a pilot program, if any changes could enhance the program’s effectiveness, or if particular lessons learned could be applied to VA construction projects more broadly. Without clear information about scalability, VA may be limited in its ability to communicate quality information about the achievement of its objectives. Such communication is part of the federal standards for internal control. We have reported that appropriate two-way stakeholder communication and input should occur at all stages of the pilot, including design, implementation, data gathering, and assessment. To that end, it is critical that agencies identify who or what entities the relevant stakeholders are and communicate with them early and often. This process may include communication with external stakeholders and among internal stakeholders. Communicating quality information both externally and internally is also consistent with federal standards for internal control. VA has begun to implement this practice, with generally successful communication with the Omaha donor group. While VA has experienced some external and internal communication challenges about the pilot, officials have taken steps to help resolve some of these challenges. External communication. VA officials and representatives from the Omaha donor group generally described excellent communication between their two parties. For example, donor group representatives told us that in-person meetings helped to establish a strong relationship that has been useful in negotiating the donation agreement and executing the project to date. Further, VA officials and donor group representatives said that all relevant stakeholders—such as the donor group’s construction manager, general contractor, and architect, as well VA’s engineer, project manager, and medical center director—were included in key meetings once the Omaha project began, and said that this practice has continued during the construction phase. Although the Omaha donor group reported overall effective relations and communications with VA, donor group representatives noted that additional public relations support from VA would have been helpful. For example, after the CHIP-IN project was initiated in Omaha, the donor group encountered a public relations challenge when news reports about unauthorized waiting lists at the Omaha medical center jeopardized some donors’ willingness to contribute to the project. While donor group representatives said this challenge was addressed when the donor group hired a public relations firm, they also explained that it would be helpful for VA headquarters to provide more proactive public relations support to the local areas where future CHIP-IN projects are located. VA officials stated that they experienced some initial challenges communicating pilot requirements to external entities that are interested in CHIP-IN donation partnerships, but officials said that in response the agency has changed its outreach approach. As discussed below, the donation commitment aspect of the pilot can be a challenge. When interested entities contact VA to request information on the CHIP-IN pilot, VA officials told us they find the entities are often surprised by the donation commitment. For example, two entities that responded to VA’s RFI told us they were not clear about the donation requirement or the expected level of donation, or both. One respondent did not understand the pilot required a donation and would not provide an opportunity for a financial return on investment. Another respondent indicated that when they asked VA for clarification about the expected project’s scope, personnel from a headquarters office and the local VA medical center could not fully answer their questions. VA officials acknowledged these challenges and said they have changed their outreach efforts to focus on certain potential CHIP-IN locations, rather than RFIs aimed at a broader audience. Further, VA officials said that when speaking with potential donors going forward, they plan to involve a small group of officials who are knowledgeable about the pilot and its donation approach. Internal communication. While VA initially experienced some challenges in ensuring that all relevant internal stakeholders have been included in the pilot’s implementation, according to officials, the agency has taken recent steps to address this concern and involve appropriate internal offices. For example, officials from the Capital Asset Management Office said they could have assisted ORP in narrowing the list of potential projects in the RFIs but were not consulted. Later, after revising the marketing approach, ORP reached out to the Capital Asset Management Office and other relevant offices for help in determining priority locations for additional CHIP-IN projects, according to an ORP official. Officials from the Capital Asset Management Office told us that with improved engagement they were able to participate more actively in discussions about the pilot. In addition, initial plans for the CHIP-IN steering committee did not include VHA representation. However, in summer 2018 ORP expanded the planned steering committee to include VHA representatives, a plan that some other VA offices told us is needed to ensure that the pilot addresses the agency’s healthcare needs and that VHA offices are informed about pilot efforts. Based on the experience with the Omaha project, the CHIP-IN donation approach can result in potential cost and time savings—through the leveraging of private-sector funding, contracting, and construction practices—according to VA officials and the Omaha donor group. Regarding cost savings, one VA official stated that using donations makes VA’s appropriated funds available to cover other costs. In addition, based on the experience with the Omaha project, other VA officials told us that a CHIP-IN project can potentially be completed for a lower cost because of practices resulting from private sector leadership. Specifically, VA estimated that the Omaha ambulatory care center would cost about $120 million for VA to build outside of a donation partnership—as a standard federal construction project. Under the CHIP-IN pilot, however, the total estimated cost of the Omaha facility is $86 million—achieving a potential $34 million cost savings. Regarding time savings, CHIP-IN projects can potentially be completed at a faster pace because of the use of certain private sector practices and because projects can be addressed earlier than they otherwise would be, according to VA officials. The use of private-sector building practices can result in cost and time savings in a number of ways, according to VA officials and the Omaha donor group, as follows: The use of private-sector building standards contributed to cost savings for the Omaha project, according to VA officials and donor group representatives. VA and the donor group negotiated a combination of industry and VA building standards. A CFM official told us that using this approach and working with the private sector donor group encouraged the design team to think creatively about the risk assessment process and about how to meet the intent of VA’s physical security standards, but at a lower cost than if they were required to build a facility using all of VA’s building standards as written. For example, when assessing the safety and physical-security risk, the donor group and VA identified a location where two sides of the facility will not have direct exposure to the public or roadway traffic. Prohibiting exposure to roadways on two sides of the facility will mean spending less money to harden (i.e., protect) the facility against threats such as vehicular ramming. According to VA officials, using the combined standards did not compromise security on the Omaha project. Involving the general contractor early on in the design for the Omaha project, an approach VA does not typically take, contributed to both time and cost savings. VA officials told us that engaging the general contractor during the project’s design stage allowed the project to begin more quickly and was also helpful in obtaining information about costs and keeping the project within budget. However, VA officials said that depending on the project and contracting method used, it might not be possible to apply this contracting practice to VA construction projects outside of the pilot program. A private-sector design review method helped to save time. The Omaha donor group used a software package that allowed all design- document reviewers to simultaneously review design documents and then store their comments in a single place. VA officials said this approach was more efficient than VA’s typical review method and cut about 18 weeks from the project’s timeline. VA officials also said use of this software was a best practice that could be applied to VA construction projects more broadly. In addition, the donor group and VA employed fewer rounds of design reviews than VA typically uses; this streamlining also helped to save time during the design process, according to VA officials. Further, VA officials said that the CHIP-IN donation approach can allow VA to address projects more quickly because they are addressed outside of VA’s typical selection and funding process. For example, VA officials told us that because of the agency’s current major construction backlog, using the CHIP-IN donation approach allowed work on the Omaha project to begin at least 5 years sooner than if the CHIP-IN approach had not been used. The Omaha project’s priority was low relative to other potential projects, so that it was unlikely to receive additional funding for construction for several years. For example, one agency official noted that even if the project was at the top of VA’s priorities, there is a backlog of 20 major construction projects worth $5 billion ahead of it—meaning the Omaha project would probably not be addressed for at least 5 years. VA officials also told us that as they consider future CHIP-IN projects, they are looking for other projects that, like the one in Omaha, are needed, but may not be a top priority given available funding and could be moved forward with a private sector donation. In addition, use of the CHIP-IN donation approach and decision to pursue an ambulatory care center contributed to an earlier start on a project to address veterans’ needs. However, as mentioned earlier, VA officials said that future construction projects will be necessary to address some needs that were part of the original replacement medical center plan. A main challenge to establishing pilot partnerships is the reliance on large philanthropic donations, according to VA officials, the Omaha donor group, and RFI respondents. In general, the potential donor pool may not be extensive given the size of the expected donations—in some cases tens or hundreds of millions of dollars—and the conditions under which the donations must be made. For example, as discussed earlier, VA officials said that when interested entities contact them about the pilot, they are often surprised by the donation commitment. When we spoke with two entities that responded to VA’s RFI, one told us that they “could not afford to work for free” under the pilot while another told us that developers are more likely to participate in the pilot if they see an incentive, or a return on their financial contribution. Also, VA officials told us that some potential project locations have not received any appropriations—making the projects’ implementation less appealing to potential donors. The Omaha donor group noted that a VA financial contribution at or above 50 percent of a project’s estimated cost is essential for demonstrating the agency’s commitment and for leveraging private-sector donations. To address challenges involving the philanthropic nature of the pilot, ORP officials told us that VA has tried to identify strategies or incentives that could encourage donor involvement. For example, the CHIP-IN steering committee is considering what incentives might be effective to encourage greater participation. One ORP official told us that such incentives could include potential naming opportunities (that is, authority to name items such as facility floors, wings, or the actual facility), although offering such incentives may require changes in VA’s authority. Further, because it may be difficult to secure donations for larger, more costly projects, some VA officials, donor group representatives, and one RFI respondent we spoke to suggested that VA consider developing less costly CHIP-IN projects—giving VA a better chance of serving veterans by filling gaps in service needs. Other VA officials, however, said they wanted to focus on larger projects because the pilot allows only five projects. Another challenge is that VA generally does not possess marketing and philanthropic development experience. VA officials told us that this makes the inherent challenge of finding donors more difficult. While VA officials have used the assistance of a nonprofit entity that has marketing expertise, they also said that going forward it would be helpful to have staff with relevant marketing and philanthropic development experience to assist with identifying donors. VA officials said this expertise could possibly be acquired through hiring a contractor, but funding such a hire may be difficult within their existing resources. As discussed above, the CHIP-IN pilot presents an uncharted approach to VA’s implementation of projects, and using CHIP-IN has aspects of an organizational transformation in property acquisition for the agency because it leverages donation partnerships and streamlines VA’s typical funding process. We have found that a key practice of organizational transformation includes a dedicated implementation team to manage the transformation process and that leading practices for cross-functional teams include clear roles and responsibilities, and committed members with relevant expertise. VA officials and Omaha donor group representatives acknowledged that a dedicated CHIP-IN team could help focus pilot implementation—and that no such team existed within the agency. ORP officials told us that the newly formed CHIP-IN steering committee would provide the necessary leadership for pilot implementation. They anticipate that a working group will be part of the committee and serve as a dedicated team for the pilot. However, as discussed below, roles and responsibilities have not been defined and staff resource decisions have not been made. Clear and documented roles and responsibilities. Several VA officials told us that responsibility for managing the overall pilot effort had not been assigned, and that they had different interpretations of which office had responsibility for leading the pilot. Some officials identified ORP as the leader, while others thought it was CFM or the Center for Strategic Partnerships. One CFM official told us that a clear definition of responsibilities is needed under the pilot along with a dedicated office or person with the ability to make decisions when an impasse across offices exists. Similarly, a senior VHA official told us that leadership roles and responsibilities for the pilot are not fully understood within the agency, which has made establishing partnerships under the pilot a challenge. For example, both VA officials and Omaha donor group representatives identified the lack of a senior-level leader for the pilot as a challenge and emphasized the need for strong pilot leadership going forward. Now that a CHIP-IN steering committee is being formed to provide pilot leadership, ORP officials intend to discuss committee members’ roles and responsibilities. This discussion was planned for the first committee meeting but was rescheduled for the next meeting in October 2018. ORP officials, however, told us that they do not expect to assign individual members’ roles and responsibilities until a future date. VA officials did not have a timeline for when committee or individual members’ roles and responsibilities would be formally documented. ORP officials said that roles and responsibilities for the pilot have not been defined because after enactment of the CHIP-IN Act, their first priority was to engage the Omaha donor group and negotiate an agreement. Later, after the Omaha project was progressing, ORP officials said they turned their attention to formalizing the pilot program and identifying additional donation partnerships. While it is important to concentrate on completion of individual projects, it is also important to plan for the overall pilot’s implementation—to help ensure that the pilot’s purpose and goals are met and in a timely manner. We have found that clarifying roles and responsibilities is an important activity in facilitating strong collaboration and building effective cross-functional teams. In addition, we have found that articulating roles and responsibilities is a powerful tool in collaboration and that it is beneficial to detail such collaborations in a formal, written document. Committed team members. Various VA offices and staff members have worked on the CHIP-IN pilot in addition to their other responsibilities, but several VA officials told us the resources currently dedicated to the pilot are insufficient. During our review, an ORP official told us that two ORP staff each spent about 4 to 6 hours per week on the pilot, as collateral duties. However, since that time, one of these two staff members has left the agency. A senior VA official told us that ORP and the Center for Strategic Partnerships could each use two to three more dedicated staff members to work solely on the pilot. While one ORP official said that additional staff would likely be assigned after other CHIP-IN projects are identified, a Center for Strategic Partnerships official said a specified percentage of staff time should be dedicated now to identifying potential donors. As mentioned above, VA officials told us they anticipate a working group will be part of the CHIP-IN steering committee and will serve as the dedicated team to implement the pilot. However, VA has not yet documented how it will staff the working group, including how it will obtain the needed expertise within its existing resources. According to one VA official, staff had not been initially dedicated to the pilot because the CHIP-IN Act did not provide resources to fund a dedicated team for the pilot, so VA has needed to implement the pilot within its existing resources. This VA official also told us that they were not certain VA could support a dedicated team with existing resources. Another official indicated that VA would need to consider how to incorporate CHIP-IN into the agency’s operations if the pilot program were expanded beyond the initial pilot and then dedicate needed resources. Dedicating a strong and stable implementation team is important to ensuring that the effort receives the focused, full-time attention needed. Team members with relevant knowledge and expertise. As previously discussed, VA officials told us that it would be helpful for a CHIP-IN team to include stakeholders with certain expertise, such as marketing and philanthropic development experience. In addition, representatives from the Omaha donor group said going forward, proactive public relations expertise is needed from VA headquarters (in particular, for external communications outside of the partnership) to quickly and positively address any incidents that could negatively impact VA’s ability to encourage donor participation in the pilot at the local level. For example, in the event of critical news reports about a local VA facility, such as what occurred in Omaha, donor group representatives said that additional public relations support would be helpful. VA officials also told us that a CHIP-IN team should be a collaborative effort across several offices. Specifically, one senior VA official said a cross-functional team with representation from ORP, CFM Operations, the Center for Strategic Partnerships, VHA, and the Office of Asset Enterprise Management (which has budget and finance expertise) would be useful in focusing and implementing the pilot. Leading practices for cross-functional teams include having members with a wide diversity of knowledge and expertise. Having a dedicated team or working group that consists of committed members with clear roles and responsibilities could assist VA in implementing the CHIP-IN pilot. For example, the working group could focus time and attention on strengthening design of the pilot program as a whole, instead of implementing projects on a piecemeal basis. Further, clearly identifying and documenting roles and responsibilities could help relevant stakeholders define and agree upon pilot objectives as well as an assessment methodology and evaluation plan. In addition, including stakeholders with relevant expertise on the dedicated team may assist VA in identifying viable projects and negotiating partnership agreements more readily. The CHIP-IN pilot is a unique, time-limited opportunity for VA to test a new way of building needed medical facilities by using non-federal funding sources—donors—to leverage federal funds. Though the first project is still under way, stakeholders have already noted benefits of the donation partnership approach, including potential cost and time savings as well as learning about private sector practices that could be applied more broadly to VA construction. However, VA is not yet collecting the information it needs to support decisions by VA or Congress about the pilot. Without a strengthened pilot design—including measurable objectives, an assessment methodology, and an evaluation plan—that can help inform decisions about the scalability of the pilot, it may not be clear to VA and Congress whether the CHIP-IN approach could be part of a longer-term strategy or how lessons learned could enhance other VA construction efforts. While leadership for the pilot had not been previously assigned, a newly formed CHIP-IN steering committee is meant to focus on the pilot’s implementation. Defining and documenting roles and responsibilities for this committee—and identifying the resources needed to effectively implement the pilot—could assist VA in partnering with additional donors and creating new opportunities to meet the urgent needs of veterans. We are making the following three recommendations to VA. The Secretary of VA should ensure that internal stakeholders—such as the CHIP-IN steering committee’s members—agree to and document clear, measurable objectives for the CHIP-IN pilot that will help inform decisions about whether and how to scale the program. (Recommendation 1) The Secretary of VA should ensure that internal stakeholders—such as the CHIP-IN steering committee’s members—develop an assessment methodology and an evaluation plan that are linked to objectives for the CHIP-IN pilot and that help inform decisions about whether and how to scale the program. (Recommendation 2) The Secretary of VA should ensure that the CHIP-IN steering committee documents the roles and responsibilities of its members and identifies available staff resources, including any additional expertise and skills that are needed to implement the CHIP-IN pilot program. (Recommendation 3) We provided a draft of this report to VA for comment. In its written comments, reproduced in appendix I, VA concurred with our recommendations and stated that it has begun or is planning to take actions to address them. VA also provided a general comment on the role of VHA in the CHIP-IN pilot, which we incorporated in our report. We are sending copies of this report to the appropriate congressional committees, the Secretary of Veterans Affairs, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions regarding this report, please contact me at (213) 830-1011 or vonaha@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix II. In addition to the contact named above, Cathy Colwell (Assistant Director), Kate Perl (Analyst in Charge), Melissa Bodeau, Jennifer Clayborne, Peter Del Toro, Shirley Hwang, Terence Lam, Malika Rice, Crystal Wesco, and Elizabeth Wood made key contributions to this report.

 

 Report 2:
While HUD has primary responsibility for addressing lead paint hazards in federally-assisted housing, EPA also has responsibilities related to setting federal lead standards for housing. EPA sets federal standards for lead hazards in paint, soil, and dust. Additionally, EPA regulates the training and certification of workers who remediate lead paint hazards. CDC sets a health guideline known as the “blood lead reference value” to identify children exposed to more lead than most other children. As of 2012, CDC began using a blood lead reference value of 5 micrograms of lead per deciliter of blood. For children whose blood lead level is at or above CDC’s blood lead reference value, health care providers and public health agencies can identify those children who may benefit the most from early intervention. CDC’s blood lead reference value is based on the 97.5th percentile of the blood lead distribution in U.S. children (ages 1 to 5), using data from the National Health and Nutrition Examination Survey. Children with blood lead levels above CDC’s blood lead reference value have blood lead levels in the highest 2.5 percent of all U.S. children (ages 1 to 5). HUD, EPA, and the Department of Health and Human Services (HHS) are members of the President’s Task Force on Environmental Health Risks and Safety Risks to Children. HUD co- chairs the lead subcommittee of this task force with EPA and HHS. The task force published the last national lead strategy in 2000. The primary federal legislation to address lead paint hazards and the related requirements for HUD is the Residential Lead-Based Paint Hazard Reduction Act (Title X of the Housing and Community Development Act of 1992). We refer to this law as Title X throughout this report. Title X required HUD to, among other things, promulgate lead paint regulations, implement the lead hazard control grant programs, and conduct research and reporting, as discussed throughout this report. The two key regulations that HUD has issued under Title X are the Lead Disclosure Rule and the Lead Safe Housing Rule: Lead Disclosure Rule. In 1996, HUD and EPA jointly issued the Lead Disclosure Rule. The rule applies to most housing built before 1978 and requires sellers and lessors to disclose any known information, available records, and reports on the presence of lead paint and lead paint hazards and provide an EPA-approved information pamphlet prior to sale or lease. Lead Safe Housing Rule. In 1999, HUD first issued the Lead Safe Housing Rule, which applies only to housing receiving federal assistance or federally-owned housing being sold. The rule established procedures for evaluating whether a lead paint hazard exists, controlling or eliminating the hazard, and notifying occupants of any lead paint hazards identified and related remediation efforts. The rule established an “elevated blood lead level” as a threshold that requires landlords and PHAs to take certain actions if a child’s blood test shows lead levels meeting or exceeding this threshold. In 2017, HUD amended the rule to align its definition of an “elevated blood lead level” with CDC’s blood lead reference value. This change lowered the threshold that generally required landlords and PHAs to act from 20 micrograms to 5 micrograms of lead per deciliter of blood. According to the rule, when a child under age 6 living in HUD-assisted housing has an elevated blood lead level, the housing provider must take several steps. These generally include testing the home and other potential sources of the child’s lead exposure within 15 days, ensuring that identified lead paint hazards are addressed within 30 days of receiving a report detailing the results of that testing, and reporting the case to HUD. Office of Lead Hazard Control and Healthy Homes (Lead Office). HUD’s Lead Office is primarily responsible for administering HUD’s two lead hazard control grant programs, providing guidance on HUD’s lead paint regulations, and tracking HUD’s efforts to make housing lead-safe. The Lead Office collaborates with HUD program offices on its oversight and enforcement of lead paint regulations. For instance, the Lead Office issues guidance, responds to questions about requirements of lead paint regulations, and provides training and technical assistance to HUD program staff, PHA staff, and property owners. The Lead Office’s oversight efforts also include maintaining email and telephone hotlines to receive complaints and tips from tenants or homeowners, among others, as they pertain to lead paint regulations. Additionally, the Lead Office, in collaboration with EPA, contributes to the operation of the National Lead Information Center––a resource that provides the general public and professionals with information about lead, lead hazards, and their prevention. Office of Public and Indian Housing (PIH). HUD’s PIH oversees and enforces HUD’s lead paint regulations for the rental assistance programs. As discussed earlier, this report focuses on the two largest rental assistance programs serving the most families with children––the Housing Choice Voucher and public housing programs. Housing Choice Voucher program. In the voucher program, eligible families and individuals are given vouchers as rental assistance to use in the private housing market. Generally, eligible families with vouchers live in the housing of their choice in the private market. The voucher generally pays the difference between the family’s contribution toward rent and the actual rent for the unit. Vouchers are portable; once a family receives one, it can take the voucher and move to other areas where the voucher program is administered. In 2017, there were roughly 2.5 million vouchers available. Public housing program. Public housing is reduced-rent developments owned and operated by the local PHA and subsidized by the federal government. PHAs receive several streams of funding from HUD to help make up the difference between what tenants pay in rent and what it costs to maintain public housing. For example, PHAs receive operating and capital funds through a formula allocation process. PHAs use operating funds to pay for management, administration, and day-to-day costs of running a housing development. Capital funds are used for modernization needs, such as replacing roofs or remediating lead paint hazards. According to HUD rules, generally families that are income-eligible to live in public housing pay 30 percent of their adjusted income toward rent. In 2017, there were roughly 1 million public housing units available. For both of these rental assistance programs, the Office of Field Operations (OFO) within PIH oversees PHAs’ compliance with lead paint regulations, in conjunction with HUD field office staff. The office has a risk-based approach to overseeing PHAs and performs quarterly risk assessments. Also within PIH, staff from the Real Estate Assessment Center are responsible for inspecting the physical condition of public housing properties. Office of Policy Development and Research (PD&R). HUD’s PD&R is the primary office responsible for data analysis, research, and program evaluations to inform the development and implementation of programs and policies across HUD offices. of the total grant amount, while the Lead Hazard Reduction Demonstration grant program has required at least a 25 percent match. For fiscal years 2013–2017, HUD awarded $527 million for its lead hazard control grants, which included 186 grants to state and local jurisdictions (see fig. 1). In these 5 years, about 40 percent of grants awarded went to jurisdictions in the Northeast and 31 percent to jurisdictions in the Midwest––regions of the country known to have a high prevalence of lead paint hazards. Additionally, in these 5 years, 90 percent of grant awards went to grantees at the local jurisdiction level (cities, counties, and the District of Columbia). The other 10 percent of grant awards went to state governments. During this time period, HUD awarded the most grants to jurisdictions in Ohio (17 grants), Massachusetts and New York (15 grants each), and Connecticut (14 grants). HUD’s Lead-Based Paint Hazard Control grant and the Lead Hazard Reduction Demonstration grant programs have incorporated Title X statutory requirements through recent annual funding notices and their grant processes. Title X contains applicant eligibility requirements and selection criteria HUD should use to award lead grants. To be eligible to receive a grant, applicants need to be a state or local jurisdiction, contribute matching funds to supplement the grant award, have an approved comprehensive affordable housing strategy, and have a certified lead abatement program (if the applicant is a state government). HUD has incorporated these eligibility requirements in its grant programs’ 2017 funding notices, which require applicants to demonstrate that they meet these requirements when they apply for a lead grant. According to the 2017 funding notices, applicants must detail the sources and amounts of their matching contributions in their applications. Similarly, applicants must submit a form certifying that the proposed grant activities are consistent with their local affordable housing strategy. HUD’s 2017 funding notices state that if applicants did not meet these eligibility requirements, HUD would not consider their applications. Additionally, Title X requires HUD to award lead grants according to the following applicant selection criteria: the extent to which an applicant’s proposed activities will reduce the risk of lead poisoning for children under the age of 6; the degree of severity and extent of lead paint hazards in the applicant’s jurisdiction; the applicant’s ability to supplement the grant award with state, local, or private funds; the applicant’s ability to carry out the proposed grant activities; and other factors determined by the HUD Secretary to ensure that the grants are used effectively. In its 2017 funding notices, HUD incorporated the Title X applicant selection criteria through five scoring factors that it used to assess lead grant applications. HUD allocated a certain number of points to each scoring factor. Applicants are required to develop their grant proposals in response to the scoring factors. When reviewing applications, HUD staff evaluated an applicant’s response to the factors and assigned points for each factor. See table 1 for a description of the 2017 lead grant programs’ scoring factors and points. As shown in table 1, HUD awarded the most points (46 out of 100) to the “soundness of approach” scoring factor, according to HUD’s 2017 funding notices. Through this factor, HUD incorporated Title X selection criteria on an applicant’s ability to carry out the proposed grant activities and supplement a grant award with state, local, or private funds. For example, HUD’s 2017 funding notices required applicants to describe their detailed plans to implement grant activities, including how the applicants will establish partnerships to make housing lead-safe. Specifically, HUD began awarding 2 of the 100 points to applicants who demonstrated partnerships with local public health agencies to identify families with children for enrollment in the lead grant programs. Additionally, HUD asked applicants to identify partners that can help provide assistance to complete the lead hazard control work for high-cost housing units. Furthermore, HUD required applicants to identify any nonfederal funding, including funding from the applicants’ partners. Appendix I includes examples of state, local, and nongovernmental funds that selected grantees planned to use to supplement their lead grants. In its lead grant programs, HUD has taken actions that were consistent with OMB’s requirements for competitively awarded grants. OMB generally requires federal agencies to: (1) establish a merit-review process for competitive grants that includes the criteria and process to evaluate applications; and (2) develop a framework to assess the risks posed by applicants for competitive grants, among other things. Through a merit-review process, an agency establishes and applies criteria to evaluate the merit of competitive grant applications. Such a process helps to ensure that the agency reviews grant applications in a fair, competitive, and transparent manner. Consistent with the OMB requirement to establish a merit review process, HUD has issued annual funding notices that communicate clear and explicit evaluative criteria. In addition, HUD has established processes for reviewing and scoring grant applications using these evaluative criteria, and selects grant recipients based on the review scores (see fig. 2). For example, applicants that score at or above 75 points are qualified to receive awards from HUD. Also, HUD awards funds beginning with the highest scoring applicant and proceeds by awarding funds to applicants in a descending order until funds are exhausted. Furthermore, consistent with the OMB requirement to develop a framework to assess applicant risks, HUD has developed a framework to assess the risk posed by lead grant applicants by, among other things, deeming ineligible those applicants with past performance deficiencies or those that do not have a financial management system that meets federal standards. However, HUD has not fully documented or evaluated its lead grant processes in reviewing and scoring the grants and making award decisions: Documenting grant processes and award decisions. While HUD has established processes for its lead grant programs, it lacks documentation, including detailed guidance to help ensure that staff carry out processes consistently and appropriately. Federal internal control standards state that agency management should develop and maintain documentation of its internal control system. Such documentation assists agency management by establishing and communicating the processes to staff. Additionally, documentation of processes can provide a means to retain organizational knowledge and communicate that knowledge as needed to external parties. The Lead Office’s Application Review Guide describes its grant application review and award processes at a high level but does not provide detailed guidance for staff as to how tasks should be performed. For example, the Guide notes that reviewers score eligible applications according to factors contained in the funding notices but does not describe how the reviewers should allocate points to the subfactors that make up each factor. Lead Office staff told us that creating detailed scoring guidance would be challenging because applicants’ proposed grant activities differ widely, and they said that scoring grant applications is a subjective process. While scoring grant applications may involve subjective judgments, improved documentation of grant review and scoring processes, including additional direction to staff, can help staff apply their professional judgment more consistently in evaluating applications. By better documenting processes, HUD can better ensure that staff evaluate applications consistently. Additionally, HUD has not fully documented its rationale for deciding which applicants receive lead grant awards and for deciding the dollar amounts of grant awards to successful applicants. In prior work examining federal grant programs, one recommended practice we identified is that agencies should document the rationale for award decisions, including the reasons individual applicants were selected or not and how award funding amounts were determined. While HUD’s internal memorandums listed the applicants selected and the award amounts, these memorandums did not document the rationale for these decisions or provide information sufficient to help applicants understand award outcomes. Lead Office staff told us that most grantees have received the amount of funding they requested in their applications, which was generally based on HUD’s maximum grant award amount. Lead Office staff said they could use their professional judgment to adjust award amounts to extend funding to more applicants when applicants received similar scores. However, the Lead Office’s documentation we reviewed did not explain this type of decision making. For example, in 2017, when two applicants received identical scores on their applications, HUD awarded each applicant 50 percent of the remaining available funds rather than awarding either applicant the amount they requested. Representatives of one of the two grantees told us they did not know why the Lead Office had not provided them the full amount they had requested. Lead Office staff told us that, to date, HUD has not considered alternative ways to award grant funding amounts. By fully documenting grant award processes, including the rationale for award decisions and amounts, HUD could provide greater transparency to grant applicants about its grant award decisions. Evaluating processes. HUD lacks a formal process for reviewing and updating its lead grant funding notices, including the factors and point allocations used to score applications. Federal internal control standards state that agencies should implement control activities through policies and that periodic review of policies and procedures can provide assurance of their effectiveness in achieving the agency’s objectives. Lead Office staff told us that previous changes to the factors and point allocation used to score applicants have been made based on informal discussions among staff. However, the Lead Office does not have a formal process to review and evaluate the relevance and appropriateness of the factors or points used to score applicants. Lead Office staff told us that they have never analyzed the scores applicants received for the factors to identify areas where applicants may be performing well or poorly or to help inform decisions about whether changes may be needed to the factors or points. Additionally, HUD has not changed the threshold criteria used to make award decisions since the threshold was established in 2003. As previously shown in figure 2, applicants who received at least 75 points (out of 100) have been qualified to receive a grant award. However, HUD grant documentation, including the funding notices and the Application Review Guide, does not explain the significance of this 75-point threshold. Lead Office staff stated that this threshold was first established in 2003 by HUD based on OMB guidance. A formal review of this 75-point threshold can help HUD determine whether it remains appropriate for achieving the grant programs’ objectives. Furthermore, by periodically evaluating processes for reviewing and scoring grant applications, HUD can better determine whether these processes continue to help ensure that lead grants reach areas of the country at greater risk for lead paint hazards. HUD has begun to develop analyses and tools to inform its efforts to target outreach and ensure that grant awards go to areas of the country that are at risk for lead paint hazards. However, HUD has not developed time frames for incorporating the results of the analyses into its lead grant programs’ processes. HUD has required jurisdictions applying for lead grants to include data on the need or extent of the problem in their jurisdiction (i.e., scoring factor 2). Additionally, Lead Office staff told us that HUD uses information from the American Healthy Homes Survey to obtain information on lead paint hazards across the country. However, the staff explained that the survey was designed to provide meaningful results at the regional level and did not include enough homes in its sample to provide information about housing conditions, such as lead paint hazards, at the state or local level. Because HUD awards lead grants to state and local jurisdictions, it cannot effectively use the survey results to help the agency make award decisions or inform decisions about areas for potential outreach. In early 2017, the Lead Office began working with PD&R to develop a model to identify local jurisdictions (at the census-tract level) that may be at heightened risk for lead paint hazards. Lead Office staff said that they hope to use results of this model to develop geographic tools to help target HUD funding to areas of the country at risk for lead paint hazards but not currently receiving a HUD lead grant. Lead Office staff said that they could reach out to these at-risk areas, help them build the capacity needed to administer a grant, and encourage them to apply. For example, HUD has identified that Mississippi and two major metropolitan areas in Florida (Miami and Tampa) had not applied for a lead grant. HUD has conducted outreach to these areas to encourage them to apply for a lead grant. In 2016, the City of Jackson, Mississippi, applied for and received a lead grant. Though the Lead Office has collaborated with PD&R on the model, HUD has not developed specific time frames to operationalize the model and incorporate the results of the model for using local-level data to help better identify areas at risk for lead paint hazards. Federal internal control standards require agencies to define objectives clearly to enable the identification of risks. This includes clearly defining time frames for achieving the objectives. Setting specific time frames could help to ensure that HUD operationalizes this model in a timely manner. By operationalizing a model that incorporates local data on lead paint hazard risk, HUD can better target its limited grant resources towards areas of the country with significant potential for lead hazard control needs. We performed a county-level analysis using HUD and Census Bureau data and found that most lead grants from 2013 through 2017 have gone to counties with at least one indicator of lead paint hazard risk. Information we reviewed, such as relevant literature, suggests that the two common indicators of lead paint hazard risk are the prevalence of housing built before the 1978 lead paint ban and the prevalence of individuals living below the poverty line. We defined areas with lead paint hazard risk as counties that had percentages higher than the corresponding national percentages for both of these indicators. The estimated average percentage nationwide of total U.S. housing stock constructed before 1980 was 56.9 percent and the estimated average percentage nationwide of individuals living below the poverty line was 17.5 percent. As shown in figure 3, our analysis estimated that 18 percent of lead grants from 2013 through 2017 have gone to counties with both indicators above the estimated national percentages, 59 percent of grants have gone to counties with estimated percentages of old housing above the estimated national percentage, and 7 percent of grants have gone to counties that had estimated poverty rates above the estimated national percentage. (For an interactive version of this map, click here.) When HUD finalizes its model and incorporates information into its lead grant processes, HUD will be able to better target its grant resources to areas that may be at heightened risk for lead paint hazards. In 2016, HUD began to incorporate new steps to monitor PHAs’ compliance with lead paint regulations for nearly 4,000 PHAs. Previously, according to PIH staff, HUD required only that PHAs annually self-certify their compliance with lead paint laws and regulations, and HUD’s Real Estate Assessment Center inspectors check for lead paint inspection reports and disclosure forms at public housing properties during physical inspections. Starting in June 2016, PIH began using new tools for HUD field staff to track PHAs’ compliance with lead paint requirements in the voucher and public housing programs. As shown in figure 4, PIH’s compliance oversight processes for the voucher and public housing programs include various monitoring tools for overseeing PHAs. Key components of PIH’s lead paint oversight processes include the following: Tools for tracking lead hazards and cases of elevated blood levels in children. HUD uses two databases to monitor PHAs’ compliance with lead paint regulations: (1) the Lead-Based Paint Response Tracker, which PIH uses to collect and monitor information on the status of lead paint-related documents, including lead inspection reports and disclosure forms, in public housing properties but not in units with voucher assisted households; and (2) the Elevated Blood Lead Level Tracker, which PIH uses to collect and monitor information reported by PHAs on cases of elevated blood levels in children living in voucher and public housing units. In June 2016, OFO began using the Lead-Based Paint Response Tracker database to store information on public housing units and to help HUD field office staff to follow up with PHAs that have properties missing required lead documentation. In July 2017, OFO began using information recorded in the Elevated Blood Lead Level Tracker to track whether PHAs started lead remediation activities in HUD- assisted housing within the time frames required by the Lead Safe Housing Rule. Lead paint hazards included in PHAs’ risk assessment scores. OFO assigns scores to PHAs based on their relative risk in four categories: physical condition, financial condition, management capacity, and governance. OFO uses these scores to identify high- and very high-risk PHAs that will receive on-site full compliance reviews. In July 2017, OFO incorporated data from the Real Estate Assessment Center into the physical condition category of its Risk Assessment Protocol to help account for potential lead paint hazards at public housing properties. Questions about lead paint included as part of on-site full compliance reviews. In fiscal year 2016, HUD field offices began conducting on-site full compliance reviews at high- and very high-risk PHAs as part of HUD’s compliance monitoring program to enhance oversight and accountability of PHAs. In fiscal year 2017, as part of the reviews, HUD field office staff started using a compliance monitoring checklist to determine if PHAs comply with major HUD rules and to gather additional information on the PHAs. This checklist included lead-related questions that PIH field office staff use to determine whether PHAs meet the requirements in lead paint regulations for both the voucher and public housing programs. In 2016, OFO and HUD field offices began using information from the new monitoring efforts to identify potential noncompliance by PHAs with lead paint regulations and help the PHAs resolve the identified issues. According to HUD data, as of November 2017, the Lead-Based Paint Response Tracker indicated that 9 percent (357) of PHAs were missing both lead inspection reports and lead disclosure forms for one or more properties. There were 973 PHAs missing one of the two required documents. OFO staff told us that they prioritized following up with PHAs that were missing both documents. According to OFO staff, PHAs can resolve potential noncompliance by submitting adequate lead documentation to HUD. OFO staff told us the agency considers missing lead documentation as “potential” noncompliance because PHAs may provide the required documentation or they may be exempt from certain requirements (e.g., HUD-designated elderly housing). While HUD has taken steps to strengthen compliance monitoring processes, it does not have a plan to identify and address the risks of noncompliance by PHAs with lead paint regulations. Federal internal control standards state that agencies should identify, analyze, and respond to risks related to achieving the defined objectives. Furthermore, when an agency has made significant changes to its processes—as HUD has done with its compliance monitoring processes—management review of changes to these processes can help the agency determine that its control activities are designed appropriately. Our review found that HUD does not have a plan to help mitigate and address risks related to noncompliance with lead paint regulations by PHAs (i.e., ensuring lead safety in assisted housing). Additionally, our review found several limitations with HUD’s new compliance monitoring approach, which include the following: Reliance on PHA self-certifications. HUD’s compliance monitoring processes rely in part on PHAs self-certifying that they are in compliance with lead paint regulations, but recent investigations have found that some PHAs may have falsely certified that they were in compliance. In November 2017, HUD filed a fraud complaint against two former officials of the Alexander County (Illinois) Housing Authority, alleging that the former official, among other things, falsely certified to HUD that the Housing Authority was in compliance with lead paint regulations. Further, PIH staff told us there are ongoing investigations related to potential noncompliance with lead paint regulations and false certifications at two other housing authorities. Lack of comprehensive data for the public housing program. OFO started to collect data for the public housing program in the Lead-Based Paint Response Tracker in June 2016 and the inventory of all public housing properties includes units inspected since 2012. In addition, HUD primarily relies on the presence of lead inspection reports but does not record in the database when inspections and remediation activities occurred and does not determine whether they are still effective. Because of this, the information contained in the lead inspection reports may no longer be up-to-date. For example, a lead inspection report from the 1990s may provide evidence that abatement work was conducted at that time, but according to PIH staff, the housing may no longer be lead-safe. Lack of readily available data for the voucher program. The voucher program does not have readily available data on housing units’ physical condition and compliance with lead paint regulations because data on the roughly 2.5 million units in the program are kept at the PHA level. According to PIH staff, HUD plans to adopt a new system for the voucher program that will include standardized, electronic data for voucher units. PIH staff said the new system (Uniform Physical Condition Standards for Vouchers Protocol) will allow greater oversight and provide HUD the ability to conduct data analysis for voucher units. Challenges identifying children with elevated blood lead levels. For several reasons, PHAs face ongoing challenges receiving information from state and local public health departments on the number of children identified with elevated blood lead levels. First, children across the U.S. are not consistently screened and tested for exposure to lead. Second, according to CDC data, many states use a less stringent health guideline to identify children compared to the health standard that HUD uses (i.e., CDC’s current blood lead reference value). PIH staff told us that some public health departments may not report children with elevated blood levels to PHAs because they do not know that a child is living in a HUD- assisted unit and needs to be identified using the more stringent HUD standard. Lastly, Lead Office staff told us that privacy laws in some states may impose restrictions on public health departments’ ability to share information with PHAs. Limited coverage of on-site compliance reviews. While full on-site compliance reviews can be used to determine if PHAs are in compliance with lead paint regulations, OFO conducts a limited number of these reviews annually. For example, in Fiscal Year 2017, OFO conducted 72 reviews of the roughly 4,000 total PHAs. Based on OFO information, there are 973 PHAs that are missing either lead inspection reports or lead disclosure forms indicating some level of potential noncompliance. HUD’s steps since June 2016 to enhance monitoring of PHAs’ compliance with lead paint regulations have some limitations that create risks in its new compliance monitoring approach. By developing a plan to help mitigate and address the various limitations associated with the new compliance monitoring approach, HUD could further strengthen its oversight and help ensure that PHAs maintain lead-safe housing units. HUD does not have detailed procedures to address PHA noncompliance with lead paint regulations or to determine when enforcement decisions may be needed. Lead Office staff told us that their enforcement program aims to ensure that PHAs have the information necessary to remain in compliance with lead paint regulations. According to federal internal control standards, agencies should implement control activities through policies and procedures. Effective design of procedures to address noncompliance would include documenting specific actions to be performed by agency staff when deficiencies are identified and related time frames for these actions. While HUD staff stated that they address PHA noncompliance through ongoing communication and technical assistance to PHAs, HUD has not documented specific actions to be performed by staff when deficiencies are identified. OFO staff told us that in general, PIH has not needed to take many enforcement actions because field offices are able to resolve most lead paint regulation compliance concerns with PHAs through ongoing communication and technical assistance. For example, HUD field offices sent letters to PHAs when Real Estate Assessment Center inspectors could not locate required lead inspection reports and lead disclosure forms, and requested that the PHA send the missing documentation within 30 days. However, OFO’s fiscal years 2015–2017 internal memorandums on monitoring and oversight guidance for HUD field offices did not contain detailed procedures, including time frames or criteria HUD staff would use to determine when to consider whether a more formal enforcement action might be warranted. Additionally, Lead Office staff said if efforts to bring a PHA into compliance are unsuccessful, the Lead Office would work in conjunction with PIH and HUD’s Office of General Counsel’s Departmental Enforcement Center to determine if an enforcement action is needed, such as withholding or delaying funds from a PHA or imposing civil money penalties on a PHA. Lead Office staff also told us that instead of imposing a fine on a PHA, HUD would rather work with the PHA to resolve the lead paint hazard. However, the Lead Office provided no documentation detailing the specific steps or time frames HUD staff would follow to determine when a noncompliance case is escalated to the Office of General Counsel. In a March 2018 report to Congress, HUD noted that children continued to test positive for lead in HUD-assisted housing in 2017. In the same report, HUD notes PIH and the Lead Office will continue to work with PHAs to ensure compliance with lead paint regulations. By adopting procedures that clearly describe when lead paint hazard compliance efforts are no longer sufficient and enforcement decisions are needed, HUD can better keep PHAs accountable in a consistent and timely manner. The standard HUD uses to identify children with elevated blood lead levels and initiate lead hazard control activities in its rental assistance aligns with the health guideline set by CDC in 2012. HUD also uses CDC’s health guideline in its lead grant programs. In HUD’s January 2017 amendment to the Lead Safe Housing Rule, HUD made its standard for lead in a child’s blood more stringent by lowering it from 20 micrograms to 5 micrograms of lead per deciliter of blood, matching CDC’s health guideline (i.e., blood lead reference value). Specifically, HUD’s stronger standard allows the agency to respond more quickly when children under 6 years old are exposed to lead paint hazards in voucher and public housing units. The January 2017 rule also established more comprehensive testing for children and evaluation procedures for HUD assisted housing. According to HUD’s press release that accompanied the rule, by aligning HUD’s standard with CDC’s guidance, HUD can respond more quickly in cases when a child who lives in HUD assisted housing shows early signs of lead in their blood. The 2017 rule notes HUD will revise the agency’s elevated blood lead level to align with future changes HHS may make to its recommended environmental intervention level. HUD’s standards for lead dust levels align with EPA standards for its rental assistance programs and exceed EPA standards for the lead grant programs. In 2001, EPA published a final rule on lead paint hazard standards, including lead dust clearance standards. The rule established standards to help property owners, contractors, and government agencies identify lead hazards in residential paint, dust, and soil and address these hazards in and around homes. Under these standards, lead is considered a hazard when equal to or exceeding 40 micrograms of lead in dust per square foot sampled on floors and 250 micrograms of lead in dust per square foot sampled on interior window sills. In 2004, HUD amended the Lead Safe Housing Rule to incorporate the 2001 EPA lead dust standards as HUD’s standards. Since this time, HUD has used EPA’s 2001 lead hazard standards in its rental assistance programs. In February 2017, HUD released policy guidance for its lead grantees requiring them to meet new and more protective requirements for identifying and addressing lead paint hazards in the lead grant programs than those imposed by EPA’s 2001 standards that HUD uses in the rental assistance programs. For example, the policy guidance requires grantees to consider lead dust a hazard on floors at 10 micrograms per square foot sampled (down from 40) and on window sills at 100 micrograms per square foot sampled (down from 250). The policy guidance noted that the new requirements are supported by scientific evidence on the adverse effects of lead exposure at low blood lead levels in children. Further, the policy guidance established a standard for porch floors––an area that EPA has not covered––because porch floors can be both a direct exposure source for children and a source of lead dust that can be tracked into the home. On December 27, 2017, the United States Court of Appeals for the Ninth Circuit ordered EPA to issue a proposed rule updating its lead dust hazard standard and the definition of lead-based paint within 90 days of the decision becoming final and a final rule within 1 year of the proposed rule. Because HUD’s Lead Safe Housing Rule generally defines lead paint hazards and lead dust hazards to mean the levels promulgated by EPA, if EPA changes its 2001 standards those new standards would be used in HUD’s rental assistance programs. On March 16, 2018, EPA filed a request to the court asking for clarification for when EPA is required to issue the proposed rule and followed up with a motion seeking clarification or an extension. In response to EPA’s motion, on March 26, 2018, the court issued an order clarifying time frames and ordered that the proposed rule be issued within 90 days from March 26, 2018. HUD’s Lead Safe Housing Rule requires a stricter lead inspection standard for public housing than for voucher units. According to HUD staff, HUD does not have the authority to require the more stringent inspection in the voucher program. While HUD has acknowledged that moving to a stricter inspection standard for voucher units would provide greater assurance that these units are lead-safe and expressed its plan to support legislative change to authorize it to impose a more stringent inspection standard, HUD has not requested authority from Congress to amend its inspection standard for the voucher program. For voucher units, HUD requires PHAs to ensure that trained inspectors conduct visual assessments to identify deteriorated paint for housing units inhabited by a child under 6 years old. In a visual assessment, an inspector looks for deteriorated paint and visible surface dust but does not conduct any testing of paint chips or dust samples from surfaces to determine the presence of lead in the home’s paint. By contrast, for public housing units, HUD requires a stronger inspection process. Lead- based paint inspections are required for pre-1978 public housing units. If that inspection identifies lead-based paint, PHAs must then perform a risk assessment. In a risk assessment, in addition to conducting a visual inspection, an inspector tests for the presence of lead paint by collecting and testing samples of paint chips and surface dust, and typically using a specialized device (an X-ray fluorescence analyzer) to measure the amount of lead in the paint on a surface, such as a wall, door, or window sill. Staff from HUD’s Lead Office and the Office of General Counsel told us that Title X did not include specific risk assessment requirements for voucher units, and HUD does not believe, therefore, that it has the statutory authority to require an assessment more thorough than a visual assessment of voucher units. As of May 2018, HUD had not requested statutory authority to change the visual assessment standard used in the voucher program. However, HUD previously acknowledged the limitation of the weaker inspection standard in a June 2016 publication titled Lead- Safe Homes, Lead-Free Kids Toolkit. In this publication, HUD noted its plans to support legislative change to strengthen lead safety in voucher units by eliminating reliance on visual-only inspections. Staff from HUD’s Lead Office and Office of General Counsel told us the agency recognizes that risk assessments are more comprehensive than visual assessments. The staff noted that, by definition, a risk assessment is a stronger inspection standard than a visual-only assessment because it includes additional identification and testing. In responding to a draft of this report, HUD cited the need to conduct and evaluate the results of a statistically rigorous study on the impacts of requiring a lead risk assessment versus a visual assessment, such as the impact on leasing times and the availability of housing for low-income families. HUD further noted that such a study could explore whether alternative options to the full risk assessment standard (such as targeted dust sampling) could achieve similar levels of protection for children in the voucher program. Requesting and obtaining authority to amend the standard for the voucher program would not preclude HUD from doing such a study. Such analysis might support a range of options based on consideration of health effects for children, housing availability, and other relevant factors. Because HUD’s Lead Safe Housing Rule contains a weaker lead inspection standard for the voucher program children living in voucher units may be less protected from lead paint hazards than children living in public housing. By requesting and obtaining statutory authority to amend the voucher program inspection standard, HUD would be positioned to take steps to ensure that children in the voucher program are provided better protection as indicated by analysis of the benefits and costs from amending the standard. HUD has taken limited steps to measure, evaluate, and report on the performance of its programmatic efforts to ensure that housing is lead- safe. First, HUD has tracked one performance measure for its lead grant programs but lacks comprehensive performance goals and measures. Second, while HUD has evaluated the effectiveness of its Lead-Based Paint Hazard Control grant program, it has not formalized plans and does not have a time frame for evaluating its lead paint regulations. Third, HUD has not issued an annual report on the results of its lead efforts since 1997. A key aspect to promoting improved federal management and greater efficiency and effectiveness is that agencies set goals and report on performance. We have previously reported that a program performance assessment contains three key elements––program goals, performance measures, and program evaluations (see fig. 5). In our prior work, we have noted that both the executive branch and congressional committees need evaluative information to help them make decisions about the programs they oversee––information that tells them whether, and why, a program is working well or not. Program goals and performance measures. HUD has tracked one performance measure for making private housing units lead-safe as part of its lead grant programs but lacks goals and performance measures that more fully cover the range of its lead efforts. In addition to our prior work on program goals and performance measures, federal internal control standards state that management should define objectives clearly and that defining objectives in measurable terms allows agency management to assess performance toward achieving objectives. According to Lead Office staff, HUD provides information on its goals and performance measures related to its lead efforts in the agency’s annual performance reports. For example, the fiscal year 2016 report contains information about the number of private housing units made lead-safe as part of HUD’s lead grant programs but does not include any performance measures on HUD’s lead efforts for the voucher and public housing programs. Lead Office staff told us HUD does not have systems to count the number of housing units made lead-safe in these two housing programs. The staff said the Lead Office and PIH recently began discussing whether data from an existing HUD database could be used to count units made lead-safe within these programs. However, they could not provide additional details on the status of all these efforts. Without comprehensive goals and performance measures, HUD does not know the results it is achieving with all its lead paint hazard reduction efforts. Moreover, HUD may be missing opportunities to use performance information to improve the results of its lead efforts. Program evaluations. HUD has evaluated the effectiveness of its Lead- Based Paint Hazard Control grant program but has not taken similar steps to evaluate the Lead Safe Housing Rule or Lead Disclosure Rule. As previously stated, our prior work on program performance assessment has noted the importance of program evaluations to know how well a program is working relative to its objectives. Additionally, Title X required HUD to conduct research to evaluate the long-term cost-effectiveness of interim lead hazard control and abatement strategies. For its Lead-Based Paint Hazard Control Grant program, HUD has contracted with outside experts to conduct evaluations. For example, the National Center for Healthy Housing and the University of Cincinnati’s Department of Environmental Health evaluated whether the lead hazard control methods used by grantees continued to be effective 1, 3, 6, and 12 years later. The evaluations concluded that the lead hazard control activities used by grantees substantially reduced lead dust levels and the original evaluation and those completed 1 and 3 years later were also associated with substantial declines in the blood lead levels of children living in the housing remediated using lead grant program funds. HUD has general plans to conduct evaluations of the Lead Safe Housing Rule and the Lead Disclosure Rule, but Lead Office and PD&R staff said they did not know when or if the studies will begin. In a 2016 publication, HUD noted its plans to evaluate the Lead Safe Housing Rule requirements and noted that such an evaluation would contribute toward policy recommendations and program improvements. Additionally, in its 2017 Research Roadmap, PD&R outlined HUD’s plans for two studies to evaluate the effectiveness of requirements within the Lead Safe Housing and Lead Disclosure Rules. However, PD&R and Lead Office staff were not able to provide a time frame for when the studies would begin. PD&R staff told us that the plans noted within the Research Roadmap were HUD’s first step in research planning and prioritization but that appropriations for research have been prescriptive in recent years (i.e., tied to specific research topics) and fell short of the agency’s research needs. By studying the effectiveness of requirements included within the Lead Safe Housing and Lead Disclosure Rules, including the cost- effectiveness of the various lead hazard control methods, HUD could have more complete information to assess how effectively it uses federal dollars to make housing units lead-safe. Reporting. HUD has not reported on its lead efforts as required since 1997. Title X includes annual and biennial reporting requirements for HUD. Staff from HUD’s Lead Office and General Counsel told us that in 1998 the agency agreed with the congressional committees of jurisdiction that HUD could satisfy this reporting requirement by including the required information in its annual performance reports. Lead Office staff told us HUD’s recent annual performance reports do not contain specific information required by law and that HUD has not issued other publicly available reports that contain the Title X reporting requirements. Title X requires HUD to annually provide Congress information on its progress in implementing the lead grant programs; a summary of studies looking at the incidence of lead poisoning in children living in HUD-assisted housing; the results of any required lead technical studies; and estimates of federal funds spent on lead hazard evaluation and reduction in HUD-assisted housing. As previously stated, the annual performance reports have provided information on the number of housing units made lead-safe through the agency’s lead grant programs, but not through the voucher or public housing programs. In March 2018, Lead Office staff told us HUD plans to submit separate reports on the agency’s lead effort, covering the Title X reporting requirements, starting in fiscal year 2019. By HUD complying with Title X statutory reporting requirements, Congress and the public will be in a position to better know the progress HUD is making toward ensuring that housing is lead-safe. Lead exposure can cause serious, irreversible cognitive damage that can impair a child for life. Through its lead grant programs and oversight of lead paint regulations, HUD is helping to address lead paint hazards in housing. However, our review identified specific areas where HUD could improve the effectiveness of its efforts to identify and address lead paint hazards and protect children in low-income housing from lifelong health problems: Documenting and evaluating grant processes. HUD could improve documentation for its lead grant programs’ processes by providing more specific direction to staff and documenting grant award rationale. In doing so, HUD could better ensure that grant program staff score grant applications consistently and appropriately and provide greater transparency about its award decisions. Additionally, periodically evaluating its grant processes and procedures could help HUD better ensure that its lead grants reach areas most at risk for lead paint hazards. Identifying areas at risk for lead hazards. By developing specific time frames to finalize and incorporate the results of its model to more fully identify areas at risk for lead paint hazards, HUD can better identify and conduct outreach to at-risk localities that its lead grant programs have not yet reached. Overseeing compliance with lead paint regulations. False self- certifications of compliance by some PHAs and other limitations in HUD’s compliance monitoring approach make it essential for HUD to develop a plan to mitigate and address limitations, as well as establish procedures to determine when enforcement decisions are needed. These actions could further strengthen HUD’s oversight and keep PHAs accountable for ensuring that housing units are lead-safe. Amending inspection standard in the voucher program. Children living in voucher units may receive less protection from lead paint hazards than children living in public housing units because HUD applies different lead inspection standards to the two programs. HUD could ensure that children in the voucher program are provided better protection from lead by requesting and obtaining statutory authority to amend the voucher program inspection standard as indicated by analysis of the benefits and costs of amending the standard. Assessing and reporting on performance. Fully incorporating key elements of performance assessment—by developing comprehensive goals, improving performance measures, and adhering to reporting requirements—could better enable HUD to assess its own progress and target its resources toward lead efforts that maximize impact. Additionally, HUD may be missing opportunities to inform the Congress and the public about how HUD’s lead efforts have helped reduce lead poisoning in children. We are making the following nine recommendations to HUD: The Director of HUD’s Lead Office should ensure that the office more fully documents its processes for scoring and awarding lead grants and its rationale for award decisions. (Recommendation 1) The Director of HUD’s Lead Office should ensure that the office periodically evaluates its processes for scoring and awarding lead grants. (Recommendation 2) The Director of HUD’s Lead Office, in collaboration with PD&R, should set time frames for incorporating relevant data on lead paint hazard risks into the lead grant programs’ processes. (Recommendation 3) The Director of HUD’s Lead Office and the Assistant Secretary for PIH should collaborate to establish a plan to mitigate and address risks within HUD’s lead paint compliance monitoring processes. (Recommendation 4) The Director of HUD’s Lead Office and the Assistant Secretary for PIH should collaborate to develop and document procedures to ensure that HUD staff take consistent and timely steps to address issues of PHA noncompliance with lead paint regulations. (Recommendation 5) The Secretary of HUD should request authority from Congress to amend the inspection standard to identify lead paint hazards in the Housing Choice Voucher program as indicated by analysis of health effects for children, the impact on landlord participation in the program, and other relevant factors. (Recommendation 6) The Director of the Lead Office should develop performance goals and measures to cover the full range of HUD’s lead efforts, including its efforts to ensure that housing units in its rental assistance programs are lead-safe. (Recommendation 7) The Director of the Lead Office, in conjunction with PD&R, should finalize plans and develop a time frame for evaluating the effectiveness of the Lead Safe Housing and Lead Disclosure Rules, including an evaluation of the long-term cost effectiveness of the lead remediation methods required by the Lead Safe Housing Rule. (Recommendation 8) The Director of the Lead Office should complete statutory reporting requirements, including but not limited to its efforts to make housing lead-safe through its lead grant programs and rental-assistance programs, and make the report publicly available. (Recommendation 9) We provided a draft of this report to HUD for review and comment. We also provided the relevant excerpts of the draft report to CDC and EPA for their review and technical comments. In written comments, reproduced in appendix III, HUD disagreed with one of our recommendations and generally agreed with the remaining eight. HUD and CDC also provided technical comments, which we incorporated as appropriate. EPA did not have any comments on the relevant excerpts of the draft report provided to them. In its general comments, HUD noted that the lead grant programs and HUD’s compliance assistance and enforcement of lead paint regulations have contributed significantly to, among other things, the low prevalence of lead-based paint hazards in HUD-assisted housing. Further, HUD said the lead grant programs and compliance assistance and enforcement of lead paint regulations have played a critical part in developing and maintaining the national lead-based paint safety infrastructure. HUD asked that this contextual information be included in the background of the report. The draft report included detailed information on the purpose and scope of HUD’s lead grant programs, two key regulations related to lead paint hazards, and efforts to make housing lead-safe. Furthermore, the draft report provided context on other federal agencies’ role in establishing relevant standards and guidelines for lead paint hazards. We made no changes in response to this comment because we did not think it was necessary for background purposes. HUD disagreed with the draft report’s sixth recommendation to request authority from Congress to use the risk assessment inspection standard to identify lead paint hazards in the Housing Choice Voucher program. As discussed in the report, HUD’s Lead Safe Housing Rule requires a more stringent lead inspection standard (risk assessments) for public housing than for Housing Choice Voucher units, for which a weaker inspection standard is used (visual assessments). In its written comments, HUD said that before deciding whether to request the statutory authority to implement risk assessments for voucher units, it would need to conduct and evaluate the results of a statistically rigorous study on the impacts of requiring a lead risk assessment versus a visual assessment, such as the impact on leasing times and the availability of housing for low-income families. HUD further noted that such a study could explore whether alternative options to the full risk assessment standard (such as targeted dust sampling) could achieve similar levels of protection for children in the voucher program. We note that requesting and obtaining authority to amend the standard for the Housing Choice Voucher program would not preclude HUD from doing such a study. We acknowledge that the results of such a study might support a range of options. Therefore, we revised our recommendation to provide HUD with greater flexibility in how it might amend the lead inspection standard for the voucher program based on consideration of not only leasing time and availability of housing, as HUD emphasized in its written comments, but also based on the health effects on children. The need for HUD to review the lead inspection standard for the voucher program is underscored by the greater number of households with children served by the voucher program compared to public housing, as well as recent information indicating that more children with elevated blood lead levels are living in voucher units than in public housing. HUD generally agreed with our remaining eight recommendations and provided specific information about planned steps and other considerations related to implementing them. For example, in response to our first three recommendations on the lead grant programs, HUD outlined specific steps it plans to take, such as updating its guidance for scoring grant applications and reviewing its grant application scoring methods to identify potential improvements. In response to our fourth and fifth recommendations to the Director of HUD’s Lead Office on compliance monitoring and enforcement of lead paint regulations, HUD noted that PIH should be the primary office for these recommendations with the Lead Office providing support. While these recommendations had already recognized the need for the Lead Office to collaborate with PIH, we reworded them to clarify that it is not necessary for the Lead Office to have primary responsibility for their implementation. HUD generally agreed with our seventh and eighth recommendations, but noted some considerations for implementing them. For our seventh recommendation about performance goals and measures, HUD noted that it will re-examine the availability of information from the current housing databases to determine whether data on housing unit production can be added to the existing data collected. HUD noted if that information is not sufficient, it would need to obtain Office of Management and Budget approval and have sufficient funds for such an information technology project. For our eighth recommendation about evaluating the Lead Safe Housing and Lead Disclosure Rules, HUD noted if its own resources are insufficient, the time frame for implementing this recommendation may depend on the availability of funding for contracted resources. Finally, in response to our ninth recommendation, HUD said that it will draft and submit annual and biennial reports to the congressional authorizing and appropriations committees and then post the reports on the Lead Office’s public website. We are sending copies of this report to the appropriate congressional committees, the Secretary of the Department of Housing and Urban Development, the Administrator of the Environmental Protection Agency, and the Secretary of Health and Human Services, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-8678 or garciadiazd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV. Under the Department of Housing and Urban Development’s (HUD) Lead-Based Paint Hazard Control and the Lead Hazard Reduction Demonstration grant programs, HUD competitively awards grants to state and local jurisdictions, as authorized by the Residential Lead-Based Paint Hazard Reduction Act (Title X of the Housing and Community Development Act of 1992). Title X requires each grant recipient to make matching contributions with state, local, and private funds (i.e., nonfederal) toward the total cost of activities. For the Lead-Based Paint Hazard Control grant and the Lead Hazard Reduction Demonstration grant programs, the matching contribution has been set at no less than 10 percent and 25 percent, respectively, of the total grant amount. For example, if the total grant amount is $3 million, then state or local jurisdictions must provide at least $300,000 and $750,000, respectively, for each grant program, in additional funding toward the cost of activities. HUD requires lead grant applicants to include information on the sources and amounts of grantees’ matching contributions as part of their grant applications. Additionally, Title X requires HUD to award grants in part based on an applicant’s ability to leverage state, local, and private funds to supplement the federal grant funds. To identify the nonfederal funding sources grantees used in the lead hazard control grants, we selected and reviewed the lead grant applications of 20 HUD grantees and interviewed representatives from 10 of these. We selected these grantees based on their geographic locations; the number of HUD lead grants they had previously received; experience with HUD’s lead hazard control grants; and whether they have received both grants from 2013 through 2017. Grantees we selected included entities at the state, municipality, and county levels. Information from our grant application reviews and interviews of grantees cannot be generalized to all HUD grantees. Based on our review of the selected grant applications and interviews of selected grantees, we found that grantees planned to use the following types of nonfederal funding sources as their matching contributions to support their lead grants activities: State and local funds. Eighteen of the 20 grantees we selected noted that they planned to use state or local funding sources to supplement HUD’s grant funds. The state and local funding sources included state or local general funds and local property taxes or fees. For example, grantees in Connecticut, Baltimore, and Philadelphia used state or local general funds to cover personnel and operating costs. Additionally, grantees in Alameda County (California), Hennepin County (Minnesota), Malden, St. Louis, and Winnebago County (Illinois) planned to use local taxes, including property taxes or fees, such as real estate recording and building permit fees, to cover some costs associated with their lead hazard control grants activities. Community Development Block Grant funds. Ten of the 20 grantees we selected indicated that they planned to use Community Development Block Grant (CDBG) program funds to cover part of the costs of their lead hazard control grants. CDBG program funds can be used by states and local communities for housing; economic development; neighborhood revitalization; and other community development activities. For example, grantees in Baltimore and Memphis noted in their grant applications that they planned to use the funds to cover costs related to personnel, operations, and training. Nongovernmental contributions or discounts. Eight of 20 grantees we selected stated that they anticipated some forms of nongovernmental contributions from nonprofit organizations or discounts from contractors to supplement the lead grants. For example, all eight grantees stated that they expected to receive matching contributions from nonprofit organizations. Table 2 summarizes the nonfederal funds by source that the 20 selected grantees planned to use, based on our review of these grantees’ applications. Furthermore, almost all of the selected grantees stated in their grant applications or told us that they expected to receive or have received other nonfederal funds in excess of their matching contributions. For example, 15 grantees stated that they generally required or encouraged property owners or landlords to contribute toward the lead hazard remediation costs. Also, grantees in Baltimore, District of Columbia, Lewiston, and Providence indicated that they expected to receive monetary or in-kind donations from organizations to help carry out lead hazard remediation, blood lead-level testing, or training. Additionally, the grantee in Alameda County (California) told us that they have received nonfederal funds from a litigation settlement with a private paint manufacturer. This report examines the Department of Housing and Urban Development’s (HUD) efforts to (1) incorporate statutory requirements and other relevant federal standards in its lead grant programs; (2) monitor and enforce compliance with lead paint regulations for its rental assistance programs; (3) adopt federal health guidelines and environmental standards for lead hazards in its lead grant and rental assistance programs; and (4) measure and report on its performance related to making housing lead-safe. In this report, we examine lead paint hazards in housing, and we focus on HUD’s lead hazard control grant programs and its two largest rental assistance programs that serve the most families with children: the Housing Choice Voucher (voucher) and public housing programs. To address all four objectives, we reviewed relevant laws, such as the Residential Lead-Based Paint Hazard Reduction Act (Title X of the Housing and Community Development Act of 1992, referred to as Title X throughout this appendix) and relevant HUD regulations, such as the Lead Safe Housing Rule and a January 2017 amendment to this rule. To examine trends in funding for HUD’s lead grant programs for the past 10 years, we also reviewed HUD’s budget information for fiscal years 2008 through 2017. We interviewed HUD staff from the Office of Lead Hazard Control and Healthy Homes (Lead Office), Office of Public and Indian Housing (PIH), Office of Policy Development and Research (PD&R), and other relevant HUD program and field offices. Finally, we reviewed our prior work and those of HUD’s Office of Inspector General. To address the first objective, we reviewed HUD’s Notices of Funding Availability (funding notices), policies, and procedures to identify HUD’s grant award processes for the Lead-Based Paint Hazard Control grant and Lead Hazard Reduction Demonstration grant programs. For example, we reviewed HUD’s annual notices of funding availability from 2013 through 2017 to identify HUD’s scoring factors for evaluating grant applications. We compared HUD’s grant award processes in 2017 with Title X statutory requirements, the Office of Management and Budget (OMB) requirements for awarding federal grants, and relevant federal internal control standards. We also interviewed HUD staff about the agency’s grant application review and award processes. To determine the extent to which HUD’s grants have gone to counties in the United States potentially at high risk for lead paint hazards, we compared grantee locations from HUD’s lead grant data for grants awarded from 2013 through 2017 with county-level data on two indicators of lead paint hazard risk from the 2011–2015 American Community Survey—a continuous survey of households conducted by the U.S. Census Bureau. We analyzed HUD’s grant data to determine the number and dollar amount of grants received by each grantee, and the grantees’ addresses. We then conducted a geographic analysis to determine whether each HUD lead grant went to a county that met at least one, both, or neither of the two commonly known indicators of lead paint hazard risk—the age of housing and poverty level. We identified these two indicators through a review of relevant academic literature, agency research, and state lead modelling methodologies. We used data from the 2011–2015 American Community Survey because the data covered a time frame that best aligned with the 5 years of lead grant data (2013 through 2017). Using its county-level data, we calculated an estimated average percentage nationwide of housing units built before 1980 (56.9 percent) and an estimated average percentage nationwide of individuals living below the poverty level (17.5 percent). We used 1980 as a benchmark for age of housing because the American Community Survey data for age of housing is separated by the decade of construction and 1980 was closest in time to the 1978 federal lead paint ban. We categorized counties based on whether their levels of pre-1980 housing and poverty were above one, both, or neither of the respective national average percentage for each indicator. The estimated average nationwide and county-level percentages of the two indicators (e.g., older housing and poverty rate) are expressed as a range of values. For the lower and upper ends of the range, we generated a 95 percent confidence interval that was within plus or minus 20 percentage points. We classified a county as above the estimated average percentages nationwide if the county’s confidence interval was higher and did not overlap with the nationwide estimate’s confidence interval. We omitted the data for 12 counties that we determined were unreliable for our purposes. We analyzed data starting in 2013 because that was the first year for which these grant data were available electronically. We also interviewed HUD staff to understand their efforts and plans to perform similar analyses using indicators of lead paint hazard risk. To assess the reliability of HUD’s grant data, we reviewed documentation of HUD’s grant database, interviewed Lead Office staff on the processes HUD used to collect and ensure the reliability of the data, and tested the data for missing values, outliers, and obvious errors. To assess the reliability of the American Community Survey data, we reviewed statistical information from the Census Bureau and other publicly available documentation on the survey and conducted electronic testing of the data. We determined that the HUD grant data and American Community Survey county-level data on age of housing and poverty were sufficiently reliable for identifying areas at risk of lead paint hazards and determining the extent to which lead grants from 2013 through 2017 have gone to at-risk areas. Furthermore, to obtain information about how HUD works with grantees to achieve program objectives, we conducted in-person site visits to five grantees located in five localities (Alameda County, California; Atlanta, Georgia; Baltimore, Maryland; District of Columbia; and San Francisco, California); and interviewed an additional five grantees on the telephone (Hennepin County, Minnesota; Lewiston, Maine; Malden, Massachusetts; Providence, Rhode Island; and Winnebago County, Illinois). In addition, we reviewed the grant applications of the 10 grantees we spoke to and an additional 10 grantees from 10 additional jurisdictions (State of Connecticut; Cuyahoga County, Ohio; Denver, Colorado; Monroe County, New York; Philadelphia, Pennsylvania; Memphis, Tennessee; San Antonio, Texas; St. Louis, Missouri; Tucson, Arizona; and State of Vermont). We selected the 10 grantees for site visits or interviews based on the following criteria: geographic variation, number of years the grantees had HUD’s lead grants, and grantees that have received both types of lead grants from 2013 through 2017. We selected the 10 additional grantees’ applications for review based on geographic diversity and to achieve a total of two applications for each year during our 5-year time frame, with at least one application from each of the two HUD lead grant programs. As part of our review of selected grant applications, we identified nonfederal funding sources used by grantees, such as local tax revenues, contractor discounts, and property owner contributions. Information from the selected grantees and grant applications review cannot be generalized to those grantees we did not include in our review. Additionally, we interviewed representatives from housing organizations to obtain additional examples of any nonfederal funding sources, such as state or local bond measures, or low-interest loans to homeowners. To address the second objective, we also reviewed HUD guidance and internal memorandums related to its efforts to monitor and enforce compliance with lead paint regulations for public housing agencies (PHA), the entities that manage HUD’s voucher and public housing rental assistance programs. In addition, we reviewed HUD’s documentation of databases it uses to monitor compliance, including the Lead-Based Paint Response Tracker and the Elevated Blood Lead Level Tracker, and observed HUD staff’s demonstrations of these databases. HUD staff also provided a demonstration of the Record and Process Inspection Data database (known as “RAPID”) used by HUD’s Real Estate Assessment Center to collect physical inspection data for public housing units. We obtained and reviewed information from HUD about instances of potential noncompliance with lead paint regulations by PHAs as of November 2017 and enforcement actions HUD has taken. We compared HUD’s regulatory compliance monitoring and enforcement approach to federal internal control standards. We interviewed staff from HUD’s Lead Office, Office of General Counsel, Office of Field Operations, and field staff, including four HUD regional directors in areas of the country known to have a high prevalence of lead paint hazards, about internal procedures for monitoring and enforcing compliance with lead paint regulations by the PHAs within their respective regions. To address the third objective on HUD’s adoption of federal health guidelines and environmental standards for lead paint hazards in its lead grant and rental assistance programs, we reviewed relevant rules and HUD documentation. To identify relevant federal health guidelines and environmental standards, we reviewed guidelines and regulations from the Centers for Disease Control and Prevention (CDC) and the Environmental Protection Agency (EPA) and interviewed staff from each agency. To identify state and local laws with different requirements than these federal guidelines and standards, we obtained information from and interviewed staff from CDC’s Public Health Law Program and the National Conference of State Legislatures. We compared HUD’s requirements to CDC’s health guideline known as the “blood lead reference value” and EPA’s standards for lead-based paint hazards and lead-dust clearance standards. Finally, we reviewed information in HUD’s 2017 funding notices and lead grant programs’ policy guidance about requirements for grantees as they pertain to health guidelines and environmental standards. We also interviewed HUD staff about how HUD has used the findings from lead technical study grants to consider changes to HUD’s requirements and processes regarding identifying and addressing lead paint hazards for the grant programs. To address the fourth objective, we reviewed HUD documentation related to performance goals and measures, program evaluations, and reporting. For example, we reviewed HUD’s recent annual performance reports to identify goals and performance measures related to HUD’s efforts to make housing lead-safe. Further, we reviewed Title X to identify requirements related to evaluating and reporting on HUD’s lead efforts. We reviewed program evaluations and related studies completed by outside experts for the lead grant programs and interviewed staff from one of the organizations that conducted the evaluations. In addition, we interviewed Lead Office and PD&R staff about the agency’s plans to evaluate the requirements in the Lead Safe Housing Rule and reviewed corresponding agency documentation about these plans. Additionally, we reviewed the Lead Office’s most recent strategic plan (2009) and annual report (1997) on the agency’s lead efforts. We compared HUD’s use of performance goals and measures, program evaluations, and reporting against leading practices for assessing program performance and federal internal control standards. Finally, we interviewed staff from HUD to understand goals and performance measures used by the agency to assess their lead efforts. We conducted this performance audit from March 2017 to June 2018 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. In addition to the contact named above, John Fisher (Assistant Director), Beth Faraguna (Analyst in Charge), Enyinnaya David Aja, Farah Angersola, Carol Bray, William R. Chatlos, Anna Chung, Melinda Cordero, Elizabeth Dretsch, Christopher Lee, Marc Molino, Rebecca Parkhurst, Tovah Rom, Tyler Spunaugle, and Sonya Vartivarian made key contributions to this report.

Question: Write a one-page summary of the report 1.

Answer: VA has pressing infrastructure needs. The Communities Helping Invest through Property and Improvements Needed for Veterans Act of 2016 (CHIP-IN Act) authorized VA to accept donated real property—such as buildings or facility construction or improvements—through a pilot program. VA has initiated one project in Omaha, Nebraska, through a partnership with a donor group. VA can accept up to five donations through the pilot program, which is authorized through 2021. The CHIP-IN Act includes a provision for GAO to report on donation agreements. This report (1) examines the extent to which the VA's pilot design aligns with leading practices and (2) discusses what VA has learned from the pilot to date. GAO reviewed VA documents, including plans for the pilot program, and visited the Omaha pilot project. GAO interviewed VA officials, the Omaha donor group, and three non-federal entities that responded to VA's request seeking donors. GAO compared implementation of VA's pilot to leading practices for pilot design, organizational transformation, and cross-functional teams. The Department of Veterans Affairs (VA) is conducting a pilot program, called CHIP-IN, that allows VA to partner with non-federal entities and accept real property donations from them as a way to help address VA's infrastructure needs. Although VA signed its first project agreement under the program in April 2017, VA has not yet established a framework for effective design of the pilot program. Specifically, VA's pilot program design is not aligned with four of five leading practices for designing a well-developed and documented pilot program. VA has begun to implement one leading practice by improving its efforts to communicate with relevant stakeholders, such as including external stakeholders in key meetings. However, the VA offices involved have not agreed upon and documented clear, measurable objectives for the pilot program, which is a leading practice. Further, VA has not developed an assessment methodology or an evaluation plan that would help inform decisions about whether or how the pilot approach could be expanded. While VA officials said they intend to develop these items as tasks for the newly formed CHIP-IN steering committee, they have no timeline for doing so. Without clear objectives and assessment and evaluation plans, VA and Congress may have difficulty determining whether the pilot approach is an effective way to help address VA's infrastructure needs. To date, the CHIP-IN pilot suggests that donation partnerships could improve construction projects, but identifying donors and establishing a team for the pilot program have presented challenges. Officials from VA and the donor group for the first pilot project—an ambulatory care center in Omaha, Nebraska—said they are completing the project faster than if it had been a standard federal construction project, while achieving potential cost savings by using private sector practices. However, VA officials said it is challenging to find partners to make large donations with no financial return, and VA's lack of marketing and philanthropic development experience exacerbates that challenge. VA and the donor group agreed that a dedicated team of individuals with relevant expertise could facilitate the pilot's implementation. The new CHIP-IN steering committee could serve this purpose, but it lacks documented roles and responsibilities. Establishing a team with clear roles and responsibilities and identifying both available and needed staff resources could assist VA in partnering with additional donors and creating new opportunities to meet veterans' needs. GAO is recommending that VA: (1) establish pilot program objectives, (2) develop an assessment methodology and an evaluation plan, and (3) document roles and responsibilities and identify available and needed staff resources. VA concurred with GAO's recommendations.