GAO

United States General Accounting Office

Report to Congressional Committees

July 2000 HUD HOUSING

PORTFOLIOS

HUD Has

Strengthened Physical

Inspections but Needs

to Resolve Concerns

About Their Reliability

GAO/RCED-00-168

Page 1 GAO/RCED-00-168 HUD’s Physical Inspection System

Contents

Letter 3

Appendixes Appendix I: Results of Baseline Inspections Conducted by

HUD’s Real Estate Assessment Center 34

Appendix II: Industries’ Concerns and HUD’s Responses 39

Appendix III: Objectives, Scope, and Methodology 43

Appendix IV: Comments From the Department of Housing

and Urban Development 45

Appendix V: GAO Contacts and Staff Acknowledgments 50

Tables Table 1: Physical Inspection Results for Properties in HUD’s

Baseline Inventory, by U.S. Region 37

Table 2: Physical Inspection Results for Multifamily Properties

in HUD’s Baseline Inventory, by Type of Financing

and Assistance 38

Figures Figure 1: Number of Multifamily and Public Housing Properties

With Inspection Scores of Less Than 60 and 60 or Higher 12

Figure 2: Physical Conditions That REAC Quality Assurance

Reviewers Found Contract Inspectors Did Not Record at

Properties GAO Visited 16

Figure 3: Percentage of Multifamily and Public Housing Property

Inspections, Sorted by Inspection Scoring Range 35

Figure 4: Percentage of Multifamily and Public Housing

Property Units, Sorted by Inspection Scoring Range 36

Abbreviations

HUD Department of Housing and Urban Development

GAO General Accounting Office

REAC Real Estate Assessment Center
Page 2 GAO/RCED-00-168 HUD’s Physical Inspection System
Page 3 GAO/RCED-00-168 HUD’s Physical Inspection System

United States General Accounting Office

Washington, D.C. 20548

Resources, Community, and

Economic Development Division

B-283608 Letr

July 25, 2000

Congressional Committees

In 1997, as part of its 2020 Management Reform Plan, the Department of

Housing and Urban Development (HUD) created the Real Estate

Assessment Center (REAC) to obtain consistent information on, among

other things, the physical condition of its public and multifamily housing. In

the past, for most programs, HUD obtained data on the physical condition

of its housing from property inspections, but there were differences in the

way the inspections were performed and in the data that HUD collected on

property conditions.

In late 1998, REAC began implementing a new electronically based system

for inspecting the physical condition of HUD’s public and multifamily

housing properties. Under the new system, HUD relies primarily on

contractors to conduct inspections but uses its own staff to monitor and

oversee the contractors through several quality assurance procedures,

including on-site reviews, to determine whether the inspections were

carried out in accordance with REAC’s standards and protocols. HUD plans

to use the data from REAC’s inspections not only to assess the condition of

properties but also to strengthen its oversight of public and multifamily

housing.

You expressed concerns about whether REAC’s inspections represent an

improvement over HUD’s prior inspection systems and whether REAC’s

inspections are reliable. Accordingly, this report

• discusses the basis for the new physical inspection standards HUD has

established for public and multifamily housing and any differences

between the new standards and inspection procedures and those

formerly used by HUD;

• summarizes the results of the first round of physical inspections and

determines what REAC’s reviews of these inspections show about their

reliability;

• determines whether REAC’s quality assurance measures are sufficient

to ensure that the inspections are reliable and that any problems with

the inspections are resolved expeditiously; and

• describes concerns expressed by housing industry representatives

about the new system and determines whether they have identified
B-283608

Page 4 GAO/RCED-00-168 HUD’s Physical Inspection System

more cost-effective systems for assessing the physical condition of

HUD’s properties.

Results in Brief Under the new physical inspection system, as in the past, the primary basis

for HUD’s physical inspection standards is the statutory requirement for

decent, safe, and sanitary housing. HUD’s new standards require that both

multifamily and public housing be decent, safe, sanitary, and in good repair.

The standards also establish more specific requirements for the major

areas of HUD housing, such as the site, dwelling units, and common areas.

While the new standards do not differ substantially from the previous ones,

HUD substantially revised the procedures used to administer the

standards. For example, it developed a detailed list of items that inspectors

are required to review at properties and specifically defined what

constitutes a deficiency for each inspectable item. During the inspections,

the inspectors identify the deficiencies that they observe at a property and

transmit their findings electronically to REAC headquarters. At REAC, data

submitted by the inspectors are checked and analyzed, and each property’s

condition is scored using a scale of 0 to 100. HUD believes that the new

system produces more objective assessments than those performed under

previous systems. These assessments provide a basis for targeting

resources to the properties that need the most attention. Moreover,

because the assessment data are centralized, HUD can track the actions

taken to correct the identified deficiencies.

According to REAC’s data, about 87 percent of the multifamily properties

and 80 percent of the public housing properties reviewed as of April 2000

received scores of at least 60, which HUD considers satisfactory. However,

when REAC quality assurance staff performed on-site follow-up reviews to

assess the adequacy of completed inspections, they often found that the

inspections were not carried out consistently with REAC’s requirements.

More specifically, the quality assurance reviewers determined during

follow-up reviews performed in 1999 and the early part of 2000 that about

35 percent of the inspections did not meet REAC’s standards. However,

because these reviews were not performed on a random basis and the

effects of the identified problems on the inspection scores cannot be

readily determined, it is unclear to what extent the problems with the

inspections affected the overall inspection results.

Although REAC deserves credit for establishing quality assurance

procedures, we found gaps or weaknesses in some of these procedures that

substantially limited their effectiveness. In particular, we found that while
B-283608

Page 5 GAO/RCED-00-168 HUD’s Physical Inspection System

REAC performed on-site reviews to assess the adequacy of the inspections,

it did not have procedures for ensuring that these reviews were performed

systematically, that problems identified during the reviews were resolved

quickly and appropriately, and that its reviews were coordinated with those

that its inspection contractors perform as part of their own quality control

programs. For example, we found that, as of March 2000, REAC had not

reviewed the adequacy of the inspections performed by 21 percent of the

active inspectors. Furthermore, REAC did not always report the results of

its reviews to inspection contractors in a timely manner, and it did not have

the systems and records needed to ensure that corrective actions are taken

after problems have been identified. REAC has recently taken a number of

actions to strengthen its quality assurance procedures. For example, it

drafted a plan that sets forth the objectives for and procedures to be used

in its quality assurance program. While this is a positive step, we are

recommending several additional actions to strengthen the quality

assurance plan. HUD agreed with us that it needs to improve its quality

assurance program and pointed out steps it is taking to do so.

Representatives of public housing authorities and multifamily housing

industry groups we interviewed also had concerns about the reliability of

REAC’s inspections. For example, they were concerned that the scores may

not necessarily reflect the actual condition of some properties, noting that,

in some instances, there were substantial differences in the inspection

scores received by properties that had undergone more than one

inspection. They were also concerned that HUD may lack the resources

needed to ensure the efficient operation of its appeals procedures. While

some housing industry groups suggested alternatives to the current

physical inspection system, none of the groups we contacted had

conducted analyses to compare the cost and accuracy of these alternatives

and HUD’s current system.
B-283608

Page 6 GAO/RCED-00-168 HUD’s Physical Inspection System

Background Nearly 4 million American families live in rental housing—either public

housing or privately owned multifamily housing—that is owned, insured, or

subsidized by HUD. To ensure that these families have housing that is

decent, safe, sanitary, and in good repair, REAC inspects these properties

annually. Public housing serves low-income families, the elderly, and

persons with disabilities and is operated by public housing authorities

using funds provided by HUD. Privately owned multifamily housing

includes properties that receive some form of rental assistance, including

Section 8, from HUD; properties whose mortgages are insured or held by

HUD; and properties that are financed by HUD.1

REAC’s new physical inspection system is based on uniform physical

condition standards and a computer-driven physical inspection protocol.

The majority of REAC’s inspections are conducted by inspectors employed

by one of the inspection contractors that HUD has hired for this service;

some inspections are, however, performed by HUD personnel. During an

inspection, a contract inspector enters observations into a hand-held

computer and then electronically transmits the data to REAC for

verification and calculation. The computer program generates a score from

0 to 100, which provides a snapshot of the condition of the inspected

property, including any health and safety violations observed during the

inspection.

To establish uniform standards that could be applied to all HUD housing,

regardless of the source of subsidy or assistance, the Department identified

hundreds of conditions or deficiencies that could be observed at any HUD

housing property. HUD grouped these observable deficiencies into five

major inspectable areas:2

• the site (including the fencing, grounds, lighting, and parking lots),

• the building exterior (including the foundation, doors, roofs, walls, and

windows),

• the building systems (including the electrical systems, elevators, heating

and air-conditioning systems, and water and sanitary systems),

1

HUD’s uniform physical condition standards apply to Section 8 project-based assisted

housing (“project-based” assistance is attached to the structure, in contrast to “tenantbased” assistance, which is attached to the resident).

2

A particular deficiency can be included in more than one of these five major inspectable

areas.
B-283608

Page 7 GAO/RCED-00-168 HUD’s Physical Inspection System

• the dwelling units (all the elements within a resident’s apartment), and

• the common areas (including the halls and stairways, community

rooms, rest rooms, utility and mechanical rooms, trash collection areas,

and day care centers).

HUD also established weights and values to quantify the severity of the

identified deficiencies and determine their relative importance. The more

severe the deficiency or the more important the inspectable item with a

deficiency, the more points are deducted for that deficiency. An area score

is then calculated from all the observed deficiencies for an inspectable

area, and the five area scores are averaged and weighted to produce a

single score for the property’s overall physical condition. Generally, a

property’s score is based on a sample of units and, less often, on a sample

of buildings. Because the score represents the property’s condition at the

time of the inspection, it does not take into account the property’s longterm capital needs—such as whether its roof should be replaced in 5 years

or its reserves are sufficient to meet such needs.

The final score and the observed deficiencies are summarized in a report

that REAC sends to the property owner or public housing authority and to

the appropriate HUD program office. HUD requires that corrective actions

be taken for properties that receive inspection scores below certain

thresholds. However, the contract inspector is required to notify the owner

or manager of any exigent (“life-threatening”) health and safety violations

at the end of the inspection. These violations must be corrected

immediately, and notice of the correction must be sent to the appropriate

program office. Health and safety violations are also noted by attaching an

alphabetic symbol to the score classifying the violations as non-lifethreatening or life-threatening.3

To ensure the reliability of the inspections, REAC has established four

quality assurance procedures. They include the following:

• Desk reviews. REAC staff review the accuracy of the inspection data,

such as the number of buildings at a property and the number of

apartment units selected for inspection. Errors are reported to the

contractor for immediate resolution. Desk reviews also allow REAC to

3

Although smoke detector violations are labeled as life-threatening, points are not deducted

for them, and they are noted by an asterisk attached to the score.
B-283608

Page 8 GAO/RCED-00-168 HUD’s Physical Inspection System

identify trends and patterns that might suggest weaknesses in the

inspection protocol.

• Field reviews of selected contract inspections. These reviews may be

collaborative (a REAC staff member accompanies a contract inspector

to observe the inspector’s performance) or follow-up (a REAC staff

member visits a property after it has been inspected to assess whether

the inspection meets REAC’s requirements).4 Both types of reviews are

used to identify inspectors who are not adhering to the inspection

protocols. Inspection contractors are then advised to take appropriate

corrective action, which may include reinspecting the property,

retraining the contract inspector, or administering disciplinary action.

• Contractor quality control programs. REAC requires each inspection

contractor to establish and maintain its own quality control program for

overseeing its inspectors and to report its findings to REAC.

Contractors, for example, perform collaborative reviews that are similar

to REAC’s reviews, survey property owners for their views on the

inspections, record complaints about inspectors’ performance, and

retrain poorly performing inspectors.

• Appeals procedures. HUD provides multifamily property owners and

public housing authorities with two avenues, technical reviews and data

corrections, for disputing inspection scores that they believe are

incorrect. The technical review process allows owners and housing

authorities to report errors committed by contract inspectors that may

result in inaccurate assessments of the condition of their properties. The

data correction process provides a means for HUD to correct inspection

data when local ordinances or building codes have not been factored

into the scoring process. REAC is responsible for processing technical

reviews, while data corrections are processed by HUD’s Multifamily

Housing and Public and Indian Housing program offices. As of April

2000, HUD had finalized the appeals procedures for public housing but

not for multifamily housing.

REAC’s goal was to obtain baseline data on the physical condition of all

HUD properties by the end of calendar year 1999. REAC completed its

baseline inspections of public housing properties on schedule but did not

finish its baseline inspections of multifamily properties. As of April 12,

2000, REAC had inspected 26,528 multifamily properties, or 92 percent of

the multifamily properties scheduled to be inspected.

4

REAC officials refer to follow-up reviews as limited quality assurance inspections.
B-283608

Page 9 GAO/RCED-00-168 HUD’s Physical Inspection System

Inspection Standards

Have Not Changed

Substantially, but

Inspection Procedures

Are Different

HUD’s new physical inspection standards, called uniform physical

condition standards, are based on a statutory requirement that HUD

housing be decent, safe, and sanitary. Although the new standards are

similar to HUD’s former physical inspection standards, the procedures for

implementing the standards have been substantially revised to address

problems with HUD’s prior inspection procedures.

New Standards Continue to

Be Based on Statutory

Requirements

Like HUD’s prior physical inspection standards, the new uniform standards

are based on statutory language that requires HUD housing to be decent,

safe, and sanitary. HUD believes that housing assisted under its programs

should be subject to uniform physical standards, regardless of the source

of the subsidy or assistance. Consequently, rather than altering the

statutory requirement for the maintenance of HUD housing, HUD aimed to

describe the statutory requirement clearly and make that definition

consistent across applicable HUD programs. The Congress established the

standard for HUD housing in the United States Housing Act of 1937 (42

U.S.C. 1437 et seq.), providing that federally assisted housing for lowincome individuals be decent, safe, and sanitary.5 This act applies to

Section 8 assisted housing and public housing. For HUD-insured

multifamily housing, the mortgagors are required by contract to maintain

the mortgaged premises in good repair and condition—a standard that,

HUD says, is very similar if not identical to the standard for decent, safe,

and sanitary housing.

HUD’s new uniform physical condition standards include requirements for

the major areas of HUD housing, such as the site, dwelling units, and

common areas. For example, the standards specify that the site—including

the fencing and retaining walls, grounds, play areas, and walkways—must

be free of health and safety hazards and in good repair. In addition, the site

must not be subject to material adverse conditions, such as abandoned

vehicles, excess accumulations of trash, infestations of vermin or rodents,

or fire hazards. Similarly, the standards specify that the property’s dwelling

units must be structurally sound and habitable and that all areas and

5

The physical condition standard currently required under the United States Housing Act of

1937 is referred to as “decent and safe.” HUD does not consider this standard substantively

different from the previous statutory standard of “decent, safe, and sanitary.”
B-283608

Page 10 GAO/RCED-00-168 HUD’s Physical Inspection System

aspects of the dwelling units must be free of health and safety hazards,

functionally adequate, operable, and in good repair.

According to HUD, the new uniform physical condition standards do not

differ substantially from the housing quality standards that the Department

formerly used for public housing and still uses for dwelling units rented to

tenants with section 8 tenant-based assistance. HUD officials told us that

the two sets of standards cover essentially the same inspectable items

except that the housing quality standards apply only to the dwelling units

while the new uniform standards apply to the entire property. HUD also

considers the uniform standards more specific and less subjective than the

housing quality standards. While the uniform standards require that, in

addition to being decent, safe, and sanitary, HUD housing must be in good

repair, HUD believes that this is the physical condition standard to which

HUD assisted housing has always been subject.

HUD Changed Its Physical

Inspection Procedures to

Address Concerns With

Prior Inspections

HUD revised its physical inspection procedures and made them uniform in

an effort to address a number of concerns about its former inspection

programs.6 First, HUD required all inspections to be performed by trained

and certified inspectors. In the past, inspections were performed by

inspectors representing several different entities, depending on the HUD

program—Section 8 contract administrators, mortgagees, HUD field

offices, or public housing authorities. According to HUD officials, these

inspectors were not uniformly trained, and different training sometimes

produced inconsistent results.

Second, HUD identified items that must be observed during inspections,

developed detailed definitions for deficiencies in each of them, and

assigned a specific number of points for each deficiency. Now, when a HUD

inspector identifies a particular deficiency, a certain number of points are

automatically deducted from the inspected property’s total physical

condition score. In the past, without such specific guidance and detailed

definitions, some inspectors might have focused on some deficiencies but

not on others. Moreover, without a point system for deficiencies, some

inspectors might have determined that a particular deficiency, such as a

cracked toilet seat, was severe enough for a unit to “fail” its inspection,

while another inspector might have attached less importance to the same

6

The new inspection procedures apply to the HUD housing covered by the uniform physical

condition standards.
B-283608

Page 11 GAO/RCED-00-168 HUD’s Physical Inspection System

deficiency and allowed the unit to “pass.” Under the new inspection

procedures, individual inspectors no longer have such broad discretion to

determine whether the dwelling units they inspect meet HUD’s standard for

decent, safe, and sanitary housing. According to HUD, the new procedures

are more objective, ensuring more consistent results.

Third, HUD centralized the reporting, scoring, and storing of property

inspection results. After inspectors electronically transmit the inspection

data they have entered into their hand-held computers to REAC, the data

are processed, the properties receive physical inspection scores, and the

data are maintained in centralized databases. Because the new scores are

based on a point system, they are scalable, allowing HUD to assess the

breadth or severity of the deficiencies found. HUD can then use the scores

to evaluate the condition of the inspected properties and to focus its

resources on the properties needing the most attention. Furthermore, since

all the data obtained during the inspections are centralized, HUD can use

the data to track the actions taken to correct deficiencies found during the

inspections. In the past, the pass/fail determinations that HUD received

from some inspection entities did not provide the information needed to

direct resources to the properties with the greatest deficiencies. Moreover,

HUD received no information on the physical condition of public housing

because it relied on public housing authorities to certify their units’

compliance with the standard for decent, safe, and sanitary housing.

Besides revising the physical inspection procedures, HUD established four

quality assurance procedures to help ensure the reliability of its

contractors’ inspections. Previously, HUD had no comprehensive,

consistent quality assurance procedures.

Although Baseline

Scores Show Most

Properties Were in

Satisfactory Condition,

REAC’s Reviews

Indicate Some

Inspections Were Not

Reliable

The results of the first round of physical inspections performed under the

new inspection system indicate that most of HUD’s public and multifamily

housing is in satisfactory physical condition. However, REAC’s quality

assurance reviews have found that a number of these inspections were not

carried out consistently with REAC’s requirements. It is unclear to what

extent the problems with the inspections affected the overall inspection

results.
B-283608

Page 12 GAO/RCED-00-168 HUD’s Physical Inspection System

Baseline Scores Indicate

That Most Properties Are in

Satisfactory Physical

Condition

As of April 12, 2000, REAC had conducted initial, or “baseline,” inspections

for 26,528 multifamily properties and over 3,100 public housing authorities

with 13,607 properties. These figures represent about 92 percent of the

multifamily properties and all of the public housing properties that REAC

intended to include in the baseline inspections.

As shown in figure 1, almost 34,000 properties, or 85 percent of those

inspected, received scores of 60 or higher. A score of 60 indicates that a

property is in satisfactory physical condition. Properties receiving

inspection scores of 60 or higher accounted for just under 1.9 million units,

or about 74 percent of the 2.6 million units receiving assistance in the

properties inspected.

Figure 1: Number of Multifamily and Public Housing Properties With Inspection

Scores of Less Than 60 and 60 or Higher

Number of properties in thousands

0

5

10

15

20

25

30

35

Less

than 60

60 or

higher

Inspection score

5982

33989
B-283608

Page 13 GAO/RCED-00-168 HUD’s Physical Inspection System

Overall, the inspection scores for multifamily properties were slightly

higher than the scores for public housing properties, but for both types of

properties, a significant percentage of the scores were 60 or higher.7 For

example, about 87 percent of the multifamily properties and 80 percent of

the public housing properties received scores of 60 or higher. The

properties receiving scores of 60 or higher accounted for 84 percent of the

multifamily units and 64 percent of the public housing units in the

properties that were inspected.8

A substantial number of properties received high inspection scores. More

specifically, about 38 percent of the multifamily properties and 25 percent

of the public housing properties received scores of 90 or higher. By

contrast, only about 2 percent of the multifamily and about 3 percent of the

public housing properties received scores of 30 or lower, an indication that

these properties may be in severe physical distress.

Despite the high proportion of satisfactory scores, a substantial number of

properties were cited during the inspections for exigent health and safety

problems. For example, almost one-third of the multifamily and public

housing properties received health and safety citations for defects REAC

categorizes as “life-threatening.” Examples of such defects, according to

REAC documents, include missing or inoperable smoke detectors,

electrical system problems, hazards, missing fire extinguishers, and the

presence of security bars. Appendix I provides additional information on

the results of REAC’s baseline inspections.

REAC Reviews Find

Problems With Inspections,

but Effect on Baseline

Results Is Unclear

The results of reviews REAC performs as part of its quality assurance

program show that, while the number of inspections that REAC rejects

during desk reviews has declined substantially, REAC has found problems

with a substantial number of the inspections for which it has performed

follow-up reviews.

7

Our analysis excludes the inspection scores for 164 multifamily properties because we

lacked complete information on the characteristics of these properties.

8

According to data provided by HUD’s Office of Public and Indian Housing, about 87 percent

of the nation’s public housing authorities received composite physical inspection scores of

60 or above. These housing authorities accounted for about 65 percent of the public housing

units.
B-283608

Page 14 GAO/RCED-00-168 HUD’s Physical Inspection System

During desk reviews, REAC staff members review whether contract

inspectors have made errors in such things as the number of buildings at a

property and the number of apartment units selected for inspection. If

errors exist, the inspection report is returned to the inspection contractor

for resolution within 48 hours. If the contractor does not resolve the

problems within 48 hours, the inspection is rejected. According to REAC,

about 10 percent of the inspections submitted to REAC in March 1999,

when REAC began tracking rejection rates, were rejected. But by April

2000, the percentage of inspections rejected during REAC desk reviews had

decreased to less than 1 percent.

However, from February 1999 through the early part of 2000, REAC quality

assurance reviewers continued to find problems with inspections when

performing collaborative and follow-up reviews. During the collaborative

reviews, quality assurance reviewers accompany contract inspectors to

judge their performance. Under follow-up reviews, reviewers test the

accuracy of completed inspections by examining building exteriors,

common areas, mechanical and electrical systems, and site conditions,

such as parking areas and fencing. They then make a judgmental

determination as to whether the original inspection met REAC’s standards.9

As of March 15, 2000, REAC quality assurance reviewers determined that

about 12 percent of the 728 initial inspections they evaluated during

collaborative reviews did not meet REAC’s standards. Furthermore, REAC

reviewers assessed 819 more initial inspections during follow-up reviews.

About 35 percent of those 819 inspections did not meet REAC’s standards.

Among the problems found during such reviews were failures of contract

inspectors to record physical deficiencies, failures to inspect all buildings,

and omissions of health and safety violations. Furthermore, the results of

these reviews, unlike the results of desk reviews, have not improved during

the first year. Between March 1999 and March 2000, REAC quality

assurance staff found little change in the percentage of inspections that

met REAC’s standards.

As part of our review, we accompanied REAC quality assurance inspectors

on 10 follow-up reviews. In each of the 10 cases, the REAC quality

assurance reviewer determined that the original inspection did not meet

9

Initially, REAC classified the inspections according to four categories—highly satisfactory,

satisfactory, less than satisfactory, or unsatisfactory—but in March 2000, REAC

consolidated the four categories into two—within standard and outside of standard.
B-283608

Page 15 GAO/RCED-00-168 HUD’s Physical Inspection System

REAC’s standards and required that a new inspection be performed. Among

the problems that the quality assurance inspectors found were adverse site

conditions that were not recorded, areas that were not included during

inspections, and health and safety violations that were not reported. The

photographs in figure 2 illustrate some of the defects in property conditions

that contract inspectors did not document during their inspections of four

of the properties we visited.
B-283608

Page 16 GAO/RCED-00-168 HUD’s Physical Inspection System

Figure 2: Physical Conditions That REAC Quality Assurance Reviewers Found Contract Inspectors Did Not Record at Properties

GAO Visited

Entryway steps with missing 

hand railing.

Perimeter fencing in need of repair.
B-283608

Page 17 GAO/RCED-00-168 HUD’s Physical Inspection System

Exposed electrical wiring 

in junction boxes.

Damaged floor in 

laundry facility.
B-283608

Page 18 GAO/RCED-00-168 HUD’s Physical Inspection System

While the results of REAC’s quality assurance reviews call into question the

adequacy of some of the inspections performed, it is not clear how the

problems identified affect the overall results of the inspections. Because

REAC performs quality assurance reviews on a targeted rather than a

random basis, it is difficult to project the results of these reviews to all of

the inspections. Furthermore, REAC has not performed any analyses to

assess how the problems identified by the quality assurance inspectors

affected the inspection scores. REAC quality assurance reviewers do not

rescore the condition of properties when performing their reviews, but in

some instances they do require the contract inspectors to reinspect the

properties. The reinspection then generates a new inspection score.

While REAC officials agreed that the percentage of inspections identified

as not meeting REAC’s standards was unacceptable, they indicated that a

precise tolerance level has not yet been established as a guide in evaluating

the reliability of the inspections. In addition, they noted that because the

classification of inspections into one of two categories was based on the

reviewers’ judgments rather than specific criteria, the results of the reviews

might not be as accurate, consistent, or reliable as they should be. REAC

officials told us that they were developing more specific criteria for

reviewers to use in assessing the inspection results.

REAC officials also emphasized that while field reviews are an important

part of the quality assurance process, their results must be considered in

conjunction with other relevant quality assurance information. They did

not believe that the results of the field reviews alone meant that a

substantial portion of the inspections performed were substandard.

Furthermore, REAC officials noted that REAC was in the process of testing

changes that have been made in the inspection protocol. These changes

include revisions aimed at clarifying the definitions for a number of

deficiencies. They believe that after the new inspection protocol is

implemented, there will be fewer problems with the quality of inspections.
B-283608

Page 19 GAO/RCED-00-168 HUD’s Physical Inspection System

Weaknesses Have

Limited the

Effectiveness of

Quality Assurance

Measures, but REAC Is

Making Improvements

REAC deserves credit for establishing quality assurance procedures to help

ensure the reliability of its contract inspections, but our review identified

certain weaknesses that substantially limited the effectiveness of some of

these procedures. More specifically, although REAC has established a welldefined process for performing desk reviews of contract inspections, we

found problems with its processes for performing field quality assurance

reviews of the inspections and for reviewing contractors’ quality control

programs. During our review, REAC began making improvements to

address a number of these weaknesses. For example, it drafted an overall

plan for carrying out its quality assurance program. However, REAC has

been slow to develop formal processes for multifamily housing owners and

public housing authority managers to request changes in their property

inspection scores.

Desk Reviews Appear

Sound, but Field Reviews

and Processes for

Overseeing Contractors

Have Weaknesses

REAC’s desk review process appears to be a well-defined and thorough

quality control mechanism for determining whether contract inspectors are

appropriately entering data into their hand held computers and complying

with other established protocols at the time they perform their inspections.

In addition to tracking each inspector’s error rate, REAC uses the

information to teach the inspection contractors about the types of data

deficiencies that can lead to rejected inspections. For example, REAC

provides contractors with a frequently updated report that shows, by

inspection standard, the types of deficiencies that could result if a standard

is not followed and the action that REAC will take if a requirement is not

met. It also uses the desk reviews to identify trends and patterns that might

suggest a weakness in a particular component of the inspection protocol.

REAC believes that these efforts have helped to lower the rejection rates

for these reviews.

In addition to desk reviews, REAC conducts on-site collaborative and

follow-up reviews to determine whether contract inspections have been

performed in accordance with its standards and protocols. However, we

found that REAC had not established systematic, clearly defined processes

for carrying out these types of reviews. The following illustrate some of the

weaknesses in REAC’s quality assurance processes:

• REAC’s business operating plan for quality assurance did not include

written policies and procedures for REAC’s overall quality assurance

program, nor did it explain how REAC’s quality assurance program

relates to or interfaces with those of the four inspection contractors.
B-283608

Page 20 GAO/RCED-00-168 HUD’s Physical Inspection System

• REAC did not have a systematic process for planning its quality

assurance reviews nor did it have criteria that effectively defined which

inspectors or inspections would be reviewed. For example, REAC set

out to review 4 percent of its contract inspections through collaborative

and follow-up reviews, but by the end of 1999, only 47 percent of the

inspection force had received reviews. Accordingly, REAC accelerated

its reviews, and by March 2000, it had increased the percentage of

inspectors reviewed to 79 percent. As of that date, it had completed

1,543 reviews, thereby approximating the 4 percent target, but 56

percent of the reviews were associated with just 53 of the more than 300

inspectors that had performed inspections as of that date.

• REAC did not have strict criteria on when collaborative and follow-up

quality assurance reviews were to take place. For example, for 374

public housing authority follow-up reviews, as of March 2000, the time

between the contractor’s original inspection and REAC’s follow-up

review ranged from 1 to 394 days, with an average elapsed time of 96

days. Several contractors were concerned that property conditions

could change between the date of the original inspection and the date of

the follow-up review, thus limiting the ability of quality assurance

reviewers to determine and evaluate conditions that existed at the time

of original inspection. One inspection contractor observed that since

REAC had waited so long to point out the inspection problems, it was

sometimes difficult to discuss performance issues credibly with

inspectors.

• REAC did not maintain the records necessary to document actions

taken by inspection contractors to address problems identified during

the reviews. For example, REAC was often unable to tell us when the

results of its reviews were referred to contractors and when contractors

responded with corrective actions. Furthermore, when documentation

was available, there were often delays in communicating the results of

reviews to the inspection contractors. For example, in mid January

2000, REAC gave contractors documentation on 75 inspections that did

not meet REAC’s standards. At least 47 of the quality assurance reviews

had taken place before December 1999, and several dated back to June

1999.

• REAC’s inspection contractors also raised concerns that they were not

being advised of quality assurance reviews that did meet REAC’s

standards so that these results could be considered in the

administration of their own quality control programs.

• REAC did not have written criteria outlining specific disciplinary

actions that should be taken against poorly performing inspectors, nor

did it have a monitoring system to record and track disciplinary actions.
B-283608

Page 21 GAO/RCED-00-168 HUD’s Physical Inspection System

We also found weaknesses in REAC’s oversight of the quality control

programs that REAC required its inspection contractors to establish. The

following are illustrative:

• Although REAC received and reviewed the written quality control plans

of its contractors, it did not perform on-site evaluations to assess the

effectiveness of the contractors’ quality control programs during 1998 or

1999. HUD’s contract management policies recommend a thorough

assessment of contractors’ quality control programs to ensure the

adequacy of quality control systems at the contractor level. These

assessments can also be used to determine the appropriate balance

between HUD’s quality assurance programs and those of the

contractors.

• While REAC’s inspection contracts require contractors to submit

monthly reports describing the results of their quality control programs,

REAC has not always received the reports and did not have a systematic

process for analyzing them and using them to monitor contractors’

performance when they were received. Furthermore, REAC did not

factor in the results of contractors’ quality control programs when

developing its own plans for carrying out quality assurance reviews.

Such coordination could help REAC better target its resources.

• REAC limited its inspection contractors to performing only one type of

review—collaborative reviews. Several contractors have informed

REAC that their quality control programs could be improved if they had

the authority to perform follow-up reviews as well. For example, they

did not believe that the collaborative reviews always give a true

indication of the quality of an inspector’s work because most inspectors

are usually more conscientious when they are being observed during an

inspection.

REAC Is Making

Improvements to Address

Weaknesses in Its Quality

Assurance Processes

During the course of our audit, REAC officials began making a number of

changes to address weaknesses we identified. REAC officials noted that

their quality assurance goal is one of continuous improvement and zero

defects in inspectors’ performance. They acknowledged that the quality

assurance process itself was not fully developed when the new inspection

system began in 1998. Rather, as they noted, the process has evolved and is

continuing to evolve over time through experience, testing, and analysis.

Among the steps REAC took during our audit to improve its quality

assurance processes were the following:
B-283608

Page 22 GAO/RCED-00-168 HUD’s Physical Inspection System

• To strengthen its planning, REAC began, in January 2000, to compare

the number of quality assurance reviews with the total number of

inspections performed by each contract inspector so that inspectors

could be targeted for reviews.

• To improve the objectivity, consistency, and credibility of its quality

assurance reviews, REAC was developing definitive criteria for rating

the quality of contract inspections, and it set a policy goal of performing

follow-up reviews within 30 days of the original contract inspections.

• Also in January 2000, REAC began conducting on-site evaluations of

contractor’s quality control programs and started reporting the results

of every quality assurance review to the contractors, including those

reviews that meet REAC’s standards.

• REAC allowed one of the four inspection contractors to expand its

quality control program by performing follow-up reviews, as well as the

collaborative reviews it was already performing.

• To improve its processes for taking disciplinary action, REAC began

developing criteria for taking specific actions against contract

inspectors. It also started to establish a system to track deficiencies for

individual inspectors and the associated disciplinary actions taken.

More recently, in April 2000, REAC officials told us that they had drafted an

overall plan for carrying out REAC’s quality assurance program. The quality

assurance plan is aimed at establishing procedures in four areas that would

allow REAC to evaluate the (1) inspection contractors’ compliance with

provisions in their contracts and their quality control programs, (2)

inspectors’ performance in applying REAC’s inspection protocol, (3)

accuracy of the inspections and the resulting scores, and (4) performance

of the physical inspection program as indicated by the precision and

replicability of the inspection protocol. REAC officials told us that the

plan’s development was based on both their own plans to develop a more

formalized quality assurance program and the suggestions we made during

our review.

REAC’s establishment of an overall plan for its quality assurance activities

is a positive step toward providing the tools needed to ensure the reliability

of its contract inspections. But because the plan is still in draft and subject

to change, it is difficult to fully assess its merits. According to our

preliminary review of the plan, it is a good start in that it establishes a

framework for REAC’s quality assurance activities. Nevertheless, we are

concerned that some of the procedures in the plan may not allow REAC to

achieve the plan’s four overall objectives.
B-283608

Page 23 GAO/RCED-00-168 HUD’s Physical Inspection System

First, the plan does not specify how REAC will use the information

collected through various quality assurance procedures to measure its

performance against each of the four objectives. For example, the plan

directs REAC, when assessing the performance of the physical inspection

system as indicated by the precision and replicability of the inspection

protocol, to measure and control the level of variance resulting from the

application of the protocol. However, the three procedures that the plan

proposes for accomplishing this task are vague. Specifically, they call for

REAC to identify variances among inspectors, identify variances caused by

inconsistencies in inspectors’ performance, and develop risk indicators and

communicate them to appropriate levels for implementation. It is unclear

precisely what data REAC will be collecting or how the data will be used to

measure the inspection protocol’s precision and replicability. Furthermore,

the plan does not discuss how REAC will report to HUD management and

the Congress on its performance in meeting each of the four objectives.

Second, although the plan recognizes the importance of the inspection

contractors’ compliance with their respective quality control programs, it

does not explain precisely how REAC’s quality assurance objectives and

procedures interface with those of the contractors. For example, the plan

directs REAC to review the contractors’ quality control activities and

requires the contractors to report regularly on their programs, but it does

not specify how REAC will use the information from these reviews and

reports to target its own quality assurance activities, such as its

collaborative and follow-up reviews.

Third, we have some questions about the criteria REAC is using to assess

its performance in meeting the plan’s four overall objectives. For example,

REAC’s criterion for assessing inspectors’ performance during

collaborative reviews allows an error rate as high as 20 percent in certain

areas before an inspector’s performance is classified as not meeting REAC

standards. Such a high error rate appears inconsistent with REAC’s goal of

zero defects in inspectors’ performance. Furthermore, the plan does not

clearly indicate what criteria REAC will use to assess inspectors’

performance when it conducts follow-up reviews. In addition, although the

plan includes steps for reviewing REAC’s processing of property owners’

and public housing authorities’ concerns about their inspection scores, it

does not specify what criteria REAC will use to determine whether these

reviews have been handled appropriately and in a timely manner.

Finally, although the plan indicates how contractors should document

inspectors’ performance for their quality control programs, it does not
B-283608

Page 24 GAO/RCED-00-168 HUD’s Physical Inspection System

specify what records REAC should maintain to document its own actions

when it identifies poorly performing inspectors during collaborative and

follow-up reviews.

HUD Has Been Slow to

Develop and Implement

Formal Processes for

Requesting Changes in

Inspection Scores

In January 2000, after nearly a year and a half, HUD finalized regulations for

providing public housing authorities with a means to request changes or

adjustments in their inspection scores. As of April 2000, HUD had not yet

finalized such regulations for multifamily property owners. HUD had been

using interim guidelines to address their concerns. However, even with the

direction provided by these interim guidelines, HUD was not able to

address many public housing authorities’ and multifamily property owners’

concerns, and requests from both remain pending, including several that

date back to May and June 1999.

As discussed in the background section of this letter, HUD has established

two procedures for public housing authorities and multifamily property

owners to appeal property inspection scores they believe are incorrect.

Specifically, HUD uses (1) technical reviews to identify errors by contract

inspectors and (2) data correction reviews to factor in local ordinances and

building codes that may not have been considered, or may have been

considered inappropriately, during inspections. For example, the owner or

manager of a property may request a data correction review when REAC’s

inspection protocol—applicable nationwide—is not entirely consistent

with a local code requirement or when an owner is incorrectly held

responsible for the condition of items (such as roads and sidewalks) that

the owner does not own and is not responsible for maintaining. Although

HUD considers these procedures critical to the REAC inspection system’s

credibility and acceptance within the housing industry, the Department had

not established the procedures when it implemented the inspection system

in October 1998. Instead, Office of Housing and REAC officials explained,

HUD’s first priority was to get the inspection system up and running and

then to establish procedures for technical and data correction reviews. As a

result, HUD only recently finalized applicable regulations for public

housing authorities, and, as of April 2000, had not yet finalized such

regulations for multifamily property owners. Consequently, it is too soon

for us to assess the effectiveness of these procedures.
B-283608

Page 25 GAO/RCED-00-168 HUD’s Physical Inspection System

As of April 2000, HUD also lacked guidelines for processing data correction

requests from public housing authorities, although it did have interim

guidelines for processing such requests from multifamily property owners.

REAC also developed interim guidelines for processing technical review

requests from both multifamily property owners and public housing

authorities, but as of April 2000, the guidelines were still in draft format.

According to the draft guidelines, the overall goal of the technical review

process is to provide owners and public housing authorities with a quick,

fair, and consistent assessment of their appeals. Moreover, as an official

from HUD’s Office of Housing noted, timely responses to the concerns of

owners and public housing authorities are important because they help

HUD confirm the existence of problems with inspection scores and

determine the need for correction. However, according to HUD’s data on

requests from property owners and public housing authorities for technical

and data correction reviews, HUD was often slow in resolving cases under

the interim guidelines. For example, as of April 2000, HUD had received

about 400 technical review requests, over half of which were still pending.

Although most of the pending requests were submitted during March and

April 2000, several dated back to May and June 1999. Furthermore, of the

requests HUD had resolved, the Department had taken 3 months, on

average, to reach a decision, and had determined, for over half, that

changes in the inspection score were warranted.10 When we asked REAC

officials why cases were not always processed in a timely manner, they

cited delays in the rulings for establishing the review processes and

insufficient staff to process the cases. In addition, one REAC official said

that some cases are more complex and take longer to process. For

example, a request from one public housing authority involved 10 different

properties and took REAC nearly 4 months to complete. REAC officials

told us that they recognize the need to reduce processing times. They said

that as REAC gains additional experience in processing requests for

technical reviews and appeals, they anticipate that the time for completing

them will decrease.

10HUD also corrected the inspection score or grade for health and safety violations for

another 1,400 properties. REAC officials told us that most of these corrections were

required because of an error in the scoring software or differences between local building

standards and REAC’s inspection standards that REAC identified during the initial months

of the inspection process.
B-283608

Page 26 GAO/RCED-00-168 HUD’s Physical Inspection System

Housing Industry

Officials Have

Concerns About the

Reliability of

Inspections and Other

Issues

Although HUD has consulted both the multifamily and public housing

industries during the development and revision of the new physical

inspection process, officials from both industries have periodically

expressed concern about the reliability of the inspections and other

aspects of the inspection process. In response to these concerns, HUD

officials have emphasized that the physical inspection process is relatively

new and still evolving and that HUD intends to continue working with the

industry to improve it. Some industry officials have suggested possible

alternatives to the current system, but none of the groups we contacted had

analyzed the costs or accuracy of these alternatives. The National Academy

of Public Administration is currently reviewing alternative performance

measurement systems for subsidized housing that would include a physical

inspection component.

Both the Multifamily and

Public Housing Industries

Have Concerns About

Aspects of the Physical

Inspection System

During the 2 years that it has been developing and refining its new physical

inspection system, HUD has met with representatives of the multifamily

and public housing industries to obtain their views on the system’s design

and their suggestions for improving certain parts of it. However, officials

from both industries still have concerns about aspects of the system, which

they believe is not yet providing a true picture of the condition of some

properties. Some of their concerns include the following:

• Some of the properties that have been inspected more than once have

received significantly different scores, even though the condition of the

properties has remained more or less the same.

• HUD is not adequately testing changes that it had made in its inspection

software to incorporate revised definitions for a substantial number of

deficiencies.

• The training and skills of some contract inspectors need to be improved.

• HUD is not giving property owners and managers enough time to

prepare and submit documentation for technical reviews, database

corrections, and appeals; these processes are complex and can lead to

confusion; and HUD may not have sufficient resources at its field offices

to implement the procedures efficiently and effectively.

In addition, representatives of some public housing organizations have a

number of other concerns. For example, they said the scoring system

considers an excessive number of deficiencies, including some that are so

minor they should not be considered; too many points are deducted for

some items relative to their importance; defects found in a very small
B-283608

Page 27 GAO/RCED-00-168 HUD’s Physical Inspection System

percentage of the items inspected could result in a property’s receiving a

failing score; and HUD imposes different oversight requirements on public

housing properties and multifamily properties that receive comparable

inspection scores.

In response to these concerns, HUD officials said they are committed to

developing a physical inspection system that accurately portrays the

condition of HUD’s subsidized and assisted properties. At the same time,

they recognize that developing such a system will be an evolutionary

process. To ensure that the system accurately portrays the condition of

HUD housing, officials said that HUD has consulted with representatives

from both industries, will continue to do so, and will make appropriate

revisions. They also cited actions HUD was taking to address the

industries’ concerns as the inspection process was evolving, such as

considering requests for technical reviews even if they are submitted after

HUD’s deadline for filing them. Concerns expressed by the housing

industry representatives and HUD’s responses to them are discussed in

greater detail in appendix II.

Housing Industry Officials

Lacked Data on Alternative

Systems, but the National

Academy of Public

Administration Is

Conducting a Review

While some of the housing industry officials we spoke with suggested

possible alternatives to HUD’s current system, they had not conducted

detailed analyses of these systems. Among the alternatives they suggested

were (1) returning to HUD’s prior system but doing a better job of verifying

the self-certified physical condition information reported by public housing

authorities; (2) turning the current inspection system over to the

owners/managers of the properties, letting the owners/managers conduct

their own inspections using the computer inspection instruments approved

by HUD, and having HUD verify the results on a random basis; (3)

establishing a physical inspection system based on local codes under which

housing agencies or owners would initiate inspections of their properties

using inspectors selected from HUD’s list of certified inspectors; and (4)

using a peer review model that would rely on standards developed, and

agreed to, by members of the housing industry and would have evaluations

of a property’s physical condition conducted by a commission or board

comprised of experts and practitioners in the field. However, the industry

groups did not have detailed information on how these systems would

work, nor did they have information for comparing these systems with

HUD’s current system in terms of cost or accuracy.

The National Academy of Public Administration is currently assessing the

effectiveness of various methods of evaluating public housing authorities
B-283608

Page 28 GAO/RCED-00-168 HUD’s Physical Inspection System

and other providers of federally assisted housing, as required under section

563 of the Quality Housing and Work Responsibility Act of 1999 (P. L. 105-

276). The act specified two primary purposes for the study. First, the study

is to identify and examine various methods of evaluating and improving the

performance of public housing authorities in administering public housing

and tenant-based rental assistance programs and of other providers of

federally assisted housing that offer alternatives to oversight by HUD.

Second, the report is to evaluate whether specific monitoring and oversight

activities currently conducted by HUD should be eliminated, expanded,

modified, or transferred to other entities to increase their accuracy and

effectiveness and to improve monitoring. According to Academy officials,

the study will include an assessment of HUD’s physical inspection portion

of performance measurement. The Academy is to provide a final report on

its findings and recommendations by September 2000.

Conclusions The establishment of a new physical inspection system is a positive step by

HUD to address weaknesses in its oversight of multifamily and public

housing properties. In particular, the establishment of uniform standards

and inspection procedures helps to address inconsistencies in the way

standards were applied to HUD properties and in the way physical

inspections were performed. Equally important, the establishment of

centralized databases for collecting information on the physical condition

of these properties provides HUD not only with detailed, readily available

data but also with a mechanism for (1) ensuring that deficiencies identified

during inspections are corrected and (2) helping HUD take appropriate

action against property owners and housing authorities that fail to provide

housing that is decent, safe, sanitary, and in good repair.

However, there are continuing concerns about the extent to which

inspections performed under the new system are reliable—concerns that

are substantiated by the results of HUD’s own quality assurance reviews.

These concerns increase the importance of HUD’s having a system of

quality assurance procedures that will allow it to assess and report on the

reliability of its inspections and take corrective actions when problems are

found. Such a system includes having effective processes for addressing

multifamily property owners’ and public housing authorities’ concerns

about the results of inspection scores.

REAC has recently taken a number of actions to strengthen its quality

assurance procedures. These include recently preparing a plan that sets

forth the objectives for and procedures to be used in its quality assurance
B-283608

Page 29 GAO/RCED-00-168 HUD’s Physical Inspection System

program. In our view, this is a positive step toward providing HUD

management, the Congress, and others with information for assessing

REAC’s progress in achieving key objectives, such as ensuring that

inspection contractors are complying with the terms of their contracts,

inspectors are performing inspections consistently with REAC’s inspection

protocol, and inspection scores accurately reflect the condition of

properties being assessed. Nevertheless, we are concerned that, in some

areas, the draft plan may not provide REAC with the information it needs to

assess its progress in meeting these objectives.

Recommendations We recommend that the Secretary of Housing and Urban Development

direct the Director of HUD’s Real Estate Assessment Center to revise

REAC’s April 2000 quality assurance plan as necessary to ensure that the

quality assurance activities it contains will provide REAC with the

information it needs to evaluate four key areas—inspection contractors’

compliance with provisions in their contracts and quality control programs;

inspectors’ performance in applying REAC’s inspection protocol; the

accuracy of the inspections and resulting scores; and the performance of

the physical inspection program as indicated by the precision and

replicability of the inspection protocol. The revisions should include

adding information to the plan that describes (1) how the information

obtained through various quality assurance procedures will be used to

assess REAC’s performance in meeting each of the plan’s objectives, (2)

how REAC’s quality assurance activities relate to activities performed by

the inspection contractors as part of their quality control programs, and (3)

what records REAC should maintain to document its actions when poorly

performing inspectors are identified during collaborative and follow-up

reviews. In addition, REAC should reevaluate whether the plan contains

appropriate criteria for quality assurance personnel to use in assessing

each of the activities covered by the plan.

Finally, to provide the Congress with timely information on REAC’s

progress in addressing concerns about the reliability of physical

inspections, we recommend that HUD periodically issue reports describing

the quality assurance activities that it has performed and the results of

these activities.

Agency Comments We provided HUD with a draft of this report for its review and comment.

HUD said while its staff and management had themselves identified many
B-283608

Page 30 GAO/RCED-00-168 HUD’s Physical Inspection System

of the items we pointed out during the course of our audit, we helped the

Department focus on specific areas and suggested improvements that

further strengthened the inspection protocol. HUD stated that the

inspection protocol has enabled the Department to determine the condition

of its entire property inventory, which represents a major modification and

improvement over what existed previously. HUD agreed with us that the

Department needs to improve its quality assurance program but pointed

out that in the past no quality assurance program, consistent standards, or

reliable inspection process existed at HUD.

In response to our recommendation that REAC revise its April 2000 quality

assurance plan as necessary to ensure that the quality assurance activities

it contains provide REAC with the information needed to assess its

performance in achieving the plan’s four key objectives, HUD pointed out

aspects of the plan that are aimed at addressing our recommendation and

noted some modifications that it had made to the plan. HUD also provided

us with an updated version of its quality assurance plan. While this version

is an improvement over the previous plan, it does not fully address our

recommendation. For example, although the plan includes some indicators

for assessing REAC’s performance in meeting the plan’s objectives, it still

does not contain indicators for some key quality assurance activities, such

as follow-up reviews for assessing inspectors’ performance and activities

aimed at assessing the inspection system’s precision and replicability.

REAC officials told us that they were still determining what indicators they

will use to assess the results of quality assurance activities, and they said

they were planning to develop additional indicators. REAC officials also

told us that although the draft plan contains requirements for reviewing

and assessing the results of contractors’ quality control programs, they

recognize that they need to improve coordination between contractors’

quality control activities and REAC’s quality assurance activities. They said

they are considering revisions to the requirements for contractors’ quality

control programs that would more clearly integrate these requirements

with REAC’s overall quality assurance program.

In response to our recommendation that HUD periodically issue reports

describing the quality assurance activities that REAC has performed and

the results of these activities, HUD said that it plans to issue a semi-annual

report summarizing the results of REAC’s quality assurance activities.

REAC officials told us that they were still deciding what information to

include in the report. The complete text of HUD’s comments and our

evaluation of them appear in appendix IV.
B-283608

Page 31 GAO/RCED-00-168 HUD’s Physical Inspection System

We conducted our work from August 1999 through June 2000 in accordance

with generally accepted government auditing standards. (See app. III for a

discussion of our objectives, scope, and methodology).

As arranged with your offices, unless you publicly announce its contents

earlier, we plan no further distribution of this report until 30 days after the

date of this letter. At that time, we will send copies to the Honorable

Andrew M. Cuomo, Secretary of Housing and Urban Development. We will

make copies available to others on request.

If you or your staff have any questions about this report, please call me at

(202) 512-7631. Key contributors to this report are listed in appendix V.

Stanley J. Czerwinski

Associate Director, Housing and

Community Development Issues
B-283608

Page 32 GAO/RCED-00-168 HUD’s Physical Inspection System

List of Requesters

The Honorable Rick A. Lazio

Chairman

The Honorable Barney Frank

Ranking Minority Member

Subcommittee on Housing and

Community Opportunity

Committee on Banking and

Financial Services

The Honorable Christopher S. Bond

Chairman

Subcommittee on VA, HUD and

Independent Agencies

Committee on Appropriations

U.S. Senate
Page 33 GAO/RCED-00-168 HUD’s Physical Inspection System
Page 34 GAO/RCED-00-168 HUD’s Physical Inspection System

Appendix I

Results of Baseline Inspections Conducted by Apendxies

HUD’s Real Estate Assessment Center ApendxiI

As of April 12, 2000, HUD’s Real Estate Assessment Center (REAC) had

completed baseline inspections of 26,528 multifamily properties and 13,607

public housing properties at over 3,100 public housing authorities. These

totals represent about 92 percent of the multifamily properties and all of

the public housing properties that REAC intended to inspect in order to

gather baseline data.

Figure 3 provides information on the percentage of multifamily and public

housing properties whose physical inspection scores fell into one of four

scoring ranges. As the figure shows, a large percentage of the inspection

scores were 60 or above. According to HUD, a score of 60 or above

generally indicates that a property is in satisfactory physical condition.

Furthermore, 38 percent of the multifamily properties and 25 percent of the

public housing properties received a score of 90 or higher. In contrast, only

about 2 percent of the multifamily properties and 3 percent of the public

housing properties received scores of 30 or below. Such scores indicate

that there may be substantial problems with the physical condition of these

properties.
Appendix I

Results of Baseline Inspections Conducted by

HUD’s Real Estate Assessment Center

Page 35 GAO/RCED-00-168 HUD’s Physical Inspection System

Figure 3: Percentage of Multifamily and Public Housing Property Inspections,

Sorted by Inspection Scoring Range

Figure 4 summarizes the percentage of multifamily and public housing

units associated with the properties whose physical inspection scores fall

into one of the four scoring ranges. About 84 percent of the multifamily

units and about 64 percent of the public housing units were in properties

with inspection scores of 60 or higher. About 2 percent of the multifamily

units, and about 9 percent of the public housing units, were in properties

with inspection scores of 30 or below.

Percentage of inspections

0

10

20

30

40

50

60

0 to 30 31 to 59 60 to 89 90 or higher

Scoring range

2% 3%

11%

17%

49%

55%

38%

25%

Multifamily

Public housing
Appendix I

Results of Baseline Inspections Conducted by

HUD’s Real Estate Assessment Center

Page 36 GAO/RCED-00-168 HUD’s Physical Inspection System

Figure 4: Percentage of Multifamily and Public Housing Property Units, Sorted by

Inspection Scoring Range

Our analysis indicates that the baseline physical inspection scores vary

according to location. Table 1 provides information on the total number of

properties inspected, the number of assisted units in these properties, and

the average inspection score for all property inspections within each of the

four regions of the country. Properties in the Northeast and Midwest

regions received lower average inspection scores than properties located in

the South and West. REAC officials cited several factors that may have

contributed to these regional variations, including the facts that properties

in the Northeast and Midwest tend to be older and weather conditions in

those regions are generally harsher. Nevertheless, REAC officials noted

that the United States Housing Act of 1937 requires that all HUD-assisted

facilities, including both assisted and public housing, be in decent, safe, and

sanitary condition and in a good state of repair.

Percentage of units

0

5

10

15

20

25

30

35

40

45

50

0 to 30 31 to 59 60 to 89 90 or higher

Scoring range

2%

9%

14%

28%

49% 48%

35%

16%

Multifamily

Public housing
Appendix I

Results of Baseline Inspections Conducted by

HUD’s Real Estate Assessment Center

Page 37 GAO/RCED-00-168 HUD’s Physical Inspection System

Table 1: Physical Inspection Results for Properties in HUD’s Baseline Inventory, by

U.S. Region

Note: “Other” refers to properties in Puerto Rico and the Virgin Islands. The average inspection score

when these properties are excluded is 78.51.

Our analysis also indicates that physical inspection scores vary according

to the type of financing and assistance received from HUD. Table 2

provides information on the total number of multifamily properties

inspected, the number of assisted units in these properties, and the average

inspection score for all property inspections according to the type of

assistance HUD provides. Properties that receive funding through HUD’s

202/811 programs received the highest average inspection score. The

202/811 programs are aimed at expanding the supply of housing and

supportive services for the elderly and handicapped. Properties whose

mortgages are “held” by HUD received the lowest average scores. HUD

assumed responsibility for these properties after they became financially

troubled, and their financial difficulties may have had an impact on their

physical condition.

Region

Total

properties

Total

assisted

units

Average

inspection

score

Northeast 7,388 698,843 74.95

Midwest 10,694 621,052 77.78

South 14,644 855,789 79.60

West 6,664 299,319 81.25

Other 581 83,261 70.42

Total 39,971 2,558,264 78.40
Appendix I

Results of Baseline Inspections Conducted by

HUD’s Real Estate Assessment Center

Page 38 GAO/RCED-00-168 HUD’s Physical Inspection System

Table 2: Physical Inspection Results for Multifamily Properties in HUD’s Baseline

Inventory, by Type of Financing and Assistance

Type of

financing

Assisted or

unassisted

Total

properties

Total

assisted

units

Average

inspection

score

202/811 Assisted 5,956 250,408 86.77

202/811 Unassisted 57 0 92.61

HUD-held Assisted 683 52,733 68.60

HUD-held Unassisted 87 0 72.62

FHA-insured Assisted 8,448 643,246 77.11

FHA-insured Unassisted 4,629 0 81.30

Noninsured Assisted 6,504 377,963 78.99

Total Not applicable 26,364 1,324,350 Not applicable
Page 39 GAO/RCED-00-168 HUD’s Physical Inspection System

Appendix II

Industries’ Concerns and HUD’s Responses ApendxiI

HUD consulted representatives of the multifamily and public housing

industries during both the development of its physical inspection system

and recent revisions to it. Nevertheless, industry officials whom we spoke

with still had concerns about the system. In general, these officials were

concerned that the inspection results do not consistently provide accurate

information on individual properties. They were also concerned that the

two processes that allow owners and managers to appeal their physical

inspection scores—technical reviews and database corrections—are too

complex and do not allow enough time for housing authorities and owners

to prepare appeals. In addition, public housing industry officials were

concerned about differences in HUD’s treatment of public housing

properties and multifamily properties that receive comparable inspection

scores. In response to these concerns, HUD officials noted that they had

been working with housing industry representatives to address their

concerns and would continue to do so.

Concerns About

Whether Inspection

Scores Consistently

Reflect Properties’

True Physical

Condition

Multifamily and public housing industry officials told us that REAC’s

inspection protocol and scoring algorithm do not consistently produce

inspection scores that accurately reflect the physical condition of the

properties being assessed. They said they were aware of several instances

in which a property’s inspection score changed substantially from the

initial inspection to a more recent one. For example, they noted that one

property received a score of 85 in 1999 and a score of 42 in 2000. The

officials believed that so much change in the score from one year to the

next, when there was little change in the condition of the property, calls

into question the reliability of the inspection system.

In addition, officials from both industries believed that the number of

points properties lose for some deficiencies is out of proportion to their

importance because of the weight that HUD assigns to the deficiencies. For

example, one public housing group identified a number of deficiencies with

point deductions that it considered inappropriate. These included fences

and retaining walls that showed signs of deterioration but still served their

intended functions, minimal accumulations of water in parking lots and

driveways, and play equipment that does not operate correctly but does not

pose a safety risk. As a result, some officials believed that the system was

not fair and generated some scores that were lower than they should have

been.

HUD officials said they are aware of significant discrepancies between

recent and first-year scores for some properties. The officials told us they
Appendix II

Industries’ Concerns and HUD’s Responses

Page 40 GAO/RCED-00-168 HUD’s Physical Inspection System

are reviewing variations in property scores and will develop remedial

alternatives as warranted. In addition, HUD officials said that pending

revisions to the inspection software should make inspection results more

accurate. As to the industries’ concerns about the number of points

associated with some deficiencies, HUD officials believed that at least

some have been addressed by changes that HUD recently made to define a

large number of deficiencies more specifically. However, the officials said

that if concerns persist after the new inspection protocol incorporating the

changes in definitions has been tested and implemented, they will consider

looking at the weights assigned to certain deficiencies.

Concerns About HUD’s

Procedures for Testing

Revised Inspection

Software

In response to the multifamily and public housing industries’ concerns, in

December 1999, HUD made changes to the definitions of a large number of

potential deficiencies that inspectors use in assessing properties’ physical

condition. Although HUD has begun testing the revised software that

incorporates these changes, industry officials believed that HUD should

review its testing procedures. The officials believed that HUD should use a

statistically valid sampling methodology for selecting test sites and that

HUD should conduct complete inspections, using both the revised and old

software simultaneously, rather than looking only at the test results for

deficiencies with revised definitions. After completing the test inspections,

they said, HUD should compare the results to determine how the revised

definitions affected the properties’ scores and whether the revised

software produces more accurate results.

According to HUD officials, HUD has modified its testing procedures twice

to address the industries’ concerns about its selection of test sites and

testing methodology. Initially, HUD conducted tests targeted to deficiencies

with revised definitions, but later it modified its procedures to test the

complete protocol at six public housing authorities. In all, HUD tested the

software at 23 developments in 15 cities. At the 17 properties where HUD

conducted tests using the complete protocol, the scores for 14 increased by

an average of about 5 points when the new software was used.

Concerns About the

Training and Skills of

Some Inspectors

Both multifamily and public housing industries said that significant

variations in the physical inspection scores received by some properties

show that certain contract inspectors need additional training in how to

implement REAC’s inspection protocol. Industry officials also believed that

variations in the scores might mean that HUD is not providing enough on-
Appendix II

Industries’ Concerns and HUD’s Responses

Page 41 GAO/RCED-00-168 HUD’s Physical Inspection System

site training for the contract inspectors. The officials further suggested that

turnover among inspectors might be negatively affecting the quality of

inspections.

In response to these concerns, HUD officials maintained that the

Department’s requirements for inspectors are sufficient to ensure that they

are qualified. The officials noted that the contract inspectors must have a

minimum of 3 years of experience in the building trades and, to be certified,

must pass tests that ensure they can appropriately characterize

deficiencies. While acknowledging that there has been some turnover

among inspectors, HUD officials said the inspection contractors have not

reported problems in hiring replacement inspectors. They also noted that

when HUD has found an inspector’s performance unacceptable, it has

revoked the inspector’s certification to perform HUD inspections. Finally,

they informed us that HUD will retrain and recertify the entire inspector

workforce in the course of implementing the latest version of its inspection

software.

Concerns About

Differences in the

Treatment of

Multifamily and Public

Housing Properties

Public housing officials are concerned that public housing properties and

multifamily properties with comparable inspection scores receive different

treatment from HUD. For example, they said HUD’s regulations designate

public housing properties that receive physical inspection scores of 60 or

lower as “troubled,” whereas multifamily properties do not receive a

comparable designation. They also noted that public housing properties

receiving overall scores of from 60 through 69 points under HUD’s Public

Housing Assessment System (which includes REAC’s physical inspections

as one of its components) must submit an improvement plan to HUD to

eliminate the deficiencies in the housing authority’s performance, whereas

no similar requirement exists for multifamily properties. In addition, they

said that multifamily properties receiving the highest physical inspection

scores are to be reinspected every 3 years, whereas top-performing public

housing authorities will be reinspected every 2 years.

Officials from HUD’s Office of Public and Indian Housing said that some of

the differences in how properties are treated, such as the designation of

public housing properties with scores of 60 or below as troubled, is based

on statutory requirements. HUD officials also said that public housing

authorities are required to file improvement plans only if their overall score

under HUD’s Public Housing Assessment System is from 60 through 69. If a

housing authority receives a score in this range on the physical inspection

component of the system, it does not need to file an improvement plan so
Appendix II

Industries’ Concerns and HUD’s Responses

Page 42 GAO/RCED-00-168 HUD’s Physical Inspection System

long as its overall score is 70 or higher. However, the rule provides that, on

a risk-management basis, a field office/program center may require that a

housing authority with a score of not less than 70 submit an improvement

plan. The officials also said they believed allowing high-performing public

housing authorities to be reinspected every 2 years was an appropriate

incentive, but that they would continue to review this and other incentives

as the new assessment system was being implemented.

Concerns About

Whether HUD Is Giving

Property Owners

Enough Time to

Prepare and Submit

Documentation for

Technical Reviews and

Database Corrections

According to the multifamily and public housing industries, HUD’s

technical review and database correction processes do not give them

enough time to seek redress for problems with inspections. The technical

review process allows owners and housing authorities to appeal their

scores when they believe contract inspectors have made errors in judgment

when assessing the condition of their properties. The database correction

process provides a means for HUD to correct inspection data when local

ordinances or building codes are not factored into the scoring process.

Specifically, industry groups are concerned that 15 days is not enough time

to complete the steps for requesting technical reviews or database

corrections. Industry groups also fear that HUD will accept appeals for

cases only when a change in score will lead to a change in the property’s

“designation” and disposition (e.g., whether the property will be referred to

a HUD office for corrective action). Finally, industry groups are concerned

about the potential for confusion resulting from HUD’s having different

processes for handling technical reviews and database corrections and the

possibility that HUD may not have enough staff to implement the processes

efficiently and effectively.

HUD officials said that the time frames specified in HUD’s regulations for

housing authorities and property owners to request technical reviews and

database corrections are short to minimize the prospects of substantial

change in a property’s condition between the time an inspection occurs and

the time a request is received. However, the officials said that HUD has

processed the requests even if it has received them after the specified time

frames. They also said that different systems are needed for technical

reviews and database corrections because HUD field offices have to be

involved in the database correction process. They also noted that as they

identify instances calling for database corrections (e.g., when they find

differences between local building codes and REAC’s standards), they plan

to modify their inspection database. Over time, corrections in the database

will reduce the burden on field office staff.
Page 43 GAO/RCED-00-168 HUD’s Physical Inspection System

Appendix III

Objectives, Scope, and Methodology ApendxiI

Congressional requesters expressed concerns about whether REAC’s

inspections represent an improvement over HUD’s prior inspection

systems and whether REAC’s inspections are reliable. In response to their

concerns, we agreed to

• discuss the basis for the new physical inspection standards HUD has

established for public and multifamily housing and any differences

between the new standards and inspection procedures and those

formerly used by HUD;

• summarize the results of the first round of physical inspections and

determine what REAC’s reviews of these inspections show about their

reliability;

• determine whether REAC’s quality assurance measures are sufficient to

ensure that the inspections are reliable and that problems with the

inspections are resolved expeditiously; and

• describe concerns expressed by housing industry representatives about

the new system and determine whether they have identified more costeffective systems for assessing the physical condition of HUD’s

properties.

We reviewed HUD’s proposed rules for establishing REAC’s physical

inspection system; these rules lay out HUD’s basis for its new physical

inspection standards. In addition, we reviewed HUD’s statutory

requirements to provide decent, safe, and sanitary housing as set forth in

the United States Housing Act of 1937, as amended. We also reviewed the

requirements in the Department’s contracts with mortgagors of HUDinsured housing. To determine how the new standards and procedures

differ from prior ones, we interviewed (1) HUD officials in REAC, the

Office of Public and Indian Housing, and the Office of Housing and (2)

representatives of public and multifamily housing industry associations.

We also compared the new standards with HUD’s housing quality

standards.

To summarize the results of the baseline inspection data, we obtained two

databases from HUD in early April of this year. These databases contain

information on the first round of inspections of properties in HUD’s

multifamily and public housing portfolios. This information includes, for

each property, the type of property inspected, the date of inspection, the

physical inspection score, and the types of exigent health and safety

violations identified during the inspection. We performed tests for

reasonableness and completeness. When we found discrepancies (e.g.,

duplicate entries or data entry errors) we brought them to HUD’s attention.
Appendix III

Objectives, Scope, and Methodology

Page 44 GAO/RCED-00-168 HUD’s Physical Inspection System

We corrected these discrepancies before conducting our own analyses. To

assess the overall physical condition of these portfolios, we determined the

extent to which properties included in the baseline inventory received

satisfactory physical ratings. To determine what REAC’s reviews of these

inspections showed about their reliability, we analyzed the results of

REAC’s collaborative and follow-up quality assurance reviews.

To determine whether REAC’s quality assurance measures are sufficient to

ensure that inspections are reliable, we interviewed REAC officials,

reviewed contract files and reports, and analyzed data on the different

quality assurance processes and appeals procedures. We analyzed

inspection contracts and visited two of the four inspection contractors,

who provided us with information on their quality control programs. We

also judgmentally selected 10 contract inspections completed at properties

in three different metropolitan areas and accompanied REAC quality

assurance inspectors on follow-up reviews to verify whether each

inspection met REAC’s standards. To determine whether inspection

problems were being resolved expeditiously, we reviewed HUD’s proposed

and final regulations for appeals and comments submitted on the

regulations, and we analyzed appeals data.

To obtain information on the multifamily and public housing industries’

concerns about REAC’s inspection system and on whether the industries

have identified more cost-effective systems for assessing properties’

physical condition, we met with representatives of associations for both

industries, including the American Association of Homes and Services for

the Aging, Council of Large Public Housing Authorities, National

Association of Home Builders, National Association of Housing and

Redevelopment Officials, National Affordable Housing Management

Association, National Center for Housing Management, National Housing

Conference, National Leased Housing Association, National Association of

Realtors, and Public Housing Authorities Directors Association. We also

discussed concerns and alternatives with a selected number of executive

directors, or their representatives, for public housing authorities. Finally,

we met with officials from the National Academy of Public Administration

to discuss their ongoing study.
Page 45 GAO/RCED-00-168 HUD’s Physical Inspection System

Appendix IV

Comments From the Department of Housing

and Urban Development ApendxiIV

Note: GAO’s comments

supplementing those in the

report text appear at the end

of this appendix.
Appendix IV

Comments From the Department of Housing

and Urban Development

Page 46 GAO/RCED-00-168 HUD’s Physical Inspection System

See comment 1.

See comment 2.

See comment 3.

See comment 4.
Appendix IV

Comments From the Department of Housing

and Urban Development

Page 47 GAO/RCED-00-168 HUD’s Physical Inspection System

See comment 5.
Appendix IV

Comments From the Department of Housing

and Urban Development

Page 48 GAO/RCED-00-168 HUD’s Physical Inspection System

The following are GAO’s comments on the Department of Housing and

Urban Development’s letter dated June 9, 2000.

GAO’s Comments 1. In response to our recommendation that HUD periodically issue

reports describing the quality assurance activities that it performs and

the results of these activities, REAC updated its quality assurance plan

to include the preparation of a semi-annual report that will summarize

the overall results of REAC’s quality assurance activities. REAC also

provided us with a 1-page document containing indicators that it had

developed for assessing the results of its quality assurance activities.

However, these indicators do not cover some key quality assurance

activities, such as (1) follow-up reviews for determining whether

inspectors performed inspections completely and correctly and (2)

activities for assessing the inspection system’s precision and

replicability. When we later met with REAC officials to discuss HUD’s

comments, they told us they were still deciding what indicators to

include in the semi-annual report, and they said they were planning to

add indicators to those included in the document they gave to us

earlier.

2. The revised quality assurance plan that HUD provided us still does not

specify how REAC will use its inspection contractors’ quality control

data and information to target its own quality assurance efforts. After

giving us the revised plan, REAC officials told us they recognize that

better coordination is needed between the contractors’ and REAC’s

own quality assurance programs. They said they are considering

revisions in the requirements for the contractors’ quality control

programs that would better integrate them with REAC’s quality

assurance program. The revisions would include more specifically

defining the information that needs to be included in the reports that

contractors prepare on their quality control activities. In addition,

REAC is looking at ways of providing more information to contractors

for them to use in strengthening their quality control efforts.

3. REAC has taken actions to help ensure that poorly performing

inspectors are dealt with appropriately. For example, it has developed

the inspector tracking system that HUD refers to in its comments, and it

has added a step to its quality assurance plan that is aimed at

determining whether an inspector’s substandard performance has been

documented and reported to the contractor. However, given the

problems we identified with REAC’s documentation of actions taken
Appendix IV

Comments From the Department of Housing

and Urban Development

Page 49 GAO/RCED-00-168 HUD’s Physical Inspection System

against contract inspectors, we believe the plan should identify the

records HUD staff need to maintain to document poor performance and

the actions taken to correct it.

4. Although REAC has developed criteria for assessing some of the

activities covered in the quality assurance plan, it is still not clear how

REAC will assess some other important activities, such as its success in

controlling the level of variance in inspection scores resulting from the

application of its inspection protocol. The plan identifies various tests

that will be performed to assess variations in inspection scores, such as

analyses of data from different inspectors for the same or similar sites,

yet it is not clear what level of variation will be regarded as acceptable.

Additionally, it is not clear what criteria will be used to assess the

results of follow-up reviews aimed at determining whether inspections

have been performed completely and correctly.

5. We continue to have concerns about the criteria REAC plans to use

during collaborative quality assurance reviews for assessing whether

inspectors’ performance meets REAC’s standards. REAC’s quality

assurance plan indicates that inspectors could have an error rate as

high as 20 percent in certain areas before their performance is

classified as not meeting REAC’s standards. We recognize that REAC

plans to correct data errors in inspections for those properties where it

has actually conducted the collaborative reviews. However, it is not

clear how REAC will determine whether the inspectors are making

similar errors at other properties that are not reviewed. At our meeting

with them to discuss HUD’s comments, REAC officials told us that it is

not their intention to allow inspectors with error rates as high as 20

percent to continue performing inspections and they would reexamine

the criteria used in the plan and define it more clearly.
Page 50 GAO/RCED-00-168 HUD’s Physical Inspection System

Appendix V

GAO Contacts and Staff Acknowledgments ApendxiV

GAO Contacts Stanley J. Czerwinski, (202) 512-7631

Richard A. Hale, (202) 512-3090

Acknowledgments In addition to those named above, Karen Bracey, Martha Chow, Mark

Egger, Bess Eisenstadt, John McGrail, Sally Moino, Jay Raple, Rose

Schuville, and Don Watson made key contributions to this report.

(385813) Letr
Ordering Information The first copy of each GAO report is free. Additional copies of

reports are $2 each. A check or money order should be made out to

the Superintendent of Documents. VISA and MasterCard credit

cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are

discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

Orders by visiting:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders by phone:

(202) 512-6000

fax: (202) 512-6061

TDD (202) 512-2537

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any list

from the past 30 days, please call (202) 512-6000 using a touchtone

phone. A recorded menu will provide information on how to obtain

these lists.

Orders by Internet:

For information on how to access GAO reports on the Internet,

send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web home page at:

http://www.gao.gov

To Report Fraud,

Waste, or Abuse in

Federal Programs

Contact one:

• Web site: http://www.gao.gov/fraudnet/fraudnet.htm

• e-mail: fraudnet@gao.gov

• 1-800-424-5454 (automated answering system)

United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. GI00