FORECLOSURE 

REVIEW

Lessons Learned 

Could Enhance 

Continuing Reviews 

and Activities under 

Amended Consent 

Orders

Report to Congressional Requesters

March 2013

GAO-13-277

United States Government Accountability Office

GAO
United States Government Accountability Office

Highlights of GAO-13-277, a report to 

congressional requesters

March 2013

FORECLOSURE REVIEW

Lessons Learned Could Enhance Continuing Reviews 

and Activities under Amended Consent Orders 

Why GAO Did This Study

Since April 2011, OCC and the Federal 

Reserve had been overseeing the 

foreclosure review, a requirement of

consent orders entered into by 14 

mortgage servicers. This undertaking

involved a review of loan files by thirdparty consultants to identify errors in 

servicing and foreclosure practices. 

More than 4 million borrowers were 

eligible for reviews. In January 2013, 

the regulators announced agreements 

with 11 of the servicers that replaced 

the reviews with a broad payment 

process to compensate borrowers in a 

more timely manner. Reviews continue 

for three remaining servicers. GAO has 

been reviewing various aspects of the 

foreclosure review process. This report 

addresses: (1) challenges to the 

achievement of the goals of the 

foreclosure review, (2) transparency of 

the process, and (3) lessons that could 

be useful for carrying out activities 

under the amended consent orders 

and continuing reviews. GAO analyzed 

third-party consultants’ sampling plans, 

reviewed regulatory guidance and 

other documents, and interviewed 

representatives of third-party 

consultants and law firms, consumer 

groups, and regulators.

What GAO Recommends

OCC and the Federal Reserve should 

improve oversight of sampling and 

consistency in the continuing reviews, 

apply lessons in planning and 

monitoring from the foreclosure review, 

as appropriate, to the activities of the 

continuing reviews and amended 

consent orders, and implement a 

communication strategy to keep 

stakeholders informed. In their 

comment letters, the regulators agreed 

to take steps to implement the 

recommendations.

What GAO Found

Complexity of the reviews, overly broad guidance, and limited monitoring for 

consistency impeded the ability of the Office of the Comptroller of the Currency 

(OCC) and the Board of Governors of the Federal Reserve System (Federal 

Reserve) to achieve the goals of the foreclosure review—to identify as many 

harmed borrowers as possible and ensure similar results for similarly situated 

borrowers. Regulators said that coordinating among foreclosure review 

participants was challenging, and consultants said that the reviews were 

complex. In spite of regulators’ steps to foster consistency, broad guidance and 

limited monitoring reduced the potential usefulness of data from consultants and 

increased risks of inconsistency. For example, GAO found that guidance was 

revised throughout the process, resulting in delays. Other guidance did not 

specify key sampling parameters for the file reviews and regulators lacked 

objective monitoring measures, resulting in difficulty assessing the extent of 

borrower harm. Good planning and collecting objective data during monitoring 

provide a basis for making sound conclusions. Without using objective measures

to assess sampling or comparing review methods across consultants, regulators’ 

ability to monitor progress toward achievement of foreclosure review goals was 

hindered.

Although regulators released more information than is typically associated with 

consent orders, limited communication with borrowers and the public adversely 

impacted transparency and public confidence. To promote transparency, 

regulators released redacted engagement letters and guidance on remediation. 

In addition, OCC released two interim progress reports. However, some 

stakeholders perceived gaps in key information and wanted more detailed 

information about how the reviews were carried out. Regulators stated they 

considered publicly releasing additional information, but expressed concerns that 

releasing detailed information risked disclosure of confidential or proprietary 

information. Further, borrowers who requested reviews experienced gaps in 

communication. For example, borrowers who submitted requests when the 

submission period opened waited nearly a year before receiving an update. 

The foreclosure review activities to date highlight key lessons related to planning, 

monitoring, and communication. GAO’s prior work shows that assessing and 

using lessons learned from previous experience can benefit the planning of 

future activities. The foreclosure review produced lessons in advanced planning 

and establishing mechanisms to monitor progress toward goals. Without 

assessing and applying relevant lessons learned, regulators might not address 

challenges in the continuing reviews or similar challenges in activities under the 

amended consent orders. In particular, regulators announced the agreements 

that led to the amended consent orders without a clear communication strategy. 

Although the regulators plan to release reports on the results of the amended 

consent orders and the continuing foreclosure reviews, neither regulator had 

made decisions about what information to provide to borrowers. GAO’s internal 

control standards and best practices indicate that an effective communication 

strategy and timely reporting can enhance transparency and public confidence. 

Absent a clear strategy to guide regular communications with individual 

borrowers and the general public, regulators face risks to transparency and 

public confidence similar to those experienced in the foreclosure review.

View GAO-13-277. For more information, 

contact Lawrance L. Evans, Jr. at (202) 512-

8678 or evansl@gao.gov.
Page i GAO-13-277 Foreclosure Review

Letter 1 

Background 4

Complexities of the Foreclosure Review Process and Limitations in 

Regulators’ Guidance and Monitoring May Have Hindered 

Achievement of Goals 10

Limited Communication Hindered Transparency for Individual 

Borrowers and the General Public 39

The Foreclosure Review Experience Could Offer Lessons for the 

Amended Consent Order Activities and Continuing Reviews 43

Conclusions 52

Recommendations for Executive Action 54

Agency Comments and Our Evaluation 55

Appendix I Objectives, Scope, and Methodology 58

Appendix II Comments from the Office of the Comptroller of the 

Currency 65

Appendix III Comments from the Board of Governors of the Federal 

Reserve System 66

Appendix IV GAO Contact and Staff Acknowledgments 69

Figures

Figure 1: Comparison of Two Third-Party Consultants’ Sampling 

Methodologies by Expected Population Error Rate, 

Sample Size, and Triggers for Additional File Reviews 21

Figure 2: Assumed Error Rates and Potential Conclusions from 

Different Sample Sizes Used by Consultants (Assuming 95 

Percent Confidence Level and 3 Percent Precision) 62

Contents 
Page ii GAO-13-277 Foreclosure Review

Abbreviations

AICPA American Institute of Certified Public Accountants

FDIC Federal Deposit Insurance Corporation

FOIA Freedom of Information Act

HAMP Home Affordable Modification Program

OCC Office of the Comptroller of the Currency

OMB Office of Management and Budget

OTS Office of Thrift Supervision

SCRA Servicemembers Civil Relief Act

TARP Troubled Asset Relief Program

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-13-277 Foreclosure Review

United States Government Accountability Office

Washington, DC 20548

March 26, 2013

Congressional Requesters

In 2011, the Office of the Comptroller of the Currency (OCC), the Board of 

Governors of the Federal Reserve System (Federal Reserve), and the 

Office of Thrift Supervision (OTS) issued consent orders against 14 

mortgage servicers after a review of these servicers’ controls over their 

foreclosure processes.1 The consent orders require the servicers to 

engage third-party consultants to review the servicers’ loan files to identify 

borrowers who suffered financial injury through errors, 

misrepresentations, or other deficiencies in foreclosure processes in 2009 

and 2010. The regulators also directed servicers to conduct an outreach 

process to enable eligible borrowers to request a review of their loan 

files.2 Together, both processes comprise the Independent Foreclosure 

Review (foreclosure review). According to regulators, the goals of the 

foreclosure review were for consultants to identify as many harmed 

borrowers as possible, to treat similarly situated borrowers across all 14 

servicers similarly, and to help restore public confidence in the mortgage 

market. The consent orders require the servicers to remediate the 

 

1

The 14 servicers that entered into consent orders with OCC, OTS and/or Federal 

Reserve were: Ally Financial, Inc.; Aurora Bank, FSB; Bank of America, N.A.; Citibank, 

N.A.; EverBank Financial Corp.; HSBC Bank USA, N.A.; JPMorgan Chase, N.A.; Metlife 

Bank, N.A.; OneWest Bank, FSB; PNC Bank, N.A.; Sovereign Bank; SunTrust Bank, Inc.; 

U.S. Bank, N.A.; and Wells Fargo Bank, N.A. and their affiliates or acquired loan servicing 

companies. The Federal Deposit Insurance Corporation (FDIC) was also a party to the 

Federal Reserve’s order with Ally Financial (GMAC Mortgage). The Dodd-Frank Wall 

Street Reform and Consumer Protection Act, Pub. L. No. 111-203, §§ 311-313, 124 Stat. 

1376, 1520-1523 (2010), eliminated OTS and transferred its regulatory responsibilities to 

OCC, FDIC, and the Federal Reserve. The transfer of these powers was completed on 

July 21, 2011, and OTS was officially dissolved 90 days later (Oct. 19, 2011). 

2

Borrowers were eligible to be included in the foreclosure review and have their loan files 

reviewed for errors if foreclosure actions took place on their primary residences between 

January 1, 2009, and December 31, 2010, by one of the participating servicers. A 

borrower who met that initial eligibility criteria and who believed he or she had been

financially injured as a result of problems during the foreclosure process could submit a 

request for review by December 31, 2012. Financial injury includes, but is not limited to, 

the following circumstances: if the mortgage balance at the time of the foreclosure action 

was more than the amount actually owed by the borrower; if the borrower was fulfilling the 

terms of a loan modification but the foreclosure sale still took place; if the foreclosure 

action proceeded while the borrower was protected by bankruptcy; or if the fees charged 

on mortgage payments were inaccurately calculated, processed or applied. 

 
 

Page 2 GAO-13-277 Foreclosure Review

financial harms suffered by borrowers as determined by consultants. 

Roughly 4.3 million borrowers of the 14 servicers were in some stage of 

foreclosure in 2009 and 2010 and, as of December 2012, consultants had 

more than 800,000 loans slated for review.

3 In a turn of events, in January 

2013, the regulators announced agreements that discontinued the 

foreclosure review with 11 of the 14 servicers.4 These agreements 

replaced the foreclosure review with a compensation framework that does 

not rely on determinations of whether borrowers suffered financial harm. 

The servicers participating in the agreements cover nearly 90 percent of 

borrowers who were eligible for the foreclosure review. In late February 

2013, the regulators publicly released amended consent orders for the 

servicers participating in the agreements that formalized the provisions in 

the agreements. The remaining three servicers, covering 450,000 

borrowers (10 percent), are continuing with the foreclosure review work.5

This report represents the second phase of our examination of the 

foreclosure review process. In a previous report, we reviewed the 

 

3

A foreclosure action includes the sale of property due to a foreclosure judgment and the 

referral of a mortgage loan into the foreclosure process. For the purposes of the 

foreclosure review, a foreclosure action is any stage of the foreclosure process. 

4

OCC and the Federal Reserve announced on January 7, 2013, that they had reached 

agreements with 10 mortgage servicers. Those servicers are Aurora, Bank of America, 

Citibank, JPMorgan Chase, MetLife Bank, PNC, Sovereign, SunTrust, U.S. Bank, and 

Wells Fargo. On January 18, 2013, OCC and Federal Reserve announced that an 

agreement had been reached with HSBC. While not part of the original consent orders 

issued in April 2011, two additional institutions, Goldman Sachs (Litton Loan Servicing LP) 

and Morgan Stanley (Saxon Mortgage Services, Inc.), also entered into consent orders 

with the Federal Reserve in 2012 that required a foreclosure review for deficient practices 

in mortgage loan servicing and foreclosure processing. The Federal Reserve announced 

on January 16, 2013, that it had reached agreements with these two servicers. 

Collectively, these 13 servicers are identified as the participating servicers in the 

agreements; however, Goldman Sachs and Morgan Stanley are outside the scope of our 

study because they were not part of the original 2011 consent orders. 

5

The servicers not participating in the agreements are Ally Financial (GMAC Mortgage), 

EverBank, and OneWest. Ally Financial’s mortgage subsidiary, GMAC or Residential 

Capital LLC, entered Chapter 11 bankruptcy proceedings in May 2012. Citing the 

regulators’ agreements with other servicers as evidence for its position that the 

foreclosure review obligation agreed to in the April 2011 consent order can be quantified 

for purposes of the bankruptcy proceeding, GMAC is seeking a court order declaring that 

the foreclosure review obligation is a general unsecured claim and that the automatic stay 

afforded under bankruptcy protection should prevent the Federal Reserve, FDIC and other 

regulators from enforcing the foreclosure review obligation. See Debtors’ Motion at 5, In re 

Residential Capital LLC (Bankr. S.D.N.Y. Feb. 27, 2013) (No. 12-12020). 
 

Page 3 GAO-13-277 Foreclosure Review

servicers’ outreach efforts to inform borrowers of the foreclosure review.6

We were in the process of reviewing other aspects of the foreclosure 

review when OCC and the Federal Reserve announced the agreements. 

We note that this report does not assess the regulators’ rationale for 

accepting the agreements. Further, we do not assess any trade-offs 

involved in the regulators’ choice to amend the consent orders with the 

servicers. We abbreviated our review plan and developed this report to 

address: (1) challenges to the achievement of the goals of the foreclosure 

review, (2) the extent of transparency in the foreclosure review process, 

and (3) lessons that could be useful for the activities under the amended 

consent orders and continuing reviews.

To review third-party consultants’ file review processes, we used a data 

collection instrument and gathered information on third-party consultants’ 

sampling plans, as contained in engagement letters between the 

servicers and consultants. We analyzed information in the consultants’ 

plans for selecting loan files, analyzing file review results, and conducting 

additional sampling of harmed borrowers, if needed. We confirmed key 

observations of our analysis in interviews and site visits with officials 

responsible for developing the sampling plans at five third-party 

consultant engagement teams, interviews with regulator staff, and with 

regulators’ examination teams that reviewed the plans. We selected these 

consultants based on the size of their respective servicer and the identity 

of the servicer’s regulator to ensure a range of perspectives. We reviewed 

relevant documents from regulators, third-party consultants, and law firms 

describing steps taken to foster consistency and accountability among file 

reviews. We also reviewed regulators’ guidance issued to third-party 

consultants, law firms, and examination teams and compared this 

information and these parties’ actions to criteria such as the regulators’ 

standard practices, regulators’ goals for the foreclosure review, and our 

previous work. We reviewed press releases and documents from 

regulators related to the foreclosure review, enforcement action 

documents available on the regulators’ websites, speeches and 

testimonies by agency officials, and agency policies regarding public 

disclosure of enforcement action information. We conducted interviews 

with regulator staff, selected third-party consultants and law firms, and 

consumer groups. For those third-party consultants we did not interview, 

 

6

See GAO, Foreclosure Review: Opportunities Exist to Further Enhance Borrower 

Outreach Efforts, GAO-12-776 (Washington, D.C.: June 29, 2012). 
 

Page 4 GAO-13-277 Foreclosure Review

we obtained written information from them to address our objectives. See 

appendix I for additional information on our scope and methodology.

We conducted this performance audit from July 2012 through March 2013 

in accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.

Mortgage servicers are the entities that manage payment collections and 

other activities associated with mortgage loans. Servicing duties can 

involve sending borrowers monthly account statements, answering 

borrowers’ inquiries, collecting monthly mortgage payments, and 

maintaining escrow accounts for property taxes and insurance. In the 

event that a borrower becomes delinquent on loan payments, servicers 

also initiate and conduct foreclosures. Errors, misrepresentations, and 

deficiencies in foreclosure processing can result in a number of harms to 

borrowers ranging from inappropriate fees to untimely or wrongful 

foreclosure. A number of federal regulators share responsibility for 

regulating the banking industry in relation to the origination and servicing 

of mortgage loans. OCC has authority to oversee nationally chartered 

banks and federal savings associations.7 The Federal Reserve oversees 

insured state-chartered banks that are members of the Federal Reserve 

System, bank and thrift holding companies, and entities that may be 

owned by federally regulated depository institution holding companies but 

are not federally insured depository institutions.8

In September 2010, allegations surfaced that several servicers’ 

documents accompanying judicial foreclosures may have been 

 

7

12 U.S.C. §§ 481, 1813(q)(1). 

8

12 U.S.C. §§ 321, 325, 1844(c)(2)(A)(i)-(ii), 1867. The Federal Deposit Insurance 

Corporation (FDIC) oversees insured state-chartered banks that are not members of the 

Federal Reserve System and state-chartered savings associations. 12 U.S.C. §§ 

1813(q)(2), 1819(a). The Bureau of Consumer Financial Protection also oversees many of 

these institutions and all mortgage originators and servicers that are not affiliated with 

banking organizations, with respect to federal consumer financial law. Pub. L. No. 111-

123, §§1024(a)(1)(A), 1025(a)-(b). 

Background 
 

Page 5 GAO-13-277 Foreclosure Review

inappropriately signed or notarized.9 In response to this and other 

servicing issues, federal banking regulators directed servicers to 

complete self-assessments of their foreclosure processes. In addition, 

banking regulators conducted a coordinated on-site review of 14 of the 

largest mortgage servicers to evaluate the adequacy of controls over 

servicers’ foreclosure processes and to assess servicers’ policies and 

procedures for compliance with applicable federal and state laws. On the 

basis of their findings from the coordinated review, OCC, OTS, and the 

Federal Reserve issued in April 2011 formal consent orders against the 

14 servicers under their supervision.10 To comply with the consent orders, 

each of the 14 servicers is required to, among other things: enhance its 

vendor management, training programs and processes, and compliance 

with all applicable federal and state laws, rules, regulations, court orders, 

and servicing guidelines. In addition, the consent orders required each 

servicer to retain an independent firm to review certain foreclosure 

actions on primary residences from January 1, 2009, to December 31, 

2010. Third-party consultants were permitted to retain outside counsel to 

provide necessary legal expertise in completing the foreclosure review.

 

9

This practice, which includes bank employees or contractors automatically signing 

foreclosure documents without verifying the details contained in the paperwork or the 

validity of the accompanying affidavits, became widely known as “robo-signing.” Failure to 

review documents filed in connection with a judicial foreclosure may violate consumer 

protection and foreclosure laws, which vary by state and which establish certain 

procedures that mortgage servicers must follow when conducting foreclosures. 

10In addition to the foreclosure review, there have been a number of other agency actions 

taken recently against servicers for alleged violations of borrower-protection provisions. In 

May 2011, the Department of Justice settled two cases against Saxon Mortgage Services 

and BAC Home Loans Servicing for allegations that the servicers wrongfully foreclosed 

upon the homes of active duty servicemembers without first obtaining court orders in 

violation of the Servicemembers Civil Relief Act. Pub. L. No. 108-189, § 303, 50 U.S.C. 

app. § 533 (2003); see United States v. Saxon Mortgage Services, Inc., No. 3:11-CV01111 (N.D. Tex. May 26, 2011); United States v. BAC Home Loans Servicing, LP, No. 

2:11-CV-04534 (C.D. Cal. May 26, 2011). The consent orders for each of these cases

dictated that damages be paid to affected servicemembers and remedial actions be taken 

by the mortgage servicers. Id. In addition, in February 2012, the Departments of Justice, 

Treasury, and Housing and Urban Development along with 49 state attorneys general 

reached a settlement with the country’s five largest mortgage servicers: Ally Financial 

(GMAC Mortgage), Bank of America, Citibank, JPMorgan Chase, and Wells Fargo. This 

agreement, known as the National Mortgage Settlement, will provide approximately $25 

billion in relief to distressed borrowers in states that signed on to the settlement and direct 

payments to participating states and the federal government. United States v. Bank of 

America Corp., No. 1:12-CV-00361 (D.D.C. Apr. 4, 2012).
 

Page 6 GAO-13-277 Foreclosure Review

Through the foreclosure review, consultants were to identify borrowers 

who suffered financial injury as a result of errors, misrepresentations, or 

other deficiencies in foreclosure actions, and recommend remediation for 

harms suffered by borrowers, as appropriate. In general, the consent 

orders identified seven areas for consultants to review: (1) whether the 

servicer had proper documentation of ownership of the loan; (2) whether 

the foreclosure was in accordance with applicable state and federal laws; 

(3) whether a foreclosure sale occurred while a loan modification was 

under consideration; (4) whether nonjudicial foreclosures followed the 

terms of the loan and state law requirements; (5) whether fees charged to 

the borrower were permissible, reasonable, and customary; (6) whether 

loss-mitigation activities were handled in accordance with program 

requirements and policies; and (7) whether any errors, 

misrepresentations, or other deficiencies resulted in financial injury to the 

borrower. Servicers proposed third-party consultants to conduct the 

foreclosure reviews. After regulators reviewed the independence of 

proposed third-party consultants, servicers and third-party consultants 

submitted engagement letters outlining their foreclosure review processes 

to the regulators for their review and approval. OCC and the Federal 

Reserve posted approved engagement letters between the servicers and 

third-party consultants on their respective websites.11 Regulators also 

posted on their websites a financial remediation framework that provided 

examples of errors covered by the consent orders and the corresponding 

compensation or other assistance that consultants could recommend 

based on their findings.12

The foreclosure review had two components: a process for eligible 

borrowers to request a review of their particular circumstances (referred 

to as the borrower outreach process) and a review of categories of files 

(referred to as the look-back review). The borrower outreach process was 

intended to complement the look-back review and help identify borrowers 

who may have suffered financial injury. The regulators required the 

servicers to inform borrowers who believed they might have been 

financially harmed due to inappropriate foreclosure that they could submit 

 

11In May 2012, one third-party consultant was dismissed by OCC due to independencerelated concerns. To date, the engagement letter with the replacement third-party 

consultant has not been released publicly. 

12For the purposes of this report, in the context of the foreclosure review we generally 

refer to such payments, compensation, and other assistance as remediation.
 

Page 7 GAO-13-277 Foreclosure Review

a request for review of their particular circumstances. The servicers 

conducted this outreach through advertising and direct mail. After several 

extensions, the final deadline for submission of these requests was 

December 31, 2012.

For the look-back review, the consent orders allowed third-party 

consultants to use statistical sampling techniques to select samples of 

files for review from various categories of loans, pursuant to regulators’ 

guidance and approval as well as 100 percent review of certain loan 

categories. Regulators established minimum sampling requirements for 

third-party consultants to conduct statistically valid sampling of the target 

population and expectations for additional sampling methods, if 

warranted, to identify as many harmed borrowers as possible for 

remediation. Bank examiners often use sampling methods to review the 

files of a financial institution’s operations. The use of sampling by 

examiners has been a regularly accepted practice and is commonly used 

as a means to review a bank’s files when a full review of all files is not 

practicable. OCC states in its handbook on sampling methodologies, 

which is geared toward sampling loan portfolios, that it is impractical or 

impossible for bank examiners to review all items or files when examining 

an area of a bank’s operations, especially if the volume of information is 

large.13 Examiners use sampling to identify a random subset of files to 

learn about the multitude of items from which those files are drawn. Upon 

drawing appropriate statistical inferences from this subset, they can state 

with a certain level of confidence that the inferences apply to the 

population as a whole. The benefits of using statistical sampling include 

the ability to generalize results to the sampled populations and to quantify 

uncertainty in estimates attributable to sampling.

In our prior report on the borrower outreach component of the foreclosure 

review, we found that regulators and servicers had gradually improved 

the communication materials for borrowers but that regulators could make 

further enhancements to the outreach efforts. First, we found that 

regulators did not consider best practices, such as using tests or focus 

groups, to assess the readability of the outreach materials and did not 

solicit input from consumer groups when reviewing initial communication 

materials. As a result, we reported that the materials at that time might 

 

13Office of the Comptroller of the Currency, Sampling Methodologies: Comptroller’s 

Handbook (Washington, D.C.: August 1998). 
 

Page 8 GAO-13-277 Foreclosure Review

have been too complex to be widely understood. Second, we found that, 

although the communication materials included information about the 

purpose, scope, and process for the foreclosure review and noted that 

borrowers may be eligible for compensation, the materials did not provide 

enough specific information about remediation, which best practices 

suggest could have encouraged more borrower responses. Third, we 

found that the outreach planning and evaluation targeted all eligible 

borrowers with limited analysis conducted to tailor the outreach to specific 

subgroups within the population. Therefore, we noted that some 

underrepresented borrowers may not have been apparent to regulators 

without further analysis of the characteristics of respondents compared to 

nonrespondents. To help ensure that all borrowers had a fair opportunity 

for review, we recommended that OCC and the Federal Reserve enhance 

the language on the foreclosure review website, include specific 

remediation information in the outreach, and require servicers to analyze 

trends in borrowers who have not responded and, if warranted, take 

additional steps to reach underrepresented groups.

In response to the recommendations that we made in our previous report, 

OCC and the Federal Reserve took steps to enhance the independent 

foreclosure review website and communication materials and conducted 

more targeted outreach. We determined that regulators implemented our 

first recommendation by including on the foreclosure review website a 

help sheet for the request-for-review form that provides tips in plain 

language, an explanation of key terms, and additional instructions to help 

borrowers fill out the form. In response to our second recommendation, 

OCC and the Federal Reserve publicly released a financial remediation 

framework and included ranges of potential payment amounts or 

categories in their communication materials and other outreach. For 

example, the regulators released 60-second radio and television 

advertisements stating that if consultants find errors, homeowners may be 

eligible for compensation or other remedies, such as refunding fees, 

stopping a foreclosure action, or making payments that could range from 

$500 to $125,000. In addition, at the regulators’ instruction, servicers 

included similar language on ranges of potential payment amounts or 

categories in other outreach materials. In response to our third 

recommendation, OCC and the Federal Reserve tailored their recent 

outreach actions to target communities based on audience 

characteristics, response data, and consumer research. Regulators 

included a wide range of minority media and a broad mix of media 

formats, including print advertisements, radio, television, and Internet. To 

tailor this outreach, a market analysis was completed to identify areas 

and ethnic groups with the greatest opportunity for increased awareness. 
 

Page 9 GAO-13-277 Foreclosure Review

Outreach materials also were made available in eight different languages. 

Regulators solicited numerous community groups for their input and 

assistance on the outreach and identified effective messengers by using 

leaders of community groups that represent minorities to deliver radio and 

television public service announcements.

In January 2013, OCC and the Federal Reserve issued joint press 

releases stating that they had reached agreements with 11 of the 14 

mortgage servicing companies subject to the April 2011 consent orders to 

discontinue the foreclosure review conducted by third-party consultants 

and to provide almost $8.8 billion in cash payments and foreclosureprevention assistance to borrowers. With this change in direction from the 

foreclosure review to an agreed-upon payment process, regulators and 

servicers moved away from identifying the types and extent of harm 

borrowers may have experienced and focused instead on assigning 

borrowers into categories based on objective criteria and issuing 

payments in what they expect will be a shorter amount of time than would 

have occurred under the foreclosure review. To explain their rationale for 

pursuing the agreements, OCC and Federal Reserve staff said they 

considered a variety of factors, including delays in payments to harmed 

borrowers, the total remediation payments expected to be made to 

borrowers, and costs of the reviews. Under the agreements, servicers will 

provide approximately $3.4 billion in direct payments to eligible 

borrowers. 

Using a framework provided by regulators and characteristics of 

borrowers’ loans, servicers will categorize borrowers, and regulators will 

develop a distribution plan and direct a payment administrator to 

distribute cash payments. As a result, all borrowers who were eligible for 

foreclosure reviews under the consent orders are expected to receive 

payments ranging from hundreds of dollars up to $125,000, depending on 

the borrower’s category. Under the agreements, servicers will also 

provide approximately $5.4 billion in foreclosure-prevention assistance to 

borrowers, such as loan modifications. To the extent practicable, 

servicers are to prioritize such assistance for borrowers eligible for the 

foreclosure review. Eligible borrowers are expected to receive notice

about the payments by the end of March 2013, whether or not they filed a 

request-for-review form, and borrowers will not need to take further action 

to be eligible for compensation. Nearly 4 million borrowers, or about 90 

percent of the eligible borrower population, are covered by the servicers 

that signed the agreements. In late February 2013, regulators released 

amendments to the April 2011 consent orders that incorporated the 

provisions of the agreements. The amended orders are publicly available 
 

Page 10 GAO-13-277 Foreclosure Review

on the regulators’ respective websites. Consultants for the servicers that 

did not reach agreements with the regulators—Ally Financial (GMAC 

Mortgage), Everbank, and OneWest—continue their foreclosure review 

activities. More than 450,000 borrowers fall under the three servicers that 

do not have amended consent orders.

• Consent Orders

In April 2011, regulators entered into consent orders with 14 mortgage servicers. 

The consent orders require the servicers to conduct foreclosure reviews by 

engaging third-party consultants to review the servicers’ loan files to identify and 

remediate the financial injuries suffered by borrowers through errors, 

misrepresentations, or other deficiencies in their foreclosure processes in 2009 and 

2010. 

• Agreements

In January 2013, the regulators announced agreements in principle with 11 of the 14 

servicers subject to the April 2011 consent orders. The parties agreed to replace the 

foreclosure review with a compensation framework that does not rely on 

determinations of whether borrowers suffered financial harm, instead requiring

participating servicers to provide cash payments and foreclosure-prevention 

assistance to borrowers.

• Amended Consent Orders

In February 2013, the regulators publicly released amended consent orders that 

formally replaced the requirements related to the foreclosure review for the servicers 

participating in the agreements. 

According to regulators, the goals of the foreclosure review were for 

consultants to identify as many harmed borrowers as possible, to treat 

similarly situated borrowers across all 14 servicers similarly, and to help 

restore public confidence in the mortgage market.14 However, regulators 

faced various challenges in accomplishing these goals.

 

14These goals were identified in publicly issued reports, guidance issued to third-party 

consultants, and during interviews with regulator staff.

Complexities of the 

Foreclosure Review 

Process and 

Limitations in 

Regulators’ Guidance 

and Monitoring May 

Have Hindered 

Achievement of Goals
 

Page 11 GAO-13-277 Foreclosure Review

According to regulator staff and third-party consultants, coordinating the 

foreclosure review process was challenging because of the large number 

of actors directly involved in the review process and the large number of 

borrowers eligible for the review. Specifically, the foreclosure review 

process included 14 servicers; 14 third-party consultant teams from 7 

different consulting and accounting firms, some with subcontractors; and 

more than 10 third-party law firms. Further, the reviews were overseen by 

local examination teams for OCC or the Federal Reserve as well as 

headquarters staff for both regulators. In addition, there were as many as 

4.3 million borrowers whose files could have been reviewed. One 

examination team informed us that the size of the review population 

presented challenges beyond those that accompany a typical consent 

order. Another challenge examiners and consultants noted was that, due 

to the deadline extensions for borrowers to submit their requests-forreview, the volume of the file review work kept changing and the total 

number of files requiring review was unknown. The original deadline for 

requests-for-review was April 2012. Regulators reported that more than

160,000 borrowers had submitted requests-for-review as of April 30, 

2012. The deadline was extended three times to December 31, 2012. 

Regulators reported that more than 510,000 requests were submitted and 

received as postmarked on or before the December 31, 2012 deadline.15

According to OCC, roughly 300,000 additional files were selected for 

review through the consultants’ look-back review process. OCC staff told 

us that more files likely would be selected if errors were found and 

consultants needed to conduct additional sampling to identify as many 

harmed borrowers as possible.

Third-party consultants and law firms told us that the size of the loan files 

and the scope of the file review made the process complicated and timeconsuming. Consultants from whom we obtained information told us that 

a typical loan file is large with many types of documents. For example, 

documents may include servicer notes on communication with the 

borrower, documents collected from the state foreclosure attorneys 

responsible for the foreclosure activities, records of fees charged and 

payments made, and, in many cases, documents assessing a borrower’s 

eligibility for loan modification and loss mitigation activities. Consultants 

 

15Federal Reserve staff informed us that the number of request-for-review forms 

submitted by borrowers for the foreclosure review, as of December 31, 2012, includes the 

gross number of forms submitted and does not account for potential duplicate forms or 

forms filed by borrowers who were not in the scope of the foreclosure review. 

Size, Scope, and 

Complexity of the 

Foreclosure Review 

Process Posed Challenges
 

Page 12 GAO-13-277 Foreclosure Review

told us that some files may contain as many as 50 documents, potentially 

comprising more than 2,000 pages. Consultants also stated that the 

reviews were challenging because they covered such a wide variety of 

complex issues, including different state foreclosure laws, federal laws 

and regulations, and guidelines for federal and servicers’ proprietary loan 

modification programs. To assess each of these areas, consultants 

developed a series of test questions—generally yes or no questions—to 

identify potential errors. The number of test questions used by third-party 

consultants to conduct the file reviews varied. For example, one 

consultant told us that they had approximately 2,600 test questions with 

more than 4,000 discrete steps, while another consultant told us they had 

16,000 test questions.16 Further, third-party consultants from whom we 

obtained information stated that their reviewers spent as many as 50 

hours to complete a full file review, although review times varied 

depending on the issue and type of review.

OCC and the Federal Reserve stated that the uniqueness of each 

servicer’s borrower population and process for recording and storing 

information on borrowers’ loan files posed challenges for defining the 

review parameters and developing a uniform review structure for all the 

consultants. For example, Federal Reserve staff told us that servicers’ 

systems have different ways to identify borrowers who are current on a 

loan modification and that the systems have varying capabilities to 

provide information about the last payment received from a borrower. As 

a result, consultants had to take different approaches to identify 

borrowers with the same characteristics. In addition, Federal Reserve 

staff told us that servicers had different concentrations of loans in 

geographic areas and that, therefore, consultants might select samples 

differently based on these concentrations. For example, if a servicer did 

not have high numbers of foreclosures in a particular state, choosing a 

large sample of loans in that state would not have been appropriate. As a 

result, OCC and Federal Reserve staff said that it was not feasible to 

design one file review process that would apply to all servicers and that it 

was necessary for third-party consultants to tailor their file review 

 

16Not every loan received a full file review. For example, regulators’ guidance states that 

requests for review received through the borrower outreach process should be reviewed 

to analyze the specific complaint raised by the borrower, whereas cases where the 

borrower submitted a more generalized complaint (i.e., “my foreclosure was mishandled”) 

were reviewed for all provisions listed in the consent orders. 
 

Page 13 GAO-13-277 Foreclosure Review

processes, particularly the review questions used to conduct the file 

review, to the unique characteristics of a given servicer.

Faced with various complexities and challenges that the foreclosure 

review posed, regulators told us that they issued guidance and took a 

number of oversight steps. First, the sections of the consent orders 

issued to servicers supervised by both OCC and the Federal Reserve 

outlining the purpose of the foreclosure review were nearly identical.17

According to OCC and Federal Reserve staff, the similarity in the consent 

orders was intended to ensure that the reviews covered the same issues 

and resulted in similar results for similarly situated borrowers. Consultants 

we interviewed said that they designed their reviews to address the 

issues as they were identified in the consent orders. Second, regulators 

issued guidance to third-party consultants to help frame the file review 

process and promote consistency in its implementation. Between May 

2011 and October 2012, regulators issued 29 joint pieces of guidance to 

third-party consultants on various topics. For example, OCC and the 

Federal Reserve jointly issued a financial remediation framework that was 

designed to be a unifying factor among all the reviews by helping to 

ensure that similarly harmed borrowers received similar remediation. 

Regulator staff said that they issued guidance in response to similar 

questions they received from multiple consultants or examination teams, 

which oversaw the reviews at the local level. For example, one third-party 

consultant we interviewed said that the reviews of issues related to the 

Servicemembers Civil Relief Act (SCRA) would likely provide fairly 

 

17The main differences in this section of the consent orders were a different order of topics 

discussed and additional language in the Federal Reserve consent orders specifying that 

borrowers who had requested loan modifications were included in the reviews in addition 

to borrowers with loan modifications under consideration as specified in the OCC orders. 

In addition, the Federal Reserve orders specified that consultants should determine 

whether the amount or rate of fees charged to borrowers exceeded what is customarily 

charged in the market.
 

Page 14 GAO-13-277 Foreclosure Review

consistent results for borrowers due, in part, to the clear guidance 

provided by regulators.18

In addition to consistent consent orders and guidance, OCC and Federal 

Reserve staff implemented regular communication mechanisms to help 

foster consistency in the reviews. Regulators had a robust system of 

regular meetings involving third-party consultants, servicers, examination 

team staff overseeing the consultants’ work, and OCC headquarters and 

Federal Reserve Board staff to discuss challenges with the file review 

process and help promote consistency among the reviews. OCC and the 

Federal Reserve met with the parties both separately and as a group and 

received weekly status reports from the consultants. For example, 

regulator staff said that weekly calls with third-party consultants provided 

consultants with an opportunity to raise areas of inconsistency that they 

had identified. These calls were also used to disseminate new guidance 

and discuss comments on any pending guidance. According to third-party 

consultants and examination team staff we interviewed, these meetings 

were helpful for sharing information among the reviews, developing a 

similar understanding of the file review process, and discussing 

challenges consultants encountered in reviewing files that may have 

affected the consistency of the results for borrowers. In addition to 

meeting with consultants and servicers, OCC and Federal Reserve staff 

also held a separate weekly meeting, which included only headquarters 

staff of each regulator, to discuss new and emerging issues or requests 

for clarification on guidance that each regulator received from its 

respective local examination teams. In addition to the weekly calls, OCC 

headquarters staff and Federal Reserve Board staff visited each of the 

consultants to observe the file review processes.

 

18SCRA restricts the foreclosure of properties owned by active duty members of the 

military. Pub. L. No. 108-189, § 303, 50 U.S.C. app. § 533 (2003). This provision applies 

to loans originated before the servicemember’s active military service. Id. For the 

foreclosure review, to assess compliance with SCRA provisions, consultants generally 

reviewed foreclosure dates and interest rates servicers charged to servicemembers. In a 

recent report we discussed, in part, eligibility for SCRA protections, extent of violations by 

depository institutions, regulators’ oversight of SCRA, and the military services’ efforts to 

educate servicemembers on SCRA. See GAO, Mortgage Foreclosures: Regulatory 

Oversight of Compliance with Servicemembers Civil Relief Act Has Been Limited, 

GAO-12-700 (Washington, D.C.: July 17, 2012). 
 

Page 15 GAO-13-277 Foreclosure Review

The April 2011 consent orders expressly allowed third-party consultants 

to use sampling techniques to identify harmed borrowers.19 According to 

regulator staff, the large number of loans in the review population as well 

as the complexity of the file review process made it difficult for 

consultants to review all the eligible loan files for errors. As a result, the 

review relied on sampling, a process of selecting units—in this case, 

foreclosure files—in a manner that regulators envisioned would allow 

consultants to identify patterns in errors. In May 2011 regulator-issued 

guidance on sampling, regulators expressly allowed third-party 

consultants to use sampling and noted that the consultants’ sampling 

methodologies should take into consideration public perception as well as 

the need to provide a high degree of certainty that borrowers who were 

financially harmed would be identified and obtain remediation.20

The May 2011 guidance outlined broad parameters for how third-party 

consultants should approach the sampling methodology to identify 

harmed borrowers and support a consistent review, such that similarly 

situated borrowers would have similar results. As part of sampling, 

regulators anticipated that third-party consultants would segment the 

review population into loan categories as some loan categories had a 

potentially higher likelihood of errors. For example, the guidance identified 

some potential high-risk loan categories, including certain states, 

foreclosure law firms, and servicing or foreclosure processing activities 

(e.g., rescinded foreclosures or foreclosure that occurred after loan 

modification) that could be associated with a higher likelihood of servicing 

or foreclosure-related errors.21 Regulator staff stated that differences 

 

19Regulators also required servicers to develop a borrower outreach process to 

complement the use of sampling to identify harmed borrowers. 

20According to regulators, the foreclosure review would also have identified servicer errors 

that may not have resulted in financial harm to borrowers. OCC staff told us that 

addressing these types of errors may require additional supervisory steps by regulators. 

OCC staff stated that for the servicers that joined the agreements that led to the amended 

consent orders, they planned to meet with consultants to discuss problematic areas of the 

servicers’ foreclosure process that may warrant additional supervisory steps.

21According to the guidance, each third-party consultant was expected to determine the 

appropriate high-risk loan categories to include in the sampling plan for the servicer they 

were reviewing. The consultants were to determine these categories based on their 

analysis of other studies of the servicer’s foreclosure practices, including internal reviews 

that had identified credible evidence of errors in the foreclosure process that may have 

resulted in financial harm to borrowers. The guidance also suggested that third-party 

consultants conduct random sampling to verify that certain categories were low-risk.

Lack of Clarity in 

Regulators’ Sampling 

Guidance Could Have 

Limited the Usefulness of 

the Information Obtained 

from the Foreclosure 

Review Process
 

Page 16 GAO-13-277 Foreclosure Review

among the servicer’s loan portfolios and servicing practices made 

requiring specific loan categories for review inappropriate, with the 

exception of three categories in which regulators anticipated that 

consultants would identify a relatively large number of errors in the files. 

For these three categories—bankruptcies, SCRA loans, and agencyreferred foreclosure cases—regulators required 100 percent review of 

files.22 The May 2011 guidance also indicated that consultants should be 

prepared to conduct a second stage of additional analysis of files with a 

certain number of servicing and foreclosure-related errors that were 

identified during the initial sampling. 

Based on our analysis of the sampling plans developed by third-party 

consultants, we found that the sampling methodologies used by 

consultants varied among the reviews.

• Expected population error rate. The sampling plans varied in their 

expected population error rate from 0 percent—that is, consultants 

expected to find few or no servicing or foreclosure processing errors 

in the sampled loan category—to 10 percent, with 8 of the 14 reviews 

assuming an error rate of zero, three assuming an expected 

population error rate of 3 percent, and three a rate of 10 percent.23

 

22The guidance suggested that third-party consultants review all loan files where a 

borrower was in bankruptcy during the foreclosure process or when foreclosure occurred, 

all cases involving servicemembers covered under SCRA, and all foreclosure cases 

referred by state or federal agencies. In subsequent guidance, the Federal Reserve 

identified additional loan categories requiring 100 percent review of all files, including 

foreclosure-related complaints filed before the borrower outreach process was launched, 

foreclosure actions where a completed request for a loan modification was pending at the 

time of the foreclosure, and foreclosure actions that occurred when the borrower was not 

in default.

23Not all of the sampling plans explicitly specified an expected population error rate. When 

the consultants developed their sampling methodology based on the regulators’ 

suggested 3 percent precision and 95 percent confidence levels and considering the 

approach outlined in OCC’s Sampling Methodologies: Comptroller’s Handbook—a 

reference that both OCC and the Federal Reserve advised consultants to consider in 

developing their sampling methodologies—a 0 percent error rate is implied. Depending on 

the population size and whether the goal of the sample was to estimate an error rate or to 

test whether an error rate exceeded the 3 percent precision threshold, consultants may 

have designed their sample using different assumptions about the distribution of the 

population. If the estimated population error rate exceeded the expected population error 

rate, the consultant would not meet their expected precision threshold and the resulting 

confidence intervals would be wider than planned. While different distributional 

approximations do not yield large differences in sample size under certain conditions, 

sample size may vary widely depending on the goals of the sample as well as parameters 

such as the expected population error rate.
 

Page 17 GAO-13-277 Foreclosure Review

• Loan category sample size. As a result of different expected 

population error rates, the plans varied in the size of the samples for 

analyzing various loan categories identified by the consultant. For 

example, some consultants selected approximately 100 loans in a 

sampled loan category for a review, and some consultants selected 

approximately 370 loans in the sampled loan categories.

• Loan categories. Based on our analysis, the loan categories used by 

consultants for their analysis varied from review to review. For 

instance, although all the consultants analyzed the files for errors 

related to loan modifications, some categorized loans by the loan 

modification program (e.g., Home Affordable Modification Program 

(HAMP) or proprietary) and others categorized loans for reasons 

modifications were denied.

• Review parameters: For similar loan categories, some consultants 

anticipated conducting 100 percent review of all files in that category 

whereas other consultants planned to sample files. For example, 

some third-party consultants planned to review all rescinded 

foreclosures, whereas others proposed reviewing a sample of those 

loans.24

According to regulator staff, differences in the sampling plans reflected 

differences in the size and characteristics of the servicers’ loan portfolios 

and data systems. Regulator staff explained that they reviewed each 

proposed sampling plan to help ensure it met the parameters outlined in 

the guidance and would result in statistically valid results. However, 

according to OCC staff, they recognized that some consultants had not 

fully implemented the sampling approach as expected, and OCC is taking 

steps to address these differences for one of the servicers that is not 

subject to an amended consent order and must continue its review. 

Our analysis of the May 2011 sampling guidance provided by regulators 

found that the guidance was ambiguous about a key parameter that 

affected consultants’ sampling methodologies and contributed to 

 

24This is distinct from those loan categories where consultants planned to conduct a 

sample of the loan category, but the number of total loans in that category was smaller 

than the minimum number of loans needed for sampling. In those cases, consultants 

reviewed all the loans in the loan category. OCC staff told us that differences in the review 

parameters for similar loan categories may result from consultants deciding to conduct 

100 percent review of some additional categories.
 

Page 18 GAO-13-277 Foreclosure Review

differences in those methodologies. Specifically, in their May 2011 

guidance, regulators did not indicate whether consultants should explicitly 

set an expected population error rate or provide consultants with direction 

on the factors they should consider when setting an appropriate expected 

population error rate to determine the size of the sample used for their 

analysis. GAO’s Financial Audit Manual and the standards of the 

American Institute of Certified Public Accountants (AICPA) have found 

that a key element of effective sampling is the determination of an 

appropriate sample size based on specified precision and reliability levels 

and an expected population error rate or frequency of errors.25

Regulators’ May 2011 guidance on sampling specified that consultants 

should use 3 percent precision and 95 percent reliability levels to 

determine their sample size.26 According to regulator staff, these precision 

and reliability levels were selected to provide a high-level of confidence in 

the sampling results. However, the guidance did not specify an expected 

population error rate for consultants to use in determining sample size. 

Generally, the expected population error rate, like precision and reliability 

levels, is determined based on professional judgment and includes 

consideration of factors such as results of prior reviews and knowledge 

about any potential risks in servicing and foreclosure processing errors.27

According to regulators, they expected consultants to find errors in their 

sampled files, because the consent orders that required the foreclosure 

 

25See GAO and President’s Council on Integrity and Efficiency, Financial Audit Manual: 

Volume 1, GAO-08-585G, (Washington, D.C.: July 2008) and AICPA, Audit and 

Accounting Guide: Audit Sampling 2012. AICPA sets ethical standards for the accounting 

profession and U.S. auditing standards for audits of private companies; nonprofit 

organizations; and federal, state, and local governments. The AICPA audit guide was 

designed to assist auditors to fulfill their responsibilities in designing and performing 

sampling in a financial audit conducted in accordance with generally accepted auditing 

standards. 

26Although not specified in the May 2011 guidance, OCC and the Federal Reserve 

advised consultants to use the OCC’s Sampling Methodologies: Comptroller’s Handbook 

for guidance on sampling. This handbook advises that sample size be calculated based 

on specific precision and reliability levels and when an error is found in a sample, the 

precision level should be recalculated because the sample will no longer satisfy the 

specified precision and reliability levels.

27The expected population error rate, also known as the assumed error rate or assumed 

population error rate, is typically based on an informed analysis of the servicer’s 

population of loans and need not be the same among all reviews. An accurate 

assessment of the expected population error rate helps to ensure that consultants design 

sample sizes sufficient for statistical testing and reporting. 
 

Page 19 GAO-13-277 Foreclosure Review

review process arose out of assessments conducted by regulators that 

identified the potential for servicing and foreclosure-related errors. Similar 

to the precision and reliability levels, there is a relationship between the 

expected population error rate and sample size needed to attain a 

specific precision level (margin of error), where the size of the sample 

generally expands as the expected population error rate approaches 50 

percent.28 When an expected population error rate is not specified, it can 

be interpreted as implying that the error rate is low or even zero—that is, 

that few or no errors are expected to be found in the sample regardless of 

whether the expected population error rate of zero is appropriate for the 

sample goals. OCC staff told us that their handbook on sampling 

methodologies—a reference their staff use for sampling and one that both 

regulators suggested consultants consider in designing their sampling 

approaches—generally assumes few or no errors will be found in a 

sample.29

Variations among the sample sizes used to analyze the various loan 

categories identified by consultants result from differences in the 

consultants’ expected population error rates and led consultants to use 

different triggers to determine when to conduct additional analysis of an 

error or errors found in a loan category.30 As shown in figure 1, Consultant 

A would conduct additional analysis of their sampled loan categories if 

 

28The sample’s variance of the estimated error rate is a function of the population error 

rate. For the same sample size selected from a large population to estimate a proportion, 

this variance is largest when the population error rate approaches 50 percent of the 

population. Therefore, when designing a sample to estimate a proportion in the 

population, the most conservative approach to estimating the sample size assumes that 

the actual population error rate is equal to 50 percent. However, when a sample is 

designed as part of an audit test against an error rate threshold, required sample size 

grows as the expected population error rate approaches the threshold. For example, a 

sample designed to test whether a population with an expected population error rate of 4 

percent is less than a 5 percent threshold is substantially greater than the sample size 

needed to test whether an expected population error rate of 1 percent is less than that 

same 5 percent threshold.

29OCC’s Sampling Methodologies: Comptroller’s Handbook bases its sample size 

calculation on a Poisson distribution. However, some consultants constructed their 

samples based on formulas that assumed a normal or binomial distribution.

30Most of the third-party consultants established a standard sample size for the loan 

categories where sampling was used for their analysis, regardless of the characteristics of 

the loans being sampled in that category. Some consultants specified more stringent 

precision or reliability requirements for certain categories.
 

Page 20 GAO-13-277 Foreclosure Review

one or more errors were found.31 In contrast, Consultant B would conduct 

additional analysis only when five or more errors were found in their 

sampled categories. Both of these approaches use the precision and 

reliability levels specified in the May 2011 guidance, with an expected 

maximum threshold of a 3-percent error rate, but relied on different 

sampling approaches and different expected population error rates to 

calculate sample size. Therefore, with the same number of errors 

identified through file reviews, some consultants would conduct additional 

analysis and some would not. According to regulator staff, when errors 

were found, they expected consultants to conduct additional sampling of 

the loan category or to review all of the loans in that category. OCC staff 

explained that after approving the original sampling methodologies, they 

recognized that their review of the plans had missed some aspects and 

that consultants were using various expected population error rates to 

calculate sample size. OCC staff told us that they were considering steps 

to try and address these differences for one of the servicers that had not 

joined the agreements to end the foreclosure review. As a result of this 

variation, the reviews could have produced inconsistent results for 

similarly situated borrowers, thereby potentially limiting achievement of 

one of the goals of the foreclosure review process.

 

31If no errors were found in the sample, consultant A would have been able to confirm the 

error rate was below 3 percent.
 

Page 21 GAO-13-277 Foreclosure Review

Figure 1: Comparison of Two Third-Party Consultants’ Sampling Methodologies by 

Expected Population Error Rate, Sample Size, and Triggers for Additional File 

Reviews

a

The regulators required 3 percent precision and 95 percent reliability levels for sampling.

b

Sample size is calculated based on the formula in OCC’s Sampling Methodologies: Comptroller’s 

Handbook. 

c

Sample size is calculated based on William G. Cochran, Sampling Techniques, 3rd ed. (New York, 

N.Y.: 1977).

d

Because the samples were designed to generalize to the population of loans, the number of loans 

with errors is used in calculating the error rate rather than the number of individual errors. Although 

some loans may have more than one error, the error rate calculation would treat loans with multiple 

errors as equivalent. For additional discussion of our results, including error rate calculations, see 

appendix I.

e

Additional sampling indicates the consultant would confer with regulators to determine whether to (1) 

draw an additional sample to establish that the error rate was below 3 percent, (2) conduct analysis of 

the error and then draw additional samples related to the characteristics of the loan with the error, or 

(3) conduct a full review of loans in the loan category from which the sample was drawn.

Our analysis also found that the May 2011 guidance on sampling did not 

include a discussion of regulators’ expectations for reporting on sampling, 

and variations among the sampling plans would have limited the types of 

information that regulators could report. For example, the guidance did 

not specify if regulators expected consultants, based on their sampling 

methodology, to be able to provide information on the error rate for the 
 

Page 22 GAO-13-277 Foreclosure Review

servicer’s whole population of eligible borrowers or for certain 

characteristics, such as high-risk loan categories (e.g., states). Our 

analysis found that 1 of the 14 reviews explicitly designed samples that 

would potentially allow consultants to generate reliable estimates of the 

error rate for certain states.32 Other consultants may have been able to 

report counts of errors by state, but some consultants sampled as few as 

one or five cases per state, and the counts would not allow regulators to 

determine whether reported errors were relatively frequent or infrequent 

in specific states. As a result, regulators could not have reported results 

by state because some reviews did not have enough cases, selected in a 

generalizable manner, from individual states to report statistically reliable 

estimates. According to regulators, reporting error rates was not a 

consideration in developing the sampling approach outlined in the 

guidance; rather, the sampling methodology was intended to find as many 

harmed borrowers as possible. However, it would be feasible to develop 

sampling approaches that could both produce error rates and identify 

characteristics of harmed borrowers.

Similarly, differences in the loan categories analyzed by the third-party 

consultants would have limited regulators’ ability to easily aggregate 

results among the reviews or present comparable servicer-specific 

information. For example, differences in how consultants organized the 

loan modification category could have impeded regulators’ ability to 

aggregate the results of the loan modification analyses conducted by 

third-party consultants or to present equivalent information among the 

servicers to the public. In addition, these differences in the loan category 

definitions as well as other differences in the consultants’ sampling 

methodologies would have made it difficult for regulators to aggregate 

results to represent the full population of eligible loans.33 One consultant 

told us that they anticipated calculating an error rate based on the number 

of files reviewed compared to the number of files with errors. This method 

 

32As consultants used different sample sizes and designs, the width of the confidence 

interval around these planned estimates varies. In addition, the width of the confidence 

intervals around these estimates depends on how different the estimated population error 

rate is from the expected population error rate that was used by a consultant to determine 

the necessary sample size.

33For example, for those consultants with loan files in multiple loan categories (i.e., 

overlapping strata) combining the number of errors to estimate an error rate at the loan 

level would not have been feasible because it could result in the same file being counted 

more than one time.
 

Page 23 GAO-13-277 Foreclosure Review

might have provided some information on errors, but could be subject to 

bias if the sample of loans reviewed were not representative of the 

population, regardless of whether the estimates from the samples were 

appropriately weighted for probability of selection or included appropriate 

confidence intervals to reflect sampling error. However, this information 

would not necessarily provide a statistically valid description of the extent 

of errors in the full population that regulators and other stakeholders 

would need to understand and assess the results.

According to regulator staff, they expected consultants to continue 

reviewing files until as many harmed borrowers as possible were 

identified, a goal of the foreclosure review process. Regulator staff told us 

that if third-party consultants’ initial analyses of sampled loans identified 

errors within a loan category, consultants were expected to analyze the 

characteristics of loans with errors to identify any patterns and use this 

analysis as the basis for a second sampling phase or to review all the 

files in a loan category. Although not explicitly stated in the May 2011 

guidance, according to OCC staff, additional analysis was warranted 

when the errors in the sampled loan category exceeded 3 percent with a 

95 percent confidence level. Although the May 2011 guidance on 

sampling did not specify the characteristics to consider for this analysis, 

Federal Reserve staff told us that identifying the characteristics to include 

would require consultants to use their judgment and they anticipated it 

would include things like the loan category itself—such as an error related 

to a certain foreclosure attorney that might warrant additional review of 

other loans that had used the same attorney—or some other 

characteristic about the loan. 

According to a few consultants, they were considering characteristics 

such as the loan product or date of the loan modification solicitation or 

foreclosure sale, as potential characteristics. Regulators told us that 

based on the results of the consultants’ analyses of the initially sampled 

loans with errors, consultants were to develop a methodology—including 

conducting additional sampling or reviewing all files with those shared 

characteristics—to identify other loans that had similar characteristics that 

could have had similar errors to use as a basis for a second review 

phase. According to Federal Reserve staff, this process would have been 

iterative—where consultants would have found errors, analyzed those 

errors, resampled and then repeated the process until as many files with 

errors as possible had been found. Third-party consultants were expected 

to discuss the results of their initial file reviews, their analysis of error 

Regulators’ Sampling 

Approach Hindered Their 

Ability to Monitor 

Achievement of a Review 

Goal
 

Page 24 GAO-13-277 Foreclosure Review

patterns, and their proposed approach for conducting additional file 

reviews with regulators prior to conducting additional reviews.34

Our analysis found that the regulators’ sampling approach did not include 

mechanisms to facilitate their oversight of the extent to which consultants 

would have reached as many harmed borrowers as possible. For 

example, the regulators’ sampling approach did not provide an objective 

method for regulators to use in determining if consultants had conducted 

sufficient reviews and could stop their review activities, except in those 

cases where there were few or no errors.

35 As we described earlier, 

consultants were using different triggers of the number of errors to 

determine if additional analysis of loan categories was required. Without a 

mechanism for regulators to use in assessing the extent to which each 

consultant had found as many harmed borrowers as possible or to 

compare the review results among the consultants, assessing which 

consultants had done enough work to identify a sufficient portion of 

harmed borrowers and which consultants needed to conduct additional 

analysis would have been difficult for regulators. According to Federal 

Reserve staff, they anticipated that consultants would continue reviewing 

files until the sampling found no additional errors. However, unless a 

large majority of the population is examined for errors, reviewing files until 

sampling finds no more errors does not necessarily imply that all or most 

errors would be identified. OCC staff told us that at the time of the 

agreements that led to the amended consent orders, regulators were 

considering developing such mechanisms.

Additional sampling activities—specifically, the use of baseline samples—

could have provided regulators with a mechanism to use in monitoring the 

extent to which consultants had identified as many harmed borrowers as 

 

34The process of conducting additional file reviews based on the analysis of the initial 

sampling results was known as the deeper dive process. According to OCC staff, at the 

time of the agreements that led to the amended consent orders, none of the consultants 

had submitted plans outlining their proposed processes for conducting additional file 

reviews.

35Although not explicitly stated in the guidance, no additional sampling would have been 

required when the error rate for a sampled loan category was below 3 percent.
 

Page 25 GAO-13-277 Foreclosure Review

possible.36 The Office of Management and Budget (OMB) standards for 

statistical surveys state that where sampling is used, it should include 

protocols to monitor activities and provide information on the quality of the 

analyzed data.37 A baseline sample could have been used to establish an 

estimate of the number of harmed borrowers among the servicer’s full 

population as part of their sampling methodology and would have 

provided an objective and consistent measure for regulators to use to 

gauge if a third-party consultant had identified a sufficient portion of 

harmed borrowers through their reviews.38 Specifically, regulators could 

have compared a statistically valid estimate of the number of harmed 

borrowers in the servicer’s total population of eligible borrowers with the 

number of harmed borrowers the consultant identified through file 

reviews. Discrepancies between the estimated number of harmed 

borrowers and the number found in the review would have helped to 

indicate the extent to which there may have been additional harmed 

borrowers who had not been identified.39 Without this type of comparison, 

regulators did not have an objective measure to help determine when a 

consultant had completed sufficient review of the servicers’ files.

 

36We use the term “baseline sample” to refer to a statistical sample that is generalizable to 

the eligible population for that servicer, that is, a random sample (e.g. simple, systematic, 

stratified, or other probability sample) drawn from among all the loans in the eligible 

population for that servicer, which would include those loans where a borrower submitted 

a request-for-review. Baseline samples can include stratification of the population prior to 

sample selection where there was no overlap of the selecting units, such as foreclosure 

files, among the strata, and can also include loan categories that were selected for 100 

percent review.

37OMB, Standards and Guidelines for Statistical Surveys (Washington, D.C.: September 

2006). This document provides 20 standards that apply to work with the statistical 

purposes of describing, estimating, or analyzing the characteristics of groups, segments, 

activities, or geographic areas.

38After completing a review of the entire set of sampled cases, consultants may have 

been able to determine error rates with proper weighting, where consultants used samples 

of loan categories or loan categories with 100 percent review that, combined, covered the 

entire population. However, if the samples were not originally designed to estimate an 

error rate with a margin of error that regulators deemed acceptable for use as an objective 

and consistent measure to gauge progress, then the error rates might not have been 

informative enough for that purpose.

39Statistical tests for difference should account for the sampling variability of the 

estimates. To test whether estimates from two different samples are different requires 

setting a significance threshold for that test and having adequate sample sizes to 

determine such differences.
 

Page 26 GAO-13-277 Foreclosure Review

In addition, the regulators’ sampling approach did not provide a clear 

mechanism for regulators to assess the extent to which consultants had 

identified the appropriate high- and low-risk loan categories to confirm 

that those categories were accurate and to signal if there were additional 

potential high-risk loan categories that had not been identified, but 

warranted additional sampling and review.40 As we noted earlier, the 

regulators’ sampling guidance suggested a number of high-risk loan 

categories consultants should consider in designing their sampling 

approach—such as rescinded foreclosure or foreclosures following loan 

modification—that were potentially associated with a higher likelihood of 

error and, depending on the servicer, warranted targeted sampling to 

verify the extent to which errors had occurred among the loans in those 

categories. However, there is evidence to suggest that other loan 

categories not included in the guidance, such as certain loan categories 

based on demographic characteristics may also be associated with higher 

likelihood of servicer errors.41 Without a mechanism to assess the 

accuracy and sufficiency of each consultant’s high- and low-risk loan 

categories, regulators would not have been able to assess the extent to 

which consultants were targeting the appropriate high- and low-risk 

categories to find as many harmed borrowers as possible.

An estimate of the overall error rate for the population—a rate that could 

have been generated using information from the baseline sample—could 

have been compared with the error rates consultants found for high- and 

low-risk loan categories to confirm that those categories were accurate 

and to signal if there were additional potential high-risk loan categories 

 

40Under certain circumstances, such as where no errors were found in the sample, 

regulators would have been able to determine that a category was low-risk (that is, the 

error rate was below 3 percent).

41Our analysis of Treasury’s HAMP loan modification data found that, holding all other 

things equal, Hispanic and Asian borrowers, compared to white borrowers, were more 

likely to have their trial modifications cancelled for missing documents, and Asian and 

black borrowers had higher cancellation risks for debt-to-income ratios of less than 31 

percent. Our analysis does not allow us to assess whether these results were associated 

with servicer error, but it does raise the possibility that other characteristics associated 

with a loan could also be associated with a greater likelihood of error.
 

Page 27 GAO-13-277 Foreclosure Review

that had not yet been identified.42 OMB has suggested that where 

different characteristics are being compared, agencies should conduct 

additional testing to help ensure appropriate statistical conclusions are 

derived from the data.43 In our prior work we have used this type of 

statistical testing to compare the performance (such as error rate) in a 

total population with the performance for subgroups—such as loan 

categories—to help distinguish among those groups.44 This can be part of 

conducting an overall risk assessment of the population to enable users 

to better focus on high- and low-risk areas. Without a mechanism to 

gauge the extent to which high-risk loan categories had been identified, 

regulators could have had difficulty assessing whether the consultants’ 

proposed activities for additional analysis were targeted at the appropriate 

high-risk loan categories to identify as many harmed borrowers as 

possible.

The May 2011 sampling guidance did not include a requirement that 

consultants develop a baseline sample, but most consultants included 

one in their sampling methodology. According to regulator staff, they had 

not considered requiring consultants to include a baseline sample in their 

methodology because the purpose of sampling was to help find harmed 

borrowers by identifying concentrations of servicing and foreclosure-

 

42The high- and low-risk loan categories could have been refined by comparing the 

estimated prevalence of errors in these groups with the estimated error rate for the overall 

population. In such a comparison, statistically higher rates in a loan category could have 

confirmed a group was high-risk and lower rates in a group could have confirmed the 

group was low-risk. To the extent higher error rates were found in a group that was 

believed to be low-risk, it may have indicated that an additional high-risk group of loans 

could have been found within that low-risk group. This type of assessment of high- and 

low-risk categories would need to be included in the sampling design to allow for statistical 

testing to detect an appropriate level of difference across categories. Regulators guidance 

provided an implicit definition of a low-risk category (an error rate below 3 percent), but did 

not clarify a statistical definition of what constituted a high risk category. In the presence of 

a small number of errors, the sample sizes that several consultants used could not confirm 

that the error rate exceeded 3 percent, but would show only that it was possible that the 

error rate exceeded 3 percent. 

43OMB, Standards and Guidelines for Statistical Surveys, September 2006. 

44For examples of our use of statistical testing to distinguish between results from a total 

population and subgroups of that population, see: GAO, Department of Homeland 

Security: Taking Further Action to Better Determine Causes of Morale Problems Would 

Assist in Targeting Action Plans, GAO-12-940 (Washington, D.C.: Sept. 28, 2012) and 

OPM Revolving Fund: Benchmarking Could Aid OPM’s Efforts to Improve Customer 

Service, GAO/GGD-92-18 (Washington, D.C.: Jan. 21, 1992). 
 

Page 28 GAO-13-277 Foreclosure Review

related errors as described in the consent orders, not to establish an error 

rate for the servicer. Our analysis found that most consultants included a 

baseline sample for the full review population in their sampling 

methodology.45 However, due to differences among the consultants’ 

baseline samples, the ability of regulators to use consultants’ baseline 

sampling results to assess consultants’ activities varies. For example, the 

samples were not designed to estimate actual population error rates and 

regulators’ guidance did not direct consultants to statistically test for 

differences in the servicer’s total eligible population and among high- and 

low-risk loan categories.46

Our analysis also found that the regulators’ approach to conducting 

repeated additional analyses to find as many harmed borrowers as 

possible potentially involved timeliness trade-offs. OMB’s standards for 

statistical analysis state that the sampling design should be appropriate to 

achieve the sampling goals; in this case regulator staff told us that the 

phase-one sampling was designed to identify patterns from among loans 

with errors to facilitate a second phase of analysis to find as many 

harmed borrowers as possible.47 Where the goal of sampling is to identify 

patterns from among loans with errors in a sample, using a larger sample 

size has the potential to provide more information to use in analyzing 

patterns and determining the appropriate next steps. For those 

consultants that used smaller sample sizes in their phase one analysis, 

for example, a sample size of 100 loans where one error would trigger 

additional analysis, finding patterns among the characteristics of those 

loans with errors may have been difficult in cases where there were few 

 

45For those reviews that used baseline samples, sample size varied from approximately 

100 loans to about 6,500 loans for reviews of the servicer’s full population. These samples 

were either drawn explicitly from the full population of loans, or drawn from the entire 

population of loans that was not subjected to 100 percent review and could be combined 

with the 100 percent review populations to cover the entire population.

46For example, the different sample sizes across consultants would result in different 

confidence intervals around their estimates. All else being equal, those consultants with 

larger sample sizes for a given loan category would find it easier to find differences among 

high- and low-risk categories using statistical tests. According to OMB, when statistical 

tests of difference are part of the expected analysis, samples should be designed so as to 

allow for tests at specified levels of difference and reliability.

47OMB, Standards and Guidelines for Statistical Surveys, September 2006. 
 

Page 29 GAO-13-277 Foreclosure Review

errors.48 As a result, these consultants may have had to conduct 

potentially time-consuming repeated sampling of certain loan categories 

where errors were found to find enough loans with errors to be able to 

analyze the findings and identify patterns that would have allowed them to 

conduct additional sampling or review. OCC staff told us that some 

consultants were considering using statistical modeling as part of their 

analysis of loans with errors to identify patterns from among those loans. 

However, in cases where the sample size is small and the number of 

loans with errors is small, the value of statistical modeling to identify 

patterns could be limited. 

According to Federal Reserve staff, due to concerns about the timeliness 

of the sampling process, regulators anticipated that they may have had to 

cut short the sampling process and provide remediation based on certain 

borrower profiles—that is, borrowers who shared characteristics 

associated with loans with a higher likelihood of error—without analyzing 

each of the loan files that shared that profile. In contrast, a larger phase 

one sample size designed with the goal of identifying patterns among 

errors may have provided consultants with more information on loans with 

errors to use as the basis for their additional analysis. In particular, this 

may have been valuable for loan categories with a relatively large number 

of loans—for example, our analysis found that in some cases a loan 

category had more than 10,000 loans and, for one review, as many as 

50,000 loans—and having more information available to use when 

analyzing the results for these categories and determining next steps may 

have been helpful. Although a larger sample size does not guarantee a 

more timely review, a larger statistical sample may have provided 

additional opportunities for analysis even with a similar proportion of 

errors. In the case of the foreclosure review, where the goal was to use 

sampling to find as many harmed borrowers as possible by analyzing 

patterns from among loans with errors, larger sample sizes may have 

resulted in a more timely process and potentially may have identified 

harmed borrowers more quickly. According to regulators, they designed 

their sampling to have a low tolerance for errors so as to find as many 

harmed borrowers as possible.

 

48The sample size of 100 loans was calculated based on the 3 percent precision and 95 

percent reliability levels specified in the regulators guidance and an expected population 

error rate of 0 percent.
 

Page 30 GAO-13-277 Foreclosure Review

According to third-party consultants, regulators’ guidance did not address 

certain aspects of the foreclosure review and consultants had to use 

additional judgment and interpretation when applying certain guidance, 

increasing the risks of inconsistency among review results. Consultants 

also noted that regulators issued critical guidance throughout the review 

process and frequently updated guidance, which expanded the scope of 

the reviews and contributed to delays. For example, the Federal Reserve 

issued three clarifications of loan modification guidance and OCC 

provided seven responses to frequently asked questions on reviews of 

loan modifications. According to regulator staff, developing the 

foreclosure review process was intentionally iterative where they 

responded to the most immediate need and used their evolving 

knowledge to help refine the guidance. Consultants said that changes to 

guidance required them to develop new test questions, re-train reviewers, 

and redo file reviews. Although regulators issued numerous pieces of 

formal guidance and informally responded to specific questions about 

review procedures that examination teams or consultants raised, 

consultants said that some of the guidance issued was not specific, 

leaving room for their interpretation and potentially contributing to 

inconsistent interpretations.

• Guidance on fees: Guidance from regulators generally directed 

consultants to consider whether the fees servicers charged for actions 

such as property inspections or lawn care services were permissible 

under the terms of the loan, followed applicable state and federal law, 

and were reasonable and customary. However, regulators provided 

additional explanation in response to requests from consultants for 

clarification and guidance on defining the terms “reasonable” and 

“customary.” Consultants for 13 of the 14 reviews indicated that they 

used investor guidelines to determine if the fees charged were 

reasonable; however, one consultant told us consultants were using 

different versions of these guidelines. In some cases, consultants 

used additional methods to evaluate whether fees charged to 

borrowers were customary. For example, one consultant evaluated 

fees against a set of benchmarks created from multiple servicers’ 

actual fee charges, while others said they benchmarked only against 

the investor guidelines.

• Guidance on remediation: Although regulators issued guidance to 

consultants on how to determine the appropriate remediation for 

different financial harms, 4 out of the 13 remediation categories would 

have required consultants to make remediation determinations on a 

case-by-case basis, risking inconsistent treatment of borrowers. For 

Broad Guidance and 

Limited Monitoring of 

Inconsistencies Increased 

Risks of Different 

Treatment for Similar 

Borrowers 
 

Page 31 GAO-13-277 Foreclosure Review

example, consultants would have had to determine remediation on a 

case-by-case basis if they found that a servicer had initiated 

foreclosure or foreclosed on a borrower who was protected by federal 

bankruptcy law. Consultants reported taking a range of approaches to 

these case-by-case determinations, including awaiting further 

guidance, developing their own guidelines, or relying on 

recommendations from their third-party law firms. According to 

Federal Reserve staff, at the time of the agreements that led to the 

amended consent orders, regulators were considering options to 

provide additional guidance to consultants for work in these areas, 

including determining remediation amounts. 

• Guidance on missing documentation: Guidance issued to 

consultants in March 2012 on how to determine whether a borrower 

suffered financial harm when key documents—such as documents 

that evidenced that foreclosure actions were taken or loan 

modification application documents—were missing indicated that 

consultants should treat those instances as errors on the part of 

servicers. However, this guidance noted that consultants should defer 

decisions on how to determine borrower remediation until regulators 

provided further guidance. In the absence of additional guidance, 

consultants indicated that they took a variety of approaches, including 

waiting to make remediation decisions, making preliminary 

considerations of whether the error might have caused financial harm, 

or working with their third-party law firm to review any applicable legal 

precedents. According to OCC staff, at the time of the agreements 

that led to the amended consent orders, they were planning to issue 

further guidance to consultants on remediation for borrowers with 

missing documents and they had informally provided additional 

direction about treatment of files with missing documents during their 

regular meetings with consultants.

According to third-party consultants, regulators missed key opportunities 

to increase the likelihood of consistent outcomes for borrowers by not 

requiring development of common criteria or reference materials that 

served as the basis for the foreclosure review process. Third-party 

consultants and their respective law firms we interviewed told us that they 

each developed their own test questions used by their file reviewers to 

determine whether any errors or financial harm occurred. Consultants 

developed the test questions based on analyses of state foreclosure laws, 

loan modification guidelines, and bank policies, among other references. 

According to OCC staff, the state law references were fairly 

straightforward and they had confidence that the third-party consultants 

and law firms would provide fairly consistent interpretations. However, 
 

Page 32 GAO-13-277 Foreclosure Review

according to third-party consultants and law firms we interviewed, 

compiling these references and using them to develop review questions 

was challenging and time consuming and, in some cases, required 

judgment or interpretation of the laws or guidelines. For example, they 

noted that certain areas of relevant state law were unsettled and 

continued to evolve as courts issued decisions. In addition, 

representatives of law firms involved in the reviews told us that each firm 

developed its own list of state foreclosure requirements and often came to 

interpretations different from those of other law firms involved with the 

foreclosure review. Consultants also indicated that some law firms had 

different interpretations of which laws were applicable. According to OCC 

staff, the scope of the reviews was limited to federal and state laws, but 

several consultants reported that they reviewed servicers’ compliance 

with certain county- or court-level requirements, whereas other 

consultants said they generally did not include these requirements. 

Although OCC and Federal Reserve staff told us that law firms were 

selected for their independence and capacity to interpret these 

documents and make these types of decisions, with multiple law firms 

developing their own interpretations of the applicable laws and 

requirements, consultants may have based their test questions on 

different interpretations of laws, which could have hindered regulators’ 

ability to achieve one of their goals for the foreclosure review, similar 

treatment for similarly situated borrowers.

Our analysis indicates that regulators missed another opportunity to 

standardize reference materials in the area of loss mitigation and loan 

modification requirements. Consultants noted challenges in compiling 

relevant loan modification program guidelines. For example, according to 

consultants, they had to compile requirements for multiple loan 

modification programs and representatives of one consultant we 

interviewed said that they had to compile requirements of 40 different 

loan modification programs that the servicer used during the 2009 to 2010 

period. In addition to numerous different programs, the guidelines for a 

single program may have changed multiple times. One consultant noted 

that they had to track and apply 28 program changes the Department of 

the Treasury (Treasury) made to HAMP between 2009 and 2010. Our 

prior work on HAMP identified instances of servicers interpreting HAMP 

guidelines inconsistently, and a consumer advocacy group report noted 

similar challenges with servicers implementing HAMP according to the 
 

Page 33 GAO-13-277 Foreclosure Review

guidelines.49 In addition, several consultants from whom we obtained 

information told us that they had to make some judgment calls when 

interpreting regulatory guidance and program guidelines. For example, 

one consultant noted that HAMP guidelines were unclear in critical 

respects, particularly regarding the nature and extent of servicers’ 

obligations to notify borrowers about steps in the HAMP process. OCC 

staff said they were aware of Treasury’s HAMP guidance and other 

servicing guidelines and made an effort to make their guidance to 

consultants consistent with these materials. OCC and Federal Reserve 

staff also stated that certain staff members had general discussions with 

Treasury staff to understand the HAMP guidelines. Consultants we 

interviewed told us that they largely relied on their internal subject-matter 

experts to interpret the relevant guidelines and did not consult with 

program experts, such as Treasury staff who designed and developed 

HAMP. As a result, third-party consultants and law firms may have 

applied different interpretations of the legal or program requirements for 

the same programs to the reviews, and the use of different reference 

materials could have reduced the likelihood of achieving the goal of 

treating similarly situated borrowers consistently.

Regulators took steps to monitor potential inconsistencies among the 

reviews, but these steps were limited and likely would have resulted in 

delays in providing remediation to borrowers. First, according to 

regulators, they closely monitored weekly reports provided by consultants 

to identify any differences in their progress that may have indicated some 

inconsistency in the foreclosure review processes. These initial reports 

included information on the number of mailings; requests for review; and 

high-level counts of file reviews started, in process, and completed and 

the number of files with borrower harm, but they did not include 

information on the specific types of errors identified in the reviews or the 

test criteria used to review files. Therefore, the usefulness of the reports 

for identifying inconsistencies was limited. According to OCC staff, they 

planned to begin requiring consultants to report on additional information, 

such as the types of errors associated with financial harm found in the 

 

49See GAO, Troubled Asset Relief Program: Further Actions Needed to Fully and 

Equitably Implement Foreclosure Mitigation Programs, GAO-10-634 (Washington, D.C.: 

June 24, 2010) and National Consumer Law Center, At a Crossroads: Lessons from the 

Home Affordable Modification Program (Boston, Mass.: January 2013).
 

Page 34 GAO-13-277 Foreclosure Review

reviews and proposed remediation amounts, which would have helped 

identify inconsistencies among reviews.50 

Second, OCC and Federal Reserve staff said that they would identify 

potential inconsistencies among the reviews by having staff from one 

examination team assist with another team’s oversight of the foreclosure 

review, but these rotations among the staff were not systematically 

organized and did not include rotations across regulators. Further, some 

examination team members we interviewed said that they had 

participated in one or two reviews for other servicers, but examiners 

overseeing reviews at a larger servicer noted that they were unable to 

participate in multiple reviews because of time constraints. In addition, 

OCC and the Federal Reserve did not provide the examination team 

members conducting these rotations with guidance on the types of issues 

to consider in assessing inconsistencies. This unsystematic approach 

limited the extent to which regulators would have been able to identify 

trends or inconsistencies. 

Third, according to regulators, they reviewed some test questions but did 

not compare them across reviews to identify inconsistencies. OCC staff 

acknowledged that inconsistencies were inherent in the foreclosure 

review because of the large number of actors and decision points and the 

subjective nature of some of the decisions consultants had to make. 

Similarly, Federal Reserve staff told us that inconsistencies among 

reviews were inevitable due to differences among the servicers’ policies, 

procedures, and systems, including their loan modification and loss 

mitigation programs. Regulator staff said they had planned to conduct 

assessments of the extent of inconsistencies affecting the outcomes for 

borrowers across the reviews after the reviews and recommendations for 

remediation were completed. However, conducting such an assessment 

after the reviews were completed could have resulted in delays in 

remediation because third-party consultants may have needed to change 

their file review questions and redo file reviews if regulators identified 

inconsistencies. In addition, regulators would have had to wait until all 

consultants had completed their reviews to conduct such an assessment. 

Consultants reported that they had estimated completing their reviews in 

 

50OCC staff told us that at the time the agreements were made to end the file review 

processes for 11 of the 14 servicers, they were close to implementing this updated 

reporting template and they plan to use this template for the continuing reviews.
 

Page 35 GAO-13-277 Foreclosure Review

different time periods and an OCC official estimated that the reviews 

would not have been completed until 2014.

Our analysis of the foreclosure review process identified challenges in 

regulators’ planning for the foreclosure review. These challenges 

underscored the importance of the following three aspects of planning 

and monitoring: (1) identifying the type and amount of information to 

report as final results during the design of data analysis; (2) consulting 

with stakeholders; and (3) assessing how well the reviews were tracking 

their goals of identifying as many harmed borrowers as possible, 

achieving consistent reviews for borrowers, and helping to restore public

confidence in the mortgage market.

As described earlier, variations among the 14 sampling methodologies 

used by third-party consultants for analysis of loan files limited the types 

of information that regulators would have been able to report. According 

to regulator staff, they did not want to make final decisions on the types of 

information they may have needed for public reporting before they had 

seen the preliminary results from the file reviews. OMB has found that in 

designing data analysis activities, including sampling, agencies should 

consider the types of data they need to collect to be able to report useful 

information on the results of their activities to the intended audience.51

The resulting sample design should have these data output requirements 

built into the structure. GAO’s internal control standards state that 

producing reliable and relevant data is important for oversight and 

management, including oversight provided by Congress.52 In addition, our 

prior work has found that public reporting of results can be important for 

strengthening public confidence in a process, and the data analysis 

strategy should be designed to include information that can be reported 

publicly.53 Providing useful and relevant public reporting of the review 

results also was a key element in renewing public confidence in the 

mortgage servicing market, a goal of the foreclosure review process. 

 

51OMB, Standards and Guidelines for Statistical Surveys, September 2006. 

52See GAO-AIMD-00-21.3.1.

53GAO, Troubled Asset Relief Program: Additional Actions Needed to Better Ensure 

Integrity, Accountability and Transparency, GAO-09-161 (Washington, D.C.: Dec. 2, 2008) 

and Quantitative Data Analysis: An Introduction, GAO/PEMD-10.1.11 (Washington, D.C.: 

June 1992).

The Foreclosure Review 

Highlighted the 

Importance of Planning 

and Monitoring Progress 

toward Goals

Data Analysis Design 
 

Page 36 GAO-13-277 Foreclosure Review

However, regulators were limited in what they would have been able to 

report because they did not plan for reporting in the design of the reviews.

As described earlier, the scope of the foreclosure review was broad and 

regulators experienced challenges in issuing guidance on the wide variety 

of complex issues covered by the reviews, resulting in delays in 

completing file reviews and potentially contributing to inconsistencies in 

the file review process. Regulators may have been able to better define 

the scope of activities and issue more complete guidance prior to 

commencing the foreclosure review process, thereby potentially reducing 

the number of revisions to the scope or guidance, by consulting with 

organizations directly responsible for or familiar with particular aspects of 

the review before initiating the foreclosure review process. For example, 

although regulators consulted with Treasury staff for their technical 

expertise on the HAMP requirements during their development of the loan 

modification and loss mitigation guidance, regulators did not have the 

benefit of additional consultations with Treasury compliance officials and 

discussions with other agencies responsible for overseeing federal loan 

modification and loss mitigation programs to help them more clearly 

define the programs covered by the consent order requirements and the 

elements to consider in assessing servicers’ evaluation of loan 

modification and loss mitigation activities—areas where regulators issued 

clarifying guidance to third-party consultants.54 Regulators issued 

additional guidance to clarify that the review of HAMP and proprietary 

loan modification programs should include programs overseen by the 

U.S. Department of Housing and Urban Development, U.S. Department 

of Agriculture, and U.S. Department of Veterans Affairs. 

In addition, regulators did not consult with community groups, such as 

national organizations representing housing counselors that have worked 

 

54Federal loan modification and loss mitigation programs are overseen by the U.S. 

Department of the Treasury and the U.S. Department of Housing and Urban 

Development, as well as the housing government-sponsored enterprises (Fannie Mae and 

Freddie Mac), among others. Our prior work has identified a number of challenges 

servicers and borrowers faced with loan modification and loss mitigation programs. For 

example, see GAO, Foreclosure Mitigation: Agencies Could Improve Effectiveness of 

Federal Efforts with Additional Data Collection and Analysis, GAO-12-296 (Washington, 

D.C.: June 28, 2012); Troubled Asset Relief Program: Results of Housing Counselors 

Survey on Borrowers’ Experiences with the Home Affordable Modification Program,

GAO-11-367R (Washington, D.C.: May 26, 2011); and Troubled Asset Relief Program: 

Treasury Continues to Face Implementation Challenges and Data Weaknesses in its 

Making Home Affordable Program, GAO-11-288 (Washington, D.C.: Mar. 17, 2011).

Stakeholder Consultation
 

Page 37 GAO-13-277 Foreclosure Review

with individual borrowers on their loan modification and loss mitigation 

applications. These consultations might have provided input on 

challenges specific servicers and borrowers experienced with the range 

of loss mitigation and loan modification activities which could have 

assisted in identifying high-risk loan categories or program elements to 

consider. For example, our prior work surveying housing counselors 

found that while assisting borrowers with HAMP applications, counselors 

experienced challenges with servicers, including missing documentation, 

lengthy decision-making processes, and miscalculations of borrowers’ 

incomes.55 Our prior work that establishes generally accepted project 

planning practices identified consulting with stakeholders as one of seven 

generally accepted practices.56 In addition, our internal control standards 

have found that consulting with external stakeholders can have a 

significant impact on the achievement of goals.57 In contrast to the 

process used to develop the loan modification and loss mitigation 

guidance, OCC and the Federal Reserve consulted with consumer 

groups while developing the remediation framework. In addition, they 

consulted with the U.S. Department of Justice in developing the 

foreclosure review guidelines related to SCRA. Although this consultation 

occurred later in the process after third-party consultants had begun 

SCRA reviews, one consultant cited the guidance that resulted from the 

consultations as evidence of a strong practice that helped promote 

consistent results among the reviews.

As we described earlier, regulators’ sampling approach did not include 

mechanisms that would allow them to objectively measure the extent to 

which consultants were on track to identify as many harmed borrowers as 

possible.58 Similarly, as we previously described, regulators had a limited 

process in place to identify inconsistencies among consultants’ file review 

 

55See GAO-11-367R. 

56See GAO, Coast Guard: Civil Rights Directorate’s Action Plans to Improve Its 

Operations Could Be Strengthened by Implementing Several Aspects of Project Planning 

and Implementation Practices, GAO-10-571T (Washington, D.C.: Apr. 27, 2010). In this 

report, we analyzed the Government Performance and Results Act of 1993 and conducted 

an external literature review to identify and adapt seven practices associated with 

generally accepted project planning management and practices.

57See GAO/AIMD-00.21.3.1. 

58For those reviews that included a baseline sample, some may not have provided 

sufficient information to generate informative estimates of the number of harmed 

borrowers because they were not designed with that intent. 

Monitoring
 

Page 38 GAO-13-277 Foreclosure Review

processes that could have affected results for borrowers. OCC and 

Federal Reserve staff told us that they were aware of some areas where 

there may have been inconsistencies, but according to Federal Reserve 

staff these areas would not have led to significant differences. However, 

in the absence of mechanisms to systematically monitor consistency, 

regulators did not have the information to verify their understanding or 

identify areas of the review with an increased likelihood of inconsistency. 

Our prior work has identified using intermediate activities or measures to 

assess progress toward intended results as an effective management 

practice to understand the extent to which activities are on track to reach 

stated goals.59 We have also identified practices that can help agencies 

successfully implement the Government Performance and Results Act 

and related results-oriented management initiatives, such as establishing 

activities during program planning and design to monitor performance 

toward these goals and using intermediate activities to analyze the gap 

between where the performance is and where it needs to be to achieve 

desired outcomes.60 We found that such activities can help management 

target areas in need of improvement and select appropriate 

methodologies to realize that improvement. In the absence of systematic 

processes to monitor the extent to which the foreclosure review was 

progressing toward its goals, regulators did not have an early warning 

mechanism to help identify problem areas where interventions, such as 

the issuance of clarifying guidance or other support, might have helped 

reorient activities and address concerns.

 

59See GAO, Agency Performance Plans: Examples of Practices that Can Improve 

Usefulness to Decisionmakers, GAO/GGD/AIMD-99-69 (Washington, D.C.: Feb. 26, 

1999). In this report, we identified and described practices that if consistently applied 

might improve the usefulness of agencies annual performance plans. For an example, see

GAO-10-634. 

60See GAO, Executive Guide: Effectively Implementing the Government Performance and 

Results Act, GAO/GGD-96-118 (Washington, D.C.: June 1996). For this report, we 

identified and described the practices most helpful to successfully implementing the 

Government Performance and Results Act and related results-orientated management 

initiatives. This report and other reports we have issued on the Government Performance 

and Results Act are intended to suggest frameworks for Congress and federal agencies to 

use in implementing the act and related management initiatives.
 

Page 39 GAO-13-277 Foreclosure Review

Regulators publicly released information on the foreclosure review 

process beyond what is typically disclosed in connection with a consent 

order, including engagement letters between servicers and consultants 

and some guidance provided to the consultants. By law, federal banking 

regulators must disclose any formal enforcement actions entered into 

under the Federal Deposit Insurance Act.61 On a case-by-case basis, 

banking regulators may consider the release of information beyond the 

mandatory disclosures.62

In an effort to promote transparency in the foreclosure review process, 

OCC and the Federal Reserve publicly disclosed some information 

related to the April 2011 consent orders. For example, in November 2011, 

OCC released redacted engagement letters between the servicers under 

its jurisdiction and the consultants contracted to conduct the foreclosure 

review. With the exception of one servicer, the Federal Reserve released 

by February 2012 redacted engagement letters for servicers under its 

jurisdiction.

63 OCC and the Federal Reserve also released the 

remediation framework for consultants to use that provided examples of 

situations in which compensation or other remediation is required for 

financial injury due to servicer errors, misrepresentations, or other 

deficiencies.

Despite these disclosures, some stakeholders perceived gaps in key 

information about how the file reviews were conducted. Regulators 

released documents, such as the redacted engagement letters and 

remediation framework, which generally described the design and 

 

6112 U.S.C. §1818(u). 

62The Freedom of Information Act (FOIA), 5 U.S.C. § 552, generally provides that any 

person has a right of access to federal agency records, unless the records, or any portion 

thereof, are protected from disclosure by one of FOIA’s nine exemptions. Records 

pertaining to the supervision of financial institutions are subject to one of FOIA’s 

exemptions. 5 U.S.C. § 552(b)(8). Despite that exemption, regulators may exercise 

discretionary disclosure authority under 12 C.F.R. § 4.12(c) and 12 C.F.R. § 261.14(c) for 

OCC and the Federal Reserve, respectively, to release records concerning financial 

institution supervision.

63The Federal Reserve released the redacted engagement letter for the last institution in 

May 2012.

Limited 

Communication 

Hindered 

Transparency for 

Individual Borrowers 

and the General 

Public

Transparency on How 

Foreclosure Errors and 

Financial Harm Would Be 

Determined Was Generally 

Lacking 
 

Page 40 GAO-13-277 Foreclosure Review

intended outcomes of the foreclosure review, but they did not disclose the 

more detailed guidance and tests consultants relied on to perform their 

reviews. As previously discussed, third-party consultants developed 

thousands of test questions to determine error and harm, and regulators 

issued a number of guidance documents to promote consistency among 

the reviews and clarify issues raised by consultants, such as how 

consultants were to construct their sample populations and how to 

address issues related to borrowers covered by SCRA. Although 

regulators released their remediation framework and provided answers to 

frequently asked questions on remediation categories and calculations, 

they did not release any additional guidance documents nor did they 

publicly disclose consultants’ test questions, which according to OCC, 

contained proprietary and supervisory information.64

To increase the transparency and credibility of the foreclosure review for 

borrowers, policy makers, and the public, among other stakeholders, 

consumer groups recommended that regulators release such information. 

According to consumer groups, without such information, the public would 

have questions and doubts about how the reviews were executed. OCC 

and the Federal Reserve staff said that they considered releasing 

additional guidance to the public, but both regulators refrained from doing 

so because of concerns that releasing detailed information risked 

disclosure of confidential or proprietary information. Moreover, test 

questions developed by consultants were numerous and complex, and 

Federal Reserve staff stated that review processes were too dissimilar to 

provide a comprehensive summary.

Borrowers who requested reviews under the foreclosure review process 

initially received limited information about the status of their individual file 

review. Borrowers received a letter acknowledging their request was 

received, but some did not receive updates until almost a year after the 

outreach program was launched, when they received a letter informing 

them of the continuing nature of the review. In letters to OCC and the

Federal Reserve, consumer groups indicated that these borrowers were 

frustrated by the lack of information on their particular file review. Eligible 

borrowers could submit requests as early as November 2011. According 

 

64OCC staff noted that they also arranged for community group representatives and 

congressional staff to meet with consultants for an overview of the testing processes.

Borrowers and the General 

Public Received Limited 

Information about the 

Progress of Reviews
 

Page 41 GAO-13-277 Foreclosure Review

to OCC staff, borrowers who submitted an accepted request-for-review 

through June 30, 2012 received a status update letter in September 2012. 

OCC staff said that the letters communicated to borrowers that their 

requests were being reviewed but that the results of the review might not 

be available for several more months. They said that the letters also 

provided a brief summary of the foreclosure review process, an Internet 

link to the interagency remediation framework, and notice of other help 

available through nonprofit organizations approved by the Department of 

Housing and Urban Development. Regulators indicated that additional 

status letters would be sent to borrowers with outstanding requests-forreview. Before the regulators halted the foreclosure review, draft letters to 

be sent to borrowers on the results of their file reviews were in 

development. However, regulators were still uncertain about specific 

information they would require be shared with both borrowers who would 

receive remediation and those who would not. Regulators have 

acknowledged the importance of transparency, but when announcing the 

agreements that led to the amended consent orders, they had not yet 

determined what information to convey beyond that which was included in 

their press releases and public websites, nor had they determined 

whether additional information would be provided to borrowers who 

submitted a request-for-review.

During the foreclosure review process, OCC released two interim reports 

that provide the public with information on the organization and conduct of 

the file review process and preliminary results, such as the number of 

requests-for-review received, for institutions it supervises. The OCC 

reports—issued in November 2011 and June 2012—summarized the 

status of actions taken to correct deficiencies in mortgage servicing and 

foreclosure processing identified in the April 2011 consent orders, 

including activities related to the foreclosure review. For example, the 

June 2012 report included the number of files selected for review, the 

number of requested reviews, and the number of reviews completed, 

among other items. These reports, according to OCC, were intended to 

build transparency in the process. The Federal Reserve did not issue 

interim reports on the foreclosure review process for institutions it 

supervised. According to Federal Reserve staff, they did not do so 

because they determined that their public release of servicers’ action 

plans provided sufficient information about how servicers were 

addressing the requirements of the consent orders and their public 

release of servicers’ engagement letters provided sufficient information 

about how the foreclosure review would be conducted. Prior to the 

announcement of the agreements that led to the amended consent orders 

and ended the foreclosure review for most servicers, OCC staff told us 
 

Page 42 GAO-13-277 Foreclosure Review

they had planned to release a final report on the results of the foreclosure 

review. After the agreements were announced, Federal Reserve staff 

indicated they expected to publish additional relevant information related 

to the foreclosure review and the agreements. However, as of February 

2013, regulators had not decided what specific information will be made 

available on the work conducted under the foreclosure review prior to the 

agreements.

While OCC and the Federal Reserve acknowledged the importance of 

transparency in the foreclosure review process, the absence of timely and 

useful communications at certain stages of the process—for individual 

borrowers as well as the general public—hindered transparency and 

undermined public confidence in the processes and results. In a 

December 2011 letter to consultants, OCC noted the importance of public 

confidence in the foreclosure review process. According to the Federal 

Reserve, the agency endorsed OCC’s letter. However, as previously 

discussed, regulators did not publicly release detailed information that 

described how consultants were to determine errors and remediation, and 

borrowers and the general public received limited information about the 

status of the reviews. As a result, consumer groups raised concerns 

about the level of transparency of the foreclosure review process and 

indicated that the absence of public information undermined credibility 

and public confidence in the process.

Our internal control standards state the importance of relevant, reliable, 

and timely communications within an organization as well as with external 

stakeholders.65 As illustrated in Treasury’s implementation of the Troubled 

Asset Relief Program (TARP), external communications can include 

posting information on its website and regular, public reporting. Our 

previous work on TARP described the importance of public reporting as a 

means to improve transparency and address potential questions of 

whether similarly situated borrowers are being treated fairly.66 For 

example, Treasury periodically issued public reports on the progress and 

performance of its TARP housing programs. These reports have provided 

information to a range of stakeholders, including Congress and the 

general public. Similarly, consumer groups recommended that regulators 

 

65See GAO/AIMD-00-21.3.1.

66See GAO-10-634. 

The Foreclosure Review 

Highlights the Importance 

of Communication to 

Transparency
 

Page 43 GAO-13-277 Foreclosure Review

provide regular, public reports on the progress and findings of the 

foreclosure review to increase transparency. Consumer groups also 

suggested that regulators provide additional information to address 

transparency-related issues for borrowers eligible for remediation through 

the foreclosure review. For example, they indicated that borrowers should 

have had access to information about the status of the review of their file 

and receive a thorough explanation of how decisions were reached. In 

addition, they recommended that the review guidelines issued by the 

regulators be made public, similar to Treasury’s release of guidance for 

HAMP. According to consumer groups, these actions, among others, 

could have increased the public’s understanding of the reviews and 

allowed borrowers to ensure their files were properly reviewed. The 

foreclosure review, unlike TARP and TARP-funded programs such as 

HAMP, was not a government program. However, as OCC described, it 

was part of a larger set of government-directed actions in response to the 

housing and mortgage crises. More publicly disclosed information about 

processes and regular reporting about the status of the reviews would 

have increased transparency and thereby public confidence in the 

reviews, given that one of the goals regulators articulated for the 

foreclosure review was to restore public confidence in mortgage markets.

The foreclosure review revealed three key lessons that could help inform 

regulators’ implementation of the amended consent orders: (1) designing 

project features during the initial stages of the process to influence the 

efficiency of file reviews, (2) monitoring progress to better ensure the goal 

of achieving intended results, and (3) promoting transparency to enhance 

public confidence. These key lessons on planning and implementation 

could help contribute to an effective process for distributing direct 

payments and other assistance as prescribed by the amended consent 

orders between servicers and regulators. In addition, these lessons could 

inform the foreclosure review process that is continuing for the servicers 

that did not reach agreements with regulators.

The foreclosure review experience suggests that a planning process to 

determine key project features, such as guidance and necessary data 

elements, for activities conducted under the amended consent orders 

could lessen the risk of changes to the planned activities, future delays, or 

rework. Our work on designing evaluations, including financial audits, has 

The Foreclosure 

Review Experience 

Could Offer Lessons 

for the Amended 

Consent Order 

Activities and 

Continuing Reviews

Sound Planning of Project 

Design Features Is Key to 

Efficiency 
 

Page 44 GAO-13-277 Foreclosure Review

found that systematic and comprehensive planning enhances the quality, 

credibility, and usefulness of the results and contributes to a more 

effective use of time and resources.67 We found that one of the first steps 

in designing a review should be to define the purpose and scope of the 

review, including defining what data will be collected and what 

comparisons will be made. In addition, evaluation questions should be 

clear and specific, and should use terms that can be defined and 

measured so that the purpose and scope are readily understood and 

feasible. Our prior work establishing a lessons-learned process also has 

found that assessing and using lessons learned from previous experience 

can provide a powerful method of ensuring that beneficial information is 

factored into the planning and work processes of future activities.68 Key 

practices of assessing lessons learned include collecting and analyzing 

information on prior activities and applying that information to future 

activities.

As the foreclosure review experience suggests, some implementation 

challenges were inevitable due to the unprecedented nature of the 

review. Nevertheless, the broad and expanding scope of the reviews and 

delays in defining key concepts could have been mitigated by more 

advanced planning from regulators, resulting in more efficient and 

effective reviews. The April 2011 consent orders provide a general 

description of the scope of the file reviews consultants were to conduct. 

According to third-party consultants, these parameters resulted in a broad 

 

67According to regulator staff, the foreclosure review was an unprecedented and unique 

action by regulators. Although it is not a program evaluation or an audit, insight from 

program evaluations and audits could have helped regulators in developing and 

overseeing the foreclosure review process—in particular, the focus on planning as a key 

step in preparing activities. In assessing the foreclosure review, we considered our prior 

work on program evaluations, government auditing standards, and financial auditing. See 

GAO, Designing Evaluations: 2012 Revision, GAO-12-208G (Washington, D.C.: January 

2012); Government Auditing Standards: 2011 Revision, GAO-12-331G (Washington, D.C.: 

December 2011); and GAO and President’s Council on Integrity and Efficiency, Financial 

Audit Manual: Volume 1, GAO-08-585G (Washington, D.C.: July 2008). 

68Our prior work has defined a lesson as knowledge or understanding gained by both 

positive and negative experiences that when studied and applied can result in a change. 

See GAO, Federal Real Property Security: Interagency Security Committee Should 

Implement A Lessons-Learned Process, GAO-12-901 (Washington, D.C.: Sept. 10, 2012) 

and NASA: Better Mechanisms Needed for Sharing Lessons Learned, GAO-02-195

(Washington, D.C.: Jan. 30, 2002). In our 2002 report, we established a lessons-learned 

process based, in part, on research done by the Naval Research Laboratory at the Navy 

Center for Applied Research in Artificial Intelligence. In 2012, we updated this work 

through a literature review and interviews with agencies.
 

Page 45 GAO-13-277 Foreclosure Review

review scope that generally covered all instances of noncompliance with 

applicable law and servicing guidelines as they related to the post-default 

residential mortgage loan borrower experience, including instances of 

noncompliance that resulted in financial harm to borrowers requiring 

remediation. Regulators may have missed opportunities to potentially 

narrow and refine the project scope—for example, through earlier 

definition of a harmed borrower or agreement on errors not resulting in 

remediation that may not have warranted additional review. Changes to 

guidance also expanded the scope of the reviews. For example, as we 

noted previously, changes to loan modification guidance contributed to an 

expanded scope for those reviews and delays in completing the work, 

including, in some cases, redoing file reviews.

According to regulator staff, developing the file review process was 

iterative; they learned as the reviews progressed, and regulators used 

that knowledge to help refine the review process. In our work on 

designing evaluations and audits, we recognize that a review process can 

be iterative and that the scope and activities of the review may change as 

work progresses and data limitations or new information arise.69

Nevertheless, conducting a planning process that involves all 

stakeholders provides an opportunity to examine preliminary information 

and pilot-test processes and procedures to help further define the scope 

of potential activities and hedge against the risk of future changes. In 

addition, assessing lessons learned by using project critiques and 

discussions with key participants and stakeholders—such as local 

examination team staff, third-party consultants and law firms, and external 

groups—could identify the root causes of strengths and weaknesses of 

the foreclosure review that could apply to the amended consent order 

activities. According to regulator staff, they are meeting with examination 

staff and third-party consultants to discuss challenges with the servicers’ 

data that may make it difficult for servicers to determine borrowers’ direct 

payment amounts under the amended consent orders. OCC staff also 

said that they had discussions with other federal agencies and consumer 

advocacy groups before announcing the agreements that led to the 

amended consent orders.

As regulators prepare to implement the amended consent orders and 

compensate borrowers, they may encounter delays and inconsistencies 

 

69See GAO-12-208G. 
 

Page 46 GAO-13-277 Foreclosure Review

similar to those associated with the foreclosure review if they miss 

opportunities to make key project planning decisions. Regulators told us 

that they anticipated borrowers would be contacted by the end of March 

2013 and that they also expected the issuance of checks to begin in April. 

However, regulators still need to make some key decisions about these 

activities in order to meet their goal of beginning borrower payments in 

April 2013, and clear guidance in several areas will facilitate the process.

• According to regulator staff, prior to contacting borrowers each 

servicer will assign borrowers to direct payment categories.70 OCC 

staff said that the servicers were instructed to use objective data and 

definitions and guidance from regulators in placing borrowers in the 

categories and that the examination teams would use the guidance 

provided to servicers to conduct their validation process. Regulator 

staff said that they agreed that clear guidance on the categorization 

and file review process and validation by examination teams are tools 

that will help the servicers and examination teams navigate this 

complex process.

• In most cases, servicers, with regulators’ approval, have engaged the 

third-party consultants to review borrowers’ files in two categories 

(SCRA and foreclosed borrowers who were not in default) to 

determine whether borrowers experienced those specific types of 

harm. According to one third-party consultant, at the time of the 

agreements that led to the amended consent orders, consultants were 

waiting on additional guidance from regulators to complete aspects of 

these reviews.

• In addition, regulators have not yet determined payment amounts for 

the different categories of borrowers. Regulator staff said that they 

could not make these decisions until servicers had completed their 

categorization process and regulators knew the number of borrowers 

in each category to allow them to divide up the total payment amount 

among the borrowers.

• Regulators also have not determined what to do with any funds that 

might be left because borrowers refused payments, did not cash their 

 

70Regulators created a new categorization framework for the agreement. This framework 

has 11 direct payment categories that are similar to the categories in the foreclosure 

review financial remediation framework. 
 

Page 47 GAO-13-277 Foreclosure Review

checks, or could not be located. According to regulators, any 

remaining funds will not be returned to the servicers.

• Further, regulators told us that the agreements specify that over the 

next 2 years servicers are required to take loss mitigation and

foreclosure prevention actions for which each servicer will earn credit 

toward fulfilling a specified obligation. These activities will provide 

assistance to borrowers covered under the consent orders and other 

borrowers. Regulator staff said they provided a list of eligible activities 

to servicers and advised them to prioritize borrowers covered under 

the consent orders for assistance. However, according to OCC staff, 

they did not provide servicers with criteria for identifying borrowers to 

whom they will offer mortgage assistance, which would help ensure 

that eligible borrowers have a consistent opportunity to be considered 

for these funds.

Without assessing past lessons learned and making decisions on key 

features of the amended consent order activities in advance, regulators 

risk having to implement changes in the planned activities or publicly 

announced timelines, which could decrease the efficiency of the process 

to distribute direct payments and other assistance.

The foreclosure review experience suggests that regulators’ process for 

monitoring third-party consultant, servicer, and examination team 

activities, including holding regular meetings and reviewing weekly 

progress reports, could provide a useful model for monitoring activities 

under the amended consent orders. According to regulators, examination 

teams will play a critical role in overseeing servicer activities under the 

amended consent orders, including testing and validating the results of 

the servicers’ categorization of borrowers. Regularly collecting and 

reviewing information on the activities and approaches of the examination 

teams could provide an opportunity to identify challenges and take steps 

to address them as the activities progress. Similarly, instituting a process 

to monitor the progress of the servicers’ loan categorization and track the 

payment administrator’s distribution of payments could help regulators 

assess the extent to which they are on target to reach their goal of 

providing notification to borrowers about their payment by the end of 

Monitoring Activities May 

Help Ensure Achievement 

of Goals
 

Page 48 GAO-13-277 Foreclosure Review

March 2013.71 OCC staff said that they continue to hold regular meetings 

with the servicers and examination teams to answer questions and share 

ideas. In addition, OCC staff told us that they intend to conduct site visits 

with each servicer, review servicers’ draft categorizations, and implement 

a reporting mechanism to oversee servicers’ activities under the amended 

consent orders.

Regulators’ experience with the foreclosure review suggests that 

identifying comparative oversight mechanisms to promote consistency 

could help achieve consistent results among key actors. As discussed 

earlier, regulators had limited and unsystematic centralized control 

mechanisms to monitor consistency among the foreclosure review 

processes and did not have the information to assess the implications of 

any differences. According to regulators, achieving consistent results for 

borrowers, so that similarly situated borrowers receive similar payment 

amounts, is a goal of the amended consent orders, as it was of the 

foreclosure review process. OCC staff stated that the direct payments 

provided under the amended consent orders will likely be more consistent 

than what would have occurred under the foreclosure review because 

servicers are using a standard framework and objective criteria to 

categorize borrowers and all borrowers in a particular category will 

receive the same payment amount. According to regulator staff, the 

categorization instructions provided to servicers, regular meetings with 

servicers and examination teams, site visits, and examination team 

verification processes will provide opportunities to review and discuss the 

results of each servicer’s categorization of borrowers. However, whether 

there is a similar process that will compare results across OCC and 

Federal Reserve supervised servicers is unclear. Furthermore, to what 

extent regulators will assess the implications of any inconsistencies 

among the reviews that consultants are continuing to conduct for the 

servicers that did not sign agreements with regulators is unknown.

Applying lessons from the foreclosure review process and our internal 

control standards to the amended consent order activities would suggest 

that mechanisms to centrally promote consistency and monitor 

agreement activities could help achieve consistent results for borrowers. 

GAO’s internal control standards state that agencies should take steps to 

 

71According to OCC staff, each servicer will make a single payment into a fund. The 

regulators will direct a payment administrator to make payments from the fund to 

borrowers.
 

Page 49 GAO-13-277 Foreclosure Review

comprehensively identify and analyze program operations to determine if 

risks exist to achieving goals—such as risks to the regulators’ goal of 

providing similar results for similarly situated borrowers.72 In our prior 

work, we found that using a horizontal review mechanism is an option to 

help mitigate risks of inconsistent results for activities conducted by 

multiple entities.73 For example, comparing servicer decision-making 

processes, including any criteria used by the servicers to categorize 

borrowers, could identify any potential differences and the extent to which 

these differences may result in different direct payment decisions for 

similarly situated borrowers. Similarly, mechanisms to provide clear and 

specific guidance—such as guidance on testing and validation of servicer 

activities—for local examination teams to use in their oversight of 

individual servicer activities could help regulators to more easily monitor 

and compare servicer activities and the results for borrowers among the 

reviews. Without using mechanisms to centrally monitor the consistency 

of servicers’ activities to categorize borrowers, regulators may risk delays 

in providing direct payments to borrowers and inconsistent results. In 

addition, without monitoring potential inconsistencies in the foreclosure 

reviews that are continuing for servicers that are not party to the amended 

consent orders, regulators will not have the information to assess whether 

those servicers’ borrowers are being treated consistently.

Lessons from foreclosure review activities conducted to date suggest that 

developing and implementing an effective communication strategy that 

includes public reporting goals could enhance the transparency of the 

activities under the amended consent orders. GAO’s internal control

standards state the importance of relevant, reliable, and timely 

communications within an organization as well as with external 

stakeholders.74 As a means to strengthen communication with external 

stakeholders and improve transparency and accountability, our work on 

 

72See GAO/AIMD-00-21.3.1.

73GAO, Opportunities Exist to Apply Lessons Learned from the Capital Purchase Program 

to Similarly Designed Programs and to Improve the Repayment Process, GAO-11-47

(Washington, D.C.: Oct. 4, 2010). In our analysis of Treasury’s oversight of the Capital 

Purchase Program under TARP we found that Treasury’s good practice of establishing 

centralized control mechanisms to help ensure consistency of activities conducted by 

multiple banking regulators helped lessen the likelihood of inconsistent results.

74See GAO/AIMD-00-21.3.1.

Communication Is Key to 

Providing Transparency 

and Enhancing Public 

Confidence 
 

Page 50 GAO-13-277 Foreclosure Review

TARP has underscored the importance of a communication strategy.75

Moreover, our prior work on organizational transformation demonstrates 

that public and private-sector leaders view establishing a communication 

strategy as especially crucial in the public sector, where policymaking and 

program management demand transparency and stakeholders are 

concerned not only with what results are to be achieved, but also with 

which processes are to be used to achieve those results.76 In addition, as 

previously discussed, public reporting is also a mechanism for external 

communication that can enhance transparency. Experiences with current 

government initiatives that are aimed at assisting struggling homeowners 

and involve institutions and mortgage-related issues similar to those of 

the foreclosure review also highlight the benefits of regular performance 

reporting. Specifically, periodic reports on the performance of and 

participation in TARP programs and scheduled reports on servicers’ 

compliance with requirements of the National Mortgage Settlement are 

intended to promote transparency and build public confidence. Because 

the foreclosure review and the subsequent activities under the amended 

consent orders—like TARP and the National Mortgage Settlement—are 

 

75GAO has made a series of recommendations aimed at improving the transparency of 

TARP by ensuring that Treasury develops a comprehensive communication strategy. 

TARP, like the foreclosure review and subsequent activities under the amended consent 

orders, is one of many activities the federal government has put in place to respond to the 

financial crisis, including the crises in the housing and mortgage markets. As such, we 

believe that similar efforts to improve communication will enhance the transparency in the 

implementation of the amended consent orders and continuing foreclosure reviews. See

GAO, Troubled Asset Relief Program: One Year Later, Actions Are Needed to Address 

Remaining Transparency and Accountability Challenges, GAO-10-16 (Washington, D.C.: 

Oct. 8, 2009); Troubled Asset Relief Program: Additional Actions Needed to Better Ensure 

Integrity, Accountability, and Transparency, GAO-09-161 (Washington, D.C.: Dec. 2, 

2008); Troubled Asset Relief Program: Status of Efforts to Address Transparency and 

Accountability Issues, GAO-09-539T (Washington, D.C.: Mar. 31, 2009); and Troubled 

Asset Relief Program: June 2009 Status of Efforts to Address Transparency and 

Accountability Issues, GAO-09-658 (Washington, D.C.: June 17, 2009). 

76See GAO, Major Management Challenges and Program Risks: Department of 

Homeland Security, GAO-03-102 (Washington, D.C.: January 2003) and Results-Oriented 

Cultures: Implementation Steps to Assist Mergers and Organizational Transformations, 

GAO-03-669 (Washington, D.C.: July 2, 2003). Part of the methodology to develop these 

reports included convening a forum of public and private-sector leaders to discuss useful 

practices from major private and public-sector organizational mergers, acquisitions, and 

transformations that federal agencies could learn from when making changes, such as 

those in response to governance challenges. The participants of the forum identified key 

practices and lessons learned regarding mergers and transformations. We considered this 

example relevant to the foreclosure review because of the significant nature of the change 

from the foreclosure review to the activities under the amended consent orders for 

distributing direct payments and other assistance.
 

Page 51 GAO-13-277 Foreclosure Review

part of the larger governmental response to the housing and mortgage 

crises, a communication strategy which incorporates plans for periodic 

public reporting may enhance transparency in the distribution of direct 

payments and other assistance and help restore confidence in the 

mortgage market.

Regulators announced the agreements that led to the amended consent 

orders without a clear communication strategy. As such, what information 

will be provided to individual borrowers and the general public about 

processes, progress, and results of activities under the amended consent 

orders is unclear. Although OCC and the Federal Reserve have provided 

some information on the amended consent orders and have plans to 

release additional information, regulators have not made key decisions on 

communicating directly with individual borrowers and the extent to which 

they will report on activities related to the amended consent orders and 

continuing foreclosure reviews. Regulators provided limited information 

on the amended consent orders through press releases and updates on 

their websites, among other ways. For example, OCC and the Federal 

Reserve issued joint press releases announcing the agreements and 

related amended consent orders, and OCC and the Federal Reserve 

posted answers to frequently asked questions on their websites on the 

agreements that led to the amended consent orders. 

In addition, regulators plan to release information about the distribution 

plan after payment amounts are determined, but staff did not describe 

plans to release additional information about the procedures servicers are 

using to categorize borrowers. As of January 2013, OCC staff said that 

they planned to release two public reports, one in April 2013 to discuss 

the direct payment process, and one in the summer of 2013 to discuss 

the foreclosure review results of servicers not covered under the 

amended consent orders. OCC staff told us, however, that they have not 

decided on the specific content of these reports. As of February 2013, the 

Federal Reserve also plans to issue public reports about servicers’ 

activities under the amended consent orders and the results of the 

foreclosure reviews at servicers not subject to the amended consent 

orders. Regulators had not decided what, if any, information will be made 

available on the results of the work conducted under the foreclosure 

review prior to the agreements. Further, OCC and Federal Reserve staff 

told us they had not made decisions about the form and specific content 

of communications, if any, directly to individual borrowers.

While the amended consent orders terminate the foreclosure review for 

most of the servicers, transparency of past and current efforts continues 
 

Page 52 GAO-13-277 Foreclosure Review

to be important to stakeholders, including Congress and consumer 

groups. In particular, members of Congress have expressed concerns 

about the lack of public reporting on the foreclosure review and a lack of 

information about how the amounts of payments and other assistance in 

the amended consent orders were determined. In addition, consumer 

groups expressed concerns about transparency and urged regulators to 

take additional steps to increase transparency and public confidence in 

the implementation of the amended consent orders, including robust data 

collection and reporting. In the absence of a clear communication strategy 

to direct external communications, including public reporting and direct 

communication with individual borrowers, regulators face risks to 

transparency and public confidence similar to those experienced in the 

foreclosure review process.

The foreclosure review was intended to identify as many harmed 

borrowers as possible, ensure that similarly situated borrowers received 

similar results, and help restore public confidence in the mortgage 

market. Ultimately, the complexity of the foreclosure reviews and 

limitations in regulators’ guidance and monitoring of the foreclosure 

review challenged their ability to achieve the stated goals. OCC and the 

Federal Reserve took a number of steps to foster consistency of this 

unprecedented review, including issuance of nearly identical consent 

orders and joint issuance of guidance documents for third-party 

consultants. However, other efforts related to guidance and monitoring 

may have exacerbated the challenges and complexities inherent in the 

process. In particular, existing guidance on sampling was ambiguous, 

leading to inconsistent sampling methodologies used by consultants, and 

did not include key oversight mechanisms to facilitate assessment of the 

extent to which consultants had identified as many harmed borrowers as 

possible. In addition, regulators’ limited monitoring of consistency of the 

consultants’ sampling methodologies and review processes increased 

risks that similarly situated borrowers would receive different results. As a 

result, the regulators risked not achieving the intended goals of identifying 

as many harmed borrowers as possible and treating similarly situated 

borrowers similarly, and this remains a challenge for the servicers 

continuing the foreclosure review. Our prior work has identified practices, 

such as assessing progress toward goals and designing such monitoring 

during the planning stage of a project, as effective management 

practices. In addition, OMB has found that in planning data analysis 

activities, such as sampling, agencies should take necessary steps to 

ensure that they have collected the appropriate data from which to draw 

conclusions. Assessing the review processes of the continuing reviews 

Conclusions 
 

Page 53 GAO-13-277 Foreclosure Review

for consistency and implementing a mechanism to assess the sufficiency 

of additional sampling activities could help mitigate risk of similarly 

situated borrowers receiving different results and facilitate oversight of the 

extent to which consultants have reached as many harmed borrowers as 

possible.

Although the regulators have terminated activity related to the foreclosure 

review for the servicers with amended consent orders, the foreclosure 

review process offers an opportunity for the regulators to leverage this 

experience to help ensure that similar difficulties are better addressed in 

future efforts. In general, identifying, assessing, and using lessons 

learned can help ensure that beneficial information is factored into the 

work processes of future activities, among other things. Therefore, 

consideration of lessons from the foreclosure review activities, such as 

advance design of data analysis activities and assessment of 

inconsistencies in the file review processes, is one way to help regulators 

improve the clarity of their guidance and ensure effective monitoring of 

progress toward results for the activities under the amended consent 

orders and the three remaining servicers still subject to the foreclosure 

review requirement that will cover 450,000 eligible borrowers. Specific 

activities include the following.

• Sound planning and additional oversight mechanisms can enhance 

project design and help ensure achievement of program goals. Our 

internal control standards on monitoring, risk assessments, and 

consultations with stakeholders, as well as generally accepted project 

management practices for agencies to use in implementing the 

Government and Performance Results Act and related management 

initiatives, emphasize the importance of planning, monitoring, and 

assessing lessons learned. Assessing the strengths and weaknesses 

of the foreclosure review process by using project critiques and 

discussions with key participants and stakeholders—such as local 

examination team staff, third-party consultants, law firms, and external 

groups—could help ensure better design and more efficient 

implementation of activities under the amended consent orders. As 

such, consideration of lessons from the foreclosure review processes 

would enhance the design, implementation, and oversight of the 

activities under the amended consent orders.

• The foreclosure review activities to date also highlight the importance 

of effective communication. We found that limited communication with 

individual borrowers and the general public hindered transparency 

and public confidence in the reviews. Further, regulators announced 
 

Page 54 GAO-13-277 Foreclosure Review

agreements that led to the amended consent orders and ended the 

foreclosure review with 11 servicers without a clear communication 

strategy, and the information that will be provided to borrowers and 

the general public remains unclear. Our internal control standards 

state the importance of external communication, and our key practices 

in implementing transformations that were identified by public and 

private sector leaders underscore the importance of a communication 

strategy. As such, the development and implementation of an 

effective communication strategy could enhance regulators’ efforts to 

ensure transparency and public confidence in the results of the 

foreclosure review as well as processes to implement the activities 

under the amended consent orders.

We are making three recommendations to the regulators:

(1) To better ensure that the goals of the foreclosure review are realized 

for servicers that are not subject to amended consent orders, we 

recommend that the Comptroller of the Currency and the Chairman of the 

Board of Governors of the Federal Reserve System, as appropriate, 

improve oversight of sampling methodologies and mechanisms to 

centrally monitor consistency, such as assessment of the implications of 

inconsistencies on remediation results for borrowers in the remaining 

foreclosure reviews.

(2) To better ensure that the goals of the amended consent orders related 

to the distribution of direct payments and other assistance are realized, 

we recommend that the Comptroller of the Currency and the Chairman of 

the Board of Governors of the Federal Reserve System identify and apply 

lessons from the foreclosure review process, such as enhancing planning 

and monitoring activities to achieve goals, as they develop and implement 

the activities under the amended consent orders.

(3) To better ensure transparency and public confidence in the activities 

under the amended consent orders and results of the continuing 

foreclosure reviews, we recommend that the Comptroller of the Currency 

and the Chairman of the Board of Governors of the Federal Reserve 

System develop and implement a communication strategy to regularly 

inform borrowers and the public about the processes, status, and results 

of the activities under the amended consent orders and continuing 

foreclosure reviews.

Recommendations for 

Executive Action
 

Page 55 GAO-13-277 Foreclosure Review

We requested comments on a draft of this report from OCC and the 

Federal Reserve. OCC and the Federal Reserve provided written 

comments that are presented in appendixes II and III, respectively. The 

regulators also provided technical comments, which we have 

incorporated into the report, as appropriate. In commenting on the report, 

OCC and the Federal Reserve both identified actions that they have 

taken or planned to implement the recommendations. 

Specifically, OCC stated that it plans to continue to ensure that its 

sampling guidance is used in the continuing reviews and that the agency 

plans to monitor for consistency. As we discussed in the report, 

developing and using objective measures to monitor and assess 

consistency among the continuing reviews is also important. With respect 

to the second recommendation, OCC stated that it plans to apply lessons 

learned from the foreclosure review to the activities under the amended

consent orders. In response to the third recommendation, OCC noted that 

it recognized the importance of providing additional information to the 

public about the processes, status, and results of the continuing reviews 

and activities under the amended consent orders. As such, OCC said it 

plans to issue at least two public reports, as we noted in the draft report. 

The Federal Reserve stated that it plans to continue to coordinate with 

OCC to provide consistent guidance during the continuing reviews. As we 

discussed in the report, developing and using objective measures to 

monitor and assess consistency among the continuing reviews is 

important. With respect to the second recommendation, the Federal 

Reserve stated that it has expanded its planning and monitoring efforts 

during the course of the foreclosure review and plans to continue to 

devote resources to planning and monitoring as it implements the 

amended consent orders. In responding to the third recommendation, the 

Federal Reserve outlined a number of steps that it and OCC are taking to 

communicate about the continuing foreclosure reviews and activities 

under the amended consent orders. These steps, such as developing a 

letter to explain why borrowers are receiving a payment and updating 

borrowers covered under the continuing reviews who submitted a 

request-for-review, will help provide information to affected borrowers. 

Moreover, the Federal Reserve and OCC said that they have committed 

to providing public reports that detail the implementation of the 

agreements. They also said that they anticipate these reports will include 

information about the findings of completed reviews, the number of 

requests for review, and the status of other activities under the consent 

orders.

Agency Comments 

and Our Evaluation
 

Page 56 GAO-13-277 Foreclosure Review

As agreed with your offices, unless you publicly announce the contents of 

this report earlier, we plan no further distribution until 30 days from the 

report date. At that time, we will send copies to interested congressional 

committees, the Board of Governors of the Federal Reserve System, 

Office of the Comptroller of the Currency, and other interested parties. In 

addition, the report will be available at no charge on the GAO website at 

http://www.gao.gov. 

If you or your staff have any questions about this report, please contact 

me at (202) 512-8678 or evansl@gao.gov. Contact points for our Offices 

of Congressional Relations and Public Affairs may be found on the last 

page of this report. Key contributors to this report are listed in appendix 

IV.

Lawrance L. Evans, Jr.

Director, Financial Markets

 and Community Investment
 

Page 57 GAO-13-277 Foreclosure Review

List of Requesters

The Honorable Robert Menendez

Chairman

Subcommittee on Housing, Transportation, and Community Development

Committee on Banking, Housing, and Urban Affairs

United States Senate

The Honorable Maxine Waters

Ranking Member

Committee on Financial Services

House of Representatives

The Honorable Luis V. Gutierrez

House of Representatives
Appendix I: Objectives, Scope, and 

Methodology

Page 58 GAO-13-277 Foreclosure Review

The objectives of this report were to assess: (1) challenges to the 

achievement of the goals of the foreclosure review, (2) the extent of 

transparency in the foreclosure review process, and (3) lessons that could 

be useful for activities under the amended consent orders and continuing 

reviews. The scope of our work was limited to the foreclosure review at 

the 14 servicers that are subject to the April 2011 consent orders. We 

were in the process of reviewing various aspects of the foreclosure review 

when the Office of the Comptroller of the Currency (OCC) and the Board 

of Governors of the Federal Reserve System (Federal Reserve) 

announced agreements with 11 of the 14 servicers to discontinue the 

foreclosure review and replace it with a broad payment process.

To assess challenges to achieving the goals of the foreclosure review 

process, we identified three areas of the review process to use as a basis 

for our analysis. Specifically, we analyzed consultants’ application of 

guidance to conduct the reviews and recommend remediation; 

consultants’ development of test questions related to loan modifications, 

state foreclosure laws, and fee reasonableness; and consultants’ 

sampling methodologies for selecting files to review. Based on our prior 

work on the foreclosure review, these were areas we identified as 

particularly challenging for ensuring consistent reviews.1 To obtain 

information on consultants’ development of test questions and application 

of regulatory guidance, we reviewed the guidance and conducted site 

visits and in-person interviews with five consultant engagement teams. To 

identify third-party consultants to interview, we selected consultants that 

were engaged to conduct reviews for servicers that are overseen by each 

regulator. In addition, we selected consultants engaged by servicers with 

a range in sizes of eligible population for the review, including some of the 

largest servicers. During the site visits we also observed demonstrations 

of the file review process and systems. In addition, we obtained 

responses to a standardized questionnaire from all of the consultants. We 

compared consultants’ processes for compiling reference materials, 

developing test questions, and applying guidance. We reviewed 

documents regulators used to monitor the file reviews, such as status 

reports and meeting agendas. We also interviewed regulator staff and 

examination teams on steps taken to promote and assess consistency in 

the reviews and the effects of any differences. To obtain information on 

 

1

See GAO, Foreclosure Review: Opportunities Exist to Further Enhance Borrower 

Outreach Efforts, GAO-12-776 (Washington, D.C.: June 29, 2012).

Appendix I: Objectives, Scope, and 

Methodology 
Appendix I: Objectives, Scope, and 

Methodology

Page 59 GAO-13-277 Foreclosure Review

third-party consultants’ sampling plans, we used a data collection 

instrument created by our statisticians. We analyzed information 

contained in engagement letters between the servicers and consultants 

for the sampling parameters consultants used to select files. We 

confirmed key observations of our analysis in interviews and site visits 

with officials responsible for developing the sampling plans at five thirdparty consultant engagement teams, interviews with regulator staff, and 

through examination teams that reviewed the plans. In addition, we 

obtained information on consultants’ plans for additional analytical

methods and confirmed other observations of our analysis through a 

standardized written questionnaire to consultants. We compared the 

information and these parties’ actions to criteria such as the regulators’ 

standard practices, stated goals for the foreclosure review, and our 

internal control standards. In addition, we reviewed our prior work on the 

Troubled Asset Relief Program (TARP), Home Affordable Modification 

Program, and reports where we established criteria on lessons-learned 

procedures, generally accepted project management practices, and 

effective management practices.2 We also referred to our sampling 

standards and several other references on sampling standards from 

OCC, the American Institute of Certified Public Accountants (AICPA), and 

the Office of Management and Budget (OMB).3 We considered these 

 

2

See for example, GAO, Troubled Asset Relief Program: Treasury Continues to Face 

Implementation Challenges and Data Weaknesses in Its Making Home Affordable 

Program, GAO-11-288 (Washington, D.C.: Mar. 17, 2011); Troubled Asset Relief 

Program: Further Actions Needed to Fully and Equitably Implement Foreclosure Mitigation 

Programs, GAO-10-634 (Washington, D.C.: June 24, 2010), and Troubled Asset Relief 

Program: Additional Actions Needed to Better Ensure Integrity, Accountability and 

Transparency, GAO-09-161 (Washington, D.C.: Dec. 2, 2008).For examples of our work 

establishing lessons learned and management practices, see for example, GAO, Federal 

Real Property Security: Interagency Security Committee Should Implement A LessonsLearned Process, GAO-12-901 (Washington, D.C.: Sept. 10, 2012); NASA: Better 

Mechanisms Needed for Sharing Lessons Learned, GAO-02-195 (Washington, D.C.: Jan. 

30, 2002); Coast Guard: Civil Rights Directorate’s Action Plans to Improve Its Operations 

Could Be Strengthened by Implementing Several Aspects of Project Planning and 

Implementation Practices, GAO-10-571T (Washington, D.C.: Apr. 27, 2010); Agency 

Performance Plans: Examples of Practices that Can Improve Usefulness to 

Decisionmakers, GAO/GGD/AIMD-99-69 (Washington, D.C.: Feb. 26, 1999); and 

Executive Guide: Effectively Implementing the Government Performance and Results Act, 

GAO/GGD-96-118 (Washington, D.C.: June 1, 1996).

3

See for example, OCC, Sampling Methodologies: Comptroller’s Handbook (Washington, 

D.C.: August 1998); GAO/President’s Council on Integrity and Efficiency, Financial Audit 

Manual: Volume 1, GAO-08-585G (Washington, D.C.: July 2008); AICPA, Audit and 

Accounting Guides: Audit Sampling 2012, (March 2012); and OMB, Standards and 

Guidelines for Statistical Surveys (Washington, D.C.: September 2006).
Appendix I: Objectives, Scope, and 

Methodology

Page 60 GAO-13-277 Foreclosure Review

practices applicable to the foreclosure review because they concern the 

use of data from samples to draw inferences about the populations from 

which the samples are drawn. To further evaluate and asses the sampling 

plans, we also compiled a list of common sampling terms based on the 

work cited above, as well as other analyses, and vetted the definitions for 

these terms internally with our methodological staff, including staff 

responsible for sampling procedures related to financial audits and 

program audits. Any data we obtained on the number of borrowers in the 

scope of the reviews or the status of the reviews were used for 

background purposes only and were not used to support our findings and 

conclusions. As such, we obtained information from regulator staff and 

the data administrator about how the data were obtained and compiled, 

but we did not assess the reliability of the data.

Statistical sampling can be a powerful tool for drawing inferences about 

populations when a full census of cases is infeasible. Well designed and 

executed samples allow researchers to make estimates of population 

characteristics and to specify uncertainty due to sampling error 

associated with those estimates.4 Parameter estimates and associated 

measures of precision (such as confidence intervals or margins of error) 

are developed from the sample data using estimation formulas consistent 

with the sampling plan actually used. For example, if the sample design 

were complex (such as using differing probabilities of selection for 

different portions of the population) and if estimation formulas for simple 

random sampling were used to produce estimates, those estimates and 

the associated confidence intervals would likely be incorrect.5

Additionally, estimates projected to a population of interest may be 

subject to bias when a sample fails to cover the population of interest, 

such as when the list used for sampling (the sampling frame) excludes 

relevant units or when data from only a portion of the sample are 

collected.

The design of an appropriate sample is intricately linked with the goals of 

the sample. These goals can include the importance of establishing the 

 

4

This report focuses on the type of error directly associated with sampling error. In 

addition, surveys and data collection for samples may be subject to other types of error 

that are not as easily quantified, such as coverage error, measurement error, or data 

processing error.

5

Sometimes estimation weights are assigned to the sample data to facilitate producing 

estimates based on a complex sample design.
Appendix I: Objectives, Scope, and 

Methodology

Page 61 GAO-13-277 Foreclosure Review

estimate within a narrow confidence interval, the need to conduct 

statistical testing against a threshold or for differences, the ability to make 

estimates to subpopulations, and the desire to report specific results. A 

sample that is designed to test against a specified tolerance level may 

use different sample sizes and a different approach to statistical testing 

than a sample that is designed to estimate the level of an attribute in the 

population. Some uncertainty is implicit in sampling as a tradeoff for such 

factors as the cost and time required to examine all of the data. However, 

according to AICPA guidelines, sampling is inappropriate if there is no 

tolerance for risk of possible erroneous decisions as a result of examining 

only a sample of the data.6

For example, figure 2 demonstrates that these two consultants interpreted 

the “precision” requirement differently. Consultant A interpreted precision 

as being the threshold for the tolerable error rate, whereas Consultant B 

interpreted this as requiring a margin of error within plus or minus 3 

percentage points of the estimate. Because Consultant A is testing 

against a threshold using the methodology specified in the OCC 

handbook on sampling methodologies (based on a Poisson distribution), 

it uses the upper bound of its one-sided 95 percent confidence interval to 

draw inferences about whether the population error rate is less than an 

established threshold value. Consultant B, in contrast, uses a two-sided 

95 percent confidence interval because it is developing point estimates. 

Although both consultants achieve the 95 percent reliability and 3 percent 

precision guidance from regulators, the consultants’ different approaches 

to sampling resulted in different sample sizes and different decision rules 

for when additional sampling or full review of a population would be 

required, even when the same number, or even proportion, of errors are 

found in the sample. For example, if 2 errors were found in Consultant A’s 

sample, one could conclude with 95 percent confidence that the error rate 

in the population was below 6.3 percent and that the 3 percent precision 

threshold had not been met. Additional sampling would be required in this 

case. If 2 errors were found in Consultant B’s sample, one could conclude 

with 95 percent confidence that the error rate in the population is between 

0.1 and 1.9 percent and no additional sampling would be required. If the 

proportion of errors in Consultant B’s sample was similar to Consultant 

A’s at approximately 2 percent, or 8 errors, one could conclude with 95 

 

6

AICPA, Professional Standards: AU Section 350, Audit Sampling (October 2012). 
Appendix I: Objectives, Scope, and 

Methodology

Page 62 GAO-13-277 Foreclosure Review

percent confidence that the error rate in the population was between 1 

and 4.3 percent, a much narrower estimate than that from Consultant A.

Figure 2: Assumed Error Rates and Potential Conclusions from Different Sample 

Sizes Used by Consultants (Assuming 95 Percent Confidence Level and 3 Percent 

Precision)

a

Because the samples were designed to generalize to the population of loans, the number of loans 

with errors is used in calculating the error rate rather than the number of individual errors. Although 

some loans may have more than one error, the error rate calculation would treat loans with multiple 

errors as equivalent.

b

GAO calculated confidence intervals based on designs presented in Consultant A’s and Consultant 

B’s engagement letters. Confidence intervals for Consultant A were calculated using the methodology 

outlined in OCC’s handbook on sampling methodologies and are based on a Poisson distribution. 

Confidence intervals for Consultant B were approximated based on a binomial distribution and 

assume a large population.

c

Additional sampling indicates the consultant would confer with regulators to determine whether to (1) 

draw an additional sample to establish that the error rate was below 3 percent, (2) conduct analysis of 

the error and then draw additional samples related to the characteristics of the loan with the error, or 

(3) conduct a full review of loans in the loan category from which the sample was drawn.

d

Consultant A used a numerical sampling approach as described in OCC’s handbook on sampling 

methodologies to test internal controls against a tolerance level of 3 percent. The expected population 

error rate was zero and the sample size according to the 3 percent precision and 95 percent 

confidence levels required by regulators was 100 loans.
Appendix I: Objectives, Scope, and 

Methodology

Page 63 GAO-13-277 Foreclosure Review

e

Consultant B used an attribute sampling approach to estimate the error rate in the population within 

a margin of error of 3 percent. The expected population error rate was 10 percent and the sample 

size with a 3 percent precision and 95 percent confidence levels was approximately 370 loans.

Neither of the samples illustrated in figure 2 was designed with the goal of 

estimating the number of harmed borrowers in the population. However, 

had samples of this size been drawn with the purpose of estimating the 

number of harmed borrowers, the width of the two-sided 95 percent 

confidence intervals around each estimate would vary. Assuming a 

population size of 100,000 cases and a binomial distribution, and that 

each loan with an error corresponds to one harmed borrower, samples 

similar to those in figure 2 would result in different ability to concisely 

estimate the number of harmed borrowers. If two errors were found in a 

sample of 100 files, a 95 percent confidence interval for the estimated 

number of harmed borrowers in the population would run between 

approximately 240 and 7,040 borrowers. For a sample of 370 cases, a 

similar proportion of errors (8 errors in a sample of 370) would result in a 

95 percent confidence interval around the estimated number of harmed 

borrowers between approximately 940 and 4,220, a much narrower 

confidence interval. To effectively use a baseline sample from the full 

population of loans to monitor whether the foreclosure reviews have 

identified the majority of harmed borrowers would require setting a 

realistic expected error rate to calculate sample size and establishing 

guidelines for an appropriate margin of error around the estimate to 

ensure that sample estimates were sufficiently precise to meet regulators’ 

goals.

To assess the extent of transparency in the foreclosure review process, 

we reviewed press releases and documents from regulators related to the 

foreclosure review. In particular, we reviewed what documents related to 

the consent orders were available on the regulators’ websites, such as 

speeches by agency officials, engagement letters, outreach materials, 

and press releases, and analyzed the content of these documents. We 

compared this documentation against agency policies on public 

disclosure of enforcement action information and our previous work on 

transparency in government programs, such as our work on TARP to 

identify any similarities and differences. Further, we reviewed reports and 

summary documentation from the National Mortgage Settlement and 

interviewed the monitor of the settlement to provide the context of a 

current example of a large-scale settlement involving similar stakeholders 

and issues similar to those of the foreclosure review. We also conducted 

interviews with regulator staff, selected third-party consultants, and 

consumer groups.
Appendix I: Objectives, Scope, and 

Methodology

Page 64 GAO-13-277 Foreclosure Review

To identify lessons learned that might be useful for the activities under the 

amended consent orders, we compared our findings from the first two 

objectives with the processes and goals regulators outlined for the 

activities under the amended consent orders and identified areas with 

similar challenges. We reviewed press releases on the agreements and 

amended consent orders from the regulators and interviewed regulator 

staff about the goals and rationale for the agreements and their plans for 

designing and implementing the activities under the amended consent 

orders. We also reviewed our internal control standards, our prior work 

identifying key practices during agency transformations, our prior work 

covering TARP and other government programs and agencies, and 

documents related to the National Mortgage Settlement.7 In particular, we 

focused on information in these reports on project design, oversight of 

progress, and communication with stakeholders. Although the foreclosure 

review and the activities under the amended consent orders are not 

government programs, we considered the steps identified in this work as 

applicable because planning, monitoring, and communication are key 

principles of any effective process.

We conducted this performance audit from July 2012 through March 2013

in accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.

 

7

See for example, GAO, Highlights of a GAO Forum: Mergers and Transformation: 

Lessons Learned for a Department of Homeland Security and Other Federal Agencies, 

GAO-03-293SP (Washington, D.C.: Nov. 14, 2002); Major Management Challenges and 

Program Risks: Department of Homeland Security, GAO-03-102 (Washington, D.C.: 

January 2003); and Results-Oriented Cultures: Implementation Steps to Assist Mergers 

and Organizational Transformations, GAO-03-669 (Washington, D.C.: July 2, 2003). Our 

work on TARP includes Troubled Asset Relief Program: Additional Actions Needed to 

Better Ensure Integrity, Accountability, and Transparency, GAO-09-161 (Washington, 

D.C.: Dec. 2, 2008) and Troubled Asset Relief Program: June 2009 Status of Efforts to 

Address Transparency and Accountability Issues, GAO-09-658 (Washington, D.C.: June 

17, 2009). Our relevant prior work on other programs includes GAO, Designing 

Evaluations: 2012 Revision, GAO-12-208G (Washington, D.C.: January 2012); Federal 

Real Property Security: Interagency Security Committee Should Implement A LessonsLearned Process, GAO-12-901 (Washington, D.C.: Sept. 10, 2012); and NASA: Better 

Mechanisms Needed for Sharing Lessons Learned, GAO-02-195 (Washington, D.C.: Jan. 

30, 2002).
Appendix II: Comments from the Office of the 

Comptroller of the Currency

Page 65 GAO-13-277 Foreclosure Review

Appendix II: Comments from the Office of 

the Comptroller of the Currency
Appendix III: Comments from the Board of 

Governors of the Federal Reserve System

Page 66 GAO-13-277 Foreclosure Review

Appendix III: Comments from the Board of 

Governors of the Federal Reserve System
Appendix III: Comments from the Board of 

Governors of the Federal Reserve System

Page 67 GAO-13-277 Foreclosure Review
Appendix III: Comments from the Board of 

Governors of the Federal Reserve System

Page 68 GAO-13-277 Foreclosure Review
Appendix IV: GAO Contact and Staff 

Acknowledgments

Page 69 GAO-13-277 Foreclosure Review

Lawrance L. Evans, Jr. (202) 512-8678 or evansl@gao.gov

In addition to the contact named above, Karen Tremba (Assistant 

Director), Bethany M. Benitez, John Karikari, Charlene J. Lindsay, 

Patricia MacWilliams, Marc Molino, Jill Naamane, Anna Maria Ortiz, 

Robert Rieke, Jennifer Schwartz, Andrew Stavisky, Sonya Vartivarian, 

James Vitarello, and Monique Williams made key contributions to this 

report.

Appendix IV: GAO Contact and Staff 

Acknowledgments 

GAO Contact

Staff 

Acknowledgments 

(250676)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations 

Public Affairs

Please Print on Recycled Paper.