GAO-01-308R Long-Range Planning for Federal Judiciary Space

United States General Accounting Office

Washington, DC 20548

January 25, 2001

The Honorable Don Young

Chairman

The Honorable James L. Oberstar

Ranking Democratic Member

Committee on Transportation and Infrastructure

House of Representatives

The Honorable Bud Shuster

House of Representatives

Subject: Federal Judiciary Space: Update on Improvement of the Long-Range

 Planning Process

This letter responds to an August 9, 2000, request from your Subcommittee on

Economic Development, Public Buildings, Hazardous Materials, and Pipeline Safety.

The Subcommittee asked that we obtain updated information on the Administrative

Office of the U.S. Courts’ (AOC) progress in implementing six GAO recommendations

that were included in our 1993 report entitled Federal Judiciary Space: Long-Range

Planning Process Needs Revision (GAO/GGD-93-132, Sept. 28, 1993). In that report,

we identified three major problems associated with AOC’s long-range planning

process: (1) the inconsistent treatment of federal districts, (2) the use of inaccurate

information in estimating space baselines, and (3) the use of an inappropriate

methodology for projecting caseloads. The report’s six recommendations identified

various steps that AOC could take to help address the problems and improve its longrange planning process.

In the late 1980s, the judiciary recognized that it faced growing space shortages,

security shortfalls, and operational inefficiencies at courthouse facilities. To address

these problems, the Judicial Conference of the United States directed each of the 94

federal judicial districts, with assistance from AOC—the administrative arm of the

judiciary—to develop long-range space plans identifying where new and additional

space was needed. The districts’ plans comprise the cornerstone of AOC’s overall

long-range planning process, which primarily involves developing projections for

future workload, personnel, and space needs for the judiciary’s 94 federal districts.

The results of AOC’s long-range planning process are used by the General Services

Administration (GSA) to develop funding requests for new or expanded space in the

districts.
2 GAO-01-308R Long-Range Planning for Federal Judiciary Space

Our initial update of AOC’s progress in implementing the six 1993 recommendations

occurred in May 1994 when we testified before the Senate Committee on

Governmental Affairs.1 At that time, we found that AOC had made progress in

improving its long-range planning process by taking action to implement two of our

six recommendations. The two recommendations involved AOC’s efforts to update

the federal districts’ long-range plans and the time periods covered by AOC’s

projections of the space needed in court-related facilities within various districts. We

also found that AOC was assessing other options to fully address the remaining four

recommendations that had been partially implemented or not yet fully addressed.

These recommendations addressed such matters as the use of improved statistical

methods for classifying districts, forecasting caseloads, and projecting personnel

needs.

Subsequently, in response to the Subcommittee’s August 2000 request, our objective

was to obtain updated information on the extent to which AOC has continued its

efforts to implement our recommendations since 1994. To accomplish this objective,

we interviewed AOC headquarters officials responsible for long-range planning and

the consultants who assisted AOC in its long-range planning efforts. We also

reviewed relevant documents including (1) prior GAO reports and testimony; (2) AOC

materials related to long-range planning, including plans submitted by two federal

districts that AOC identified as illustrative examples of recently completed plans; and

(3) appropriate sections of a May 2000 study prepared by Ernst & Young (EY) on the

judiciary’s space and facilities program.2

On December 13, 2000, we used the slides that appear in enclosure I to brief the

Subcommittee on the results of our work. The slides include AOC’s oral comments.

In addition, AOC’s written comments on this letter and the slides are included in

enclosure II. As agreed with the Subcommittee, the transmittal of this letter and its

two enclosures represents the culmination of our work on this assignment.

Results in Brief

The enclosed slides show that, since 1994, AOC has continued its efforts to improve

the long-range planning process and has made progress in implementing our six

previous recommendations. Of the six recommendations, AOC has fully

implemented five and partially implemented one.

Specifically, AOC’s actions for implementing the five recommendations primarily

involved

 1

Federal Judiciary Space: Progress Is Being Made to Improve the Long-Range Planning Process

(GAO/T-GGD-94-146, May 4, 1994).

2

Independent Assessment of the Judiciary’s Space and Facilities Program, Ernst & Young, May 2000.
3 GAO-01-308R Long-Range Planning for Federal Judiciary Space

• using a statistical classification technique, known as cluster analysis,3 to create

groups of federal districts with similar characteristics;

• relying on an automated computer program called AnyCourt, which is based on

the U.S. Courts Design Guide (the Design Guide), to verify that districts’ estimates

of space baselines are consistent with the Design Guide;

• using a standard statistical forecasting technique, known as AutoRegressive

Integrated Moving-Average (ARIMA),4 as a basis for developing more accurate

caseload projections;

• helping districts improve their personnel projections by comparing their

projections to AOC personnel projections and discussing the results with districts

when large deviations occurred; and

• providing GSA with data related to 10-year projections of the districts’ estimated

space needs to support the judiciary’s request for congressional approval of funds

to build new court-related facilities and modify existing buildings.

Regarding the remaining partially implemented recommendation related to updating

the districts’ plans, AOC has not made as much progress as it had anticipated when

we testified in 1994 on AOC’s progress in implementing our recommendations. AOC’s

current goal is to update the districts’ long-range plans every 3 to 5 years, but it has

not been able to meet this goal. AOC said that it encourages, but cannot compel, the

districts to update their plans and that the need to modify plans for which funding

was delayed impeded its ability to update some other districts’ plans. Nevertheless,

AOC said that districts with space needs have been treated fairly and consistently.

AOC mentioned that it considers the updating of districts’ long-range plans to be an

important effort and that it has focused its efforts on updating plans from those

districts that have construction projects in the judiciary’s 5-year plan, so that the

Congress and GSA can have the most accurate and up-to-date space requirements for

those projects. Also, according to AOC, the judiciary is considering whether to

reinstate regular site visits so more consistent updating of the districts’ plans can

occur.

In addition to its efforts to implement our recommendations, AOC is currently

considering how to best implement other improvements, including some that were

suggested by EY, to the long-range planning process, such as using more advanced

techniques to forecast caseloads. Although AOC’s actions to date have improved the

overall process, additional study would be needed to determine whether other

opportunities exist that could further help improve the long-range planning process.

However, as agreed with the Subcommittee, we did not follow up on them in order to

give AOC time to fully implement the improvements it already has planned or has

underway.

 3

Cluster analysis is the generic name for several statistical procedures that are used to create groups or

“clusters” of similar entities.

4

ARIMA is an advanced statistical technique that can be used to predict future trends from time series

data (e.g., monthly or yearly data on past activity).
4 GAO-01-308R Long-Range Planning for Federal Judiciary Space

AOC’s Comments and

Our Response

In December 2000, we obtained oral comments on a draft of the enclosed slides from

AOC’s Assistant Director, Office of Facilities and Security, and other responsible AOC

officials. The officials generally agreed with the information presented in the slides.

As a result of our discussions with AOC, we made some technical corrections and

clarifications to the slides, where appropriate.

In addition to oral comments, in January 2001, AOC’s Associate Director provided us

written comments on a draft of this letter and the enclosed slides. In his written

comments, AOC’s Associate Director stated that he was pleased that we recognized

the judiciary’s progress in improving the process since we completed our initial

review in 1993, including AOC’s efforts to address and implement our

recommendations, and that AOC has worked diligently over the past 10 years to

develop a sound long-range planning process. AOC’s Associate Director said that the

process was the first of its kind to establish a systematic approach to space and

facilities planning in the federal government and that it was being used by other

agencies as a model. Furthermore, he mentioned that the value of the process was

nationally recognized in 1998, when AOC received GSA’s Annual Achievement Award

for Real Property Innovation. He said that despite these achievements, however,

AOC will continue to refine and improve the process. In addition to the written

comments, AOC provided some technical clarifications to the letter and the enclosed

slides, which we included where appropriate.

We performed our work from August 2000 to December 2000 in accordance with

generally accepted government auditing standards. We are sending copies of this

letter to the Chairman and Ranking Member, Senate Committee on Environment and

Public Works; the Honorable Jane R. Roth, Chair, Judicial Conference Committee on

Security and Facilities; the Honorable L. Ralph Mecham, Director, AOC; the

Honorable Thurman M. Davis, Sr., Acting Administrator of GSA; the Director of the

Office of Management and Budget, and other interested congressional committees.

The letter will also be available on GAO’s home page at http://www.gao.gov. Major

contributors to this letter included David Sausville, Martin de Alteriis, James M.

Fields, Anne Hilleary, and Casey Brown. If you have any questions about this letter

or the enclosures, please contact me at (202) 512-8387 or at ungarb@gao.gov.

Bernard L. Ungar

Director, Physical Infrastructure

Enclosures - 2
Enclosure I

Federal Judiciary Space: Update on Improvement of the Long-Range Planning Process5 GAO-01-308R Long-Range Planning for Federal Judiciary Space

Federal Judiciary Space

Administrative Office of the U.S. Courts

Update on Improvement of the

Long-Range Planning Process
Enclosure I

Federal Judiciary Space: Update on Improvement of the Long-Range Planning Process6 GAO-01-308R Long-Range Planning for Federal Judiciary Space

Presentation Overview• Objective• Scope and methodology• Background• Results in brief• Status of Administrative Office of the U.S. Courts’ (AOC) efforts to

implement GAO’s recommendations

• Results of Ernst & Young’s (EY) May 2000 study• Other AOC actions to improve the long-range planning process• Issues that could merit further inquiry• AOC’s comments and our response
Enclosure I

Federal Judiciary Space: Update on Improvement of the Long-Range Planning Process7 GAO-01-308R Long-Range Planning for Federal Judiciary Space

Objective• Provide updated information on AOC’s progress in implementing

GAO’s six recommendations intended to help improve the longrange planning process.
Enclosure I

Federal Judiciary Space: Update on Improvement of the Long-Range Planning Process8 GAO-01-308R Long-Range Planning for Federal Judiciary Space

Scope and Methodology• Scope

• AOC headquarters offices, Washington, D.C.

• Methodology

• Interviewed

• AOC officials responsible for long-range planning and• consultants that assisted in AOC’s statistical analyses.

• Reviewed relevant documents, including

• prior GAO reports and testimony;• appropriate sections of EY’s May 2000 report concerning the

judiciary’s space and facilities program;

• materials related to long-range planning, such as plans

prepared by two federal districts; and

• AOC’s written responses to GAO’s questions.
Enclosure I

Federal Judiciary Space: Update on Improvement of the Long-Range Planning Process9 GAO-01-308R Long-Range Planning for Federal Judiciary Space

Background• The purpose of the judiciary’s long-range planning process is to

project future workload, personnel, and space needs for federaldistrict courts.

• Each of the 94 federal districts is expected to prepare and

update a long-range plan (LRP).

• AOC, the administrative arm of the judiciary, assists the districts

in their planning efforts.

• The results of the process are used by the General Services

Administration to develop funding requests for new or expandedspace.
Enclosure I

Federal Judiciary Space: Update on Improvement of the Long-Range Planning Process10 GAO-01-308R Long-Range Planning for Federal Judiciary Space

Background: Overview of the Long-RangePlanning Process• To assist the districts, AOC provides workbooks that contain

• workload forecasts,• formulas for developing personnel projections based on

workload forecasts, and

• the districts’ previous personnel projections.

• Districts initially update their existing LRPs to send to AOC. The

updates include

• personnel projections for 5, 10, 20, and 30 years and• current space and current unmet space needs.

• AOC estimates future space needs based on personnel projections

and space baselines, reviews updates, and provides feedback todistricts.

• Districts finalize their updates after discussions with AOC.
Enclosure I

Federal Judiciary Space: Update on Improvement of the Long-Range Planning Process11 GAO-01-308R Long-Range Planning for Federal Judiciary Space

Background: Prior GAO Work• In 1993, GAO evaluated the reasonableness of the judiciary’s

process for projecting long-range space needs.

• We identified three major problems in our report.• We made six recommendations to address the problems.

• In 1994, we testified that the judiciary had made progress in

implementing our recommendations.
Enclosure I

Federal Judiciary Space: Update on Improvement of the Long-Range Planning Process12 GAO-01-308R Long-Range Planning for Federal Judiciary Space

Results in Brief• Since 1994, AOC has continued its efforts to improve the longrange planning process.

• Of the six recommendations, AOC has

• fully implemented five and• partially implemented one.

• AOC is also working to implement other improvements to the

process, such as reinstating site visits and using more advancedforecasting techniques.

• Although AOC’s actions have improved the long-range planning

process, additional study would be needed to determine whetherother opportunities exist to further help improve the process.
Enclosure I

Federal Judiciary Space: Update on Improvement of the Long-Range Planning Process13 GAO-01-308R Long-Range Planning for Federal Judiciary Space

Results in Brief

Problem s identified in1993 report Recomm endations Recomm endation status and AO C’s actions

Issues that couldm erit further inquiry

1a) AO C should prepare updated spaceplans for all districts whenever changesare m ade to the assum ptions that affectstaff and space allocations.

Status: Partially Im plem entedAOC’s actions : Established a goa l to update theplans every 3 to 5 years, but the goal is not beingm et for som e districts. The judiciary is consideringth e reintroduction of site visit plannin g sessions sothat m ore consistent updating can occur.

The reasons whysom e districts do notupdate their plans andthe possible effects onlong-range planning

1. Not all districts weretreated consistently.

1b) AO C should devise a m ethod forclassifying districts that would considercase com plexity in addition to caseload.

Status: Fully Im plem entedAOC’s actions : Adopted a statistical classificationm ethod that G AO recom mended. W henperform ing the classification, AO C uses 45variables, several of which m easure caseco m plexity.

AOC ’s approach in

using cluster analysisto develop personnelprojections fromworkload forecasts

2. Districts’ baselines towhich future space needswere added did notalways accurately reflectcurrent space needs.

2) AO C should revise the long-rangeplanning process to re quire th atbaselines reflect the relationships am ongcaseloads, staff needs, and spacerequirem ents and should include am echanism to verify that deficitsrepresent actual unm et space needs.

Status: Fully Im plem entedAOC’s actions : Checks the baselines using acom puter program that is based on the standardsin the U.S. Courts Design Guide. In addition, itreview s and discusses deficits w ith AO C programmanagers.

No issues

3a) AO C should identify and use astandard statistical technique that wouldgenerate accurate caseload projectionswith defined confidence intervals.

Status: Fully Im plem entedAOC’s actions : Uses a standard statisticalforecasting technique (known as ARIM A) thatgenerates confidence intervals.

Possible additionalforecastingim provem ents

3b) AO C sh ould reduce th e subjectivityof the process by elim inating the use ofarbitrarily selected regression m odelsand by verifying the inform ation providedby local representatives.

Status: Fully Im plem entedAOC’s actions : Uses an acceptable m odelingtechnique (ARIM A) and com pares local personnelforecasts to its own data.

The extent to whichdistricts’ final spaceprojections differ fromAOC ’s suggestedprojections

3. M ethodology used forprojecting caseloads wasnot statisticallyacceptable.

3c) AO C should lim it the time spancovered by the projections to 10 years.

Status: Fully Im plem entedAOC’s actions : Provides G SA with projectproposals based on its 10-year spacerequirem ents.

No issues
Enclosure I

Federal Judiciary Space: Update on Improvement of the Long-Range Planning Process14 GAO-01-308R Long-Range Planning for Federal Judiciary Space

Status of AOC’s Efforts to Implement Six GAO

Recommendations Since 1994

Status of therecommendations 1994 2000Fully implemented 2 (Recommendations

1a and 3c)

5 (Recommendations

1b, 2, 3a, 3b, and 3c)

Partiallyimplemented

2 (Recommendations

3a and 3b)

1 (Recommendation 1a)

Not yet fullyaddressed

2 (Recommendations

1b and 2)

0

Total 6 6

Note: In 2000, we reclassified Recommendation 1a, which dealt with updating districts’ space plans, as “partially implemented” because AOC has been unable to update the plans regularly in recent years.
Enclosure I

Federal Judiciary Space: Update on Improvement of the Long-Range Planning Process15 GAO-01-308R Long-Range Planning for Federal Judiciary Space

Recommendation 1a: AOC should prepare updated space plans for

all districts whenever changes are made to the assumptions that affectstaff and space allocations.

Status: Partially Implemented

• AOC stated that it would begin to update the plans for all districts biennially

starting in 1994. However, AOC’s data show that more than 20 plans have

not been updated since 1994.

• The current goal is to update all plans every 3 to 5 years; however, AOC

said that this goal is not being met.

• AOC stated the following:

• It encourages districts, but cannot compel them, to update their plans. It has

recently focused attention on those districts expressing space needs.

• Districts with space needs are being treated fairly and consistently.• Recent delays in construction funding have had an impact on updating.• Updating district plans is an important effort, and the judiciary will consider

whether to reinstate regular site visits so more consistent updating of districts’plans can occur. It plans to pilot new updating procedures in three districts byMarch 2001.
Enclosure I

Federal Judiciary Space: Update on Improvement of the Long-Range Planning Process16 GAO-01-308R Long-Range Planning for Federal Judiciary Space

Recommendation 1b: AOC should devise a method for classifyingthe districts that would consider case complexity in addition tocaseload.

Status: Fully Implemented

• In response to our recommendation, AOC is now using a statistical

classification technique known as cluster analysis.1

• AOC’s cluster analysis uses 45 variables to create groups of

districts with similar characteristics.

• 30 variables are related to court activities, such as the number

of criminal and civil cases per district judge.

• 15 variables are demographic, such as population and income.

• AOC now accounts for case complexity in some of the court-related

variables, including

• weighted caseload and• the number of defendants per criminal case.

1Cluster analysis is the generic name for several statistical procedures that are used to create

groups or “clusters” of similar entities.
Enclosure I

Federal Judiciary Space: Update on Improvement of the Long-Range Planning Process17 GAO-01-308R Long-Range Planning for Federal Judiciary Space

Recommendation 2: AOC should revise the long-range planning process to

require that baselines reflect the relationships among caseloads, staff needs,and space requirements and include a mechanism to verify that deficitsrepresent actual unmet space needs.

Status: Fully Implemented

• AOC stated that it recently started routine checks of districts’ space

baselines, using an automated computer program, called theAnyCourt model, that is based on the U.S. Courts Design Guide.

• AnyCourt can assess whether a baseline (existing space plus

unmet space needs) is consistent with what the Design Guidewould specify for existing personnel.

• AOC said it investigates any discrepancies between the baselines

and the Design Guide’s standards.

• As part of its investigation, AOC said it reviews and discusses

the baselines with AOC program managers who are familiarwith the courthouses.
Enclosure I

Federal Judiciary Space: Update on Improvement of the Long-Range Planning Process18 GAO-01-308R Long-Range Planning for Federal Judiciary Space

Recommendation 3a: AOC should identify and use a standard

statistical technique that would generate accurate caseloadprojections with defined confidence intervals.

Status: Fully Implemented

• AOC now uses a standard statistical forecasting technique

known as AutoRegressive Integrated Moving-Average(ARIMA)2 modeling that generates caseload projections with

confidence intervals.3

• AOC is developing procedures to use additional techniques

that could yield more accurate forecasts.

2ARIMA is an advanced statistical technique that can be used to predict future trends from time

series data (e.g., monthly or yearly data on past activity).

3Confidence intervals can measure one aspect of the quality of forecasting models.
Enclosure I

Federal Judiciary Space: Update on Improvement of the Long-Range Planning Process19 GAO-01-308R Long-Range Planning for Federal Judiciary Space

Recommendation 3b: AOC should reduce the subjectivity of the long-range

planning process by (1) eliminating the use of arbitrarily selected regressionmodels and (2) verifying the information provided by local representatives.

Status: Fully Implemented

• AOC implemented part (1) of this recommendation by instituting

ARIMA modeling.

• Concerning part (2) of this recommendation, AOC

• compares the personnel projections provided by local

representatives to its own statistical data and

• when large deviations occur,

• discusses the data with the districts and• attempts to help districts develop improved predictions.

• However, the districts finalize the projections based on their own

knowledge and understanding of local conditions.
Enclosure I

Federal Judiciary Space: Update on Improvement of the Long-Range Planning Process20 GAO-01-308R Long-Range Planning for Federal Judiciary Space

Recommendation 3c: AOC should limit the time span covered by

the projections to 10 years.

Status: Fully Implemented

• AOC provides GSA with project proposals that are based on 10-

year space requirements.

• The proposals also include 30-year staff projections, which are

used for site selection.
Enclosure I

Federal Judiciary Space: Update on Improvement of the Long-Range Planning Process21 GAO-01-308R Long-Range Planning for Federal Judiciary Space

Results of EY’s Study• In May 2000, EY published its study on AOC’s Space and Facilities

Program, which included a review of the long-range planningprocess.

• EY reported that the process was “essentially sound” but could be

improved.

• Some of EY’s recommendations related to the issues that we

identified as possibly meriting further inquiry. Among other things,EY recommended that AOC should

• use more advanced forecasting techniques,• improve its use of cluster analysis or replace it with an

alternative statistical technique, and

• establish a more structured approach to obtaining input from

knowledgeable district representatives.
Enclosure I

Federal Judiciary Space: Update on Improvement of the Long-Range Planning Process22 GAO-01-308R Long-Range Planning for Federal Judiciary Space

AOC Plans to Improve the Long-RangePlanning Process• AOC is currently working to implement other improvements to the

long-range planning process. For example, AOC stated that it is

• evaluating advanced techniques designed to reduce the error

margins in forecasting caseloads and

• exploring different ways to apply cluster analysis.

• Furthermore, the judiciary is considering the reinstatement of site

visits.

• AOC said that it will need at least 6 months to implement these new

improvements.
Enclosure I

Federal Judiciary Space: Update on Improvement of the Long-Range Planning Process23 GAO-01-308R Long-Range Planning for Federal Judiciary Space

Issues That Could Merit Further Inquiry• Issue 1. Why are some districts not updating their plans, and what

effect does this have on AOC’s long-range planning? (Related to

Recommendation 1a)

• Issue 2. Does AOC’s cluster analysis, as implemented, provide a

reasonable basis for developing personnel projections fromworkload forecasts? (Related to Recommendation 1b)

• Issue 3. Do opportunities exist for AOC to further improve

workload forecasting techniques? (Related to Recommendation3a)

• Issue 4. To what extent do the districts’ final space projections

differ from AOC’s suggested projections? (Related toRecommendation 3b)
Enclosure I

Federal Judiciary Space: Update on Improvement of the Long-Range Planning Process24 GAO-01-308R Long-Range Planning for Federal Judiciary Space

AOC’s Comments and Our Response• On December 8, 2000, we briefed AOC.• AOC generally agreed with our findings.

• AOC stated that the judiciary is considering actions to address

the only recommendation that has not been fully implemented.

• The reinstatement of site visits is being considered so more

consistent updating of the district plans can occur.

• As a result of our discussions with AOC and the information it

provided, we made some

• technical corrections and• clarifications.
Enclosure II

Federal Judiciary Space: Update on Improvement of the Long-Range Planning

Process

25 GAO-01-308R Long-Range Planning for Federal Judiciary Space

(393006)
Orders by Internet

For information on how to access GAO reports on the Internet, send an e-mail message

with “info” in the body to

info@www.gao.gov

or visit GAO’s World Wide Web home page at

http://www.gao.gov

To Report Fraud, Waste, and Abuse

in Federal Programs

Web site: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: 1-800-424-5454