FEDERAL JUDGESHIPS

The General Accuracy of 

District and Appellate 

Judgeship Case-Related 

Workload Measures

Statement of David C. Maurer, Director 

Homeland Security and Justice

Statement for the Record to the 

Subcommittee on Bankruptcy and the 

Courts, Committee on the Judiciary, 

U.S. Senate

For Release on Delivery 

Expected at 10:30 a.m. EDT

Tuesday, September 10, 2013

GAO-13-862T

United States Government Accountability Office
 

Page 1 GAO-13-862T 

Mr. Chairman and Members of the Subcommittee:

I appreciate the opportunity to submit this statement on our work on caserelated workload measures for district court and courts of appeals judges. 

My statement today is based on work completed and reported in 20031

and discussed in testimony on June 17, 2008,

2 and September 30, 2009.3

My statement is focused exclusively on these workload measures; we 

have not assessed or taken a position on the Judicial Conference’s 

pending request for additional judgeships.

Biennially, the Judicial Conference, the federal judiciary’s principal policymaking body, assesses the judiciary’s needs for additional judgeships.4 If 

the conference determines that additional judgeships are needed, it 

transmits a request to Congress identifying the number, type (such as 

courts of appeals or district courts), and location of the judgeships it is 

requesting.

In assessing the need for additional judgeships, the Judicial Conference 

considers a variety of information, including responses to its biennial 

survey of individual courts, temporary increases or decreases in case 

filings, and other factors specific to an individual court. However, the 

Judicial Conference’s analysis begins with the quantitative case-related 

workload measures it has adopted for the district courts and courts of 

appeals—weighted case filings and adjusted case filings, respectively. 

These two measures recognize, to different degrees, that the time 

demands on judges are largely a function of both the number and 

complexity of the cases on their dockets. Some types of cases may 

demand relatively little time and others may require many hours of work. 

Generally, each case filed in a district court is assigned a weight 

 

1

GAO, Federal Judgeships: The General Accuracy of the Case-Related Workload 

Measures Used to Assess the Need for Additional District Court and Courts of Appeals 

Judgeships, GAO-03-788R (Washington, D.C., May 30, 2003).

2

GAO, Federal Judgeships: General Accuracy of District and Appellate Judgeship CaseRelated Workload Measures, GAO-08-928T (Washington, D.C., June 17, 2008).

3

GAO, Federal Judgeships: The General Accuracy of District and Appellate Judgeship 

Case-Related Workload Measures, GAO-09-1050T (Washington, D.C., Sept. 30, 2009).

4

The Chief Justice of the United States presides over the conference, which consists of 

the chief judges of the 13 courts of appeals, a district judge from each of the 12 regional 

circuits, and the chief judge of the Court of International Trade. The conference meets 

twice a year.

 
 

Page 2 GAO-13-862T 

representing the average amount of judge time the case is expected to 

require. The weights are relative to one another: the higher the case 

weight, the greater the time the case would be expected to require. For 

example, on average, a case with a relative weight of 2.0 would be 

expected to require twice as much judge time as a case with a weight of 

1.0. In the courts of appeals, all case filings are weighted equally at 1.0, 

except for pro se case filings—those in which one or both parties are not 

represented by an attorney—which are discounted.

Using these measures, individual courts whose past case-related 

workload meets the threshold established by the Judicial Conference may 

be considered for additional judgeships. These thresholds are 430 

weighted case filings per authorized judgeship for district courts and 500 

adjusted case filings per three-judge panel of authorized judgeships for 

courts of appeals (courts of appeals judges generally hear cases in 

rotating panels of three judges each). Authorized judgeships are the total 

number of judgeships authorized by statute for each district court or court 

of appeals.

The Judicial Conference relies on these quantitative workload measures 

to be reasonably accurate measures of judges’ case-related workload. 

Whether these measures are reasonably accurate rests in turn on the 

soundness of the methodology used to develop them. This statement 

summarizes information and recommendations from our 2003 report and 

related updates to the methodology including (1) whether the judiciary’s 

quantitative case-related workload measures were reasonably accurate 

measures of district judges’ and courts of appeals judges’ case-related 

workload, and (2) actions the judiciary has taken to implement 

recommendations from our 2003 report.

Our 2003 report and 2008 and 2009 testimonies were based on the 

results of our review of documentation provided by the Federal Judicial 

Center (FJC) and the Administrative Office of the U.S. Courts (AOUSC) 

on the history and development of the case-related workload measures 

and interviews with officials in each organization. The scope of our work 

did not include how the Judicial Conference used these case-related 

workload measures to develop any specific request for additional district 

or courts of appeals judgeships. In addition, in August 2013, we met with 

FJC and AOUSC officials to discuss any updates that have been made to 

the methodologies for developing case-related workload measures. While 

we obtained some updated information for the purposes of this statement, 

we did not evaluate these methodologies or the judiciary’s efforts in this 

area. This work was conducted in accordance with generally accepted 
 

Page 3 GAO-13-862T 

government auditing standards. Those standards require that we plan 

and perform the audit to obtain sufficient, appropriate evidence to provide 

a reasonable basis for our findings and conclusions based on our audit 

objectives. We believe that the evidence obtained provides a reasonable 

basis for our findings and conclusions based on our audit objectives.

The demands on judges’ time are largely a function of both the number 

and complexity of the cases on their dockets. To measure the caserelated workload of district court judges, the Judicial Conference has 

adopted weighted case filings. The purpose of the district court case 

weights was to create a measure of the average judge time that a specific 

number and mix of cases filed in a district court would require. 

Importantly, the weights were designed to be descriptive, not 

prescriptive—that is, the weights were designed to develop a measure of 

the national average amount of time that judges actually spent on specific 

cases, not to develop a measure of how much time judges should spend 

on various types of cases. Moreover, the weights were designed to 

measure only case-related workload. Judges have noncase-related duties 

and responsibilities, such as administrative tasks, that are not reflected in 

the case weights.

With few exceptions, such as cases that are remanded to a district court 

from the court of appeals, each civil or criminal case filed in a district court 

is assigned a case weight. For example, in the 2004 case weights—which 

are still in use—drug possession cases are weighted at 0.86, while civil 

copyright and trademark cases are weighted at 2.12. The total annual 

weighted filings for a district are determined by summing the case weight 

associated with all the cases filed in the district during the year. A 

weighted case filing per authorized judgeship is the total annual weighted 

filings divided by the total number of authorized judgeships. The Judicial 

Conference uses weighted filings of 430 or more per authorized judgeship 

as an indication that a district may need additional judgeships. Thus, for 

example, a district with 460 weighted filings per authorized judgeship, 

including newly requested judgeships, could be considered for an 

additional judgeship. However, the Judicial Conference does not consider 

a district for additional judgeships, regardless of its weighted case filings, 

if the district does not request any additional judgeships.

Background 
 

Page 4 GAO-13-862T 

In our 2003 report, we found the district court case weights approved in 

1993 to be a reasonably accurate measure of the average time demands 

a specific number and mix of cases filed in a district court could be 

expected to place on the district judges in that court. The methodology 

used to develop the weights used a valid sampling procedure, developed 

weights based on actual case-related time recorded by judges from case 

filings to disposition, and included a measure (standard errors) of the 

statistical confidence in the final weight for each weighted case type. 

Without such a measure, it is not possible to objectively assess the 

accuracy of the final case weights.

At the time of our 2003 report, the Subcommittee on Judicial Statistics of 

the Judicial Conference’s Judicial Resources Committee had approved 

the research design for revising the 1993 case weights, with a goal of 

having new weights submitted to the Judicial Resources Committee for 

review in the summer of 2004. The design for the new case weights relied 

on three sources of data for specific types of cases: (1) data from 

automated databases identifying the docketed events associated with the 

cases; (2) data from automated sources on the time associated with 

courtroom events for cases, such as trials or hearings; and (3) consensus 

of estimated time data from structured, guided discussion among 

experienced judges on the time associated with noncourtroom events for 

cases, such as reading briefs or writing opinions.

As we reported in 2009, according to FJC, the subcommittee wanted a 

study that could produce case weights in a relatively short period of time 

without imposing a substantial record-keeping burden on district judges.5

FJC staff provided the subcommittee with information about various 

approaches to case weighting, and the subcommittee chose an eventbased method—that is, a method that used data on the number and types 

of events, such as trials and other evidentiary hearings, in a case. The 

 

5

GAO-09-1050T.

Accuracy of 

Judiciary’s Workload 

Measures Could Not 

Be Determined

1993 District Case Weights 

Reasonably Accurate, but 

Accuracy of 2004 Case 

Weights Could Not Be 

Statistically Determined
 

Page 5 GAO-13-862T 

design did not involve the type of time study that was used to develop the 

1993 case weights. Although the proposed methodology appeared to 

offer the benefit of reduced judicial burden (no time study data collection), 

potential cost savings, and reduced calendar time to develop the new 

weights, we had two areas of concern—the challenge of obtaining 

reliable, comparable data from two different data systems for the analysis 

and the limited collection of actual data on the time judges spend on 

cases. 

First, the design assumed that judicial time spent on a given case could 

be accurately estimated by viewing the case as a set of individual tasks or 

events in the case. Information about event frequencies and, where 

available, time spent on the events would be extracted from existing 

administrative databases and reports and used to develop estimates of 

the judge time spent on different types of cases. For event data, the 

research design proposed using data from two databases (one of which 

was new in 2003 and had not been implemented in all district courts) that 

would have to be integrated to obtain and analyze the event data. FJC 

proposed creating a technical advisory group to address this issue. In 

August 2013, FJC officials told us that the process of integrating the two 

data systems, though labor-intensive, was successful and resulted in 

accurate data. However, we have not reviewed the integration process for 

the two data systems, so we cannot determine the effectiveness of this 

process or whether accurate data resulted.

Second, we reported that the research design did not require judges to 

record time spent on individual cases, as was done for the 1993 case 

weights. Actual time data would be limited to that available from existing 

databases and reports on the time associated with certain courtroom 

events and proceedings for different types of cases. However, a majority 

of district judges’ time is spent on case-related work outside the 

courtroom. The time required for noncourtroom events—and some 

courtroom events that did not have actual time data available—would be 

derived from structured, guided discussion of groups of 8 to 13 

experienced district court judges in each of the 12 regional circuits (about 
 

Page 6 GAO-13-862T 

100 judges in all).6 The judges would develop estimates of the time 

required for different events in different types of cases within each circuit 

using FJC-developed “default values” as the reference point for 

developing their estimates. These default values would be based in part 

on the existing case weights and, in part, on other types of analyses. 

Following the meetings of the judges in each circuit, a national group of 

24 judges (2 from each circuit) would consider the data from the 12 circuit 

groups and develop the new weights.

The accuracy of judges’ time estimates is dependent upon the experience 

and knowledge of the participating judges and the accuracy and reliability 

of the judges’ recall about the average time required for different events in 

different types of cases—about 150 if all the case types in the 1993 case 

weights were used. In 2003, we found that these consensus data could 

not have been used to calculate statistical measures of the accuracy of 

the resulting case weights. Thus, we concluded that the planned 

methodology did not make it possible to objectively, statistically assess 

how accurate the new case weights are—weights whose accuracy the 

Judicial Conference relies upon in assessing judgeship needs.

In August 2013, AOUSC officials stated that, since 2005, for purposes of 

determining the need for an additional authorized judgeship, a district’s 

weighted case filings per authorized judgeship is calculated by including 

the potential additional judgeship. For example, if a district had total 

weighted filings of 4,600 and 9 authorized judgeships, and it planned to 

request 1 additional judgeship, its weighted filings per authorized 

judgeship, for purposes of the judgeship request process, would be 460. 

Without including the potential additional judgeship in the calculation, the 

weighted case filings would be about 511. AOUSC officials stated in 

August 2013 that the judiciary adopted the proposed methodology in 

2004 and does not have plans to update the 2004 district court case 

weights.

 

6

In August 2013, FJC officials stated that, for the 2004 case weight methodology, there 

were 15 criminal case event types and 13 civil case event types. Actual event frequency 

data from administrative databases and reports were used in the case weight 

methodology for all criminal and civil case event types. Actual event time data from 

administrative databases and reports were used for 4 of the criminal and civil case event 

types, such as a jury trial. Time estimates from the judge discussion process were used 

for 11 criminal case event types and 9 civil case event types—including all 6 

noncourtroom events, such as preparation for trial.
 

Page 7 GAO-13-862T 

In 2003, we found that the principal quantitative measure the Judicial 

Conference used to assess the need for additional courts of appeals 

judgeships was adjusted case filings. The measure is based on data 

available from standard statistical reports for the courts of appeals. The 

adjusted filings workload measure is not based on any empirical data 

regarding the time that different types of cases required of appellate 

judges.

The Judicial Conference’s policy is that courts of appeals with adjusted 

case filings of 500 or more per 3-judge panel may be considered for 1 or 

more additional judgeships. Courts of appeals generally decide cases 

using constantly rotating 3-judge panels. Thus, if a court had 12 

authorized judgeships, those judges could be assigned to four panels of 3 

judges each. In assessing judgeship needs for the courts of appeals, the 

conference may also consider factors other than adjusted filings, such as 

the geography of the circuit or the median time from case filings to 

disposition.7

Essentially, the adjusted case filings workload measure counts all case 

filings equally, with two exceptions. First, cases refiled and approved for 

reinstatement are excluded from total case filings.8 Second, pro se cases

are weighted at 0.33, or one-third as much as other cases, which are 

weighted at 1.0. For example, a court with 600 total pro se case filings in 

a year would be credited with 198 adjusted pro se case filings (600 x 

0.33). Thus, a court of appeals with 1,600 filings (excluding 

reinstatements)—600 pro se cases and 1,000 non-pro se cases—would 

be credited with 1,198 adjusted case filings (198 discounted pro se cases 

plus 1,000 non-pro se cases). If this court had 6 judges (allowing two 

panels of 3 judges each), it would have 599 adjusted case filings per 3-

judge panel, and, thus, under Judicial Conference policy, could be 

considered for an additional judgeship.

 

7

At the time of our 2003 report, FJC had suggested that adjusted case filings may not be 

an appropriate measure for the Court of Appeals for the D.C. Circuit, given the distinctive 

characteristics of the administrative agency appeals that were a major source of that 

court’s caseload. Details on the FJC analysis for the D.C. Circuit can be found in our 2003 

report: GAO-03-788R. In August 2013, AOUSC reported that the judiciary does not have 

plans to develop an authorized judgeship workload formula specific to the Court of 

Appeals for the D.C. Circuit.

8

Such cases were dismissed for procedural defaults when originally filed, but “reinstated” 

to the court’s calendar when the case was later refiled. The number of such cases, as a 

proportion of total cases, is generally small.

Accuracy of Courts of 

Appeals Case-Related 

Workload Measure Could 

Not Be Assessed
 

Page 8 GAO-13-862T 

The current court of appeals workload measure, which, AOUSC officials 

stated, was adopted in 1996, represents an effort to improve the previous 

measure. In our 1993 report on judgeship needs assessment, we found 

that the restraint of individual courts of appeals, not the workload 

standards, seemed to have determined the actual number of appellate 

judgeships the Judicial Conference requested.9 At the time the current 

measure was developed and approved, using the new benchmark of 500 

adjusted case filings resulted in judgeship numbers that closely 

approximated the judgeship needs of the majority of the courts of 

appeals, as the judges of each court perceived them. The current courts 

of appeals case-related workload measure principally reflects a policy 

decision using historical data on filings and terminations. It is not based 

on empirical data regarding the judge time that different types of cases 

may require. On the basis of the documentation we reviewed for our 2003 

report, we determined that there was no empirical basis for assessing the 

potential accuracy of adjusted case filings as a measure of case-related 

judge workload.

In our 2003 report, we recommended that the Judicial Conference of the 

United States

• update the district court case weights using a methodology that 

supports an objective, statistically reliable means of calculating the 

accuracy of the resulting weights, and

• develop a methodology for measuring the case-related workload of 

courts of appeals judges that supports an objective, statistically 

reliable means of calculating the accuracy of the resulting workload 

measures and that addresses the special case characteristics of the 

Court of Appeals for the D.C. Circuit.

Neither of these recommendations has been implemented, and in August 

2013, AOUSC officials stated that the judiciary does not have plans to 

update the 2004 district court case weights or the 1996 court of appeals 

adjusted filings weights. With regard to our 2003 recommendation for 

updating the district court case weights, we reported that FJC agreed that 

the method used to develop the new case weights would not permit the 

calculation of standard errors, but that other methods could be used to 

 

9

GAO, Federal Judiciary: How the Judicial Conference Assesses the Need for More 

Judges, GAO/GGD-93-31 (Washington, D.C., Jan. 29, 1993).

Judiciary Has Taken 

No Action to Address 

Our 2003 

Recommendations 
 

Page 9 GAO-13-862T 

assess the integrity of the resulting case weight system.10 In response, we 

noted that the Delphi technique to be used for developing out-of-court 

time estimates was most appropriate when more precise analytical 

techniques were not feasible and the issue could benefit from subjective 

judgments on a collective basis. More precise techniques were available 

for developing the new case weights and were to be used for developing 

new bankruptcy court case weights.

In our 2003 report, we also concluded that the methodology the Judicial 

Conference decided to begin in June 2002 for the revision of the 

bankruptcy case weights offered an approach that could be usefully 

adopted for the revision of the district court case weights.11 The 

bankruptcy court methodology used a two-phased approach. First, new 

case weights were to be developed based on time data recorded by 

bankruptcy judges for a period of weeks—a methodology very similar to 

that used to develop the bankruptcy case weights that existed in 2003 at 

the time of our report. The accuracy of the new case weights could be 

assessed using standard errors. The second part represents 

experimental research to determine if it is possible to make future 

revisions of the weights without conducting a time study. The data from 

the time study could be used to validate the feasibility of this approach. If 

the research determined that this was possible, the case weights could be 

updated more frequently with less cost than required by a time study. We 

concluded in 2003 that that approach could provide (1) more accurate 

weighted case filings than the design developed and used for the 

development of the 2004 district court case weights, and (2) a sounder 

method of developing and testing the accuracy of case weights that were 

developed without a time study. However, we have not reviewed the 

effectiveness of this methodology or confirmed whether the judiciary 

implemented this approach.

With regard to our recommendation on improving the case-related 

workload measure for the courts of appeals, the Chair of the Committee 

on Judicial Resources commented that the workload of the courts of 

 

10We have not reviewed in detail the materials FJC has posted on its website with regard 

to the methodology actually used to develop the revised case weights approved in 2004. 

However, those materials indicate that FJC essentially followed the design we reviewed 

and that standard errors were not computed for the final weights.

11See GAO, Federal Bankruptcy Judges: Weighted Case Filings as a Measure of Judges’ 

Case-Related Workload, GAO-03-789T (Washington, D.C., May 22, 2003).
 

Page 10 GAO-13-862T 

appeals entails important factors that have defied measurement, including 

significant differences in case-processing techniques. We recognized that 

there were significant methodological challenges in developing a more 

precise workload measure for the courts of appeals. However, we stated 

that using the data available, neither we nor the Judicial Conference 

could have assessed the accuracy of adjusted case filings as a measure 

of the case-related workload of courts of appeals judges.

The Ranking Member of the Subcommittee on Bankruptcy and the Courts 

has requested that we conduct a full review of the case-related workload 

measures for district court and courts of appeals judges, including a 

follow-up on our 2003 recommendations. Such a review will allow us to 

evaluate the judiciary’s methodology and efforts over the last 10 years.

Mr. Chairman, this concludes my statement for the record.
 

Page 11 GAO-13-862T 

For further information about this statement, please contact David C. 

Maurer, Director, Homeland Security and Justice Issues, on (202) 512-

9627 or maurerd@gao.gov. In addition to the contact named above, the 

following individuals also made major contributions to this testimony: 

Chris Currie, Acting Director; David P. Alexander, Assistant Director; 

Brendan Kretzschmar; Jean M. Orland; Rebecca Kuhlmann Taylor; and 

Janet G. Temko.

Contact and Staff 

Acknowledgments 

(441185)
This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations 

Public Affairs

Please Print on Recycled Paper.