Federal Bankruptcy Judges: Measuring Judges' Case-Related Workload
Published: 2009-06-16T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Judicial Conference of the United States, the federal judiciary's principal policymaking body, uses 1,500 annual weighted case filings per authorized judgeship (judgeship position) in a bankruptcy court as an indicator of the need for additional bankruptcy judgeships for that court. Total annual weighted case filings for any specific bankruptcy court is the sum of the weights associated with each of the cases filed in the court in a year. Total annual weighted case filings per judgeship represent the estimated average amount of judge time that would be required to complete the cases filed in a specific bankruptcy court in a year. In May 2003 GAO testified on whether weighted case filings were a reasonably accurate measure of the case-related workload of bankruptcy judges. The accuracy of weighted case filings rests in turn on the soundness of the methodology used to develop them. GAO's work focused on whether the methodologies used to develop the current case weights and to revise and update those weights were likely to result in reasonably accurate measures of bankruptcy judges' case-related workload. This statement is based on GAO's May 2003 testimony on weighted case filings as a measure of bankruptcy judges' case-related workload and documentation provided by the Federal Judicial Center (FJC) in June 2009 on subsequent efforts to update the current weighted filings measure.
section_title: What GAO Found
paragraphs: In May 2003 GAO reported that the methodology used to develop the case-related workload measure for federal bankruptcy judges--weighted case filings--were likely to result in reasonably accurate workload measures. The current study to revise those weights, begun in 2008, uses the same methodology as the study used to develop the current case weights and, as designed, is also likely to result in reasonably accurate workload measures. (1) The time demands on bankruptcy judges are largely a function of the number and complexity of the cases on their dockets, with some cases taking more time than others. To measure these differences, the Judicial Conference uses weighted case filings, which are a statistical measure of the average estimated judge time that specific types of bankruptcy cases are expected to take. Each case filed is assigned a weight, and the total weight of all cases filed in a bankruptcy court divided by the number of judgeships for that court provides a measure of the total average case-related workload per judgeship. (2) In assessing the need for new bankruptcy judgeships, the Judicial Conference relies on the weighted case filings to be a reasonably accurate measure of case-related bankruptcy judge workload. Whether the weighted filings are reasonably accurate depends in turn upon the soundness of the methodology used to develop the case weights. (3) On the basis of the documentation provided for our review and discussions with FJC and Administrative Office of the U.S. Courts officials, GAO concluded in 2003 that the case weights, as approved by the Judicial Conference in 1991 and 1996, were likely to be reasonably accurate. (4) The original case weights are now 18 years old. Changes in the intervening years in case characteristics, case management practices, and the implementation of new statutory or procedural requirements, such as the many changes in 2005 Bankruptcy Abuse Prevention and Consumer Protection Act, may have affected the continued accuracy of the current case weights. (5) To the extent that the case weights now understate or overstate the total time demands on bankruptcy judges, use of the weights could potentially result in the Judicial Conference understating or overstating the need for additional bankruptcy judgeships. (6) In 2008, the Federal Judicial Center began a study to revise the current case weights that is designed to collect data on the time bankruptcy judges spend on cases filed during 5, 10-week data collection periods from May 2008 through May 2009. Each active and recalled bankruptcy judge is to participate during one of the five reporting periods. This study design permits the development of new case weights based on the same type of objective time data as the current weights, which we found to be reasonable.

section_title: Letter
paragraphs: I am pleased to be here today to discuss the results of our 2003 review and  assessment of bankruptcy court weighted case filings, the workload  measure the Judicial Conference first considers in assessing the need for  additional bankruptcy judges. My statement today also briefly discusses  the judiciary’s ongoing study to update the current bankruptcy case  weights.
Weighted filings are a statistical measure of the estimated average amount  of judge time that specific types of bankruptcy cases are expected to take.  For example, a business chapter 7 bankruptcy case with assets of $50,000  to $499,999 is expected to take about twice as much judge time as a  nonbusiness chapter 7 case with assets of $50,000 to $499,999. We  assessed whether weighted case filings were a reasonable means of  measuring bankruptcy judges’ case-related workload and assessed the  methodology of proposals to update the current case weights.
My statement today is based on the results of our 2003 review of  documentation provided by the Federal Judicial Center (FJC) and the  Administrative Office of the U.S. Courts (AOUSC) and interviews with  officials in each organization as well as selected updates conducted in  June 2009. We conducted our work in accordance with generally accepted  government auditing standards. In summary, my statement includes the  following major points:    The time demands on bankruptcy judges are largely a function of the  number and complexity of the cases on their dockets. Not all cases  necessarily take the same amount of judge time. Some types of cases  may take more judge time than others.
In assessing the need for new bankruptcy judgeships, the Judicial  Conference relies on the weighted case filings to be a reasonably  accurate measure of case-related bankruptcy judge workload. Whether  weighted case filings are a reasonably accurate workload measure  rests in turn on the soundness of the methodology used to develop the  case weights.
On the basis of the documentation provided for our review and  discussions with FJC and AOUSC officials, we concluded that weighted  case filings, as approved by the Judicial Conference in 1991 and  amended in 1996, were likely to be a reasonably accurate means of  measuring the case-related workload of bankruptcy judges.
The original case weights are now about 18 years old and were based  on time data that are now about 21 years old. Changes in the  intervening years in such factors as case characteristics, case  management practices, or new statutory and procedural requirements,  such as the implementation of the 2005 Bankruptcy Abuse Prevention  and Consumer Protection Act (the Bankruptcy Reform Act), may have  affected whether the case weights continue to be a reasonably accurate  measure of case-related judge workload. Some of these changes may  have increased the time demands on bankruptcy judges and others may  have reduced time demands. To the extent that the case weights may  now understate or overstate time demands on bankruptcy judges, the  weights could potentially result in the Judicial Conference understating  or overstating the need for new bankruptcy judgeships.
The Judicial Conference’s Committee on the Administration of the  Bankruptcy System has approved a revision of the current weights, a  study currently underway, whose methodological design is essentially  identical to that used to develop the current case weights—a  methodology we concluded in 2003 was reasonable.
The accuracy of the case weights is also dependent upon accurately  assigning each case filed in each bankruptcy court to the appropriate  case weight category. AOUSC said that its staff took a number of steps  to ensure that individual cases were assigned to the appropriate case  weight category. These steps are described in appendix I. We did not  evaluate how effective these measures may be in ensuring data  accuracy.
subsections: section_title: Background
paragraphs: Biennially, the Judicial Conference, the federal judiciary’s principal  policymaking body, assesses the judiciary’s needs for additional  judgeships. If the Conference determines that additional judgeships are  needed, it transmits a request to Congress identifying the number, type  (courts of appeals, district, or bankruptcy), and location of the judgeships  it is requesting.
The demands upon judges’ time are largely a function of both the number  and complexity of the cases on their dockets. Some types of cases may  demand relatively little time, and others may require many hours of work.  The federal judiciary has developed workload measures for bankruptcy  judges to estimate the national average amount of a judge’s time that  different types of cases may require. Individual judges may actually spend  more or less time than this average on specific cases within each type— such as personal chapter 7 bankruptcy cases with assets of less than  $50,000 or chapter 13 cases with liabilities of $50,000 or more (see app. II).
In assessing the need for additional bankruptcy judgeships in a bankruptcy  court, the Judicial Conference first considers the court’s weighted case  filings. The Judicial Conference has established 1,500 annual weighted  case filings per authorized judgeship as an indicator of a bankruptcy  court’s potential need for additional judgeships. This represents about  1,500 annual hours of case-related judge time. The Conference’s policy for  assessing bankruptcy judgeship needs recognizes that judges’ workloads  may be affected by factors not captured in the bankruptcy-weighted case  filings. Examples of such factors include historical caseload data and filing  trends; geographic, economic, and demographic factors in the bankruptcy  district; and the availability of alternative solutions and resources for  handling a court’s workload, such as assistance from judges outside the  district. However, our analysis focused solely on the weighted case filings  workload measure.
Each case filed in a bankruptcy court is assigned a case weight. The case  weight statistically represents the national average amount of judicial  time, in hours, each type of bankruptcy case would be expected to require.  The case weights are based on a 1988-1989 study in which bankruptcy  judges completed diaries on how many hours they spent on specific types  of cases and noncase-related work. Total annual weighted case filings for  any specific bankruptcy court is the sum of the weights associated with  each of the cases filed in the court in a year. Total annual weighted case  filings per judgeship represent the estimated average amount of judge time  that would be required to complete the cases filed in a specific bankruptcy  court in a year.
Weighted case filings per judgeship is the total weighted filings divided by  the number of authorized judgeships. For example, if a bankruptcy court  had 5,100 weighted case filings and three authorized judgeships, the  weighted case filings per judgeship would be 1,700. Because this exceeds  the 1,500 threshold, the Judicial Conference would consider this court for  an additional judgeship. However, it should be noted that the Judicial  Conference’s policy is to consider additional judgeships only for those  courts that request them. Thus, if a court would otherwise be eligible for  an additional judgeship, but did not request one, the Judicial Conference  would not request a judgeship for that court.
section_title: How the Case Weights Were Developed
paragraphs: The Federal Judicial Center (FJC) developed the weights, adopted by the  Judicial Conference in 1991, based on a 1988-1989 time study in which 272  bankruptcy judges (97 percent of all bankruptcy judges in those years)  recorded the time they spent on specific cases for a 10-week period.  Unlike the District Court time study, whose goal was to follow each  sample case from filing to disposition—a “case tracking” study—this study  was a “diary study” in which judges recorded in a time diary the hours  spent on each case in the study and for other judicial work for the 10-week  period. This period of time may or may not have covered the entire life of  the case from filing through disposition. Appendix III includes a more  detailed comparison of case-tracking and diary time studies as methods of  capturing judge time spent on specific cases.
The case weights were developed using a two-step process. First, time  data were collected from 272 judges (97 percent of the total of 280  bankruptcy judges at the time of the study). The judges recorded the time  they spent on a sample of cases and other judgeship work over a 10-week  period. The judges were subdivided into five groups and the recording  time period for each group was staggered over a 1-year period. Second, the  researchers assessed the relative impact on judicial workload of different  types of cases—that is, which types of cases seemed to take more or less  time—and developed individual case weights for specific case categories.  The basic case weight computations involved calculating the average  amount of time spent on cases of each type during each month of their life.  These averages were then summed to determine the total amount of time  for each case type.
Once the case weights had been created, total weighted case filings were  calculated for each bankruptcy court. Then, weighted caseloads were  transformed into initial estimates of required judgeships. These initial  estimates were adjusted to account for factors other than those covered  by the case weight calculation, such as the court’s case management  practices and the time required to travel to divisional offices. After all  adjustments, the study concluded that bankruptcy judges spent about  1,280 hours annually on direct case-related work and an average of 660  hours on matters not directly related to specific cases (e.g., on court and  chambers administration, work-related travel, and other matters related to  the judicial role).
When it approved the case weights in 1991, the Judicial Conference stated  that it expected that in addition to other judicial duties, a bankruptcy court  should have at least 1,500 annual case-related hours per judgeship to  justify additional judgeships. The federal work year is 2,080 hours per  year, based on a 40-hour work week. Assuming that judges spent 1,500  hours annually on cases, there would remain 580 hours for federal  holidays, annual leave, training, and noncase-related administrative tasks.  Of course, the actual time that individual judges spend on case-related and  non case-related work will vary.
section_title: Assessment of Case Weight Methodology
paragraphs: Overall, the methodology used to develop the bankruptcy case weights  appears to be reasonable. The methodology included a valid sampling  strategy, a very high participation rate among bankruptcy judges, and a  reasonable means of adjusting for such factors as missing data. A notable  strength of the methodology was the high participation rate by judges—97  percent of the bankruptcy judges at the time of the study. Thus,  participating judges represented almost the entire universe of bankruptcy  judges that could be included. The sampling period was not limited to a  single time of year, thus minimizing potential bias due to variations in case  filings by time of year. FJC researchers systematically used the reported  time data to develop the case weights and made an effort to address all  known limitations in the data. In computing the case weights,  assumptions, and adjustments needed to be made to account for time data  that were not linked to specific cases, missing data, and other factors.  Both the assumptions and the methods used to make these adjustments  appeared to be reasonable. It is important to note that the case weights  were designed to estimate the impact of case filings on the workload of  bankruptcy judges. Noncase-related time demands, such as time spent on  court administration tasks, are not included in the case weights. The  Judicial Conference focuses its analysis of the need for additional judges  primarily on the demands that result from caseload, not noncase-related  tasks and responsibilities.
Potential limitations of the methodology included the possibility of judges  using different standards and definitions to record their time. Although the  judges had written instructions on how to record their time, judges may  have varied in how they interpreted case-related and noncase-related  hours. To the extent this occurred, it may have resulted in the recording of  noncomparable time data among judges. Because some cases require  longer calendar time to complete than others, not all cases in the sample  were completed at the end of the 10 weeks in which judges recorded their  time. In particular, the study captured only a small portion of the total time  required for very large business bankruptcies. Where the cases were not  completed, it was necessary to estimate the judge time that would have  been required to complete the case. However, the method used to make  these estimates was also reasonable.
section_title: Amending the Case Weights—”Mega” Chapter 11 Cases
paragraphs: The size and time demands of chapter 11 business bankruptcies vary  considerably. The bankruptcy case weights, which the Judicial Conference  approved for use in 1991, included a weight of 11.234 hours for chapter 11  business filings involving $1 million or more and a weight of 4.021 hours  for chapter 11 business filings with assets between $50,000 and $99,999.
In 1996, a new method was used for measuring the workload required for  very large (“mega”) chapter 11 business cases. This measure was also  developed by the FJC and approved by the Judicial Conference’s  Bankruptcy Committee. The mega cases were defined as “those involving  extremely large assets, unusual public interest, a high level of creditor  involvement, complex debt, a significant amount of related litigation, or a  combination of such factors.” The Administrative Office of the U.S. Courts  defines mega chapter 11 cases as a single case or set of jointly  administered or consolidated cases that involve $100 million or more in  assets and 1,000 or more creditors. Mega chapter 11 cases are distinct  from other large chapter 11 cases in that they generally involve a larger  number of associated filings and extend over a longer period of time.
The 1991 case weights did not fully reflect the judge time required for  these very large, complex bankruptcy filings. The weighting scheme was a  particular problem for the Southern District of New York and the District  of Delaware, both of which have a high number of mega cases. At the time  of the 1988-1989 bankruptcy time study, the highest value for chapter 11  cases in the bankruptcy administrative database was $1 million or more.
Subsequently, changes were made to the database, which now includes  several subcategories for cases above $1 million, the highest being $100  million and above. Also, the time study estimated the judge time required  by cases for the first 22 months after the case was filed, a period which  may not have encompassed the entire calendar time required to dispose of  the case. Both of these factors contributed to the inability to create case  weights for the mega chapter 11 cases.
Beginning in 1996, the adjustment of weighted case filings to account for  mega chapter 11 cases was implemented in the two districts where most of  these cases have been filed—first in the Southern District of New York and  later in the District of Delaware. FJC’s research suggested there was no  clear linear relationship between asset size and judge time in mega chapter  11 cases. Instead, FJC selected an adjustment method using data routinely  collected on docketed events in bankruptcy cases, such as docketed  hearings. The method used to adjust the case weights for mega chapter 11  cases consists of a preliminary weighted caseload computation, followed  by a ratio adjustment step. The preliminary weighted caseload is the sum  of the bankruptcy case weights for each case filing associated with the  mega chapter 11 cases. For example, if a mega case consisted of two  consolidated cases, one with assets of between $50,000 and $99,999  (weight: 4.021) and one with assets greater than $1 million (weight:  11.234), the preliminary case weight would be 15.255 (4.021 plus 11.234).  In the Southern District of New York, this preliminary case weight is  adjusted by the ratio of docketed events per weighted case-hour for mega  chapter 11 cases to the docketed events per weighted case- hour for  nonmega chapter 11 cases involving more than $1 million in assets. In the  District of Delaware, where mega chapter 11 cases tended to have a larger  number of consolidated filings, several ranges of the number of associated  filings are used to classify mega chapter 11 cases. For each range, a  separate docketing ratio adjustment is calculated in the same manner as it  is for the District of Southern New York. In both districts, the final step is  to report these calculations over a period of several years and use the  average value across the years as the adjusted weighted caseload for mega  chapter 11 cases. The purpose of this final step is to moderate the effect of  fluctuations in the number of mega chapter 11 cases filed from year to  year.
subsections: section_title: Assessment of Mega Case Weighting Method
paragraphs: The methodology used to adjust the weighted caseload for mega chapter  11 cases, specifically the ratio adjustment step, cannot be thoroughly  assessed because there are no objective time data to use for comparison.  The FJC selected this methodology after extensive research on other  possible methods. The overall strategy of applying a ratio adjustment using  auxiliary information, followed by use of a multiyear average, is a  reasonable approach.
section_title: 2002 and 2008 Research Designs for Updating the Bankruptcy Case Weights
paragraphs: In June 2002, the Judicial Conference Committee on the Administration of  the Bankruptcy System decided to begin a study to create new bankruptcy  case weights. The preliminary design for the study had a two-phase  structure. In the first phase, a diary time study would be conducted, and  the time study data would be used to develop new case weights. In the  second phase, research was planned to assess the possibility of developing  “event profiles” that would allow future updating of the weights without  the necessity of conducting a time study for each update. Future updating  of the weights could include revision of case weight values and/or  developing case weights for new case categories. The data from the time  study could be used to validate the feasibility of the new approach.The  preliminary design for this study appeared to be reasonable. In the first  phase, new weights would be constructed using objective data from the  time study. The second part represented experimental research to  determine if it would be possible to make future revisions to the weights  without the requirement of conducting a time study. If the research  determined this were possible, it would then be possible to update the  case weights more frequently with less cost than required by a time study.
If bankruptcy reform were enacted during the course of the new  bankruptcy time study, FJC officials said they would recommend halting  the time study and allowing some period of time for the implementation of  the new law before restarting the study. This was a prudent plan because  the law had many provisions affecting personal bankruptcy filings and  personal bankruptcy filings represent the vast majority of bankruptcy  filings. The FJC did begin collecting data for new case weights in 2005, but  terminated the effort soon after the Bankruptcy Reform Act was enacted.
It is possible, indeed likely, that the Bankruptcy Reform Act’s many new  provisions have affected the time that bankruptcy judges spend on cases.  For example, there are new objections that can be filed that require  hearings. These include a U.S. Trustee’s objection to the debtor’s  exemption from credit counseling certification. Under the Bankruptcy  Reform Act, debtors who file for bankruptcy are required to complete  credit counseling prior to filing. The extent to which new provisions in the  Bankruptcy Reform Act affect bankruptcy judges workload depends, of  course, on the frequency with which they are invoked and the time it takes  to address them.
Although nonbusiness (personal) bankruptcy filings accounted for more  than 96 percent of total bankruptcy filings both before and after the  implementation of the Bankruptcy Reform Act, the Act initially had a  dramatic effect on bankruptcy filings. Total personal bankruptcy filings in  2004 were 1,563,145 and in calendar years 2005 were almost a half million  higher at 2,039,214. By contrast, in calendar year 2006, the first full  calendar year after the Bankruptcy Reform Act became effective, personal  bankruptcy filings were 597,965—a drop of about 71 percent compared to  2005 filings and about 62 percent compared to 2004 filings. Personal  bankruptcy filings have since grown to 1,153,412 in the 12 month period  ending March 31, 2009. Thus, it was prudent for the FJC to suspend its  2005 time study because it would likely take some time for the filings  under the new law to normalize, and there would inevitably be issues  about the law’s implementation that would need to be addressed.
The FJC has again initiated a new case weight study that includes data  collected over 5 10-week reporting periods from May 2008 through May  2009. This study, like its predecessors, is a time study in which  participating bankruptcy judges record the time they spend on cases and  other judicial activities during their assigned reporting period. Each active  and recalled bankruptcy judge is to participate during one of the 5  reporting periods. The FJC has dropped the second part of the 2002  design, which was to collect assess whether an event-based approach  could be used to more frequently update the case weights. The FJC said  that the experience in the 2005 study indicated that the supplemental  information about judges’ time reports—which was very detailed and  keyed to specific case events—was the most burdensome to provide.  These data elements were not included in the 2008 study in order to  simplify the process, reduce the burden on judges, and contribute to  keeping judges’ participation rate in the 2008 study high, since 125 judges  had already participated in the 2005 study and would be asked again to  participate in the 2008 study. Moreover, the FJC said that the information  from the suspended 2005 study provides the necessary foundation for the  exploratory work on the event-based method, which the FJC still intends  to do  Mr. Chairman, this concludes my prepared statement, I would be pleased  to respond to any questions that you or other members of the  Subcommittee may have.
section_title: Contacts and Acknowledgments
paragraphs: For further information regarding this testimony, please contact William  Jenkins, Jr., at (202) 512-8777. Individuals making key contributions to this  testimony included David Alexander, Leyla Kazaz, and Geoffrey Hamilton.
section_title: Appendix I: Quality Assurance Steps the Judiciary Takes to Ensure the Accuracy Of Case Filing Data for Weighted Filings
paragraphs: All current records related to bankruptcy filings that are reported to the  Administrative Office of the U.S. Courts and used for the bankruptcy court  case weights are generated by the automated case management systems in  the bankruptcy courts. Filings records are generated monthly and  transmitted to AOUSC for inclusion in its national database. On a quarterly  basis, AOUSC summarizes and compiles the records into published tables,  and for given periods, these tables serve as the basis for the weighted  caseload determinations.
In responses to written questions, AOUSC described numerous steps taken  to ensure the accuracy and completeness of the filings data, including the  following:    Built-in, automated quality control edits are done when data are  entered electronically at the court level. The edits are intended to  ensure that obvious errors are not entered into a local court’s database.  Examples of the types of errors screened for are the district office in  which the case was filed, the U.S. Code title and section of the filing,  and the judge code. Most bankruptcy courts have staff responsible for  data quality control.
A second set of automated quality control edits are used by AOUSC  when transferring data from the court level to its national database.  These edits screen for missing or invalid codes that are not screened  for at the court level, such as dates of case events, the type of  proceeding, and the type of case. Records that fail one or more checks  are not added to the national database and are returned electronically  to the originating court for correction and resubmission. Monthly  listings of all records added to the national database are sent  electronically to the involved courts for verification.
Courts’ monthly and quarterly case filings are monitored regularly to  identify and verify significant increases or decreases from the normal  monthly or annual totals.
Tables on case filings are published on the Judiciary’s intranet for  review by the courts.
Detailed and extensive statistical reporting guidance is provided to  courts for reporting bankruptcy statistics. This guidance includes  information on general reporting requirements, data entry procedures,  and data processing and reporting programs.
Periodic training sessions are conducted for bankruptcy court staff on  measures and techniques associated with data quality control  procedures.
In addition to the quality control procedures listed above, AOUSC  indicated that an audit was performed in 1997 by Clifton Gunderson L.L.C.,  a certified public accounting firm, to test the accuracy of the bankruptcy  statistical data maintained by bankruptcy courts and the AOUSC. The firm  compared individual case records in 11 courts nationwide with data in the  national database for cases filed in 1993, 1994, and 1995 for completeness  and accuracy. Excluding problems in one district, the overall match rate of  all statistical data elements captured exceeded 97 percent, and the fields  with most mismatches were not relevant to the bankruptcy weighted  caseload. AOUSC was unaware of any other efforts to verify the accuracy  electronic data to “hard copy” case records for bankruptcy courts. AOUSC  noted that it did not have time to seek detailed information from the  individual bankruptcy courts on this issue within the short time available  to respond to our questions.
section_title: Appendix II: Bankruptcy Case Weights and Confidence Intervals for All Cases Except “Mega” Chapter 11 Business Filings
section_title: Appendix III: Measuring Judicial Workload Using the Collection of Time Study Data
paragraphs: The current Bankruptcy Court and District Court workload measures were  developed using data collected from time studies. The District Court time  study took place between 1987 and 1993, and the Bankruptcy Court time  study took place between 1988 and 1989.
Different procedures were used in these two time studies. The Bankruptcy  Court time study protocol is an example of a “diary” study, where judges  recorded time and activity details for all of their official business over a 10  week period. The District Court time study protocol is an example of a  “case-tracking” study, where a sample of cases were selected, and all  judges who worked on a given sample case recorded the amount of time  they spent on the case. Time studies, in general, have the substantial  benefit of providing quantitative information that can be used to create  objective and defensible measures of judicial workload, along with the  capability to provide estimates of the uncertainty in the measures.
subsections: section_title: Estimating Judge Time in Diary and Case Tracking Studies
paragraphs: At the conclusion of a case-tracking study, total time spent on each sample  case closed during the study period is readily available by summing the  recorded times spent on the case by each judge who worked on the case.  For a given case type, the summed recorded times can be averaged to  obtain an estimate of the average judicial time per case for that case type.
For a diary study, however, it is necessary to make estimates of judicial  workload for all cases that were not both opened and closed during the  data collection period. This estimation step requires information from the  caseload database, and thus the accuracy of estimates depends in part on  the accuracy of the caseload data. Two kinds of information are required  from the caseload database: case type and length of time the case has been  open.
With the diary approach, the total judicial time that is required for lengthy  case types is estimated by combining “snap shots” of the time required by  such cases of different ages. Thus, in theory, reducing accurate weights for  lengthy case types is not problematic. In practice, however, difficulties  may be encountered. For example, in the 1988-1989 bankruptcy time  study, the asset and liability information for cases older than 22 months  was inadequate and appropriate adjustments had to be made. In addition,  difficulties may arise if only a small number of cases of the lengthy type  are in the system. This is an issue FJC said it is considering as it finalizes  how to assess the judicial work associated with mega cases in the  upcoming bankruptcy case-weighting study.
section_title: Comparing Case- Tracking Studies and Diary Studies
paragraphs: Each study type has advantages and disadvantages. The following outlines  the similarities and differences in terms of burden, timeliness of data  collection, post-data collection steps, accuracy, and comprehensiveness.
subsections: section_title: Burden on Participants
paragraphs: Each study type places burden on judicial personnel during data  collection. It is not clear that one study type is less burdensome than the  other. The diary study procedure requires more concentrated effort, but  data are collected for a shorter period of time.
section_title: Timeliness of Data Collection
paragraphs: Data collection for a diary study can be completed more quickly than for a  case-tracking study.
Post Data Collection Steps  More effort is needed to convert diary study data to judicial workload  estimates than case tracking study data. Also, the accuracy of estimates  from diary study data depends in part on the accuracy and objectivity of  the information in the caseload database.
section_title: Data Accuracy
paragraphs: It is not clear that one study type collects more accurate data than the  other study type. Some of the Bankruptcy Court case-related time study  data could not be linked to a specific case type due to misreporting errors  and/or errors in the caseload database. Some error of this type likely is  unavoidable because of the requirement to record all time rather than  record time for specific cases only. However, it is plausible that a diary  study collects higher quality data, on average, because all official time is to  be recorded during the study period; judicial personnel become  accustomed to recording their time. In contrast, the data quality for a case- tracking study could decline over the study’s length; for example, after a  substantial proportion of the sample cases are closed, judicial personnel  could become less accustomed to recording time on the remaining open  cases.
section_title: Comprehensiveness and Efficiency
paragraphs: In theory, a case-tracking study collects more comprehensive information  about judicial effort on a given case than a diary study, because data for a  sampled case almost always are collected over the duration of the case.  (Data collection may be terminated for a few cases that remain open, or  are reopened, many years after initial filing.) For case types that  simultaneously stay open for a long period and require a substantial  amount of judicial effort, it is possible that a diary study would not be able  to produce suitable estimates of judicial workload due to a lack of data.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.