Source: https://e-discoveryteam.com/2012/03/11/perspective-on-legal-search-and-document-review/
Timestamp: 2019-04-18 20:57:52+00:00

Document:
For millennium writings were on paper. For centuries the legal profession depended upon writings, referred to in the law as documents, as the key evidence to resolve disputes in a fair and just manner. Losey, Mathematical Formula for Justice Proves the Importance of ESI in Civil Litigation, Chapter 4 of Electronic Discovery (West 2010). Paper documents were well-known and mastered by every lawyer and judge who swore an oath to uphold the law.
This all changed in a historical blink of the eye. In just one generation documents have dematerialized and transformed into a dizzying array of digital media.
Many see this as a much more profound cultural revolution than that precipitated by Guttenberg, which took centuries to play out, not decades. George L. Paul and Jason R. Baron, Information Inflation: Can the Legal System Adapt? 13 RICH. J.L. & TECH. 10 (2007). Paul and Baron’s well known article explains how writing co-evolved with civilization over the past 50 centuries or longer, with a slow but steady increase in information as our writing technologies slowly improved. They point out that this all changed about 25 years ago when Mankind invented a totally different form of electronic writing, free from physical confines, that triggered a Big Bang like explosion of a new Universe of virtually unlimited information. Paul and Baron predicted the legal profession will necessarily have to significantly change and adapt new strategies of practice to cope with this information revolution.
Documents originally created on paper still exist in our society, but are rare. Zubulake v. UBS Warburg LLC, 217 F.R.D. 309, (S.D.N.Y. 2003) (FN 5 cites Wendy R. Liebowitz, Digital Discovery Starts to Work, Nat’l L.J., Nov. 4, 2002, at 4 reporting that in 1999, ninety-three percent of all information generated was in digital form).
The wide variety of computer systems currently in use, and the rapidity of technological change, counsel against a limiting or precise definition of electronically stored information. Rule 34(a)(1) is expansive and includes any type of information that is stored electronically.
Electronically stored information is potentially discoverable under Fed. R. Civ. P. 34 or its state equivalents. Organizations must properly preserve electronically stored information that can reasonably be anticipated to be relevant to litigation.
The legal profession has been severely stressed by the rapid, ever-accelerating advances in technology. The changes in writing and resulting information explosion have been the key stressors. ESI is not only changing and evolving new into forms every year, but is now multiplying at an exponential rate that is almost beyond comprehension. See Eg., Rowe Entm’t, Inc. v. William Morris Agency, Inc., 205 F.R.D. 421, 429 (S.D.N.Y. 2002) (explaining that electronic data is so voluminous because, unlike paper documents, “the costs of storage are virtually nil. Information is retained not because it is expected to be used, but because there is no compelling reason to discard it”), aff’d, 2002 WL 975713 (S.D.N.Y. May 9, 2002); Data, Data Everywhere (The Economist, March 2010); Baron and Losey E-Discovery: Did You Know video.
Most lawyers are unfamiliar with ESI and the complex systems that store it. They prefer the familiar paper and alphabetical filing cabinets. They are paper lawyers living in a digital world.
Although electronic discovery is becoming extraordinarily important in civil litigation, it is proving to be enormously expensive and burdensome.
[The articles in this Supplement] suggest that if participants in the legal system act cooperatively in the fact-finding process, more cases will be able to be resolved on their merits more efficiently, and this will help ensure that the courts are not open only to the wealthy. I believe this to be a laudable goal, and hope that readers of this Journal will consider the articles carefully in connection with their efforts to try cases.
Even though many scholars, jurists, and practitioners recognize the problems created by the inability of lawyers to keep pace with technology, most law schools still only train students in paper evidence and discovery. Students graduate unprepared to handle the ESI where the truth of past events is now stored. William Hamilton, The E-Discovery Crisis: An Immediate Challenge to our Nation’s Law Schools, Chapter 39 of Electronic Discovery (West 2010); Shannon Capone Kirk, Kristin G. Ali, Teach Your Children Well: A Case for Teaching E-Discovery in Law Schools, Chapter 38 of Electronic Discovery (West 2010); Shira Scheindlin, Ralph Losey, E-Discovery and Education, Chapter 33 of Electronic Discovery (West 2010); Ralph Losey, Plato’s Cave: Why most lawyers love paper and hate e-discovery and what this means to the future of legal education, Chapter 32 of Electronic Discovery (West 2010).
The law remains as dependent as ever upon documents to prove the truth, but the vast majority of lawyers are untrained and unprepared to handle the electronic documents upon which the world is now built. E-Discovery Competence is a Fundamental Ethical Challenge Now Faced by the Legal Profession, Chapter 34 of my book Electronic Discovery (West 2010). In fact, most lawyers, even those who specialize in litigation, dislike e-discovery and try their best to avoid it. See my blogs: Tell Me Why?, and Spilling the Beans on a Dirty Little Secret of Most Trial Lawyers. Lawyers are trained and prepared instead to handle paper documents following systems developed in the 20th Century.
These old paper-based legal search and review methods are one dimensional and linear in nature. They typically follow a sequential Bates stamp organizational model created in the 1890s. The simple paper evidence discovery processes worked pretty well for decades before computers. It should be noted, however, that even before technology moved away from paper typing machines to computers in the 1980s, the discovery processes were already severely taxed by the growing volumes of paper documents generated from the 1960s forward. The increase in paper volume was caused by another technological innovation, the photocopy machine and by ever more complex transactions. Still, the legal profession coped somehow for the rest of the Twentieth Century. Lawyers added more numbers to the Bate stamps and used larger teams of lawyers and paralegals to manage the additional papers. They were still on familiar ground.
The linear systems developed in the 19th and 20th Centuries for the discovery and production of documents continue to be used today by most attorneys for both ESI and paper discovery. See Eg. the D’Onofrio saga of four opinions by Judge Facciola which describes the processes used in this case and his many orders resolving disputes, including an order requiring production of a sample of the 9,413 documents listed on the privilege log. D’onofrio v. SFX Sports Group, Inc. 247 F.R.D. 43 (D.D.C. 2008); D’onofrio v. SFX Sports Group, Inc. 254 F.R.D. 129 (D.D.C. 2008). The cases are described in Chapter 27 of my book Electronic Discovery (West 2010). This use of old paper based systems with today’s ESI is a big mistake and the primary reason e-discovery is so expensive.
The old linear review methods involved serial culling of documents down to a final production set. The process generally required multiple reviews of the same document for different purposes. It was inefficient. It was expensive. Moreover, the quality control of human eyes on paper did not work with high volumes of documents. This is shown by the latest scientific experiments where the agreement rate among professional legal reviewers was found to be just less than 50%. Cormack, Grossman, Hedin, Oard, Overview of TREC 2010 Legal Track (February 21, 2012).
This tradition of multiple manual reviews, with only limited computer assistance, typically on a linear based review platform, still continues today. But it is too inexpensive and inefficient with high volumes of ESI. This will only get worse as the amount of information continues to grow exponentially. Jason Baron, the Head of Litigation for the National Archives and Records Administration, which is in charge of all federal records, including white house email, explains this as a problem of scale. He projects the number of White House emails will soon exceed One Billion. He estimates it would cost over $2 Billion to search that many emails. That assumes a team of one hundred full-time lawyers working over fifty-four years and a very low billing rate of $100 per hour. Information Inflation: Can the Legal System Adapt? Supra at pgs. 13-15. It also assumes computer-assisted review tools, but following the old paper-based linear review models. Also see: Baron, Jason, E-discovery and the Problem of Asymmetric Knowledge (Presentation at the Mercer Law School Ethics in the Digital Age Symposium, Nov. 2008).
Plaintiff’s counsel conceded at the hearing that the task of searching Plaintiff’s records for relevant emails in response to Defendants’ discovery request was entrusted to a junior associate. It is apparent that the associate worked with little or no direction or supervision. The search terms used by the associate were inadequate — they did not even include the term “phone” — and, as a result, she failed to locate or perceive the significance of the emails about which Defendants now complain.
Also see Danis v. USN Communications, Inc., 2000 WL 1694325 (N.D. Ill. 2000) ($10,000 fine imposed against CEO personally when the young general counsel he hired to supervise ESI preservation was grossly negligent); Mt. Hawley Ins. Co. v. Felman Production, Inc., 2010 WL 1990555 (S.D. W. Va. May 18, 2010) (serious mistake made resulting in waiver of privilege in spite of sophisticated counsel with very elaborate processes and safeguards).
Some are looking for an answer to these expense issues by keeping the old processes, but outsourcing the work of manual review to less expensive contract lawyers. They are called “contract lawyers” because the law firm that represents the client does not employ them. Instead, they work for the firm or some other company under a contract to do only review work. These contract lawyers may be located in India or other countries, or may be down the street from your office, or down the hall.
Even if you accept the abilities of contract lawyers to adequately perform the task of the first level relevance review (which I do, with qualifications), this is still just a stopgap measure. The volumes of ESI are increasing exponentially. Outsourcing may help with expenses to a point, but is still futile as a stand-alone long-term strategy, still just treading water in the midst of a flood. Indeed, when this outsourcing strategy was employed by the Department of Justice to try to reduce the costs of a privilege review, it still cost $9.09 per file to do a review at a total cost of $6,000.000. In re Fannie Mae Securities Litigation, 552 F.3d 814, (D.C. App. Jan. 6, 2009). The government reviewed 660,000 files at a cost of $6,000,000 using contract lawyers to respond to a non-party subpoena. The Order denying the government’s motion for cost-shifting was upheld by the appellate court.
The answer does not lie in modifying the system somewhat to employ cheap labor to do manual review. Outsourcing may still be part of the answer, and is likely to remain as a viable long-term strategy. But outsourcing only works when combined with the new systems and technologies like predictive coding, and only when properly supervised and performed by highly skilled professionals. The answer is not outsourcing. The answer is a whole new system for e-discovery.
Our current linear, confrontative, one-dimensional, largely manual, costly, Bates stamp approach to discovery must be replaced with a cooperative, iterative, largely automated, predictive coding based, proportionality cost-controlled, hash value approach.
For more on hash values and e-discovery, see: Losey, R., HASH: The New Bates Stamp, 12 Journal of Technology Law & Policy 1 (June 2007). For more on cost control see my blog Bottom Line Driven Proportional Review. For more on the new approaches in general and predictive coding in particular, see eg. Chris Dale’s Judge Peck’s Predictive Coding Opinion – reporting the reaction and the case and articles cited therein.
This entry was posted on Sunday, March 11th, 2012 at 4:51 pm and is filed under Evidence, Lawyers Duties, Metadata, New Rules, Review, Search, Technology.	You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.
The cost of $6,000,000 to review 660,000 documents is extraordinary. Assuming the review attorneys doing the work were billed out at $150 an hour that is 40,000 hours of work for a review rate of 16.5 documents per hour. While I do not believe documents per hour is the best metric for determining reviewer efficency, 16.5 documents per hour is incredibly slow.
re: Jeffrey Schilling’s comment above. The statistics quoted above are for DoJ “files”, not documents. A file could theoretically consist of a single document but is more likely to consist of dozens or more. Once you expand from files to documents, the rate of review is more reasonable.
The decision not to allow cost-shifting is less reasonable, in my opinion. Cost-shifting, especially from third-party respondents, is a necessary control to reduce the moral hazard of frivilous or unbalanced demands for documents.

References: v. 
 v. 
 v. 
 v. 
 v. 
 v.