A Value-Added Ranking of Law Schools

Rankings have become an important, if not essential, element of the law school environment since U.S. News & World Report first began publishing law school rankings in 1987. While several alternative rankings have begun to gain traction in recent years, for better or for worse, the U.S. News ranking has become the gold standard, as well as a proxy for determining a law school’s quality and value.In the ideal, good ranking systems help consumers of information determine quality and value. For example, prospective law students — who are principally concerned with post-graduation outcomes — need a mechanism to assess marginal costs and benefits associated with each institution of legal education. Prospective employers also rely upon educational rankings in directing their recruitment efforts to students attending top-quality institutions. Thus, a time-invariant relationship between current and prior year scores, either peer review scores or overall rankings, would be highly problematic, not simply for the U.S. News ranking system, but for all of legal education. This is because it would evince a failure of the field to provide potential and actual stakeholders with a reliable measure of institutional quality and value. As demonstrated by several recent studies, including this study, there is strong evidence to suggest at least serial correlation if not time-invariance in the U.S. News peer rankings of law schools. The extent to which this phenomenon exists will be further explored in this study, specifically:Q1: To what extent are reputational peer reviews of American law schools over time associated with changes in organizational performance and institutional differences on the basis of student characteristics, institutional selectivity, bar passage rates, and expected cost of attendance?Q2: To what extent are reputational peer reviews of American law schools since 2008 associated with prior-year peer review scores?Importantly, this study employs the use of value-added modeling and measures from the education policy literature, perhaps for the first time ever, to the context of legal education to examine the following research question:Q3: Controlling for differences between and within institutions, to what extent do indicators of a law school’s performance exceed or fall short of its predicted performance?In service of answering these research questions, this study utilizes the ABA Standard 509 Required Disclosures Reports from each of the 203 ABA-accredited law schools from 2008 and 2011 to 2014. Over this time frame, the data set surveys these nationally-accredited law schools and records their institutional responses to over 400 variables relating to key metrics of student inputs (admissions selectivity, and incoming student GPA and performance on the LSAT), student outcomes (degree completion rates, graduating student bar passage rates, and employment within 9 months), and value (average financial aid awards, tuition, and ancillary costs). The corresponding annual U.S. News rankings, including peer review scores, for each institution were then merged into the data set for analysis.This study extends the theory beyond its previous considerations — looking at both inputs and outputs to legal education simultaneously in order to reveal discrepancies in the peer assessment ranking — in light of the post-Recession context of American legal education. Results from this study illustrate the degree to which the current year peer review scores can to an overwhelming degree be explained by prior year peer review scores, and by comparison, rely very little on the specific attributes of a law school that change from year to year and should be considered in a law school’s reputational score. Given the persistent nature of these trends, it may be reasonable to deduce a continuing information bias within peer reviewers and even the U.S. News methodology, impacting the reliability of measures and perceptions of quality and value of legal education. As such, this study proffers a new ranking based on value-added modeling measures. This ranking system is a truly unique contribution to the rankings literature in that it assigns a value only to that for which the law school can be said to add value (i.e. bar passage and employment prospects), and not those indicators specific to the student for which a law school was not responsible or cherry-picked to improve standing in the U.S. News rankings (i.e. LSAT and GPA scores). It is hoped that this study, and its first-ever law school value-added rankings, will spur the national discourse about the value of legal education, and the requirement of good data to assess the present conditions in legal education.