metadata: authors: 0: author: Ramos Maqueda,Manuel
1: author: Chen,Daniel Li
count: World
docna: 0: docna: The Role of Justice in Development : The
            Data Revolution
docty: Policy Research Working Paper
subtopic: Judicial System Reform,Law and Justice Institutions,Justice for the Poor,Crime and Society,Social Policy,Regulatory Regimes,Legal Reform,Legal Products,Legislation,Common Property Resource Development
teratopic: Social Development,Law and Development,Governance,Rural Development,Public Sector Development
abstracts: cdata!: This paper summarizes the empirical
            evidence on the role of justice in economic development,
            conflict, and trust in institutions. It finds that justice
            institutions play a significant role in economic
            development, particularly through their impact on credit
            markets and firm growth, the protection of vulnerable
            populations, their capacity to deter violence, and their
            influence over people’s trust in formal institutions. The
            paper then considers the promise of administrative data,
            machine learning, and randomized controlled trials to
            enhance the efficiency, access, and quality of justice. The
            paper concludes by discussing new avenues for research and
            the potential for data to improve the functioning of justice
            systems in the age of COVID-19.
repnme: repnme: The Role of Justice in Development : The
            Data Revolution
display_title: The Role of Justice in Development :
            The Data Revolution
listing_relative_url: /research/2021/06/33230967/role-justice-development-data-revolution
topicv3: Justice m1326106 840,Detention, Prison and Corrections Reform
            m1326383 2495,Personal Property Rights m1326571 1907,Access to Justice m1327477 942,Crime m1439217 3041
publishtoextweb_dt: 2021-06-29T00:00:00Z
dois: 10.1596/1813-9450-9720
historic_topic: Social Development,Law and Development,Governance,Rural Development,Public Sector Development
origu: DIME Governance & Inst Building (DIME3)
no_of_pages: 47
author_keys: 0: author_key: 1656196
1: author_key: 167904
model_extractions: Policy Research Working Paper 9720 The Role of Justice in Development The Data Revolution _Manuel Ramos Maqueda Daniel L. Chen_ Development Economics Development Impact Evaluation Group June 2021
Policy Research Working Paper 9720

# **Abstract**

This paper summarizes the empirical evidence on the role of justice in economic development, conflict, and trust in institutions. It finds that justice institutions play a significant role in economic development, particularly through their impact on credit markets and firm growth, the protection of vulnerable populations, their capacity to deter violence, and their influence over people’s trust in formal institutions.

The paper then considers the promise of administrative data, machine learning, and randomized controlled trials to enhance the efficiency, access, and quality of justice. The paper concludes by discussing new avenues for research and the potential for data to improve the functioning of justice systems in the age of COVID-19.

This paper is a product of the Development Impact Evaluation Group, Development Economics. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at mramosmaqueda@worldbank.org and dchen9@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **The Role of Justice in Development: The Data Revolution**

_Manuel Ramos-Maqueda and Daniel L. Chen_<sup>_⇤_</sup>

**Keywords:** Justice, Governance, Legal Institutions, Data, Development, Covid-19 **JEL Classification:** D02, K0, O1, O38, H7

> _⇤_ Manuel Ramos-Maqueda, mramosmaqueda@worldbank.org (corresponding author); Daniel L. Chen, dchen9@worldbank.org. We are grateful for comments and contributions from Bilal Siddiqi, Manaswini Rao and Justin Haccius. We are also grateful to Nayantara Vohra and Jane Tien for excellent research assistance. The findings, interpretations, and conclusions in this paper are entirely those of the authors. They do not necessarily represent the views of the World Bank Group, its Executive Directors, or the countries they represent.
The Presence of Keywords in Academic Articles Evolution of Institutions v. Other Topics in NBER's Abstracts <!-- Start of picture text --> !<br>!<br>i}<br>n !<br>ae] !<br>$ 9.0%-<br>a1<br>xI<br>Dl<br>51<br>5 Keywords<br>is}<br>« 7.0% | -® education<br>3 -® health<br>G \ © institution<br>Ss !<br>oO I<br>Dl<br>£ 1<br>5 5.0% 1<br>° !<br>oO I<br>oO !<br>!<br>!<br>!<br>| 1<br>1995 2000 2005 2010 2015 2020<br>Year<br><!-- End of picture text -->

Source: data scraped by the authors from NBER's website. It contains the abstracts from all of the articles published up to November 2019
1 The Role of Justice in Development: Empirical Evidence 3 The institutionalist definition of the rule of law focuses on property rights and the efficient administration of justice. This mainly includes laws and regulations, the police, and the judiciary. Each of these institutions is under the control of different branches of government: laws are developed by the legislative branch, the police are controlled by the executive branch, and the judiciary is, in principle, independent. In this paper, we specifically focus on the judiciary. The judicial branch may include both formal and informal institutions that aim to resolve conflicts among citizens, or between citizens, corporations and/or the state.

The lack of high-frequency data is no longer an issue for many judiciaries. Recent years have seen an increase in the availability of data in judicial systems around the world. Many countries have developed electronic case management and e-filing systems. This has led to the creation of massive databases that track every characteristic of each case. In fact, this is by definition a characteristic of the fair administration of justice: everything that happens in a case needs to be tracked to prove the evidence behind fact-based decisions. Even though this data is readily available to policy makers, it has rarely been exploited to evaluate policies or improve the functioning of the judiciary. In other words, courts have become data-rich but information-poor.

In the era of big data, the data revolution opens an immense range of opportunities. These opportunities should be used by policy makers and researchers to understand how to improve the access, quality and efficiency of justice. We examine specific ways in which data has been used, or could be used, by government officials and researchers to make more progress in responding to the “small, concrete questions” that currently dominate the field of development economics. By providing responses to these questions, we will be able to rigorously evaluate the impact of rule of law institutions on the prosperity of developing nations in future years.

In this paper, we divide our review into three parts. Part 1 studies the empirical evidence evaluating the impact of justice on development outcomes. Our review focuses on the impact on poverty reduction and firm growth, the prevention of conflict and violence, as well as on the prevalence of corruption and citizens’ trust in institutions. Part 2 discusses the importance of the data revolution in justice. We focus on the potential for judicial administrative and text data to improve the functioning of judicial institutions throughout the world, particularly in developing countries. We argue that the increase in data availability in judiciaries globally, especially since many have been compelled to transition to data-driven systems in light of the COVID-19 pandemic, has the potential to bring the rule of law back to center stage in the development agenda. Part 3 explores new avenues for future research as well as directions for policy makers to leverage the ongoing expansion of data and e-justice solutions in order to address some of today’s most pressing needs.

# **1 The Role of Justice in Development: Empirical Evidence**

In this section we review the empirical evidence that evaluates to what extent justice matters for development outcomes. Specifically, we focus on three main aspects of development: economic growth, conflict and violence, and trust in institutions.

Overall, empirical evidence shows that the judiciary matters–not only for its intrinsic value, but also for its impact on development outcomes. A more efficient judiciary promotes economic growth through better enforcement of contracts and more secure property rights, while also leading to healthier business environments. The judicial system can incentivize entrepreneurship, have a positive effect on firms’ outcomes and growth, and even affect the output of other industries that heavily rely on contracting. A more effective judiciary may also contribute to reducing violence, by creating a deterrent effect on criminal acts and increasing citizens’ trust in the quality of institutions. Formal institutions can protect vulnerable populations in contexts where customary laws are biased against them, as well as reduce violent resolution of disputes through alternative dispute resolution
1 The Role of Justice in Development: Empirical Evidence 4 (ADR) in contexts with a weak rule of law. Demand-side interventions such as legal aid clinics can also protect vulnerable populations, reducing domestic violence and child abuse. Even though the association between a stronger judiciary and lower corruption may sound evident, there has yet to be experimental or quasi-experimental evidence documenting this causal relationship.

# **1.A How Does Justice Impact Poverty Reduction and Economic Growth?**

Underlying the analysis of welfare effects of any law, policy or large-scale macroeconomic shocks is the application of justice theories such as utilitarianism. Therefore, the design and enforcement of laws and policies have welfare ramifications not just in aggregate but also in terms of inclusiveness. In the subsections below, we present and summarize the findings of empirical literature that sheds light on both aggregate as well as distributive welfare effects resulting from justice institutions.

First, we summarize the impact of justice through contract enforcement on individual and business outcomes. Effective contract enforcement forms the basis of well-functioning markets–credit, insurance and other financial markets, in particular–by reducing moral hazard. In the absence of well-functioning formal enforcement institutions, local and informal (party specific) enforcement can lead to inequality, inequity and increased uncertainty. Second, we focus on literature investigating effects on credit markets, through enabling loan recovery, and on firms. Firms are affected not only through contract enforcement, which leads to increased relational (rather than competitive) contracting, differential access to credit based on firm size or value, but also through access to resources, trade, and provision of security to ensure efficient allocation of resources. Third, we review the evidence regarding the impact of justice on entrepreneurship and the overall business environment. Finally, we review the impacts of legal aid on vulnerable populations. According to the literature, legal aid may be an effective avenue to improve the productivity and well-being of citizens in developing as well as developed countries.

## **The Impact of E↵ective Contract Enforcement on Firm Growth**

At the beginning of the century, the literature established a positive relationship between institutional quality and economic development (Acemoglu, Johnson, and Robinson 2001; Pande and Udry 2005; Rodrik 2000; Rajan and Zingales 1996). This literature distinguished two types of institutions: property rights institutions and contracting institutions. Contracting institutions refer to the legal framework that supports private contracts and reduces transaction costs, whereas property rights institutions are those that constrain government and elite expropriation. Acemoglu and Johnson (2005) distinguish between the effects of property rights institutions and contracting institutions on growth. Relying on cross-country data, these authors find that, while strong institutional protection of property rights has a positive effect on economic growth, the effect of contracting institutions is not robust. They suggest that contracting institutions only matter for the form of financial intermediation and not for long-term growth, for which property rights institutions matter more. The latter mitigates the risk of expropriation, and therefore is more important for long-term growth, whereas the contracting risks are better mitigated by the parties themselves (through altering or renegotiating contractual terms) in the presence of weak institutions. In contrast, Nunn (2007) establishes that contract enforcement—measured as judicial quality—is essential for investment, explaining more of the pattern of trade than physical capital and skilled labor combined.

In recent years, a nascent literature using more robust empirical methods and better data sets has found a large impact of contracting institutions on firm growth (Amirapu, 2017; P. Chakraborty, 2016; Ahsan, 2013; Chemin, 2012). In reviewing the literature on contract enforcement, we sum-
1 The Role of Justice in Development: Empirical Evidence 5 marize the effects of the speed of judicial institutions, which lends strength to the enforcement of contracts in the event of violation, on various business and market outcomes. Specifically, an environment of effective contract enforcement promotes competitive credit markets, increases firm productivity by lowering contracting costs, and supplements other market oriented reforms (Ahsan, 2013; Sequeira, 2016).

There are four main mechanisms highlighted in the literature that show a causal impact between contracting institutions and firm outcomes. First, weak enforcement might hinder firm-to-firm trade. There might be weaker incentives to cooperate in a contractual agreement if courts are slower and less reliable at enforcing contracts (Chemin, 2012). Second, weak enforcement might affect firms’ incentives to invest and distort production decisions. Crawford, Klein, and Alchian (1978) demonstrate that slow courts will reduce investment if there is room for post-contractual opportunistic behavior by a firm’s partner once investment costs are sunk. Weak enforcement might also distort the input materials used by firms, as firms rely solely on trusted sources rather than on the most efficient sources, avoid purchasing the inputs altogether, or switch to a different production process (T. Chakraborty et al., 2018). Third, slow enforcement increases the opportunistic behavior of borrowers. Creditors might respond to this strategic behavior by reducing the availability of credit (Aberra and Chemin, 2018; Jappelli, Pagano, and Bianco, 2005). Fourth, better judicial enforcement might increase the probability of providing a loan as well as the recovery of loans.

The magnitude and direction of these causal links are established in the recent empirical literature, with a particular focus on India. Chemin (2012) measures the effects of faster courts on firms’ contracting behavior and performance in India. The author exploits spatial variation in the implementation of a legal reform amending the code of civil procedure, which sped up trials in courts. The findings suggest that the reform led to fewer breaches of contract, encouraged investment, and facilitated access to finance. This is consistent with the first and second mechanisms according to Aboal, Noya, and Rius (2014) provide a systematic review of the causal link between contract enforcement and investment. They conclude that the empirical evidence establishing this link is weak, based on reviewing 19 empirical studies published between 1990 and 2010. Given the period of review, many of the studies take a broad, macro view of the relationship, rather than deep diving into the micro-foundations. Further, some of the studies reviewed conflate protection of property rights and general rule of law with contract enforcement, leading to the general conclusion of positive association between “contracting” institutions and capital accumulation. There has been an increase in empirical causal studies since 2010, which point to a more established connection between better judiciaries and better firm outcomes.

firm productivity. Ahsan (2013) studies the complementarity between the effects of trade liberalization and judicial efficiency on firm productivity (TFP). The author uses spatial variation in judicial efficiency (at lower level courts) as a key impediment or enabler to help pass-through of input tariff reduction on large firm productivity. His findings suggest that a relation-specific (as opposed to generic) input supply chain leads to the holdup problem, where the firm under-invests in the production of such inputs lest the buyer backs out. He finds that for a 10 percentage point drop in input tariffs, firms located in the 75th percentile of judicial efficiency gain an additional 3.6 percentage points in productivity when compared to those located in median judicial efficiency areas.

Similarly, Amirapu (2017) examines the effect of slow courts on the performance of in India. Using variation across industries in their reliance on contracts, along with variation across Indian states in the average speed of courts, the author finds a strong positive effect of court efficiency on firm growth. Transactions involving relationship-specific investments are more exposed to post-contractual opportunism and hence have greater need for efficient contract enforcement. The
1 The Role of Justice in Development: Empirical Evidence 6 paper finds that the interaction between state level court efficiency and industry level relationshipspecificity is highly predictive of future growth in India’s formal manufacturing sector.

Using microdata on Indian manufacturing plants, the authors show that in states with weaker enforcement–as measured by judicial lags–production and sourcing decisions appear systematically distorted. Among plants in industries that tend to rely more heavily on inputs that require customization, those in states with more congested courts shift their expenditures away from intermediate inputs, whereas they find the opposite in industries that tend to rely on standardized inputs. The authors find that weak enforcement exacerbates a holdup problem that arises when using inputs that require customization, distorting both the intensive and extensive margins of input use. Poor contract enforcement distorts production in two ways. There is a direct impact: if a producer uses a supplier for which there is a severe contracting friction, use of that input will be distorted, which will directly lower productivity. There is also an indirect impact: a severe contracting problem might lead a producer to switch to a more costly supplier or less efficient technology to avoid the friction. On average across states, the boost to productivity is roughly 5%, and the potential gains of court improvements for the states with the most congested courts are roughly 10%. The authors conclude that economic benefits to improving courts may be large. Preliminary results suggest that, for each year reduction in the average age of pending cases, the state’s aggregate productivity would increase by about 3%.

# **Justice E↵ects on Credit Markets**

There is a general consensus in the literature on the positive effects of strong justice institutions on credit markets. Both the efficiency and quality of courts have been positively associated with better credit availability and the development of credit markets. The consensus also extends to the mechanism behind this effect: prevention of moral hazard through proper enforcement of the credit contracts and increasing competition between lenders. However, caution must be exercised in examining how increasing competition among lenders ultimately affects borrowers’ welfare, which would depend on the presence of well-developed insurance markets. The papers that address this issue (Horioka and Sekita, 2011; T. J. Besley, Burchardi, and Ghatak, 2012; Kranton and Swamy, 1999) present cautionary evidence regarding the effects of competition between lenders on credit outcomes.

Cristini, Moya, and Powell (2001) analyze how variations in the effectiveness of the legal system across different provinces in Argentina have affected the development of credit markets. They find strong results that provinces with poor legal enforcement have less credit available to borrowers and banks’ non-performing loans are higher. The authors conclude that improvements in Argentina’s legal system would result in a significant increase in the availability of credit. Jappelli, Pagano, and Bianco (2005) also evaluate this relationship by using panel data from Italian provinces. The authors show that provinces with longer trial durations or larger backlogs have less developed credit markets and lower credit availability.

Also in Italy, Schiantarelli, Stacchini, and Strahan (2016) use detailed firm level panel data to show that firms are more likely to default against banks when legal enforcement is weak. A similar conclusion is reached by Ponticelli and Alencar (2016), who measure the effects of proper enforcement of Brazilian bankruptcy reform on firm performance using congestion in courts as the enforcement variable. However, since congestion can be endogenous, the authors use "extra jurisdiction" as the instrument. “Extra jurisdiction” is a measure of the number of neighboring municipalities that do not qualify to become an independent judicial district, increasing the congestion of existing courts. The authors find that municipalities that are one standard deviation below the mean with
1 The Role of Justice in Development: Empirical Evidence 7 respect to potential extra jurisdiction have 28.3% less congested courts and a 5% larger increase in secured loans per firm. In other words, firms with lower potential extra jurisdiction experienced higher increase in secured loans to manufacturing firms, and higher increase in firm investment and output after the introduction of the reform. Given these results, the authors make the case that an efficient judiciary is a necessary precondition for firms to benefit from financial reforms.

In India, Chemin (2009) examines the effects of judicial trial duration on credit markets, agricultural development, and manufacturing performance, using measures of procedural complexity of judicial institutions as instruments. The author finds that slower judiciaries reduce access to credit markets in the agriculture sector, leading to depressed agricultural outputs. In addition, contractintensive sectors of the economy, such as registered manufacturing, are negatively affected by weak judiciaries. Thus, these papers suggest that improvements in legal enforcement of contracts would result in a significant increase in the availability of credit and, as a result, spur economic outcomes.

Judicial efficiency also affects interest rate spread across countries, as shown by Laeven and Majnoni (2005). The paper uses a measure of the ex-ante cost of bank credit to study the relationship between judicial efficiency and the cost of financial intermediation. The authors find that judicial efficiency is an important determinant of interest rate spreads across countries. They suggest that improvements in judicial efficiency and judicial enforcement of debt contracts are critical to lowering ness to lend. Shvets (2013) studies whether the creditor’s willingness to lend to a firm is affected by the quality of courts that protect the creditor’s rights. The author analyzes loans to 11,000 Russian firms between 1996 and 2002. Court quality is measured through the share of the court’s decisions appealed. Using panel data regressions and matching firms and courts, the authors find that creditors make more loans when their rights are protected by courts with lower appeal rates.

Visaria (2009) evaluates a mechanism to improve the functioning of credit markets, India’s Debt Recovery Tribunal (DRT). The author uses staggered roll out in the establishment of the Debt Recovery Tribunal–an alternate contract enforcement system to the existing judicial system of courts–across India and a minimum claim amount by the lender to study the effect on debt recovery, delinquency and, finally, the cost of credit for Indian firms using difference-in-differences. The author finds that DRT increases the probability of timely payment of loan installment by 28 percentage points.

Nonetheless, as previously mentioned, the credit market consequences of justice may be heterogeneous. For instance, Kranton and Swamy (1999) study the impact of the introduction of civil courts in colonial India on agricultural credit markets. The introduction of courts increased competition among lenders. However, it lowered farmers’ welfare by reducing the lenders’ ability to underwrite farmers’ investments during bad shocks. Horioka and Sekita (2011) use a panel survey of consumers in conjunction with judicial data by court district in Japan to estimate that better judicial enforcement increases the probability of being rationed in some cases and decreases loan size (contrary to expectation) but increases the probability of bankruptcy (as expected). Thus, the authors argue that better judicial enforcement facilitates the recovery of loans but may sometimes be socially harmful.

In addition, Besley, Burchardi, and Ghatak (2012) examine the effects of improving property rights to facilitate the use of fixed assets as collateral, as has been posited in much of the finance and credit markets literature. Using a structural model on Sri Lankan data, they find that the effects are nonlinear and heterogeneous based on borrower wealth, which in turn, also depends on competitiveness in the credit markets. Their theory and evidence suggest that there might be significant effects on interest rates and profits from improving property rights. However, these appear to come mostly from increased effort rather than increased levels of borrowing. Using the context of the 2009 DRT
1 The Role of Justice in Development: Empirical Evidence 8 legal reform in India, LilienfeldˆaToal, Mookherjee, and Visaria (2012) show that faced with inelastic credit supply, stronger enforcement of lender rights reduces credit access to small borrowers in favor of the wealthy.

# **Justice E↵ects on Entrepreneurship and Business Environments**

Empirical evidence demonstrates the impact of justice on entrepreneurship and healthier business environments. Lichand and Soares (2011) explore the creation of Special Civil Tribunals in the Brazilian state of S ̃ao Paulo during the 1990s. These tribunals increased the geographic presence of the justice system, simplified judicial procedures, and increased the speed of adjudication of disputes. According to the authors, these new courts achieved considerable success and improved the business climate throughout the country. Microenterprises, creditors and the general population rushed to these new courts, leading to a greater resolution of contractual disputes, a better business environment, and greater recovery of defaulted loans by creditors. The results find that these special civil tribunals led to increased entrepreneurship among individuals with higher levels of education. The authors estimate that, as a consequence of the creation of Special Civil Tribunals, the number of people starting new businesses in Brazil increased by 10% in just 10 years.

judicial performance (measured in time-to-disposition and unpredictability of cases) stunts economic and social growth, resulting in lower per capita income, higher poverty rates, fewer private economic activities, and poorer public infrastructure. Jurisdictions with lower judicial performance also experienced higher crime rates and more industrial riots.

Similar effects have been recorded in two empirical studies in India and Pakistan. In 2002, India modernized its antiquated Code of Civil Procedure. Chemin (2012) shows that simple changes, such as limiting adjournments, imposing time limits, and encouraging out-of-court settlements, decreased the time it took to resolve cases and, in doing so, led to a number of positive changes in the economic sphere, including greater investments in firms. Commercial firms that benefited increased their investment by 7.5% . The author also finds that slower judiciaries are associated with poorer outputs in manufacturing and agricultural trade.

A different study by the same author evaluates an innovative delay reduction program implemented in Pakistan (Chemin, 2009). In this “Access to Justice Programme,” judges were trained on modern case flow management techniques, and a concerted effort was made to resolve the backlog of older cases. This improvement had a positive effect on factors related to economic development, including greater entrepreneurship in pilot districts. The author suggests that this reform, which only cost 0.1% of GDP, translated into an increase of 0.5% of GDP.

in Mexico by reducing the idiosyncratic risk faced by firm owners. The data comes from the Mexican economic census of 1998, and the findings are robust to instrumenting for legal quality using historical conditions.

quality of adjudication in Senegal. The reform gave judges the duty and powers to conclude pretrial proceedings within a four-month deadline. The length of the pretrial stage decreased by 42.9 days, the number of pretrial hearings was reduced, and there was no adverse impact on quality. The paper documents positive firm-level effects.

intermediate goods suppliers, with distributors, with lenders and others. The outcomes range from ease of entry, firm size, output, investment (especially efficient allocation of resources), productivity, and growth. A few papers quantify the consequences of lack of justice on firms, particularly in an
1 The Role of Justice in Development: Empirical Evidence 9 environment of strife and conflict that leads firms to misallocate labor towards protection rather than towards productive tasks (T. Besley and Mueller, 2018). On the other hand, firms that are associated with the executive or the military (or those in power, more broadly) benefit from monopolistic distortions to the disadvantage of other firms (Guidolin and La Ferrara, 2007; Brown and Huang, 2017).

# **Legal Aid for Vulnerable Populations**

The of justice can only be realized if citizens and have access to justice institutions. According to the World Justice Project (2019), 5.1 billion people–approximately twothirds of the world’s population–lack meaningful access to justice. This includes 1.4 billion people who have a criminal, civil, or administrative justice problem they cannot solve, and at least 253 million people who live in extreme conditions of injustice.

Interventions that empower clients of the judicial system may yield especially when the clients are unaware of their legal rights. Scholarship shows most of these interventions strengthen clients’ ability to obtain a remedy for their grievances. In fact, in the parts of the world where formal legal services are out of reach for people with modest means, they hold great promise as an alternative model of providing affordable and accessible advice or representation. Legal aid may also provide valuable assistance to vulnerable populations in high-income countries. The papers in this section show that legal aid may affect a wide range of outcomes. This assistance may be valuable to address essential aspects of citizens’ well-being across contexts.

In rural Kenya–a setting with very low access to formal institutions but with many land disputes–Aberra and Chemin (2018) gave access to the legal system to a treatment group by offering the services of a free lawyer for 2 years. The authors find relevant impacts of access to justice on farmers’ incentives to exert efforts and invest. Those who received the free lawyer increased the number of days worked on their land by 15% in comparison to those who did not. In addition, their investment increased by 21%, access to credit increased by 56%, and agricultural production increased by 42%. This experiment, which took place in a location where most people are small-scale farmers working and investing on their plot of land, provides evidence of the potential economic impact of providing access to legal aid for citizens with limited access to justice. These results are consistent with another study in Liberia where Sandefur and Siddiqi (2013) offered paralegals to a randomly selected treatment group. The authors find that this offer led to an increase in food security and household well-being.

Legal aid in housing-related interventions produced remarkable results in the United States. Seron et al. (2001) track the results of a legal assistance program for low-income tenants in New York City’s Housing Court. They find that legal counsel produces significant differences, as tenants who received aid are less likely to be evicted or denied rent abatement or repairs by landlords. These changes in outcomes are independent of the merits of the clients’ case. Greiner, Pattanayak, and Hennessy (2013) observe a similar result when they offer limited or full legal assistance to occupants facing eviction in Boston. Approximately two-thirds of occupants in the treated group and one-third of occupants in the control group retained possession of their units at the end of litigation. Moreover, treated-group occupants received more payments or rent waivers worth a net of 7.5 months of rent per case than their control-group counterparts, on average.

In addition, predictive analytics may be relevant to understand when legal aid may be more helpful to resolve disputes. In Mexico, the rate of settlement in specialized labor courts remains low because of overconfidence on the client-side and low-quality legal representation (Sadka, Seira, and Woodruff, 2017). Many plaintiffs spent more on legal fees than what they recovered. Quality legal aid and clear predictions about the length of the trial are both effective in correcting these erroneous
1 The Role of Justice in Development: Empirical Evidence 10 perceptions regarding their entitlement and the importance of particular types of evidence. As a result, more litigants chose to settle, freeing up space in court proceedings.

However, legal help may need to be dispensed with caution. Aberra and Chemin (2018) warn that offering free legal help on a large scale may flood the dockets, offsetting the positive effects of intervention. Furthermore, both the practitioner’s lack of competency and the candidate’s reluctance to cooperate could compromise the quality of aid. In relation to this, Chemin (2020) finds that reforms improving access to the legal system must be accompanied by simultaneous improvements in court efficiency to generate an effect on economic outcomes.

# **1.B How Does Justice Impact Conflict and Violence?**

This section summarizes literature that studies the empirical relationship between justice institutions and conflict. We define conflict here to mean civil strife and physical violence as opposed to contractual disputes that are covered in the earlier sections.

nations. Researchers have hypothesized different ways in which the judiciary could address–or redress–the prevalence of violence. The endogeneity of conflict and institutions has hindered robust responses to this question, even though in recent years there has been a rise in the empirical evidence related to it (Blattman and Miguel, 2010).

Ethnic nationalism has been popularly viewed as “the leading source of group cohesion and (by extension) intergroup civil conflict” (Blattman and Miguel 2010). Institutions resolving conflicts need to take into account the potential causes of conflict for preventive policies. Examples of preventive institutions include ADR, Community Driven Reconstruction (CDR), Cognitive Behavioral Therapy (CBT) and Social-Emotional Learning (SEL). Conflicts, in themselves, affect the welfare of citizens and businesses although not always in ways commonly imagined. Sometimes, a first-best approach does not yield an efficient outcome, especially when there are winners and losers from Justice institutions must also safeguard the rights of minority groups, women, and children, who are more vulnerable to violence in conflict settings. This can be done through specific interventions targeted to these groups or by creating an overall environment of trust in state institutions in the presence of biased informal institutions. In addition, justice institutions play a role in promoting crime deterrence and desistance from crime.

Given the endogeneity of in relation to institutions and the quality of justice, careful research examining the causal relationship is limited but growing. Administrative or other types of data are hard to come by in conflict zones and existing data are likely to be manipulated or destroyed. The papers reviewed in this section employ a host of non-experimental designs to examine this relationship and estimate the economic cost of conflict and violence. Additionally, individual level interventions like mediation, cognitive behavioral therapy and legal clinics can be examined using experimental methods where possible, as demonstrated by some of the studies introduced below.

## **Socio-Economic Costs of**

economic development. Although the volatility of violence makes categorizing its effects difficult, the papers in this section identify some of the main costs and provide numerical estimates. Consequences range from a decline in per capita GDP, lower firm performance (through misallocation of labor towards unproductive uses such as guard labor), and monopolistic distortions (where certain firms that benefit from conflict thrive at the expense of others).
1 The Role of Justice in Development: Empirical Evidence 11 On the macro-level, research has documented that outbreaks of violence lead to a shrinking GDP and lower firm performance, whereas the conclusion of violence spurs economic growth. In a case study of Basque Country, Spain, Abadie and Gardeazabal (2003) find the terrorist activities in the 1960s led to a 10 percentage point decline in per capita GDP and the truce in the late 1990s led to Basque firms performing more positively than non-Basque firms, which reverses at the end of the On a micro-level, labor misallocation may provide one explanation. predation, firms threatened by violence divert more resources from productive uses to privatizing security. Firms also reduced their presence in Afghanistan districts that experienced major increases in violence (Toomet et al., 2018). Perennial violence may also deter the development of formal financial markets, as people hold onto cash and spend less mobile money (Blumenstock, Callen, and Ghani, 2015).

Monopolistic distortion also follows after conflicts, often a byproduct of weak rule of law. For instance, during the Angolan Civil War, some diamond-mining firms were able to exploit the event of the sudden death of the rebel leader, whose group depended on diamond resources for financing (Guidolin and La Ferrara, 2007). Dube, Kaplan, and Naidu (2011) observe a similar selective accrual of benefits in the aftermath of US-backed coups around the world. The asset prices of certain companies rose from an average return of 9% over 4 days to 13% over 16 days.

rise to favoritism for the in-group and bias for the out-group(s). Examining the Israeli-Palestinian conflict’s effect on judges in Israeli courts, Shayo and Zussman (2011) show that the presence of ingroup bias in judicial decision-making correlates with the intensity of violence of the previous year. In a subsequent study, Shayo and Zussman (2017) highlight the long-term effects of conflict: even after the conflict ended, the bias remains positively associated with the past intensity of violence due to locational dynamics.

# **Conflict Prevention and Post-Conflict State Rebuilding Interventions**

Another question arises when we assess the costs of What are the strategies for supporting post-conflict recovery? Many studies agree that sustainably implemented preventive interventions—such as CDR programs—can effectively build social cohesion, lower disputes, and improve self-control. In post-conflict settings, social cohesion plays a crucial role in fostering resilience and the capacity for good governance. At the very center of a community’s cohesion is its members’ willingness to form partnerships. Fearon, Humphreys, and Weinstein (2009) find participation in community-wide public goods games a good measure for that willingness. In their examination of a CDR project in 42 northern Liberian communities, members of treatment communities are more inclined to cooperate, demonstrated by a 5.8 percentage point change in individual contributions.

Dispute resolution holds another key to supporting communities affected by protracted conflict. Since rebuilding formal mechanisms is time-consuming, many organizations channel their resources towards shaping informal practices. Community-level initiatives in this area see mixed results. Blattman, Hartman, and Blair (2014) make two observations about ADR’s short-term impact on behavior: first, the Liberian communities with mass educational campaigns on ADR had higher resolution of land disputes and lower violence, even when these programs reached only 15% of adults; second, these communities also experienced more extrajudicial punishment and higher non-violent disagreements.

In addition to cohesion and order, the management of residual hostilities presents another challenge to a peaceful future. Rarely is the end of a conflict immediately succeeded by a complete cessation of violence. Labor policy scholars have identified a few approaches to prevent individuals
1 The Role of Justice in Development: Empirical Evidence 12 from returning to violent lifestyles. Rehabilitating high-risk men via agricultural training, capital inputs, and counseling produced remarkable results in Liberia (Blattman and Annan, 2016). As farm employment increased, participants withdrew from mercenary activities. CBT is also useful for improving patience, self-regulation, and creating a non-criminal identity and lifestyle (Blattman, Jamison, and Sheridan, 2017). The authors further find cash a powerful reinforcing incentive, as interventions providing both cash and CBT prolonged reduction in crime by a year, whereas the effect of standalone interventions (providing only cash or CBT) dissipated over time.

# **Interventions Involving Disadvantaged Groups Vulnerable to Violence**

While the previous section traces the major prevention and resolution practices targeting conflict-ridden regions at large, specific segments of our society merit independent examination. Due to sources of oppression including socio-economic status, gender, race, or sexual orientation, disadvantaged groups are exposed to greater risks in the wake of unrest and violence. How law and justice, the very foundation of stability and reconstruction, interacts with these groups and addresses the challenges they face provides a useful perspective for studying the underlying causes of interpersonal and collective violence. Recognizing that individuals may hold membership within multiple social groups and their aggregate identities converge in unique experiences, this section does not limit itself to rigid characterizations.

Owen and Portillo (2003) assess the of legal aid clinics targeting economically disadvantaged women in Ecuador. They show that women who had access to these clinics were 20% more likely to receive an award in favor, 10.4% more likely to receive payments for child support, and experienced lower domestic violence than those who did not. Thus, unencumbered access to judicial institutions or mediation can protect vulnerable populations by reducing domestic violence. Sandefur, Siddiqi, and Varvaloucas (2015) investigate the consequences of providing free legal aid to police detainees in Sierra Leone. The results are no less noteworthy than civil legal aid: the release of detainees without charge increased by 13% and the share of inmates held on remand reduced by 20%. Although legal aid did not visibly reduce bribes or police extortions, suggestive evidence indicates it still removed some constraints on police resources.

Besides legal clinics, as Cappelletti, Garth, and Trocker (1982) point out, institutions such as community justice centers, arbitration, and public interest litigation are another way legal professionals can join forces to serve disadvantaged groups. As providers of police and legal services, these centers can also act as a source of employment by recruiting officers from these groups. AllWomen’s Justice Centers (WJCs) in Peru are an example of this. Kavanaugh, Sviatschi, and Trako (2018) find the opening of a WJC in Peru increased reporting and prosecutions for gender-specific crimes by 40% and reduced gender-based violence by 10%. WJCs also impact more than one generation, demonstrated by a substantial increase in human capital investments, enrollment, attendance, and test scores among children. In Papua New Guinea, J. Cooper (2019) finds that the presence of community police officers, particularly female officers, increases the probability of violence against women being reported, while reducing the perceived prevalence of violence against women.

The rule of law not only protects vulnerable groups from harm, it also boosts their well-being and health. For example, in Sub-Saharan Africa, legal origins and judicial mechanisms ensuring the enforcement of property rights enable women to negotiate safe sex practices with strength and lessen their risk of contracting HIV (Anderson, 2018). Anderson reports that female HIV rates are significantly higher in common law Sub-Saharan African countries compared to civil law ones. Common law is associated with weaker female marital property laws, which leaves women lower bargaining power within the household. The author demonstrates that women are therefore less able to negotiate safe sex practices and are thus more vulnerable to HIV in common law countries,
1 The Role of Justice in Development: Empirical Evidence 13 compared to their civil law counterparts. At the individual level, reducing these groups’ vulnerability necessitates a holistic approach that removes the particular barriers they encounter in acquiring resources and power. At the very root is their impeded access to justice, often engendered by a lack of awareness or physical access, reinforced by a mistrust of formal institutions or a fear of backlash for seeking help. To rectify these factors, modern interventions usually involve providing immediate legal assistance for short-term relief, and education and CBT programs to improve longterm outcomes and reduce recidivism.

Apart from rehabilitating former offenders, legal institutions are also actively cultivating a societywide atmosphere of trust and increasing disadvantaged groups’ bargaining power. In India, lower caste and female entrepreneurs gain more from better judicial enforcement of contract rights than their higher caste and male counterparts, possibly because informal dispute resolution mechanisms are dominated by upper-caste men (T. Chakraborty et al., 2018). The same phenomenon occurs in Liberia (Sandefur and Siddiqi, 2013). Without legal aid, litigants with relatively lower social positions paid more bribes to overcome the biases they suffered at the hands of informal institutions. They are more likely to use formal forums for better protection of their rights.

# **The E↵ect of Justice on Crime and Recidivism**

A vast amount of literature in economics and related disciplines has studied crime prevention. There are recent literature reviews evaluating the empirical evidence on crime deterrence (Chalfin and McCrary, 2017; Nagin, 2013) as well as crime desistance (Doleac, 2020). In this section we summarize the main conclusions from such literature reviews and highlight the judiciary’s role on preventing crime.

and McCrary (2017) organize the literature on crime deterrence into three general categories. First, they study how crime responds to the probability that an individual is apprehended, which mostly falls on the side of the police, either via police manpower or policing intensity. Second, the authors evaluate the sensitivity of crime to changes in the severity of criminal sanctions including sentence enhancements, habitual offender laws, capital-punishment regimes, and discontinuities in the severity of sanctions faced by particular individuals. The third category examines the responsiveness of crime to local labor-market conditions.

Overall, and McCrary consistent evidence that crime is responsive to police manpower and to the existence of attractive legitimate labor-market opportunities. The responsiveness of crime to the severity of criminal sanctions–more directly related to the judiciary–is less clear. Deterrence effects coming from prison sentences, unlike the other two, are experienced in the future, which may reduce the importance given to it by individuals who are myopic or have a high discount rate. In addition, potential offenders may also be less aware of changes in prison sentences than they are of those in policing and local labor market conditions. Nonetheless, the authors find some evidence of deterrence induced by policies that target specific offenders with sentence enhancements (Helland and Tabarrok, 2007; Drago, Galbiati, and Vertova, 2009).

Beyond crime deterrence, crime desistance has also been subject to extensive analysis, and is of great importance for the prevention of violence. In the United States, two-thirds of those released from prison are arrested again within three years, and half re-incarcerated (Durose, A. D. Cooper, and Snyder, 2014). This seems to be driven by a group of individuals who repeatedly cycle through the system (Rhodes et al., 2016). The question is then: what is the best way to handle those who have already committed one or more crimes? What interventions can reduce their rate of reoffending, and increase social welfare? Doleac (2020) summarizes the existing empirical evidence on those interventions that may reduce recidivism and improve social welfare.

According to such review, there are various ways in which the judiciary and prisons may play
1 The Role of Justice in Development: Empirical Evidence 14 a significant role on crime desistance. For instance, two studies find that increasing non-carceral punishments such as fines or probation (in the context of DUI and traffic offenses) has a deterrent effect on reoffending (Hansen, 2015; Gehrsitz, 2017). Another intervention such as replacing short prison sentences or pre-trial detention with electronic monitoring also has net benefits on reducing recidivism–possibly due to the avoidance of the criminogenic effects of prison–and may improve secondary school completion for young offenders (Di Tella and Schargrodsky, 2013; Monnery, Henneguelle, and Kensey, 2016; Østergaard Larsen, 2017). On the other hand, the evidence on increasing prison sentences for those on the margin of being incarcerated shows mixed effects on future offending and employment.

There are two studies highlighting the importance of the difference between time served in prison relative and the initial, expected sentence (Bushway and Owens, 2013; Monnery, 2016). Conditional on the actual time served, those whose recommended sentences are longer tend to be more likely to recidivate after released.

There is also substantial evidence that cognitive behavioral therapy is effective at reducing recidivism (Voorhis et al., 2004; Pearson et al., 2016; Barnes, Hyatt, and Sherman, 2017; Heller et al., 2017), and some evidence showing that drug courts (as an alternative to regular courts) can reduce recidivism (Prins and Officer, 2015). In addition, courts may increase access to jobs by providing rehabilitation certificates, shifting the risk of hiring someone from the employer to courts (Leasure and Stevens Andersen, 2016).

In a cross-country analysis using micro-data from 25 European countries, N. H. Mocan, Bielen, and Marneffe (2018) find that higher judicial quality, measured via the appointment procedures of judges and prosecutors, generates a two-fold effect: first, it diminishes individuals’ criminal and dishonest behaviors; and second, it reduces tolerance for unlawful behavior and increases regard for the rule of law, impacting society as a whole.

# **1.C Do Better Judicial Institutions Enhance Trust and Reduce Corruption?**

The fundamental basis of modern democracies is the separation of the executive, legislative and judicial branches of government. When deriving the trias politica model, Montesquieu realized that “every man invested with power is apt to abuse it, and to carry his authority as far as it will go” (Montesquieu, 1748). In order to prevent this abuse, a system was designed in which “power should be a check to power”, and the three branches would be separated, autonomous and independent from each other. In particular, the judiciary would hold accountable the other two branches and ensure fair resolution of conflicts among citizens. Even though the judiciary should hold accountable the other two branches of government, the empirical relationship between these two concepts is not so clearly defined.

Conceptually, the judiciary should play a crucial role in monitoring corruption as well as ensuring the integrity and quality of justice. A weak justice sector provides the ideal environment for corruption to expand and persist without being monitored. Moreover, when the justice sector institutions are seen as part of the corruption problem, access to a fair trial, and thus the quality of justice, are severely undermined. A corrupt judiciary opens the room for the powerful and wealthy to escape prosecution and conviction, and it may also preclude the rest of the population from their rightful access to a fair trial.

on actors varying from governments to private firms to individuals (Olken and Pande, 2012). Not only does it stunt economic growth, it might also invite political turmoil and diminish the quality of life. This section reviews literature on the effects of regulatory institutions as well as overall
1 The Role of Justice in Development: Empirical Evidence 15 governance structures imposing checks and balances on various components of state institutions. More empirical evidence on the impact of the judiciary on monitoring corruption would be valuable. There is substantial evidence of the pervasive effects of corruption, yet the extent to which better judiciaries could reduce the incidence of corruption is yet to be documented.

# **Justice through Regulatory Institutions**

Well-functioning democracies impose many checks and balances on corruption, ranging from independent judiciaries to a free press to robust regulatory systems (Mcmillan and Zoido, 2004). This section examines how different types of regulatory institutions facilitate or hamper justice delivery. Wielding powers resembling those of courts, regulatory and other quasi-judicial institutions set standards for industries or provide redress for complainants. Despite their limited toolkit, these institutions effectively reduce the burden of courts with overflowing dockets. At the same time, they cannot be relegated to mere alternatives. Regulatory and quasi-judicial organs strengthen the rule of law in unique ways. For example, the absence of a strong regulatory system often invites cronyism and collusion; firms that are politically connected derive an unfair share of benefits through oligopolistic as well as oligarchic practices (Faccio, 2006). This becomes apparent when the political environment is fraught, as firms with better political connections suffer more from the negative rumors on the President’s health than less-connected firms (Fisman, 2001). Although the regulatory systems operate under the ambit of the executive, they prevent monopolistic distortions by private players or capture by state organs.

A just society starts with leadership that does not use its power for personal gains, and one of the most relevant tools of increasing accountability is auditing. Usually handled by an independent agency, auditing verifies individuals’ or organizations’ financial records. When performed adequately, random audits and result disclosures improve the quality of democratic processes by providing voters reliable evaluations of the incumbent administrations’ political performance. This effect is amplified when the audit is conducted just before elections. If the auditor’s reports reveal any violations, incumbents are less likely to get reelected (Ferraz and Finan, 2008). Finan, Ferraz, and Avis (2018) support this conclusion. They find randomized government audits in Brazil reduced future corruption by 8% and increased the probability of legal retribution by 20%.

Bobonis et.al’s (2016) investigation of municipal audits in Puerto Rico provides consistent results, with a caveat. In the short-term, they confirm that timely audits reduced municipal corruption levels by 67%. However, these significant disciplining and sanctioning effects of timely audits dissipated in the long term, as timely and untimely audits lead to similar levels of corruption. In addition, incumbents in municipalities with an earlier timely audit are more likely to be elected, which suggests that voters select responsive but corruptible candidates to office. Therefore, audits do not automatically deter corruption, unless they are carried out in a timely and sustained manner over the long term.

Similar to auditing, disclosure laws are central to curbing corruption. Unlike auditing, disclosure laws require individuals or corporations to divulge information. After examining the financial and business disclosures of members of parliaments around the world, Djankov et al. (2010) find that more adequate disclosure requirements are associated with less corruption and stronger governments. The adequacy of such requirements depends on whether the results are made public and whether the content of the disclosure includes more diverse items than just income and wealth levels. For instance, public disclosures that cover assets, liabilities, and income sources are more effective checks on the politicians’ personal motivations or undue influence from the executive.

Procedural formalism, however, has some drawbacks. Analyzing data from 109 countries, Djankov et al. (2003) conclude that a robust regulatory ethos over dispute resolution pervades almost
1 The Role of Justice in Development: Empirical Evidence 16 all legal systems, especially in civil law and developing countries. Contrary to the view that heavily regulated adjudications ensure fairness, strong formalism is correlated with longer proceedings, less consistent and fair decisions, and greater corruption.

The rise of technology also produces repercussions in the promotion of political and corporate accountability. Social media holds great promise as a tool for detecting and preventing corruption. Enikolopov, Petrova, and Sonin (2018)’s research on the Russian Federation provides a snapshot of how news media disciplines private-sector wrongdoing. Blog posts exposing corrupt practices in state-controlled firms not only caused a decrease in these firms’ market returns, they are also associated with higher management turnover and lower minority stakeholder conflicts.

Another way to combat corruption is improving the governance of social services (Reinikka and Svensson, 2005). In Uganda, public funds were often captured by local officials. After introducing a newspaper campaign exposing this practice, schools in areas with higher newspaper penetration received more funds and were able to allocate them to nonwage items such as textbooks, meals, or uniforms. This in turn caused an increase in enrollment and test scores of students.

In sum, regulatory systems provide powerful antidotes for countries and industries caught in the grip of corruption. However, although unregulated systems frequently fall prey to bribery and corruption, not all regulations are productive. Amirapu and Gechter (2017) discuss one mechanism through which excessive labor regulations worsen economic outcomes and increase corruption. Burdened by higher regulatory costs, Indian firms in intensely regulated industries actually experience worse levels of corruption. One explanation is that inspectors—the primary task force for enforcing labor regulations in India—are threatening firms by overreporting violations to extract more bribes.

# **E↵ects of Justice on Trust and Corruption**

Wielding the power of judicial review, courts in modern democracies represent another key component in the system of checks and balances. The integrity of the judiciary is fundamental to the delivery of justice, citizens’ well-being, and a nation’s stability. Thus, enhancing citizens’ trust in judicial institutions is a potential way to ensure other civil and political processes run smoothly. Empirical evidence in fact shows the role of the judiciary to enhance trust in formal institutions, as well as its association with lower crime and democracy promotion.

Pakistani state courts improved citizens’ trust in the state. Citizens reported higher willingness to use state courts, greater trust in them, and larger allocations of funds to these courts. Choi (2010) offers suggestive evidence on courts’ influence on crime reduction and democracy promotion. From 1984 to 2004, his survey of 131 countries indicates that impartial judicial systems are consistently better equipped to redress the grievances of citizens, cultivating a perception of legitimacy and lessening their tendency to turn to terrorist activities.

In a study based on judicial data from the EU Justice Scoreboard, Gutmann and Voigt (2017) found that citizens’ perception of judicial independence at the national level is negatively associated with the presence of formal legislation considered conducive to judicial independence. The authors suggest that this puzzle is explained by cultural traits: countries with high levels of generalized trust (and to a lesser extent individualistic countries) show increased levels of de facto judicial independence, while also having reduced levels of de jure judicial independence.

On the contrary, previous studies, mostly from India and Pakistan, demonstrate how corruption affects judicial decision-making in favor of the executive.

In India, Aney, Dam, and Ko (2017) study whether judges respond to incentives by ruling in favor of the government in the hope of receiving jobs after retiring from the court. They find that
1 The Role of Justice in Development: Empirical Evidence 17 pandering incentives have a causal effect on judicial decisions. They also find that authoring judgments in favor of the government is positively associated with being appointed to a prestigious post-Supreme Court job. Also in India, Poblete-Cazenave (2020) finds that politicians in power get special treatment in courts when facing criminal accusations. The author finds two opposite effects of winning office, depending on the political alignment with the state ruling party. Winners from the state ruling party are more likely to get their pending criminal cases closed without conviction while they are in office, whereas winners from other parties are less likely to get their pending cases closed without conviction during the same period.

idential appointment to appointment by peer judges, rulings in favor of the government decreased significantly and the quality of judicial decisions improved.

dermines the independence of judicial decision-making and may be indicative of corruption. Greater independence in the appointment of judges may reduce this impact. The question is then whether more independent and stronger judiciaries may not only improve the fairness of judicial decisions, but also reduce the presence of corruption in the public administration.

# **1.D Summary of Evidence**

velopment, particularly through their impact on economic growth and credit markets, the protection of vulnerable populations, their capacity to deter violence and their influence over people’s trust in institutions.

First, we review the evidence on the impact of justice on economic outcomes. The evidence finds that contract enforcement influences moral hazard, can increase the competitiveness of credit markets, and plays an important role in firms’ productivity. Weak enforcement affects firms’ incentives to invest, thus reducing productivity, and distorts the production materials used by firms, leading them to switch to more costly suppliers or less efficient technologies.

We consensus on the positive impact of judicial institutions on credit markets, both via credit availability and the development of credit markets. The two main mechanisms discussed are the prevention of moral hazard and increased competition across lenders. Judicial institutions may have downstream outcomes in other sectors as well. For example, in India the negative effect of slower judiciaries on access to credit markets in the agricultural sector leads to a subsequent reduction in agricultural output Chemin (2009).

Judicial institutions also entrepreneurship and the overall business environment. For example, new tribunals that increased the presence and the efficiency of the justice system in Brazil led to increased entrepreneurship and healthier business environments (Lichand and Soares, 2011). Similar effects on entrepreneurship were found from a training program for judges in Pakistan (Chemin, 2009).

Greater access to justice through legal aid can play a crucial role in the well-being of vulnerable populations, even in very different contexts. In the United States, legal aid is shown to reduce the likelihood of eviction for those at risk and increase tenants’ likelihood of retaining possession of their units. In developing country contexts such as rural Kenya or Liberia, legal aid may increase farmers’ investments and the productivity of their work, and even increase recipients’ food security and economic well-being. Given the size of the global justice gap, i.e. 1.5 billion citizens who lack access to justice, the capacity of justice institutions to enhance the well-being of poor citizens in developing countries may be worthy of particular attention.

Second,
1 The Role of Justice in Development: Empirical Evidence 18 violence. Even though the literature in this field is scarce–mostly due to the endogeneity of conflict with institutions–in recent years a growing body of literature has focused on the ways to redress The evidence at the macro level between institutions and is still limited. There is evidence that conflict affects justice, for example, via the impact of in-group bias in judicial decisionmaking, as in the case of Israel (Shayo and Zussman, 2017). On the opposite direction, there is suggestive evidence that an increase in judicial quality may reduce criminal and dishonest behaviors, increase regard for the rule of law and reduce tolerance of unlawful behavior (Choi, 2010).

Different interventions at the micro level have proven to be effective at reducing violence in the resolution of disputes. An important justice-related one is alternative dispute resolution, which in post-conflict Liberia led to increased resolution of land disputes and lower violence, despite an increase in non-violent disagreements. Evidence also shows that legal aid clinics play a crucial role at protecting vulnerable populations at risk of violence. For example, legal aid clinics may reduce domestic violence and increase human capital investments. An example of this are Women Justice Centers in Peru (Kavanaugh, Sviatschi, and Trako, 2018). These clinics attended by women substantially reduced gender-based violence and increased children educational performance throughout the country. Beyond legal clinics and informal institutions, formal justice institutions may be particularly important at protecting vulnerable populations in contexts where customary laws are biased against them.

At the interpersonal level, there is consensus that interventions such as CBT, substituting short prison sentences or pre-trial detention for electronic monitoring, drug courts (rather than actual courts), or making courts (rather than employers) responsible for rehabilitation certificates are ways in which judicial institutions can reduce crime propensity and recidivism.

Third, we review the evidence linking justice, trust and corruption. Even though the relationship between justice and corruption may sound evident, we find no empirical evidence demonstrating the role of the judiciary in monitoring or deterring corruption. On the contrary, the literature highlights the influence that the executive may have on the judiciary, which undermines judicial independence and may be indicative of corruption. Greater independence may be achieved by preventing Presidential appointment of judges, as in the case of Pakistan (Mehmood, 2020).

The evidence suggests that impartial judicial systems contribute to cultivating a perception of legitimacy and lessen people’s tendencies to turn to terrorist activities (Choi, 2010). In addition, disclosure laws that require individuals or corporations to divulge information are associated with less corruption and stronger governments (Djankov et al., 2010).

Courts also matter for people’s trust in formal institutions. ciency may increase willingness to use courts, trust in them, and allocations to courts in comparison to other informal institutions.

Our summary of the evidence warrants two important caveats. First, the impact of the judiciary will depend upon the type of regime that prevails in a country. Most empirical evidence presented in this paper studies the impact on democratic regimes. However, the effect under other systems of government may be significantly different. Autocratic regimes oftentimes ensure that the judicial system is subservient and highly responsive to the government. In such contexts, it is unclear whether a more efficient or stronger judiciary will actually benefit society as a whole. Ginsburg and Moustafa (2008) found little reason to believe that judicial reform will lead to political transitions.

Second, most empirical evidence is concentrated in a limited set of regions. South Asia, the United States and Europe are better represented in empirical research related to the judicial system. In contrast, Sub-Saharan Africa and Latin America are underrepresented in these studies. Microeconomic evidence on judicial reforms is just growing in recent years. Greater empirical evidence would be highly valuable to understand the broader impact of the rule of law on development.
2 The Data Revolution in Justice 19

# **2 The Data Revolution in Justice**

Recent years have witnessed an increase of data availability in judicial systems around the world. Many countries have developed electronic case management and e-filing systems. This has led to the creation of massive databases that track every characteristic of each case. In fact, this is by definition an element of the fair administration of justice: everything that happens in a case needs to be tracked to demonstrate the evidence behind facts-based decisions. Even though this data is readily available to policy makers, it has rarely been exploited to evaluate policies or improve the functioning of the judiciary.

The underutilization of data raises many questions: What if we could harness its power to understand what sectors of the population lack access to justice? Taking it one step further, what if we could use the text of decisions to reduce judicial biases, to revamp justice systems that unfairly target gender and racial minorities? What if we could use high-frequency data to understand and address why cases get backlogged, thereby removing a main barrier to justice in the developing world? In the era of COVID-19, what if we could use high-frequency data to predict the future congestion of courts, and plan ahead to be prepared for what is yet to come? Ultimately, what if we could use data to assess the specific impact of new laws on individuals and firms, and promote best practices among different countries?

By responding to these questions, the data revolution can help the rule of law reclaim its role in development planning. More importantly, by using data more effectively, judiciaries around the world, and particularly those in developing countries, will be able to improve their performance, address deficits in the quality and accessibility of justice, and contribute to prosperity. In this section, we explore the opportunities that arise from the expanded data systems in justice. We discuss how data can help assess social prejudice and integrity in justice systems in ways that have not been possible until recently. As COVID-19 moves justice into virtual courtrooms and greater electronic processing of cases, this section assesses the opportunities that arise to evaluate and improve crucial aspects of the efficiency, quality and integrity of courts.

# **2.A The Expansion of Data Availability**

Judiciaries generate vast amounts of data on a daily basis. Historically, this data has been collected in paper form. Judicial data has rarely been analyzed quantitatively, and little information has been extracted from the overall patterns of the data. According to the literature, the reasons are twofold: first, judicial officers have historically not been concerned with performance indicators; and second, legal scholars have prioritized qualitative over quantitative analysis of the functioning of the judiciary (Dakolias, 1999). However, this trend has been changing in recent times. Judiciaries around the world are establishing e-filing and case management systems where every aspect of each case is recorded. Others have also established electronic calendars, electronic notification systems, electronic jurisprudence, virtual courts, and other technological platforms that facilitate the efficient functioning of the judiciary. These technologies, particularly case management systems, tend to generate vast amounts of data that can be utilized to evaluate the functioning of the court. Moreover, as large backlogs of cases becomes a significant problem for the efficiency of the judiciary in developing countries, interest is growing around performance metrics that will improve the functioning of the judiciary. The expansion of data availability in judiciaries around the world creates valuable opportunities to measure the overall performance of the judiciary, identify key areas for improvement and evaluate the impact of justice reforms.

The adoption of technological platforms that generate judicial data has affected both developed and developing countries. The United States, Canada, European countries, and Australia are at the
2 The Data Revolution in Justice 20 forefront of e-justice systems. Even so, electronic systems have also been implemented or are in the process of implementation in Latin America (Chile, Argentina, Uruguay, Peru, the Rep ́ublica Bolivariana de Venezuela, and some Mexican states, among others), Africa (Kenya, Rwanda, Lesotho, etc.) and Asia (Malaysia, Singapore, and many others). The increased availability of judicial data can be used to improve the functioning of judiciaries, define what works in justice reforms, and produce rigorous evidence on how the judiciary may affect development outcomes. That said, this paper does not establish standard performance indicators. Indicators have already been discussed on areas such as access to justice; expedition and timelines; equality, fairness and integrity; independence and accountability; and public trust and confidence (Reno et al., 1997). Moreover, we do not cover issues of transparency or accessibility of the data. The recommendations are targeted to everyone who may have access to judicial data. We also acknowledge that not all countries may have the institutional capacity or infrastructure to create data ecosystems, but even countries without electronic systems in place may benefit from understanding what may be possible were they to develop such data systems.

The extent to which each of these countries can utilize the approaches outlined in this paper will depend on the extent to which e-judiciaries are integrated, the sources of data available, as well as the quality and accuracy of this data. Common concerns surrounding representativeness and exhaustivity of the data will be present in contexts where not every case is added to the database, which may impose a burden on those with lower digital access.

The main sources of data are automated registrars, which generate a database that includes every case in a specific court, division (for example, civil or criminal cases), and country. These databases tend to include at least the following information: type of case and tribunal, the stage of the case, the dates of filing and resolution, reason for claim or accusation, resolution of the case, information about judges and parties involved, and the history of the case. Oftentimes, these cases can be merged to the judgment text. In an ideal scenario, this data will also be complemented by other sources of data to create a larger ecosystem. Case data can be linked to users or firms through tax or national IDs to evaluate the potential effect of policies or reforms on the affected parties. Case data can also be linked to other records, such as police and prison records, or to geographic areas, to better understand regional access. Finally, the data should be complemented by legal needs or court-user surveys to evaluate satisfaction with the functioning of courts, and to consider both (1) the needs of those who have access to courts and (2) of those who have a legal need but resort to other methods to resolve it.

two main limitations impeding utilization of this data. The first is difficulties in merging different databases with different definitions, IDs, or relationships for each case (Schauffler, 2014). Nonetheless, in contexts where data cannot be merged, valuable insights will still arise from comparing the results of different sources. For example, learning about access to justice in remote regions can be achieved by comparing the outcomes from legal need surveys in a region to case-level data from courts. The second limitation is related to human capital. Judicial officers are rarely experts on data analysis, and engineers generally lack the domain expertise necessary to understand the functioning of the law. In courts without sufficient human capital to take advantage of available data, training bureaucrats to learn the necessary skills may be a valuable long-term investment for improving the functioning of courts. Alternatively, there may be room for non-governmental organizations, international organizations or even private companies to meet this need.
2 The Data Revolution in Justice 21

# **2.B Measurement, Diagnostics and Experimentation**

In this section, we present an initial framework to make better use of judicial data. Justice systems should start by measuring indicators of performance, quality, integrity and accessibility of justice. These measurements can also be used to improve the deployment of resources, and therefore enhance efficiency and reduce the backlog of cases. Courts should then provide diagnostics on those areas that are most valuable to their service. For example, databases can be used to identify the main causes for adjournments, understand what the barriers to justice are, or evaluate which cases would benefit the most from mediation. Finally, researchers and policy makers should take advantage of data systems to rigorously evaluate the impact of justice reforms. The random assignment of cases to judges behaves as an exogeneous shock that allows to evaluate the impact of judicial performance on the well-being of citizens and firms. By keeping track of innovations in the judiciary and experimenting with potential solutions, data systems generate the opportunity to understand the impact of reforms on people’s lives and the prosperity of nations.

## **Measurement and Diagnostics**

Despite the growing availability of case-level data, most courts around the world are underutilizing the opportunities created by these databases. Many courts are not even producing performance or management reports that summarize key variables, such as the number of incoming cases, time to disposition, and case clearance rate. Correspondingly, most courts allocate resources without taking into account important efficiency measures that may be gleaned from stronger empirical data, for example by evaluating the backlog or productivity of specific courts. As data systems become available and more courts decide to catalogue data systematically, greater opportunities to evaluate a court’s efficiency will arise.

The very basic usage of case management systems consists in measuring what happens in courts. To begin with, judiciaries could create indicators that evaluate the efficiency, access, quality and integrity of each court. A full list of indicators is available at “Court Performance Around the World: A Comparative Perspective,” which includes, among others, the following variables: number of cases filed, disposed and pending; clearance rate and congestion rate; average duration of a case; and number of judges per habitant across regions (Dakolias, 1999). Court-user surveys help create indicators of quality, integrity and public trust in the judiciary. Databases can also be used to diagnose the main issues and problems the judiciary faces. Many courts in developing countries are worried about their large backlog of cases, which limits citizens’ access to justice and erodes their trust in formal institutions. Databases can be used to identify the main causes of delay and adjournments in legal cases.

Similar diagnostics can be used to evaluate an array of outcomes that a judiciary cares about, such as who uses formal institutions and how they make use of them, the level of user satisfaction via court-user surveys, which legal needs in a region remain unfulfilled, the main barriers to accessing justice and how to address them. Apart from diagnosing problems, data systems should also be used to coordinate and deploy resources in an efficient and effective manner. For example, the assignment of cases to judges can be automated so that they take into account the workload and backlog of cases for each judge, but also to ensure the random assignment of cases to judges. Databases can also be used to evaluate the capacity of judges and mediators. Then governments can train or incentivize those who are less capable or less willing to improve performance. Finally, databases could be used to determine which courts and cases may benefit more from ADR. Advanced systems can detect which cases are more likely to be successfully resolved through mediation and automatize the redirection of these cases to mediation.
2 The Data Revolution in Justice 22 A bureaucratic strand of literature evaluates case flow management and staffing adequacy, highlighting the importance of management practices in court performance. For courts to move through their dockets quickly, Buscaglia and Dakolias (1996) already proposed the adoption of data-driven decision-making, a method tested by Ecuador and Argentina during a period of skyrocketing pendency rates in the 1990s. Among the most constructive changes are active management of cases, clear timelines for processing, and adequate mechanisms to monitor progress. Dakolias and Said (1999) study the bottom-up approach to implementing judicial reforms. Lessons from the pilot projects on improving the judiciary in Colombia, Peru, Argentina, and Ukraine demonstrate that judicial reforms work best when implemented in lower courts. A change in culture and management practices and support from stakeholders from different sectors of society, including legal professionals, businesses, and NGOs, also enhance these reforms’ reach.

Databases may also be used to diagnose the overall functioning of the system and the capacity of different legal actors involved. For example, Carmignani and Giacomelli (2009) argue that the number of lawyers has a large positive effect on civil litigation across Italian provinces from 2000 to 2005. This conclusion is challenged by Yeung and Azevedo (2011), who find that human or material resources do not fully explain the differences in efficiency across Brazilian courts. Instead, it correlated more closely with court management practices. Mitsopoulos and Pelagidis (2010) suggest that the main problems in the Greek judicial system are actually failings in the system’s own design: insufficient judicial organization and accountability, excessively burdensome procedures, and lack of competition in the provision of legal services. Moreover, Coviello, Ichino, and Persico (2015) show that judges carry out work in short, interrupted segments–what they define as task juggling. When judges juggle tasks, it lowers their productivity substantially. In particular, those judges who juggle more trials at once instead of working sequentially on few of them over time, take longer to complete their portfolios of cases.

These diagnostics may be an essential tool to understand the main areas for improvement in the justice system. The establishment of comprehensive databases and performance indicators are the first step towards understanding what works, or what needs to be improved, in a country’s justice system. Policy-making can then benefit from targeted goals and recommendations that are based on solid measurements.

# **Experimentation**

Apart from measurement and diagnostics, data systems provide the ideal opportunity to experiment with policy options in order to rigorously evaluate the impact of justice reforms. Innovative policies can be implemented in a staggered fashion to provide the opportunity to rigorously evaluate them. Alternatively, performance incentives and other cost-effective measures can be tested via randomized controlled trials (RCTs). For policies that can have a relevant long-term impact, iteratively testing and evaluating their impact through RCTs can lead to a better understanding of the benefits and trade-offs of such policies.

Legal scholars and judges have long made arguments about laws and regulations and their arguments with theories about the effects of these legal rules. A particularly challenging dimension of studying the effects of legal rules is that many other aspects of society are correlated with the presence of legal rules, so it is difficult to determine cause or effect. Much like medicine a century ago, prior to clinical trials, theories most often lacked rigorous causal evidence.

Randomizing judicial decisions, however, violates our notion of justice and equal treatment before the law. On the other hand, randomizing case assignment to judges generates a retrospective “clinical trial”, the first example of which was only published as recently as 2006 (Kling, 2006), where the policy question was whether longer sentence lengths affected subsequent labor outcomes
2 The Data Revolution in Justice 23 and earnings. Randomizing sentences is unethical, but randomizing cases to judges who are systematically harsher or more lenient than others generated the inference on the long-run impacts of sentences lengths. Thus, in countries where cases are randomly assigned, the random assignment itself can be used as an exogenous source of variation to evaluate the impact of judicial decisions. This method has become widely used since the credibility revolution.

For instance, Dobbie and Song (2014) investigate the causal effect of consumer bankruptcy–one of the largest social insurance programs in the United States–on debtors by exploiting the random assignment of bankruptcy filings to judges. The authors find that debt relief increases individuals’ earnings and employment and decreases mortality and foreclosure rates. In Norway, Dahl, Kostol, and Mogstad (2013) evaluate the existence and importance of family welfare cultures, where the receipt of a welfare program by one generation causes increased participation in the next generation. The authors exploit the random assignment of judges to applicants for disability insurance whose cases are initially denied, by comparing the implications of being assigned an appeal judge who is systematically more lenient than a harsher one. They find strong evidence for a causal link across generations, where the adult child’s participation increases if the parent also participated in the insurance program.

Another policy question that has been answered by leveraging the random assignment of cases to judges is the impact of pretrial detention on defendants. Arnold, Dobbie, and Yang (2017) use the detention tendencies of quasi-randomly assigned bail judges to evaluate the impact of pretrial detention of subsequent defendant outcomes. They find that pretrial detention–due to comparably harsher judges–decreases formal sector employment and the receipt of government benefits.

This “judge leniency” design may be applied to other characteristics of judicial decisions. For example, what is the impact of the speed of justice? These kinds of questions can be studied in administrative data where random case assignment exist. Judges predicted to be fast also tend to be fast in other cases. The causal effect of faster case resolution can be studied by linking to long-term outcomes where the data infrastructure permits.

Overall, experimentation brings the opportunity to implement rigorous causal evidence to the legal realm. Whether it is via RCTs or leveraging the random assignment of cases to judges, “clinical trials” can bring important lessons on why rule of law matters for development outcomes, and perhaps more importantly, on what is the impact of specific policies and reforms on the litigants’ well-being.

# **2.C Machine Learning Applications**

In this section, we discuss avenues to leverage machine learning to improve judicial decision-making and increase the effectiveness of justice. Moreover, we propose a multistep approach to leverage administrative data to analyze disparities in judicial decisions. We also explore the potential of machine learning and natural language processing (NLP) techniques to process large quantities of text to improve the knowledge and efficiency of justice systems. These techniques currently allow researchers and policy makers to analyze texts to an extent that was not humanly possible before. Such opportunity has arisen not only due to the increase in the large volumes of data available, but also thanks to the recent development of computational tools that make it possible to process and analyze such large and complex data.

## **Applications of Administrative Data**

Even though the adoption of machine learning tools in the justice system is in its early stages, the preliminary results showcase the potential of these methods while also highlighting the risks that
2 The Data Revolution in Justice 24 their inappropriate use may present. Recent research examines the potential of machine learning techniques to “improve human decision”, in specific by evaluating the potential of machine learning to decide whether defendants should await trial at home or in jail in different US urban cities (Kleinberg et al., 2017). The authors focus on bail decisions, in which judges have to assess whether the defendant will flee or commit a new crime if released, and trade off these risks against the cost of incarceration. In brief, the judges have to decide, based on their prediction, what a defendant would do if released. This specific prediction task makes an algorithm particularly promising for this job.

The authors use a large data set of cases heard in New York City from 2008 to 2013 to build an algorithm that assesses the risk of fleeing or recidivism. The algorithm then makes a prediction based only on data available to the judges at the time of the bail hearing. After comparing results, the authors find that the algorithm improves judicial decisions, and has the potential to reduce crime as much as 25% without changing the numbers of people waiting in jail. By evaluating the results, they realize that judges release many defendants the algorithm ex ante identifies as very high risk, and that stricter judges do not jail the riskiest defendants first. These results are not restricted to New York City, as the authors report similar findings in a national data set as well. Moreover, the authors show the potential of machine learning tools to diagnose why judges mispredict.

The results from the aforementioned study suggest a promising avenue for future research on the application of machine learning to predict the risk of recidivism and improve the precision in judicial decisions. According to the United Nations (2018), the proportion of prisoners held in detention without being sentenced for a crime has remained almost constant between 2003–2005 and 2014–2016, and still affects almost one third of all prisoners. Thus, the importance of potential developments in this field may be particularly relevant all over the world to better identify those who should stay in prison–or at home–while expecting a trial.

In addition, clear predictions about the length of a trial or expected gains from it may be helpful to improve litigants’ decisions on their cases. In Mexico, the rate of settlement in mediation in specialized labor courts remains low because of overconfidence on the client-side and low-quality legal representation (Sadka, Seira, and Woodruff, 2017). Many plaintiffs spend more on legal fees than what they recover in court. Quality of legal aid and predictions about the case are both effective in correcting these erroneous perceptions regarding their entitlement and the importance of particular types of evidence. Sadka, Seira, and Woodruff (2017)find that providing information to litigants reduces the level of overconfidence of litigants, and nearly doubles the overall settlement rate. Nonetheless, the increase in settlement rates only occurs for the subset of cases for which the plaintiff is present to receive the information, as the information provided to lawyers does not have an effect. Thus, administrative data may also be able to improve litigants’ expectations of a case, identify the optimal path to resolve a specific case and–as in the case of Mexican labor courts–lead more litigants to settle, freeing up space in court proceedings.

Nonetheless, the implementation of machine learning tools in the judicial sector entails relevant risks that may not be overlooked. Inappropriate application of these tools can lead to decisions that may violate due process or that may discriminate based on race, gender or other characteristics. This may be of particularly high risk when the tools represent a black box with no clear explanation of how it works (Rudin and Ustun, 2017). Without appropriate safeguards, letting machines make judicial decisions could amplify existing biases and discrimination.

# **Analyzing Disparities in Justice Systems**

Equal treatment before the law is an essential feature of democratic societies. According to the Universal Declaration of Human Rights, “All are equal before the law and are entitled without any discrimination to equal protection of the law.” As a consequence, everyone must be treated equally
2 The Data Revolution in Justice 25 under the law regardless of race, gender, color, ethnicity, religion, disability, or other characteristics, without privilege, discrimination or bias. Judges are responsible for following and interpreting the law. Thus, they play a crucial role in ensuring de facto equality of treatment under the law. De jure equality–that is, equality in laws and regulations–will not lead to de facto equality under the law if those responsible for applying the law are still affected by biases along gender, race, or other dimensions of discrimination.

Despite this principle, there is ample social evidence documenting arbitrariness, unfairness, and discrimination in judicial decision-making. Judges become more politicized before elections and more unified during war (Daniel L Chen, 2016; Berdejo and Daniel L. Chen, 2017). Politics and race also appear to influence judicial outcomes (Schanzenbach, 2005; Mustard, 2001; Steffensmeier and Demuth, 2000; Albonetti, 1997; Zingraff and Thomson, 1984; Abrams, Bertrand, and Mullainathan, 2013; Boyd, Epstein, and Martin, 2010; Shayo and Zussman, 2017), as does masculinity (Daniel L. Chen, Halberstam, and Yu, 2017), defendent birthdays (Daniel L. Chen and Philippe, 2019), football game outcomes (Eren and N. Mocan, 2016; Daniel L. Chen, 2017), time of day (Danziger, Levav, and Avnaim-Pesso, 2011; Daniel L. Chen and Eagel, 2016), name (Daniel L. Chen, Frankenreiter, and Yeh, 2016), and shared biographies (D. Chen et al., 2016) or dialects (Daniel L. Chen, Kozbur, and Yu, 2015). These biases affect the quality of decisions and may undermine citizens’ confidence in the judiciary.

There are also various papers showing clear judicial biases in laboratory environments. Judges, as humans, are also affected by behavioral biases such as anchoring, framing, hindsight bias, representative heuristics, egocentric bias, snap judgments, and inattention (Guthrie, J. J. Rachlinski, and Wistrich, 2000; J. Rachlinski et al., 2009; J. J. Rachlinski, Wistrich, and Guthrie, 2013). Thus, the primary question is not whether these problematic features of the legal system exist. Rather, the dilemma facing policy makers is what, if anything, can be done.

Predictive judicial analytics holds the promise of increasing the fairness of the judiciary. Much empirical work observes inconsistencies in judicial behavior. By predicting judicial decisions—with more or less accuracy depending on judicial attributes or case characteristics—machine learning offers an approach to detecting when judges are most likely to allow extralegal biases to influence their decision making. Thus, data may be used to understand, and address, judicial biases in decisionmaking.

As one of the key institutions that enforces social contracts and resolves the judicial system plays a vital role in addressing disparities. Based on previous literature, we propose the following framework to measure and analyze disparities in a systematic manner:

1. **Data before Decisions:** Using data before the date of the decision, does adding control variables–such as race–measured prior to a decision affect the size of the measured disparity? As an example of this, Roland G. Fryer (2016) assesses whether racial differences in police use of force arise from omitted variables associated with race.

2. **Data on Decision-Makers:** Using data with the decision-maker’s identifier, how much of the aggregate disparity is driven by which and how many of the decision-makers? For instance, Goncalves and Mello (2020) identified racial bias at the police officer–level and found that the entire discrepancy within a unit could be explained by the behavior of just 40% of the force. Hence, it is important to distill the differences to an individual level to gather a comprehensive understanding of the problem and facilitate targeted interventions.

3. **Data on Decision-Makers Linked to Preceding Decision-Makers:** Using linked data in a sequential setting across multiple decision-makers (lawyers, administrators, translators, police), how much do disparities increase or decrease at each decision node? With multi-agency
2 The Data Revolution in Justice 26 linked data from arrest to sentencing, Rehavi and Starr (2014) found that racial disparities in federal criminal sentences are no longer statistically significant. By controlling for the prosecutors’ initial charge decision, the authors found that initial charge decisions contribute to aggregate disparities, thereby highlighting a key antecedent decision-maker, namely, the prosecutor.

4. **Data after Decisions:** Using data after the date of the decision and (allowing for) different thresholds for decision-makers, are decision-makers more often “wrong” for certain groups? Arnold, Dobbie, and Yang (2017) applies this framework to bail judges. A more lenient bail judge should have a higher fraction of defendants failing to appear at court relative to a stricter bail judge. This is in fact the case for white defendants but not for black defendants.

5. **Machine Learning to Predict Decision-Makers:** Using data after the date of the decision, how does the predicted decision-maker compare with the actual decision-maker? Kleinberg et al. (2017) shows for bail judges that the predicted judge makes fewer errors than the actual judge. Using machine learning to predict bail charges could lead to a 24.8% reduction in crime and a 42% reduction in jailing. They also show that a machine learning algorithm would have reduced racial disparities.

6. **Impact of Extraneous Factors:** Using extraneous factors, are decision-makers more inattentive/affected by extraneous factors for certain groups? Eren and N. Mocan (2016) show that judges are more affected by football game outcomes when making decisions for black defendants. Daniel L. Chen, Frankenreiter, and Yeh (2016) finds that judges assign 8% longer sentences to defendants whose first initials match their own. Judges are more affected by such same-initial effects when making decisions for black defendants categorized by the police as “Negro” rather than “Black.” 7. **Long-Run Consequences of Decisions:** Using linked administrative data after the decision, what are the long-term consequences of decision-maker disparities of consequences of decision-maker disparities? Kling (2006) used the random assignment of cases to judges to assess the causal effects of sentencing lengths on subsequent life outcomes of defendants. Many studies have followed.

8. **Machine Learning to Understand Decision-Makers:** Using the text of judges’ decisions, are stereotypes reflected in their writings predictive of decisions, disparities, and early predictability? Elliott Ash, Daniel L. Chen, and Ornaghi (2020) show that word embeddings trained at the judge-level reveal differences in gender attitudes across judges as reflected in their writings. These gender attitudes are then predictive of how judges vote on gender rights cases, reverse decisions made by female judges, assign authorship to female judges, and cite female judges.

In addition, Dunn et al. (2017) conceptualize the notion of early predictability. The basic idea is that machine learning could be used to automatically detect judicial indifference–i.e., instances when the judges appear to ignore the circumstances of the case when making decisions. This information could then be used to trigger debiasing information or other interventions to prevent decisions that would undermine the fair and non-arbitrary operation of the justice system.

More generally, machine learning techniques can be used on data of any sort, and in the context of a legal decision, a wide range of data–from the weather to judge characteristics–have proven informative. Given the textual nature of the law, and the importance of argumentation and reasongiving to legal decision-making, there is a substantial amount of textual data that can be used to
2 The Data Revolution in Justice 27 examine how legally relevant and legally irrelevant factors affect legal outcomes. For example, Elliott Ash and Daniel L. Chen (2018) uses judges’ writings to predict the average harshness and racial and sex disparities in sentencing decisions. That work finds that the information contained in written opinions can improve significantly on naive prediction of punitiveness and disparity.

This information could be used to aid decision makers in ways that reduce bias in the system. For example, training programs could be targeted toward biased judges, either with the goal of debiasing or to help them learn how to use the hearing process to better advantage. Simply alerting judges to the fact that their behavior is highly predictable in ways that may indicate unfairness may also be sufficient to change their behavior. Informing judges about the predictions made by a model decision maker could help reduce judge-level variation and arbitrariness. Potential biases that have been identified in prior decisions or writing could be brought to a judge’s attention, where they could be subjected to higher order cognitive scrutiny.

Thus, data not only makes it possible to measure and understand judicial bias, but also provides different avenues to address these biases and reduce existing discrimination. Leveraging such opportunities may be essential to ensure that not only de jure equality exists, but also that there is de facto equality in the interpretation and application of the law.

# **Applications of Text as Data**

Recent innovations have generated new opportunities for empirical research on the delivery of justice. Court proceedings and rulings are now increasingly digitized, allowing the construction of large-scale data sets. The increase in text availability is particularly promising in the legal field, where legal documents are meticulously documented and play an essential role in judicial decisions. Additionally, computer scientists have developed a slate of machine learning tools that can produce interpretable data from unstructured text–including written judicial opinions–making it possible to analyze a quantity of text that would be far too large for humans to read.

The question of how to analyze texts has gained importance in social science research in recent years (Gentzkow, Kelly, and Taddy, 2017). In the past, the most common approach was qualitative, with either a deep reading of the text or a subjective coding of important themes (see Glaser, Strauss, and Strauss (2017) for an example of the latter approach). However, these approaches lack a rigorous method of replication (Ricoeur, 1981; DiMaggio, 1997) and more formal methods to analyze texts have been developed (Andrade, 1995; Mohr, 1998). Topic modeling discovers underlying topics and themes through an inductive method (Blei, Ng, and Jordan, 2003; Blei, 2012; DiMaggio et al., 2013; Mohr and Bogdanov, 2013). Another family of models learns the features of text that are predictive of some outcome, such as political ideology (Gentzkow, Kelly, and Taddy, 2017; Jelveh, Kogut, and Naidu, 2014; Elliott Ash, 2016; Elliott Ash, Morelli, and Van Weelden, 2015; Osnabr AŒgge,<sup> ̃</sup> Elliott Ash, and Morelli, 2020).

Recent approaches have gone beyond the traditional network or topic methods by mapping word relations into a high-dimensional vector space (Mikolov et al., 2013; Pennington, Socher, and Manning, 2014). This approach positions connected words close to each other in the space and can be used to recover relevant dimensions in language. A rich literature in computational social science has begun to apply these methods in many contexts (Spirling and Rodriguez, 2020).

to legal documents (Carlson, Livermore, and Rockmore, 2015; Leibon et al., 2018). Ganglmair and Wardlaw (2017) apply a topic model to debt contracts; Elliot Ash, MacLeod, and Naidu (2019) implement a syntactic parser to extract legal commitments and entitlements from union contracts. Daniel L Chen and Elliot Ash (2019) construct document embeddings for federal courts and show they recover differences between courts, over time, and across topics.
2 The Data Revolution in Justice 28 As online collections of court decisions grow, many options become available to use text for legal research. For instance, legal scholars may be able to trace evolving interpretations of legal concepts such as fault, causation, or damages, or to examine patterns in how courts handle specific types of cases or parties (Liebman et al., 2019). Given that word embeddings measure correlations between words, they can be used to detect biases in language. Elliott Ash, Daniel L Chen, and Ornaghi (2018) analyze gender bias in the language of US Circuit Court judges, finding that slanted judges vote more conservatively in gender-related cases.

Even further, NLP may be used not only to identify and understand human biases, but also to mitigate them. For instance, a possibility may be to prevent prosecutors from seeing irrelevant information about a case (such as race or ethnicity) when making an initial decision on whether to charge someone. The San Francisco District Attorney’s Office has begun to use an algorithm that automatically redacts race-related information from free-text case narratives (Williams, 2019). This approach has been described as “blind charging”, and hopes to reduce the potential racial bias in prosecutors’ initial decisions on whether to charge someone.

The promise of NLP may also be of particular relevance to support legal actors’ knowledge of jurisprudence. Legal search engines already compile jurisprudence and identify decisions that are similar to the one at hand; decisions that apply similar laws and regulations; or decisions with similar case patterns, among other characteristics. Thus, these tools bring opportunities to improve the quality and consistency of legal arguments and judicial decisions, as well as to improve the training of lawyers, judges and prosecutors. Summarization tools may also be able to extract patterns of interest, reducing the amount of work spent on inputting case information into case management systems. Even further, NLP may be able to automatize basic judicial decisions, such as those that only require human revision of documents. For instance, NLP is already being used to improve the accuracy and efficiency of formal adjudication in the United States (Engstrom et al., 2020). A tool called Insight is used to parse text in draft decisions to flag potential errors. The tool identifies weaknesses in draft opinions that are suggestive of policy noncompliance or internal inconsistencies in the decision. Thus, it aims to ensure that adjudicators properly go through the analysis required by regulations.

# **2.D Courts and Data in the Age of COVID-19**

The COVID-19 pandemic presents an extraordinary challenge for communities far beyond health care systems. Justice systems around the world have endured great upheavals as courts canceled or postponed proceedings, staggering backlogs have built up, and entrenched stakeholders have struggled to incorporate new initiatives. Amid the crisis, one notable development is the justice sector’s rapid adoption of remote work technologies, many of which were developed before the pandemic but became an operational lifeline during lockdowns. In many cases, they allowed the legal community to continue dispensing services to resolve disputes, redress violations, and empower individuals and communities from afar.

As more countries gradually come to terms with the pandemic, the transition to data-driven systems and technological applications allows for opportunities to strengthen the administration of justice moving forward. For courts, COVID-19 fundamentally altered the way trials are conducted. Starting in March 2020, courts in many countries switched to holding hearings on web-based platforms with two-way audio-visual communication (Sourdin and Zeleznikow, 2020).

congestion in courts as adjourned hearings overlap with newly scheduled hearings. Added to this, employment problems, debt, and bankruptcy are expected to drive an increase in the demand for justice as the pandemic continues to unfold (Steven and De Langen, 2020). This presents significant
2 The Data Revolution in Justice 29 risks to the effective delivery of justice throughout the world. In such a context, data systems should inform effective planning to prepare the human and physical capital that might be needed to address a future spike in cases. Machine learning tools–or even basic data analysis–may allow to estimate or predict future caseload, understand its regional variation, and identify the main reasons for adjournments. Data may also be used to understand the level of risk and urgency that a case has for the parties involved, and thus effectively prioritize cases. Data-informed decision-making therefore becomes crucial as the risk of court congestion increases during and after the pandemic.

Data-driven technologies may also be used to bring courts closer to citizens, making them more efficient and increasing trust in the institutions. For instance, given the limitations for in-person services in courts, the Chilean judiciary has developed a digital platform, Conecta, which allows citizens to access courts’ services through a video call, chat or WhatsApp. Conecta’s goal is to facilitate access to court services for the general population, as well as to generate greater confidence in the justice system among citizens. In addition, the platform aims to optimize resources that can reduce the workload of court officials. By bringing different data sources together in one platform, Conecta allows courts to effectively respond to citizens’ queries, share updates on their cases, and provide a more efficient service to litigants.

Technologies can bring this opportunity to the next level, for instance, by introducing virtual courts to resolve fast or urgent matters. Before the pandemic, Canada introduced the Civil Resolution Tribunal (CRT), which is an online tribunal that allows online dispute resolution, mostly related to small claims, property disputes and traffic accidents. Other countries such as India, Pakistan, and South Africa have rapidly introduced virtual courts to resolve urgent matters as a result of the pandemic. These technologies filled in a crucial gap in access to justice as courts closed, yet they may also bring risks to effective dispute resolution going forward. For example, a study in the United States finds that the switch to video in bail hearings in Cook County led to an average increase of 51% in overall bail amounts (Diamond et al., 2010). A rigorous evaluation of the trade-offs between faster and–potentially–more accessible justice, on the one hand, and the risks associated with online communication, cyber-security, and disparities in access, on the other, should be undertaken before adopting these technologies going forward.

The data resulting from this switch to digital systems may also open new opportunities. For instance, recordings of online hearings represent another method to evaluate and understand biases in judicial decisions. Using court recordings and 3D Virtual Reality technology to alter the defendants’ race, Bielen, Marneffe, and N. Mocan (2018) conducted a study to understand courtroom biases. The authors found that while evaluators are harsher towards defendants of their own race during the guiltinnocence decision, in the sentencing phase they tend to be more lenient towards defendants of their own race. In terms of overall bias, it was found that minorities are more likely to get convicted and are given stiffer punishments on conviction. Thus, recordings of online hearings can be leveraged to understand aspects such as racial or gender bias in hearings, and to explore potential solutions. These data may also be used to develop ways of improving the effectiveness and quality of hearings, as well as to address deficits in communication and equal representation.

One of the most salient problems facing legal systems around the world is still the lack of access to formal justice mechanisms. During the pandemic, those without legal representation remain defenseless against criminal activities. In a recent briefing, the New York University Center on International Cooperation (CIC) articulates the three factors contributing to the global justice gap: extreme conditions of injustice, including slavery, statelessness, or insecurity; unreported civil or administrative justice problems that cannot be resolved; and lack of identity or proof of property ownership that result in legal exclusion (Steven and De Langen, 2020). For justice systems to be more conscious of the gap’s human dimensions, the CIC calls for the collection of more person-centered justice data emphasizing the legal needs and victimization of the poorest and most disadvantaged members of
3 Future Directions 30 society. With better data, finite resources can be allocated to the most urgent justice needs and the people least able to access justice.

# **3 Future Directions**

In this paper, we discuss how the data revolution in justice systems may bring ample opportunities for policy makers and researchers alike to improve the justice sector and identify the mechanisms through which justice contributes to development outcomes. The following section proposes a way forward for researchers and policy makers to 1) expand the empirical evidence regarding the role of justice in development and 2) leverage data for better judiciaries.

# **3.A New Avenues for Research**

judiciaries as well as the mechanisms through which better judiciaries may influence development outcomes.

Most research is concentrated on the of the judiciary, rather than on its quality, integrity, or public access. A better understanding of the relative impact of each of these indicators would be beneficial to identifying the main barriers and opportunities to improve justice. Despite numerous academic discussions on the topic, it is unclear whether trade-offs exist between greater efficiency, quality, and access to justice. For instance, it remains an open question whether the speedier resolution of cases comes at the expense of procedural defects, or whether wider access might increase case filings and lead to a slower resolution of cases. In addition, the relative importance of each metric for different legal actors may vary widely, as judges, lawyers, and litigants might have different interests and incentives. Thus, evaluation of judicial reforms should take into consideration how key justice metrics interact and the relative importance of each of these metrics for citizens and policy makers in distinct contexts.

The closing of courtrooms has highlighted the importance of understanding how to leverage technology well. Technological applications should be subject to rigorous testing and evaluation to understand how they affect key justice metrics. Researchers should leverage NLP methods to assess the impact of automatizing the revision of judicial decisions, improve legal training, and address judicial biases. In addition, given the extent of the global justice gap, the capacity of these innovations to bring the judiciary closer to citizens is worthy of future research and evaluation.

search, a subject which is at once complex and essential to addressing instability in fragile, conflictprone and violent (FCV) settings. Given the rise in violent conflict since 2010 and the expectation for FCV countries to host up to two-thirds of the world’s extreme poor by 2030 (World Bank), further empirical evidence evaluating the role of the rule of law and the judiciary in these settings becomes of paramount importance to build safer and more prosperous societies. Future research should evaluate the relationship between formal and informal dispute resolution mechanisms and their comparative advantages in providing effective resolutions that contribute to long-term peace. The role that unresolved grievances and inaccessible justice systems may play in future conflict also deserves further attention. In contexts with a weak rule of law or a large presence of organized crime, the effect of strengthening the judiciary and its impact on peace, economic outcomes or corruption still needs to be empirically studied. Moreover, further research is needed to better understand how judiciaries affect the relationships between ethnicities. Ethnic nationalism being “the leading source of group cohesion and intergroup civil conflict” (Blattman and Miguel, 2010), a better judicial sys-
3 Future Directions 31 tem could theoretically prove very relevant at reducing ethnic tensions and deterring conflict, yet there is insufficient evidence on the subject.

A new avenue of research should evaluate the potential of a stronger judiciary to reduce corruption. To what extent can a stronger judiciary hold accountable corrupt politicians and government officials? What interventions could achieve greater judicial independence–and would they lower corruption in society? What is the specific impact of a corrupt judiciary over development outcomes such as poverty, violence and overall corruption? These questions remain unanswered. Big data may offer new ways to evaluate the independence or quality of the judiciary, for example, by identifying outlier decisions in comparison to similar decisions based on fact patterns and legal precedent. Empirical research on these topics may become promising in the fight against corruption.

For empirical evidence to truly contribute to development, greater diversity in the countries and contexts of study is essential. Our review shows that few countries–mostly in South Asia, Europe, and North America–concentrate most research on justice. There is little research on the role of justice in Sub-Saharan and North Africa, Latin America, the Middle East, and East and Southeast Asia. Extending the geographical presence of research is important for a variety of reasons. On the research side, it provides greater external validity to ongoing research and brings new perspectives, contexts and challenges into consideration. On the policy side, these regions tend to concentrate a significant number of the poor as well as the conflict-afflicted areas.

Challenges related to the unreliability and unavailability of data pose a conundrum: countries that stand to gain the most from data-driven technologies lack the resources and data infrastructure to take advantage of them. However, with sufficient training, support, and testing of new innovations to collect and analyze data there is the capacity to overcome such challenges. Addressing these will be essential to ensuring that the promise of technological innovations and data-driven decision-making reach those who need it the most.

# **3.B Data for Better Judiciaries**

To exploit the potential of the data revolution, countries should start by creating a data ecosystem that relates different sources of data to each other, thereby improving the universe of analyses that becomes possible. This requires planning ahead and making an up-front investment in data infrastructure whose benefits may not be realized until the long term. Realizing the promise of data requires another essential asset: human capital. In the legal field, engineers generally lack domain knowledge whereas lawyers tend to lack the technical skills to wrangle and analyze data. Thus, staff training and hiring may be essential to ensuring that data is used responsibly to develop effective innovations. It is through investment in human capital that countries may leverage the promise of data in justice systems for the greater good.

Once the appropriate data ecosystem and the human capital is established, we propose the following paths for the use of data, each of which are explored in greater detail below: 1) access to e-justice for citizens; 2) data for better court performance; and 3) data for better knowledge.

First, the data revolution can bring better access to e-justice for citizens and companies. Given the global justice gap in access to justice, making justice more accessible for vulnerable populations is particularly relevant in developing countries. Data may be used to identify gaps in access to justice, for example, by comparing legal need surveys to the actual disputes that are brought to courts. This may help policy makers better understand if there are specific disputes that elude the justice system, where better access to legal means of resolving disputes might be the main gap. For instance, if surveys indicate that a territory is deeply affected by domestic violence, but few cases of domestic violence are resolved in courts, this will provide evidence of a gap in cases of domestic violence being resolved in courts. Data systems may also be used to inform citizens of their prospective out-
3 Future Directions 32 comes in mediation as compared to courts, allowing them to decide the most appropriate mechanism to resolve their dispute (Sadka, Seira, and Woodruff, 2017). In terms of data-driven innovations, the Chilean example of Conecta—a platform developed during the COVID-19 pandemic to provide citizens with greater access to courts via WhatsApp—is just one among many which illustrates the potent capacity of data and creativity to facilitate electronic justice. Data integration helps bring together the documents and history of a case via the ease of sending a WhatsApp message. A comprehensive data infrastructure also introduces the ability to build chatbots that improve institutional capacity to respond to citizens’ queries and demands. This is just one example, in addition to many others—such as e-arbitration or electronic hearings—where data can bring justice closer to citizens and improve the effectiveness of the legal system while enhancing people’s trust in institutions.

Second, the data revolution may improve court performance. Data has the promise of improving the efficiency, fairness, and effectiveness of justice systems. It may be used to identify the main bottlenecks of courts, i.e., the reasons why cases get backlogged or why adjournments take place. Data systems have the potential to facilitate a better deployment of resources by diagnosing and forecasting future needs in courts and litigants’ needs. In addition to providing diagnoses, there are data-driven solutions that may improve court performance. An example is dashboard management tools and nudges to court actors. These apps allow for a wide range of options: from providing personalized feedback on best and worst-performing metrics, to motivating better performance via behavioral nudges Cooke et al. (2018). These applications can be enhanced by bringing AI to act as a support tool for court actors. AI can help predict future congestion in courts, of particular importance due to the pandemic’s effect on adjournments; detect early cases at high risk of gender-based violence; or potentially help predict recidivism (Kleinberg et al., 2018). These innovations may come with risks and trade-offs that need to be evaluated and calibrated, yet the promise of the applications make them worthy of a proper assessment. Finally, data may also be used to identify and address judicial biases and inconsistencies, including gender and racial discrimination or snap judgments, among others. We have presented in this paper an 8-step approach by which data integration may help identify such biases.

Third, the data revolution may be used to create better knowledge within and across justice systems. Data has the promise to improve the quality and accessibility of judicial decisions, and promote best practices among legal actors. As a diagnostic tool, data may help evaluate the text of judicial decisions, identify inconsistencies in writings, and create tools for better training of judges. Specific innovations may include, for example, legal search engines that allow judges and court actors to find cases with similar fact patterns or legal citations, thus speeding up the process–and ideally the quality–of legal decisions. This data may also be used to train judges based on the history of their past decisions in comparison to their peers. In addition, data may not only help legal actors develop a knowledge base for their own decisions, but may also help to create and share knowledge across peers. For example, the sharing of advice in a systematic manner across mediators or judges may create a set of best practices that others in the profession can use to improve the quality and efficiency of their services. This knowledge can scale up beyond cases in a country, and actually refer to laws across countries. The data revolution may bring the possibility to identify best legal practices or "missing laws" in any specific country by developing an empirically-based method to analyze and compare existing regulations to enhance a country’s legal system.

Overall, by collecting data, measuring indicators, evaluating performance and testing innovations, judiciaries around the world can improve the efficiency, access to and quality of justice. Data systems and technological innovations bring the opportunity to improve the functioning of justice systems while also evaluating the impact of justice reforms. Such opportunities should not be overlooked, particularly in developing countries, given the crucial role that justice and legal institutions play in people’s lives and the prosperity of nations.