# Information Technology and the Efficiency of the Brazilian Judiciary System

André Andrade, Luiz Antonio Joia

<figure><img src="image_1.png" /></figure>

## To cite this version:

André Andrade, Luiz Antonio Joia. Information Technology and the Efficiency of the Brazilian Judiciary System. 11th International Conference on Electronic Government (EGOV), Sep 2012, Kristiansand, Norway. pp.126-136, 10.1007/978-3-642-33489-4_11. hal-01543575

HAL Id: hal-01543575

https://inria.hal.science/hal-01543575

Submitted on 21 Jun 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Information Technology and the Efficiency of the Brazilian Judiciary System

André Andrade<fnref n="1" /> and Luiz Antonio Joia<fnref n="2" />

andre.andrade@amaisb.com.br

luiz.joia@fgv.br

**Abstract.** This article presents an analysis of the impact of Information Technology (IT) investments in the efficiency of the Brazilian Judiciary System. In order to conduct this investigation, it was adopted the case study method to deal with the complexity of the aforementioned phenomenon. The organizational structure and the informatization trajectory of the Brazilian Judiciary System, the legal framework for electronic lawsuits, as well as the role of the National Council of Justice (NCJ) on the automation of the Brazilian Judiciary System form the basis for understanding the context. A quantitative analysis of the correlation between IT investment and the efficiency of the courts shows a potential positive influence of IT on reducing the duration and cost of lawsuits, thereby increasing the operational and financial efficiencies of the Brazilian Judiciary System.

**Keywords:** Judiciary System; e-Government; Brazil; electronic lawsuit; IT investment

## 1. Introduction

As the computerization of the Brazilian Judiciary System in Brazil evolves, e-government becomes an important tool to promote the access of Brazilian citizens to justice. In the meantime, units of the Brazilian Judiciary System are investing in Information Technology (IT) to build the infrastructure necessary to provide e-government services [1]. The adoption of the New Public Management paradigm in Brazil has identified e-government as a path to be followed by the Judiciary System. The continuing development of e-government increases the need for a restructuring of the state to provide these services in terms of routines and processes that need to be eliminated or modified through the use of IT tools [2],[3].

Strategic planning of the Brazilian Judiciary System, coordinated by the National Council of Justice (NCJ), focuses on IT as a tool for solving the efficiency problems of the Brazilian Judiciary System [4]. IT investment of the Brazilian Judiciary System might be evaluated by several indicators from political goal-based ones, such as governance, to technical-based ones, such as software performance. Transparency,

<footnote n="1">¹Brazilian School of Public and Business Administration at Getulio Vargas Foundation, Rio de Janeiro, Brazil</footnote>

<footnote n="2">²Brazilian School of Public and Business Administration at Getulio Vargas Foundation, Rio de Janeiro, Brazil</footnote>
info-inclusion, equity, quality, efficiency, capability, accountability, maturity, infrastructure, standardization, interoperability, availability and usability are just some of these performance indicators [5],[6],[7],[8],[9].

As Brazil is implementing its latest judicial reform, led by the NCJ, empirical research becomes very important to guide these IT implementation initiatives in order to assess the actual impact of Information Technology [10],[11],[12]. In its latest phase, Brazilian judicial reform has incorporated not just legal changes but also new elements related to management issues and investment in equipment, buildings and IT [4]. Among these new elements are attention to administration and management and investment in equipment, buildings, and IT.

However, the relationship between IT investments undertaken by the Brazilian Judiciary System as planned by the NCJ and the efficiency of the Brazilian courts of justice has not been researched in Brazil so far. Thus, this article aims to examine the correlation between IT investment and efficiency of the Brazilian courts assessed by two indicators, namely operational efficiency and financial efficiency.

## 2. Method

The case study method, described by [13], was chosen to conduct this research, in order to explore and describe a phenomenon in its own context, when the boundaries between phenomenon and context are not clearly evident, and in which multiple sources of evidence are used [13],[14],[15]. The phenomenon to be studied - the correlation between IT investment and efficiency of the courts - is intricately connected to political, social, historical, and personal issues, providing wide ranging possibilities for questions and adding complexity to the case study [16]. The case study method conducted here follows four stages, namely designing the case study, conducting the case study, analysis of the evidence, and report writing [17],[18].

The unit of analysis for the case study is the Brazilian Judiciary System and the sub-units of analysis for the quantitative research developed were the State Courts [19]. Given the wide variety of the administrative units of the Brazilian Judiciary System and their differences, it was necessary to select a population that can be compared. There are 27 State Courts in Brazil that are similar in their attributions, which provide a large sample with similar characteristics in order to isolate the phenomenon under scrutiny. This choice allows a comparison between the sub-units of analysis, but does not remove the appearance of a single case study [13].

Although the choice of the case and sub-units of analysis is restrictive, it does not imply abandoning the possibility of generalizing the results obtained. The choice of the Brazilian state court as an object of study was necessary to establish efficiency comparisons. However, the results obtained in relation to the state courts can be generalized for the entire Brazilian Judiciary System, or even other countries, at least those with similar procedural dynamics.

The case study followed an exploratory/explanatory approach [13],[14], in which the analysis of the evidences was undertaken in three stages [20]. First, the organizational structure of the Brazilian Judiciary System is unveiled in order to make clear where the phenomenon under research is located, via the analysis of several documents issued by the Brazilian Judiciary System. Second, the informatization
process and stage of the Brazilian Judiciary System are set forth, including the
implementation of the electronic lawsuit, through the analysis of documents
developed by the NCJ. Finally, data was collected for a quantitative analysis, by using
the annual report "Justice in Numbers" issued by the National Council of Justice [21],
in order to investigate a statistical correlation between the IT investments and the
efficiency of the State Courts of Justice in Brazil, via data analysis of time series
available from 2004 to 2010.

The congestion of lawsuits in courts (percentage of lawsuits waiting for a judge’s sentence) is the best available proxy for operational efficiency, because it embodies both the regional characteristics and the number of lawsuits in a given administrative unit. It also provides the efficiency characteristics, as the court performance in judging the lawsuits that come before it. Besides, the cost per lawsuit judged (average cost of each lawsuit judged in an administrative unit) is the best proxy for financial efficiency that can be calculated from NCJ indicators.

As such, these are the two indicators used in this work to assess the IT-enabled
efficiency of the Brazilian Judiciary System.

3. The Brazilian Judiciary System

The Brazilian Judiciary System consists of a complex combination of nature of
lawsuit, physical location and level of jurisdiction [12]. The jurisdiction for deciding a
certain issue is based on a combination of the right in dispute (nature of lawsuit),
location of the dispute (physical location) and level of jurisdiction of the judge (level
of jurisdiction). Although the explanation may be simple, the multitude of possible
combinations offers a complexity of options that goes beyond the number of
administrative autonomous units of the Brazilian Judiciary System.

Because Brazil is a federative republic, the basis of the Brazilian Judiciary System
lies at state level. The macro-organizational structure of the Brazilian Judiciary
System is established in Title IV, Chapter III, Section I, Article 92 of the Brazilian
Constitution (Figure 1): "Art. 92. The organs of the Judiciary Power are as follows: I -
the Federal Supreme Court; I - the National Council of Justice; II - the Superior Court
of Justice; IV - the Courts of Appeal and Labor Assizes; V - the Courts of Appeal and
Electoral Assizes; VI - the Courts of Appeal and Military Assizes; VII - the Courts of
Appeal and State, Federal District and Territorial Assizes" [23]. Due to this division,
Brazil has more than 100 autonomous administrative judiciary units [4]. Besides the
federal and state justice, there are three more specialized instances of jurisdiction:
labor, electoral and military.

With 26 states and a federal district, each one with its own structure for almost all
of the 5 different instances of jurisdiction, coordination is no simple task. In this
fragmented environment, information systems were developed in an uncoordinated
manner, according to the local internal needs of the organizations [1]. Rarely were the
different stakeholders interested in the Judiciary administration heard [4]. This
resulted in several concurrent and non-interoperable systems for lawsuit automation,
with low knowledge-sharing and high costs [1].
In 2004, Constitutional Amendment No. 45 created the National Council of Justice (NCJ) and established the constitutional right to a “reasonable” duration of the judicial process (Brasil, 2010b). While the “reasonable” duration established a constitutional right with no adequate means to ensure its accomplishment, the creation of the NCJ was a response to the social demand for an external control of the Brazilian Judiciary System, given the broad autonomy of its units [1].

The attributions of the NCJ include defining strategic planning and the goals and programs for institutional evaluation of the Brazilian Judiciary System. Strategic planning could be found in units of the Brazilian Judiciary System since the early 1990s [24]. Most of these initiatives were isolated and/or discontinued in the course of time. The isolation was given by the fragmented structure of the Brazilian Judiciary System and the discontinuity was caused by the short duration of the administrations of the Judiciary units, limited to a two-year term [22].

<figure><img src="image_3.png" /><figcaption>Figure 1: Organizational Structure of the Brazilian Judiciary System</figcaption></figure>

## 4. IT in the Brazilian Judiciary System

According to [25], there are three stages in the virtualization of working processes. First, the value chain is still physical, though there is the use of electronic tools such as word processors, spreadsheets and simple databases. Second, automation becomes part of the activities associated with the execution of working processes. Third, the value chain is fully digital with intensive use of IT.

The automation of the Brazilian Judiciary System is more than three decades old [1]. However, during this time there was hardly any coordination between the various individual initiatives. Indeed, until recently no coordinated IT planning was detected and separate information systems were developed for each and every unit [26]. This problem is even more acute in the state courts, given the administrative autonomy of each state.
The evolution of the computerization of the Brazilian Judiciary System is fully compatible with Tapscott’s model [1]. First, judges and civil servants use word processors and simple databases to type decisions and hearings and register information on the progress of lawsuits. Second, the courts implement information systems to control the progress of lawsuits, which [25] defines as “control of working processes”, and early steps of automation. Third, the virtualization of lawsuits referred to as electronic lawsuits takes place, when the courts start to implement a fully digital value chain, with intensive use of IT, including e-government tools.

All the administrative units of the Brazilian Judiciary System have completed the first phase, and most of them have also implemented phase two. In rare cases, part of the lawsuits of a given court is not controlled through an information system. All of them have initiated the third phase (digital value chain), though none of them have completed it yet. It is expected that, by the end of 2012, the administrative units of smaller states will have completed this task, with all their lawsuits in electronic format

The increasing computerization of the Brazilian Judiciary System, motivated among other things by the desire to speed up judicial lawsuits, is a solution involving technical procedures rather than new lawsuit routines per se [27], [28]. Undoubtedly, the most important initiative in the field of e-government by the Brazilian Judiciary is the so-called electronic lawsuit. Although it is known by this name, it is a lawsuit in a differentiated physical medium with the same rules as traditional lawsuits, rather than a new type of lawsuit. Instead of the court dockets being on paper, the lawsuit is processed using electronic means. In other words, the standard principles and rules of judicial lawsuits are maintained, though documents that were stored on paper (and often generated via electronic media) are stored and managed electronically.

After the enactment of Federal Law 11.419/06 the implementation of the electronic lawsuit has been essentially pragmatic. The control of routines such as the distribution of initial briefs, fulfillment of court orders, accompaniment of lawsuits, publication routines, scheduling of hearings, among others, undoubtedly benefit from computerization [29]. The first instances of electronic lawsuits were implemented by isolated courts in the various autonomous units, often based on previously available lawsuit information systems [1]. In many cases, there is more than one system in place in each court [1].

In order to clear up this problem, the NCJ developed the Digital Judicial Lawsuit (PROJUDI) [22]. However, the autonomous administrative units of the Brazilian Judiciary System use different versions of the system and have a high degree of freedom to customize them. This situation repeats the same model of decentralized development, with high costs and low knowledge sharing, although on a common platform. The unification of the platform can contribute to reducing the problems of communication and interoperability that need to be overcome in order to achieve the fifth (seamless) stage of e-government initiatives, in accordance with UN/ASPA standards [30].

**5. IT Investment and the Brazilian State Courts Efficiency**
Although many existing works address many aspects of the problem of efficiency of the Judiciary, none of them links efficiency to the use of IT [10],[11],[12]. Given the constraints already discussed in the methodology, data analysis was conducted to establish if there is a reliable correlation between IT investment and state court efficiency. IT investment in the Brazilian Justice System refers to any of the possible items previously referred to: hardware, storage, software purchase, system development, network, Internet access, management and training.

The analysis of the correlation of IT investment and court efficiency was conducted by considering two dimensions, namely operational efficiency and financial efficiency [31],[32]. As already said, the congestion of the courts was selected for measuring operational efficiency, while the cost of lawsuits judged evaluates financial efficiency [33]. The selection of both the dimensions and their measurement variables is justified by the NCJ strategies and goals [34],[35]. The definition of the variables and its abbreviations are those provided by the NCJ, where available [34]:

* State population (H1) – number of inhabitants, according to data from the National Institute of Geography and Statistics (IBGE).

* Court budget (Dpj) – expenditure of a court in a given year, excluding expenses from previous periods.

* IT investment (Ginf) – all investment in IT resources, including those funded by third-parties.

* Total of sentences (Sent) – number of judicial sentences handed down in a given year.

* Congestion (tc) – Number of lawsuits awaiting a judge's sentence in relation to lawsuits in progress (lawsuits awaiting judgment plus new lawsuits). It is calculated by using the equation: tc = 1 - (Sent / (CN + Cpj)), where CN is the number of new cases in a given year, and Cpj is the number lawsuits carried over without judicial sentence from the previous year.

* Cost per lawsuit judged (DpjSent) – average cost of lawsuits judged in a given year. This is calculated by dividing the court budget (Dpj) by the number of judicial sentences handed down in a given year (Sent).

It is important to note that the higher the congestion (tc) is, the lower the operational efficiency. The same happens with the cost per lawsuit judged, i.e. the higher the cost, the lower the financial efficiency. Because of this, both variables are expected to have a negative correlation on IT investment. In other words, IT investment is supposed to lead to a reduction in both congestion and costs.

Brazilian states differ greatly in terms of environment and local conditions and feature a broad variance in important indicators such as population, number of municipalities, revenue, budget, and others. It is important to note that the budgets of the states in Brazil are heavily influenced by transfers from the federal government, especially in poorer states, and therefore do not necessarily reflect the economic activity of the state. The budgets of the State Courts (Dpj) are a percentage of the state budget (@ GT), which in 2010 ranged from 3.5% to 12.1% (G2) [21]. IT investment (Ginf) is more irregular and varied in 2010 between 0.1% and 4% (infl) of the court budget (Dpj) [21]. The tool chosen to reduce the regional inequalities was balancing the variables by the state's population (H1). This is expected to narrow environmental
differences, since it is impossible to isolate all local variables that affect the functioning of the state courts to calculate its efficiency.

IT investment per capita (GinfH1) was calculated for each of the seven available years (2004 to 2010) using the equation GinfH1 = GInf / H1. The measurement of the congestion was limited to the regular courts, given the inherent difficulty in comparing these data with data both from appeal courts or small claims courts. No transformation was made to congestion (tc) or cost per lawsuit judged (DpjSent) because these are proportional measures. Then, to reduce the effects of annual fluctuations, all the variables were converted into an average for the seven available years. An average for Ginf for each state was found by adding up the results for Ginf and dividing by seven (µGinfH1 = (Σ GinfH1) / 7). The same was done to find an average for the congestion (tc) for each state (µtc = (Σ tc) / 7) and an average of cost per lawsuit judged (DpjSent) for each state (µDpjSent = (Σ DpjSent) / 7).

The analysis of the histograms of the transformed variables showed that none of them revealed normal distribution. Thus, Spearman's rho correlation was adopted because it does not require that data are from a normal population. The variable chosen to proxy the IT investment (µGinfH1) was then tested to establish if there was a negative correlation with the proxies for state court operational efficiency (µtc) and financial efficiency (µDpjSent), according to the hypothesis stated previously, namely that IT investment has a positive influence on efficiency. As a result, the one-tailed test of significance was selected and undertaken via SPSS Version 13 (see Tables 1 and 2).

Correlations

<table><thead><tr><th></th><th></th><th></th><th>mGinfH1</th><th>mtc</th></tr></thead><tbody><tr><td rowspan="3">Spearman's rho</td><td rowspan="3">mGinfH1</td><td>Correlation Coefficient</td><td>1,000</td><td>-,538**</td></tr><tr><td>Sig. (1-tailed)</td><td>.</td><td>,001</td></tr><tr><td>N</td><td>27</td><td>27</td></tr><tr><td rowspan="3">mtc</td><td rowspan="3">Correlation Coefficient</td><td>Correlation Coefficient</td><td>-,538**</td><td>1,000</td></tr><tr><td>Sig. (1-tailed)</td><td>,001</td><td>.</td></tr><tr><td>N</td><td>27</td><td>27</td></tr></tbody></table>

<caption>Table 1: Spearman's rho correlation between average population-weighted IT investment (µGinfH1) and operational efficiency (µtc)</caption>

<footnote>**. Correlation is significant at the 0.01 level (1-tailed).</footnote>
<table><caption>Correlations</caption><thead><tr><th></th><th></th><th>mGinfH1</th><th>mDpjSent</th></tr></thead><tbody><tr><td rowspan="3">Spearman's rho</td><td>mGinfH1</td><td>Correlation Coefficient</td><td>1,000</td><td>-,425**</td></tr><tr><td></td><td>Sig. (1-tailed)</td><td>.</td><td>,008</td></tr><tr><td></td><td>N</td><td>27</td><td>27</td></tr><tr><td rowspan="3">mDpjSent</td><td>Correlation Coefficient</td><td></td><td>-,425**</td><td>1,000</td></tr><tr><td>Sig. (1-tailed)</td><td></td><td>,008</td><td>.</td></tr><tr><td>N</td><td></td><td>27</td><td>27</td></tr></tbody></table>

<caption>Table 2: Spearman's rho correlation between average population-weighted IT investment (µGinfH1) and cost efficiency (µDpjSent)</caption>

The analysis of the outputs considered the highly restrictive significance level of alpha = .01. This means that the odds that the correlation is a chance occurrence are no more than 1 in 100. It was also adopted Cohen's criteria for interpretation of a correlation coefficient [36].

In both cases, there is a medium to high correlation by Cohen's criterion (see Table 3) within the confidence interval: -0.538 between the average spending on IT per capita (µGinfH1) and the measure of operational efficiency - average congestion (µtc) - with a ρ-value (denoted by Sig.) of 0.001; and -0.425 between the average spending on IT per capita (µGinfH1) and the measure of financial efficiency - average cost per case decided (µDpjSent) - with a ρ-value (denoted by Sig.) of 0.008. Therefore, one should reject the null hypothesis that ρ = 0, i.e. reject the hypothesis of no correlation and support the hypothesis of its existence. Besides, both are negative correlations. As such, assuming that the relationship is causal in the sense that spending on information technology influences congestion and cost, and not the other way around (although any expense does influence the cost), one cannot reject the hypothesis that IT investment has a positive effect on the efficiency of the Brazilian Judiciary System.

<table><caption>Table 3: Cohen's criteria for interpretation of a correlation coefficient</caption><thead><tr><th>Correlation</th><th>Negative</th><th>Positive</th></tr></thead><tbody><tr><td>Small</td><td>-0.3 to -0.1</td><td>0.1 to 0.3</td></tr><tr><td>Medium</td><td>-0.5 to -0.3</td><td>0.3 to 0.5</td></tr><tr><td>High</td><td>-1.0 to -0.5</td><td>0.5 to 1.0</td></tr></tbody></table>

<footnote>**. Correlation is significant at the 0.01 level (1-tailed).</footnote>
## 6. Conclusion

The main goal of this research was to analyze e-government being enabled by IT investment of the Brazilian Judiciary System via the investigation of the impact of IT investments undertaken by the Brazilian Judiciary System on the efficiency of the Brazilian Courts of Justice. For this reason, an embedded single case study [13] was performed. Besides, this investigation intended to tackle one of the themes of the eGovRTD2020¹, namely assessing the value of government IT investments.

Analysis of the role of the NCJ in the strategic planning of the Brazilian Judiciary System shows that it focuses on the efficiency of the courts and its capacity to judge the lawsuits in a timely manner. In order to accomplish this, the NCJ has chosen IT as one of the main tools (management being the other one).

As such, the NCJ’s focus on operational and financial efficiency established the parameters for defining the variables to evaluate the effects of IT investment on the Brazilian Judiciary System. The existence of a medium to high correlation between IT investment and both measures of efficiency (average congestion and average cost per lawsuit judged) within a narrow confidence interval, shows that IT investment has a clearly positive influence on the efficiency of the Brazilian State Courts. As such, faster and less expensive lawsuits can become a reality and foster access to justice.

However, there are some research limitations in this work that must be addressed. How long does it take for the IT investment to mature and have the expected effects on efficiency? It is a difficult question to answer mainly if one doesn’t know the structure of the IT investment. Investment in software development, notably in the early stages, has uncertain returns. Investment in training has a more rapid return. The solution adopted here was to consider all the time series available (seven years) to evaluate the results of medium-term investment in IT. This approach was designed to reduce the effects of the considerable fluctuations in IT investment from year to year within the same court but must be considered as a research limitation.

Another research limitation in this study was the use of data directly related to the activities of the Brazilian Judiciary System. It was the best approach available for the purposes intended, despite the fact that the currently available data do not cover all aspects of efficiency, such as correct and non-biased decisions, and promoting access to justice.

Despite that, the analysis of the data related to the role of IT investments undertaken by the Brazilian Judiciary System shows that there is an opportunity for the Brazilian Judiciary System to provide higher levels of e-government services through the Internet, such as electronic lawsuits. However, providing access to justice through e-government is no easy task, as there are several restrictions to the use of e-government services, particularly by the lower classes [7],[37].

<footnote n="1">¹ eGovRTD2020 is a project co-funded by the European Commission under the 6th Framework Programme of Information Society Technologies. It seeks to project the scenario of e-government in 2020 and thereby identify future strategic research fields for the development of e-government and the public sector per se. See at http://www.egovrtd2020.org/EGOVRTD2020/navigation/work packages/wp4_roadmapping/it_value</footnote>
Lastly, future research initiatives in this realm must investigate not just the impact of IT investments on the efficiency of the Brazilian Judiciary System but also on the accountability of the Brazilian State Courts. To [38], this concept may be understood as a question of democracy. That is, the more advanced the democratic stage, the greater the interest in democracy. Thus, government accountability tends to follow the advance of democratic values such as equality, human dignity, participation and responsibility. As such, it is important to further investigate the role of IT on this issue.

## References

[1] Andrade, A., Mallet, J. P., & Fleury, N. (2008). Modelos concorrentes de automação processual. *Revista de Direito das Novas Tecnologias, 4*, 21-30.

[2] Ruediger, M. A. (2003). Governança Democrática na Era da Informação. *Revista de Administração Pública, 37*(6), 1257-1280.

[3] Fountain J. (2005). *Building the Virtual State: Information Technology and Institutional Change*, Brookings Institute Press, Washington D.C.

[4] Andrade, A. (2009). The Strategic Planning and ICT in the Brazilian Justice. *Proceedings of the 3rd International Conference on Theory and Practice of Electronic Governance, ICEGOV 2009*, ACM ICPS 322, Bogota.

[5] Pinho, J. A., Iglesias, D. M. Iglesias, & Souza, A. C. P. (2005). Governo Eletrônico, Transparência, Accountability e Participação: o Que Portais de Governos Estaduais no Brasil Mostram. *Anais do XXIX Enanpad*, Brasília.

[6] Soares Junior, J. S., & Santos, E. M. (2007). Governança Eletrônica: uma Perspectiva Sociotécnica das Organizações Públicas a partir da Padronização e Interoperabilidade. *Anais do XXXI Enanpad*, Rio de Janeiro.

[7] Joia, L. A. (2007). Inclusão Digital no Brasil: um Modelo Heurístico de Natureza Dinâmica. In: Martins, P. (Org.). *Estado e Gestão Pública: Visões do Brasil Contemporâneo*. Editora FGV, Rio de Janeiro.

[8] Magoutas, B., & Mentzas, G. (2009). Refining, Validation and Benchmarking of a Model for E-government Service Quality. In: Wimmer, M. A. et al (eds), *EGOV 2009, LNCS 5693*, 139-150.

[9] Solar, M., Astudillo, H., Valdes, G., Iribarren, M., & Concha, G. (2009). Identifying Weaknesses for Chilean E-government Implementation in Public Agencies with Maturity Model. In: Wimmer, M. A. et al (Eds.), *EGOV 2009, LNCS 5693*, 151-162.

[10] Hammergren, L. (2009). *Diagnosing Judicial Performance: Toward a Tool to Help Guide Judicial Reform Programs*. Working Paper, World Bank, New York. Retrieved March, 15, 2009, from
[26] Andrade, A. (2008). Por que a Justiça não se Comunica? Um Problema de Estrutura Organizacional. *Alfa-Redi: Revista de Derecho Informático/Computer Law Magazine*, AR: RDI, 121.

[27] Duarte, F. C. (2002). Reforma do Judiciário: por um Novo Paradigma. Vol. II. Curitiba, Juruá.

[28] Silva, A. (2004). *A Reforma do Judiciário*. Del Rey, Belo Horizonte.

[29] Marcacini, A., & Costa, M. (2008). Questões sobre a vigência do parágrafo “único” do artigo 154 do CPC. Retrieved May 22, 2008, from http://augustomarcacini.cjb.net/.

[30] Ronaghan, S. A. (2002). Benchmarking “E-government: a Global Perspective”. United Nations Division for Public Economics and Public Administration, and American Society for Public Administration, New York, May 2002. Retrieved May, 19, 2009, from http://unpan1.un.org/intradoc/groups/public/documents/UN/UNPAN021547.pdf.

[31] Smith, P., & Street, A. (2005). Measuring the Efficiency of Public Services: the Limits of Analysis. *Journal of the Royal Statistical Society: Series A (Statistics in Society)*, 168(2), London, Mar., 401-417.

[32] Mandl, U., Dierx, A., & Ilzkovitz, F. (2008). *The effectiveness and efficiency of Public Spending*. Economic Papers. 301, European Commission Directorate-General for Economic and Financial Affairs, Brussels, Feb.

[33] Vera Institute for Justice. (2003). *Measuring Progress Toward Safety and Justice: A Global Guide to the Design of Performance Indicators across the Justice System*. Vera Institute for Justice, New York.

[34] Brasil. (2009). Conselho Nacional de Justiça. *Resolução n.º 70*, Mar. 18.

[35] Brasil. (2010). Conselho Nacional de Justiça. Metas Prioritárias 2010. Retrieved May 29, 2010, from http://www.cnj.jus.br/index.php?option=com_content&view=article&id=10350&Itemid=1125.

[36] Cohen, J. (1988). *Statistical Power Analysis for the Behavioral Sciences*, Academic Press, New York, 2nd edn.

[37] Joia, L.A. (2005). A Framework for Developing Regional E-Government Capacity-Building Networks. *Information Technology and International Development, 2*(4), 61-73.

[38] Campos, A.M. (1990). Accountability: Quando poderemos traduzi-la para o português? *Revista de Administração Pública, 24*(2), 30-50, Feb.-April.