In the context of reforms of public administration, pressure has been put on public organizations to improve efficiency, by reducing budgets, raising expectations about the quality of services provided, and introducing performance-based management techniques (Boyne et al., 2005; Osborne and Gaebler, 1993). Assessment of performance of public organizations is now a key aspect in public administration (Boyne, 2004). In recent decades, procedure and technical innovations have been adopted in Brazilian courts. The most important of these innovations are those supported by information and communication technologies (ICTs) and, more specifically, the electronic judicial process. It is important to understand the extent of the effects caused by and the resources involved in the adoption of an innovation by the judiciary, as exemplified by the electronic judicial process, because it is a subject that has been under-explored (Sousa and Guimaraes, 2014).

The objectives of this paper are: (a) to develop and test a theoretical and empirical model that explains the relationships between resources, innovation, and performance in courts and (b) to measure the observed performance (variation in efficiency and adoption of innovations) of 24 Brazilian labor courts as a function of the resources and level of innovation in those courts. The study examines the attributes that may have an impact in the performance of Brazilian labour courts and thus contributes to knowlegde about judicial administration, providing evidence for improvement of the management of courts and the development of better policies for the judiciary. The judiciary attached to labor courts was chosen because it has the following relevant characteristics: shorter length of the judicial process (Dias Júnior, 2004) and; high level of innovation adoption (Costa, 2008).

The Brazilian labor courts are comprised of: (i) first trial courts, or courts in the first-degree, where a judge sitting alone decides the outcome of a case; (ii) Regional Labor Courts, or courts in the second-degree; and (iii) the Superior Labor Court, or the court of final appeal. The judicial process in courts in the first-degree begin when an application is assigned to a judge. The first stage requires an attempt at conciliation. If conciliation is successful, the judge records the conclusion. If conciliation is unsuccessful, the judge decides the case. Courts in the second-degree courts can judge new applications or may receive appeals against decisions of the courts in the first-degree. The new cases relate to specific issues that arise in situations such as collective bargaining among others. New cases may be decided by a judge sitting alone, while appeals are reviewed by a panel of judges that constitute the appeal court if one party appeals a decision. Rulings of the courts in the second-degree may be appealed, in which case they are referred to the Superior Labor Court.

There are in Brazil 1,568 courts in the first-degree and 24 Regional Labor Courts (TST 2015). 32.6% of lawsuits are presented in electronic format. There are 3,371 labor judges, 36,997 permanent officers and administrators of the courts. In 2013 the court system handled 7.9 million lawsuits, 3.9 million pending cases and 4 million new cases (CNJ, 2014).

The literature on the performance of the judiciary identifies several variables that contribute to the provision of legal services. The production frontiers approach has been developed using the technique called Data Envelopment Analysis (DEA) in several studies that evaluate the efficiency of courts (Deyneli, 2012; Kittelsen and Førsund, 1992; Lewin, Morey and Cook, 1982; Pedraja-Chaparro and Salinas-Jimenez, 1996; Schneider, 2005; Yeung and Azevedo, 2011). The assessment of efficiency through stochastic frontier analysis, a different technique that was used by Castro (2011) and Schwengber (2006), is less commonly used.

The DEA technique allows the assessment of performance through relative composite measures of efficiency, and does not require previous knowledge of prices, costs, and relationships among the variables. In addition, it makes it possible to assess one organizational decision-making unit (DMU) of analysis in comparison to other units to identify the most productive ones. These most productive units are then used as references, also described as “best practice” (Zhu, 2009). In this study, each court is regarded as a decision-making unit (DMU).

The index proposed by Malmquist (1953) makes it possible to investigate the variation in technical efficiency and technological progress in a dynamic manner over time. It also makes it possible to decompose the multiplicative effect on the total productivity variation index into separate components of technical efficiency and innovation adoption - technological change (Cooper, Seiford and Tone, 2007). The variation in efficiency, also known as catch-up, is measured by the distance between the index of a given court and the production frontier. Thus, catch-up represents the extent to which the efficiency of the court has improved or deteriorated over time (Cooper, Seiford and Tone, 2007). Technological change (known as frontier-shift), which is a result of the adoption of innovations, is assessed by the displacement of the optimal frontier of pooled resources, and is measured by the geometric mean of change between two periods (Färe et al., 1994). This index was used in research on the judiciary by Kittelsen and Førsund (1992). The advantage of using this index is that the impact of the adoption of innovation can be evaluated in a longitudinal approach.

Turning to the other possible approach, stochastic frontier analysis requires the development of a more rigorous econometric model to explain the extent of variation in judicial performance by the variation in the productive resources of the judiciary. This technique makes it possible to assess the factors that affect the efficiency and the decomposition of the error, to identify systematic effects on efficiency (Castro, 2011). This technique also makes it possible to compare performance with operations that are a reference. So, we decided to take advantage of the strengths from both approaches, using Malmquist index, a non-parametric modeling, and stochastic frontier, a parametric one.

Box 1 presents a non-exhaustive summary of the variables found in the literature that are employed in the model used in this study. The first column contains indicators that exhibit dependency relationships in parametric models, in addition to the resource or result in non-parametric models. The second column shows the variables. The third column lists the studies that support the findings that are listed.

Box 1 Source: Elaborated by the authors

The basic quantitative model used in these studies, increased or reduced, mainly depending on the availability of data, has the relationship presented in equation 1.

y = f ( P e n d i n g ,   n e w , M a g i s t r a t e , S t a f f ) (1)

Where Y, the dependent variable, is understood as completed/resolved cases. The independent variables are Pending, pending cases; New, new cases assigned to judges; Magistrate, the number of judges; and Staff, the number of support staff. The variables Pending and New are related to the workload of the judge.

The application of Information and Communication Technologies (ICT) by the judiciary have been studied both in relation to the effect on performance and terms of the adoption of innovation (Sousa and Guimaraes, 2014). The recurring themes are the impact of these technologies on judicial performance (Crunkilton, 2009; Joia, 2008, 2009; McKechnie, 2003; Rosa, Teixeira and Pinto, 2013; Saman and Haider, 2013; Velicogna, Errera and Derlange, 2011, 2013), e-government (Joia, 2008, 2009; McKechnie, 2003), systems and databases (Hara, 2007; Rosa, Teixeira and Pinto, 2013), and the electronic judicial process (Velicogna, Errera and Derlange, 2011, 2013).

Studies that address performance in the judiciary can contribute to the evaluation of innovation through the measurement of parametric and non-parametric relationships among the main resources of the judiciary and their performance. Studies that aim to evaluate the effect of innovation on judicial outcomes use measures that have already been tested and validated in different contexts (Sousa and Guimaraes, 2014). The pressure caused by the increase in judges’ workload has been shown to have an impact on the performance of courts (Beenstock and Haitovsky, 2004; Dimitrova-Grajzl et al., 2012; Lewin, Morey and Cook, 1982; Rosales-López, 2008), so that the following hypothesis can be formulated:

The organization size variable is traditionally used in innovation adoption studies (Boyne et al., 2005; Damanpour and Schneider, 2009; Damanpour and Wischnevsky, 2006; Rogers, 2003). As suggested by Rogers (2003), this variable would be positively related to innovation adoption, and therefore, large courts would be more likely to adopt an innovation. On the performance perspective is expected that large organizations would be more efficient (Tulkens, 1993), in this way, large courts would increase performance (Rosales-López, 2008). Thus, the following hypothesis is suggested:

To assess innovation in labor courts, the variable electronic process index (II) was used as an indicator and represents the percentage of electronic processes adopted by the labor judiciary, together with, as a proxy measure, total investment in ICT. These variables for measuring innovation in the judiciary have severe limitations, but it is difficult to identify indicators that assess the indirect and final impacts, as has previously been noted by Rauen (2013).

The variable investment in ICT represents the share of resources invested by the court in activities involving the electronic process, and the electronic process index is related to the ability of the court to operationalize innovation. There is no consensus on the relationship between ICT investment and organizational performance. The paradox, first identified by Solow (1987) is that there is a negative relationship between these variables, even though there are studies that indicate that such investments may contribute to improve performance in organizations (Stratopoulos and Dehning, 2000). In the public sector, investment in ICT can improve the efficiency of the services provided (Foley and Alfonso, 2009; Lee and Perry, 2002). Nevertheless in the judiciary there is evidence that this investment has both positive (Soares and Sviatschi, 2010) and negative (Buscaglia and Ulen, 1997; Deyneli, 2012) impacts on court performance. Thus hypotheses 3 and 4 are proposed as follows:

Investment in training in courts can be an important resource by providing capability related to legal and managerial knowledge. Training can be a stage of the ICT implementation process (Crunkilton, 2009; Rosa, Teixeira and Pinto, 2013; Velicogna, Errera and Derlange, 2011, 2013). In addition, the adoption of technology without training may limit its impact on performance (Aragão, 1997; Buscaglia and Ulen, 1997; McKechnie, 2003; Velicogna, 2007). Yeung and Azevedo (2011) have found evidence that judicial administration - management training is one of the variables - is positively correlated with performance. As a result, the following hypothesis is proposed:

The labor judiciary in Brazil is composed of 24 courts divided into jurisdictions: 1. Rio de Janeiro; 2. São Paulo (Capital), covering the metropolitan area of São Paulo, part of the metropolitan area of the Baixada Santista region and Ibiúna city; 3. Minas Gerais; 4. Rio Grande do Sul; 5. Bahia; 6. Pernambuco; 7. Ceará; 8. Pará and Amapá; 9. Paraná; 10. Federal District and Tocantins; 11. Roraima and Amazonas; 12. Santa Catarina; 13. Paraíba; 14. Acre and Rondônia; 15. São Paulo (Campinas) cities included in the 2^nd jurisdiction; 16. Maranhão; 17. Espírito Santo; 18. Goiás; 19. Alagoas; 20. Sergipe; 21. Rio Grande do Norte; 22. Piauí; 23. Mato Grosso; and 24. Mato Grosso do Sul.

This study uses census data, covering all 24 labor court jurisdictions for courts in both the first and second-degree, and draws on secondary data available in the Justice in numbers database of the Brazilian National Council of Justice (CNJ) and data collected from the websites of the courts for the period between 2003 and 2013. It is important to notice that this Database evolved overtime, improving the collecting data method and including more variables and analysis.

The analyses were performed in two steps: 1) construction, analysis, and processing of the database and 2) performance optimization modelling and constructing a classification of courts using the Malmquist index based on DEA and econometric modelling using stochastic frontier analysis.

The collected data were structured into a two-dimensional database. First, these data were analyzed regarding measurement units, transformations, missing values, and outliers. Next, descriptive statistics and a correlation matrix were used for data and model analysis. Finally, the model parameters were estimated. The Deap Version 2.1 software was used for the estimation of the DEA results, and the Frontier Version 4.1 software was used for the estimation of the stochastic frontier parameters. The R software was used for the analysis of the secondary data, descriptive statistics, and correlation analysis.

The courts were classified according to technical efficiency and innovation adoption. For the analysis of the relationships between resources, innovation, and performance, panel data models of Malmquist indices were developed using DEA programming (nonparametric), and the variable parameters were estimated using stochastic frontier analysis (parametric). Data from the CNJ made it possible to break down some variables and distinguish between courts in the first and second-degrees. Therefore, the data were evaluated in these two degrees, at both the chamber level and court level. Box 2 shows the variables used in the study.

Box 2 Source: Elaborated by the authors.

The variables Staff, ICT, II, IE, and Size enrich the analysis of the resources and capabilities that impact performance in the judiciary and may be useful in the evaluation of inefficiency. The variable IE is not available in the Justice in numbers database of the CNJ and was collected from the website of each court. The variables Staff and ICT are not separated into first and second-degree. The variable II is called the electronic process index by the CNJ and was first published in 2009.

The variables related to resources and internal capabilities of the previous model were used as the resources (inputs) - variables Magistrate and Staff - and the variable Y was used as the result (output), divided between courts in the first and second-degree. These variables are already established in the literature, and it was not possible to add other variables in the Malmquist model as the data was missing. The model follows the suggestion of Charnes, Cooper and Rhodes (1978), which assumes constant returns to scale with product orientation, already applied in the public sector, and which is also one of the models used in the literature on performance in the judiciary. This approach is appropriate for the Brazilian context, given that, according to the civil law characteristics adopted in Brazil, decisions tend to conform to codes that are different from common law, where the emphasis is on jurisprudence (Deyneli, 2012; Yeung and Azevedo, 2011). The orientation toward product - maximizing product while maintaining the resource base - is justified because, in addition to having been widely adopted in the literature, there is little flexibility to change the resource base of courts, for example by dismissing judges or support staff (Yeung and Azevedo, 2011).

Resources, capabilities, and results were divided by the judge’s workload. The weighting by the workload, according to Yeung and Azevedo (2011), is critical for controlling the variation between courts. The workload indicator, as provided by the CNJ (2013) in the Justice in numbers database, is understood in Brazil as follows:

W o r k l o a d i t = N e w i t + P e n d i n g i t + I R i t + P R i t M i t (2)

Where Workload is the average workload per judge in each court and period, IR is the amount of internal resources, and PR is the amount of pending internal resources, per given court and period. This definition of workload is more complete in that it includes legal resources that were ignored by Beenstock and Haitovsky (2004), and without which the workload may incur measurement problems. In addition, it has the advantage of being more parsimonious than the definition proposed by Costa and collaborators (2006). The terms New, Pending, and Magistrate were defined previously. The stochastic frontier is based on the Cobb-Douglas function, which is also applied to the judiciary by Castro (2011) and Schwengber (2006). The stochastic frontier model is stated on Equations 3 and 4.

l n Y i t M a g i s t r a t e i t = β 0 + β 1 l n M a g i s t r a t e i t + β 2 l n S t a f f i t M a g i s t r a t e i t + β 3 l n N e w i t M a g i s t r a t e i t + β 4 l n P e n d i n g i t - 1 M a g i s t r a t e i t + T i m e + v i y + u i t (3)

Where i = 1, 2, 3, ... 24 representing the units of analysis, i.e., the 24 Brazilian regional labour courts, and t represents the time period; vit is a random variable that represents the error and is independent of uit . The term uit refers the inefficiency effects model, i.e. a random variable with normal truncated distribution. The variable Time is a vector representing dummies for the years investigated.

u i t = δ 0 + δ 1 l n I C T i t + δ 2 W o r k l o a d i t + δ 3 l n I E i t + δ 4 I I i t + δ 5 S i z e + w i t (4)

To test the model and calculate the maximum likelihood we used: (a) the suggestion of Battese and Coelli (1995) to replace σ_u ² and σ_v ² with σ²= σ_v ²+σ_u ² and γ= σ_u ²/ σ², where γ has a value between 0 and 1; and (b) the likelihood ratio test on γ to verify the adequacy of the model.

There are some variables for which data is not available for the entire period of this research, from 2003 to 2013. The Staff1 and Staff2 variables are available only after 2009. Therefore, 144 missing values were computed, and the variable Staff was analyzed as the sum of Staff1 and Staff2. The following missing data were identified: (a) from 2003 to 2006, there are no data available for the variables II and Size; (b) from 2007 to 2008, there are no data available for the variable II; and (c) for the other years, see box 3.

Box 3 Source: Research data.

Table 1 shows the descriptive data for the variables used in this study. The data exhibit high heterogeneity among courts. The court with the largest number of magistrates (422) has nearly 20 times the number of the smallest (22), which justifies the use of Spearman’s correlation (non-parametric), the adjustment of the variables for workload and the number of magistrates, and the transformation by natural logarithm. A high correlation was found between the following variables: (a) Magistrate with Staff, ICT, Pending, and New; (b) Staff with ICT, Pending, and New; and (c) Pending and New. With the division by the workload and the number of magistrates, the correlation decreased, remaining high only between Magistrate and Staff. The use of both variables in the model is justified because it is a specification widely used in the literature. Adjustments were also important to reduce the extreme values found.

The difference in the number of new cases between the court receiving the lowest number (10,502) and the court receiving the highest number (778,679) is more than 74 times. The average number of cases resolved in the courts in the first-degree is more than four times the number in the courts in the second-degree. The workload of the courts in the first-degree is approximately 1.5 times greater than that in courts in the second-degree. In relation to the innovation index (II), whereas some courts have almost all processes in electronic format, others have an index close to zero. The investment in ICT seems to be the most heterogeneous variable, given that the difference between the court that invested the most in one year in the analyzed period (R$ 883.395.206,00) and the court that invested the least (R$ 131,373,00) is more than 6,000 times.

Table 1

	Y2	Y1	Magistrate	Staff	ICT	Workload	Workload2	Workload1	II	II2	II1	IE	Pending	New
Y2	1.000
Y1	.902***	1.000
Magistrate	.929***	.933***	1.000
Staff	.893***	.913***	.951***	1.000
ICT	.566***	.567***	.590***	.590***	1.000
Workload	.462***	.530***	.344***	.398***	.380***	1.000
Workload2	.782***	.641***	.642***	.609***	.470***	.540***	1.000
Workload1	.332***	.424***	.217***	.279***	.298***	.971***	.373***	1,000
II	-.007	-.096	-.022	.000	.100	-.060	.117	-.066	1.000
II2	-.069	-.143	-.056	-.027	.092	-.057	.125	-.069	.875***	1.000
II1	.015	-.080	-.008	.025	.111	-.036	.144	-.038	.999***	.865***	1.000
IE	.323***	.345***	.357***	.381***	.485***	.105	.188**	.025	.364***	.370***	.375***	1.000
Pending	.840***	.853***	.808***	.816***	.560***	.747***	.690***	.642***	-.087	-.077	-.074	.279***	1.000
New	.925***	.983***	.939***	.914***	.574***	.543***	.670***	.433***	-.063	-.125	-.046	.339***	.858***	1.000
Obs	264	264	264	264	262	264	264	262	115	114	108	156	264	264
Aus	0	0	0	0	2	0	0	2	149	150	156	108	0	0
Mean	25791.92	116058.70	125.261	1372.72	R$ 33700000,00	2026.368	1474.923	2165.27	0.163	0.088	0.191	514616.80	129148.8	145617.9
SD	29962.69	125061.60	100.840	1099391	R$ 115000000,00	630.102	674.533	684.474	0.277	0.229	0.312	492819.30	154567.2	153011.8
Min	758	4594	22	225	R$ 131373,00	7.676.964	277	850	0	0	0	0	12966	10502
Max	130007	632574	422	5533	R$ 883395206,00	4054.775	3411.192	4388	1.000	1	1	2812157	797987	778679

Source: Research data. Obs.: number of valid observations; Aus: missing data; SD: standard deviation. *** p≤ 0.01. ** p≤ 0.05

The objective of the study was to develop and validate theoretical and empirical models that explain the relationships between resources, innovation, and performance in the judiciary and to measure the performance of the labor courts through Malmquist indices and stochastic frontier analysis. The data panel that was used covered 24 Brazilian labor courts between 2003 and 2013, comprising the entire period available in the Justice in numbers database.

Two methods were used, taking advantage of both parametric and non parametric approaches. The Malmquist indices indicate improvement in the index related to innovation adoption in 16 courts, whereas the other seven courts failed to improve performance as a result of innovation adoption. For the period 2003 to 2013, labor courts had more variation in performance related to innovation adoption than related to technical variation. Comparing the calculated indices with the resource variables, a significant and positive relationship was found between the index related to innovation adoption and the variables investment in training and electronic process index, while there was a negative correlation with investment in ICT.

The stochastic frontier analysis confirmed three of the five hypotheses, indicating the following: workload positively impacted performance, with coefficients approximately seven times greater than those related to human resources; court size impacts efficiency; and investment in training helped reduce inefficiency in all models where its inclusion was possible. The following hypotheses were not confirmed: the electronic process index was not statistically significant, and investment in ICT contributed to increased inefficiency, indicating that innovation adoption does not necessarily translate into performance gains and that investments in ICT made because of the standardization phase initiated in 2012 have not yet impacted performance.

The present study fills a gap in the research, develops and tests theoretical and empirical quantitative models of a panel of courts to explain the relationships between resources, innovation and performance in the judiciary. Therefore, the results contribute to knowledge that can planning and resource allocation in the courts, and at a broader level can support policy making for courts. Given that the development and adoption of innovation are an on-going process, these results represent important feedback to those responsible for judicial management and innovation in courts.

The study was limited to data available in a database available in Brazil. The Vtec index analyses performance as a function of innovation adoption but does not specifically identify which innovation and to what extent this contribution occurs. Consequently, other innovations not discussed in this study may influence performance, given that the electronic process index was statistically positive, although low (0.225), indicating that other variables, i.e. other aspects of court professionalization, not studied may impact the Vtec index. Investment in training was similarly significantly positively correlated, although low (0.226). this means that other variables may help to explain performance. The variable investment in ICT covers large investments over the entire cycle, including the acquisition, development, and maintenance of technologies. The variables are available only at the aggregate level of the court.

The use of variables at the chamber level is suggested for future studies. On a micro level, the models may show relationships that are undetected at the aggregate court level, as examined in this study. Analysis of the optimum judge workload, beyond which there is declining performance, is also suggested. Moreover, given the complex nature of measuring the performance of courts, other approaches, for example, the development of a scale to evaluate innovation and performance through latent, not directly observable variables, such as orientation toward innovation, and a more flexible and less formal management model, may shed new light to the court management theme.