Relating Efficiency with Service Compliance Indices in Public Transportation Using Slack-based Measure Data Envelopment Analysis and Shadow Prices

<jats:p>In many countries, bus operators are private companies whose service has been leased by government agencies. These agencies develop service compliance indices or measures to keep track of factors such as passenger satisfaction, frequency, and regularity but do not necessarily include the objectives of the operators in the assessment. In this paper, we used slack-based measure data envelopment analysis (SBM) to investigate whether it is possible for a bus operator to be efficient (from a private perspective) and match required standards of frequency and regularity. In doing so, data collected from two major bus operators in Santiago, Chile has been used comprising 99 services. The results show that when private objectives, namely revenues, are included in the analysis, bus operators do not necessarily seek to improve the regularity of their service. Moreover, it was found that some bus services are on the efficient frontier while keeping low performance measure standards. Using the shadow prices of the models, it was also found that improving the performance measures will be hard for many bus services unless there is a significant change in factors that are not under control of the operators (i.e., number of stops, length of the route, etc.). This shows the difficulty of correctly aligning the private objectives of operators with agencies’ objectives.</jats:p>

section_content: Regularity and frequency are two of the key factors in the success of a bus transportation system. Regular and frequent services reduce passengers' waiting time, improve utilization of the fleet, and reduce the operational cost of the operators. Agencies usually keep track of these two performance measures to evaluate the performance of their systems. 

In many countries, bus operators are private companies whose service has been leased. In Chile, for instance, the Transantiago integrated system comprises eight bus operators and a metro/subway agency. Each bus operator is a private company that manages and operates a certain number of routes in specific zones of the Santiago metropolitan area. A local metro agency (Transantiago) monitors the regularity and frequency by means of two service compliance indices: the regularity compliance index (ICR) and frequency compliance index (ICF). The ICF measures the effective frequency delivered by each service in each direction, while the ICR is based on the coefficient of variation (CV) and measures the regularity [1]. In both cases, the indicators vary from 0 to 1. 0 as they are basically evaluated as percentage of compliance with established values. For instance, the ICF is constructed as the percentage of programmed bus trips effectively satisfied (1. 0=100% of compliance), and the ICR converts ranges of the CV into values between 0 and 1. 0, with 1. 0 being the maximum compliance. 

technology". That makes this improvement impractical as these are factors that are not under control of the operators. Thus, that fact would be one of the reasons why operators perform poorly in regularity. 

The rest of the paper is organized as follows. The next section presents an overview of the use of DEA for bus efficiency. As this topic is extensive, the section presents only main references related to or similar to the operating measures that we have included in this study. Section 3 briefly describes the SBM DEA technique that is used in the evaluation of the case study. The case study data and context are provided in Section 4. Section 5 presents the major results and findings. Finally, Section 6 draws conclusions.
section_name: INTRODUCTION

section_content: Data envelopment analysis (DEA) [4] is an operations research-based technique for measuring the relative performance of organizational units (decision-making units or DMU) by means of obtaining technical efficiency from multiple inputs and multiple outputs. The technique benchmarks DMUs and defines a measure of efficiency. Due to its flexibility to accommodate multiple outputs and inputs and because it does not require an a-priori relationship of inputs and outputs, it has been applied in the past to measure the efficiency of different transportation problems such as public transportation, bus transit, traffic systems, and airlines, among other transportation applications. Moreover, by presenting multiple transportation applications, Ozbek et al. [5] have illustrated the benefits of using DEA for comparative performance measurement in transportation problems. 

In terms of bus performance, DEA has also been extensively used to measure and compare public transportation efficiency. Chu et al. [6] is one of the oldest studies on DEA to measure transit efficiency. In their study, they use the Charnes, Cooper & Rhode's model or CCR [4] to develop two different models, one for the efficiency (using expenses and revenues) and one for what they call effectiveness (using operating variables as well as other exogenous variables), pointing out the importance of keeping track of both measures separately when evaluating public agencies. De Borger et al. [7] provide an extensive literature review and assessment of different frontier models up to year 2000, including DEA models. They found that there are exogenous aspects such as the organization of the decision-maker, the type of regulation, and the contract design, which are important factors of inefficiency, more so than congestion. In addition, Chu et al. [6] highlight the negative relationship between effectiveness and efficiency as well as the lack of consensus on the selection of inputs and outputs. Regarding the latter, they found that the most traditional selections Penalties are applied to operators when the standards presented in the operational plans are not achieved. The penalties are fines that are stipulated in the contracts. The aim of such penalties is to make sure that bus operators provide a regular and frequent service. This has allowed a certain improvement in the indicators of performance. In particular, Beltran et al. [2] have found, using data from all Transantiago operators, that the ICF improved from 0. 75 in 2008 to 0. 95 in 2011, and that the ICR improved from 0. 74 in 2008 to 0. 83 in 2011. However, this also shows that while the ICF has achieved the standard, the ICR is still far from the target value (0. 90). 

While these measures are used, as mentioned, to evaluate the success of a bus transportation system, bus operators have different ways in which they measure their efficiency. For instance, the efficiency of the operator may be related to how their resources, namely buses, costs, or schedules, are being used to achieve some private goals which are not necessarily aligned with the regularity and frequency measures. 

In this paper, we attempt to measure whether the efficiency of the service in a route is linked to achieving the minimum goals of ICF and ICR. Our hypothesis is that if all services that have low ICR and ICF standards are not operating efficiently and all services with high ICR and ICF standards are efficient, there is room for improving the service and for attempting to reach the standards required by the government agencies. However, if there are services in which at least one of the indices is low and the service is efficient, this implies that the services are already being operated efficiently and it is not under control of the operators to reach the minimum standards required by the agency. 

To make this assessment, we use data collected from two major bus operators in Santiago, Chile, for a total of 99 bus services. A bus service in our case consists of two directions for the same bus routewhich is how Transantiago measures the ICR and ICF. That is, each bus route has two services: upstream direction and downstream direction. The data includes operating variables as well as the referred indices. Using the Slack-Based Measure Data Envelopment Analysis model (SBM) [3] we have been able to assess the ICF and ICR both when a private output is included and when not. The results show that when a private objective is included (namely revenues) the number of efficient bus services increases compared to the cases in which only the ICF and ICR are accounted as outputs. It was also found that in both cases a portion of services with low ICR is already on the efficient frontier. This implies that, at least for these services, it would be difficult to achieve the standards. By using the shadow prices of the DEA models, it was also found that improving the ICR compliance service measure to achieve the required standard implies not only the addition of resources but also a "change of inefficient units, because it produces at least as many outputs using the same or lower number of inputs as all of the identified inefficient units. 

The first data envelopment analysis model (the CCR model) was developed by Charnes, Cooper, and Rhodes [4]. The model is based on measuring the efficiency of a DMU by means of an efficiency rate relatively to other DMUs. This original problem is a fractional non-linear programming problem that was linearized, making it a popular tool for measuring efficiency and productivity. After the CCR model was developed, other DEA models were proposed incorporating some particular characteristics. For instance, Banker, Charnes, and Cooper's model (BCC) [13] is similar to the CCR model but takes into account variable returns to scale. The SBM or slack-based measure [3] is based on slacks that seek to reduce inputs and increase outputs simultaneously, and can have both constant and variable returns to scale version. 

Compared to the CCR and BCC models, the SBM model has the advantage of being an additive model with units-invariance, non-oriented and non-radial. That is, the objective function seeks to simultaneously improve both inputs and outputs, and the model does not force the inputs and outputs to improve uniformly or equal-proportionally letting the maximum possible improvement in each dimension be computed by the model. This makes it particularly useful when including ratios or indices in inputs and outputs because it makes the model dimensionally free [14]. 

The SBM model computes the ratio of the average inputs reduction to the average output increase while using the constraints of the CCR model, adding s - j and s + i to indicate respectively the slacks of j-th input and i-th output, and a variable t for model linearization. The linearized SBM model proposed by Tone (2001) can be formulated as follows:

,,,, y s ty i m 0 1 2

,,,,,, ;,,, ;,,,

where x is the efficiency, s - j is the slack of the j-th input, s + i is the slack of the i-th output, m k is the contribution of the k-th DMU, t is the model linearization factor, x j0 is the j-th input of DMU under analysis, y i0 is the i-th output of the DMU under analysis, x jk is the j-th input of inputs are related to capital, labor, and energy variables, while for the outputs the selection is more problematic. 

After De Borger et al. 's review, several other models have been developed. Lao & Liu [8] have used DEA in combination with geographic information systems to measure efficiency of bus lines as well as spatial effectiveness (measured through passenger coverage). Boame [9] used DEA to compute the technical efficiency of bus operators in Canada by means of the CCR model using inputs such as fleet size, average speed, labor hours, fuel, and outputs such as revenue per km within a bootstrapping approach in order to compute confidence intervals of the efficiency estimators. They later used Tobit regression to identify the sources of efficiency, which they relate to average speed and the mix of buses (capacity). Tsamboulas [10] used the CCR model to measure the performance of transit systems in Europe at a city scale. Aggregated variables such as number of employees, amount of fuel spent by each transit system, and the number of vehicles operated by each system were used as inputs, while vehicle-kilometers traveled and number of passengers served were used as outputs for two different models. As in Chu et al. [6], Tsamboulas [10] uses vehicle-kilometers traveled as the output for efficiency and number of passengers served as effectiveness. Barnum et al. [11, 12] changed the focus of the analysis by changing the type of DMU. That is, they used the CCR model to compare subunits of a transit system (bus routes of a transit system) instead of comparing the performance of multiple transit systems, using seat hours and seat kms as inputs, and passenger ridership, on-time performance, span of service, and frequency as outputs. As in Boame [9], they also accounted for the effect of environmental (exogenous) variables, but they used linear regression instead of Tobit regression.
section_name: BUS EFFICIENCY USING DEA

section_content: Data envelopment analysis (DEA) is a nonparametric mathematical programming method used to measure relative efficiency of decision making units (DMUs) in a system with multiple variables, known as inputs and outputs. That is, efficiency is measured as the ratio of a combination of outputs to a combination of inputs. A DMU reaches 100% efficiency only if none of the inputs or outputs can be improved without worsening some of its other inputs or outputs and without violating the constraints of relative efficiency ≤1 for all DMUs, so that using the same weights for other DMUs none of them can obtain an efficiency measurement superior to 1 as well. This efficiency is measured with respect to a hypothetical efficient unit, as a weighted average of efficient units to serve as benchmarking for The values obtained for the ICF vary between 0 and 100% as they represent a percentage of compliance of the operating plan. Penalties are then applied whenever certain levels of accomplishment are not reached. For instance, a minimum of 90% ICF jpdt is defined for the operator to not be penalized, while a penalty of USD 190 is applied if the ICF is between 85% and 90%, and USD 1,140 if the ICF is below 85%.
section_name: SLACK-BASED MODEL FOR DATA ENVELOPMENT ANALYSIS

section_content: This indicator is aimed at protecting users' waiting times from being affected by an increase of the time between buses or the unpunctuality of the services. The index has two sub-indices, one for the time between buses and one for the punctuality. The use of each sub-index depends on the type of service. For instance, low-demand services are measured through the punctuality sub-index while high-demand services use the time between buses. In both cases, the ICR is computed as the ratio of the number of intervals with incidents over the number of observed intervals. An incident is considered whenever the observed bus interval does not fall within the programmed time interval at the checkpoints (ICR-I for services with a frequency of 15 buses/hour or more) or whenever the waiting time exceeds certain waiting time threshold (ICR-E for services with a frequency between 6 and 15 buses/ hour). In the first case, the programmed time interval includes a reasonable slack between 3 and 10 minutes depending on the programmed time interval [22]. In the second case, the programmed waiting time accounts for a factor based on the coefficient of variation and includes a slack of 3 and 10 minutes used in the ICR-I. Again, as the ICF, the ICR is subject to penalties that vary according to the degree of compliance. The penalties are similar to the ICF: no penalty if the ICR is 90% or more, USD 190 if the ICR is between 75% and 90%, or USD 1,140 if the ICR is below 75%.
section_name: Regularity compliance index

section_content: The data was obtained based on three months of operations by two major bus operators in Santiago that together currently provide more than 200 bus route services. For these two operators, GPS data is stored and monitored through a software system that of the k-th DMU, y ik is the i-th output of the k-th DMU, n is the number of inputs, m is the number of outputs, and z is the number of DMUs. 

As in the CCR model, it is possible to take into account the variable returns to scale imposing the following constraint for :

.
section_name: CASE STUDY 5.1 Case study data: input-output selection

section_content: Santiago's System of Public Transportation (Transantiago) is a city-wide integrated bus-metro transportation for the 6. 68 million inhabitants in Santiago, Chile. The system started its operations in 2007 with 4,489 buses [1, 15] and new metro lines. The system was pushed to start with an insufficient bus frequency, collapsing only a few months later [16, 17, 18]. It was upgraded to correct problems related to organization, implementation, design of infrastructure, and the incentive system [1, 15, 19, 20]. By 2013, the system handled 186 trains with a total of 108 stations on 5 metro lines. The number of buses was about 6,500, operating on a total of 2,766 km (31% more than in 2007) with 11,271 bus stops in the city, and 212 kms of dedicated bus lanes (136% more than in 2007) [21]. By 2013, the number of transactions per day was 5,596,675 split between buses (59. 5%) and metro (40. 5%). 

The bus system has been leased to 7 companies which operate different routes and zones, with three of them handling 60% of the total number of buses. About 23,800 people work for the operators, mainly bus drivers (70% of the total workforce). 

In spite of the effort invested in recent years to improve the operations, the system has consolidated, but it has not resolved most of its major problems [2]. One of such problems is the lack of regularity in the headways. Transantiago measures regularity together with frequency using two different indicators, the ICR ('Índice de Cumplimiento de Regularidad' -Regularity Compliance Index) and the ICF ('Índice de Cumplimiento de Frecuencias' -Frequency Compliance Index). Both indicators are compliance-based. That is, they are measured against an operational plan which is revised by the Transantiago Agency. Also, as it was mentioned in the Introduction section, both indicators are scaled to 0-1. 0, 1 being the maximum value.
section_name: TRANSANTIAGO'S COMPLIANCE INDICES

section_content: The index measures the number of effective services performed by the bus line operator against the number of planned services. The objective is to control passengers' waiting time by controlling the number of buses that should be running according to the operating plan. The ICF is therefore computed as follows: and outputs were available were included in the evaluation. This resulted in 99 DMUs, where each DMU is a bus service (2 directions per route, as explained in the introduction). The SBM model was run using a code written in Matlab for two different sets of outputs. In the first set, only ICR and ICF indices were included, while in the second set the operating revenues for each service were also included as an output. That is, we evaluated two different sets of models. The first one includes only social (public) objectives represented by the ICR and ICF indices as outputs, and available seatkm, speed, and operating costs as inputs. The second set includes not only the social objectives but also revenues as outputs and available seat-km, speed, and operating costs as inputs. For each of the sets, three DEA models were used to obtain the efficiency: SBM model with variable returns to scale (SBM VRS), SBM model with constant returns to scale (SBM CRS), and for benchmark the CCR model was also included.
section_name: Frequency compliance index

section_name: Efficiency assessment

section_content: The first analysis included the computation of efficiency using the available seat-km, speed, and operating costs as inputs, and ICR and ICF as outputs. As these indices were set by the agency, the use of these outputs can be considered as evaluating social objectives only. Figure 1 shows the efficiencies obtained using three DEA models: SBM VRS, SBM CRS, and CCR. The darkness of the color indicates higher efficiency values. It can be noticed that both the SBM CRS and the CCR model identify 11 efficient DMUs while SBM VRS identify the same 11 DMUs but add 3 more. However, CCR tends to overestimate efficiency compared to the SBM models. In such cases, CCR provides efficiencies which are about 56%-66% higher on average than those found by SBM models (0. 723 compared to 0. 445 and 0. 420, see Table 2 ). This can be traced to the problem of using indices as outputs, as pointed out in Section 3.
section_name: Results using ICR/ICF as outputs

section_content: The model included the computation of efficiency using the same inputs as in the models with ICR and ICF outputs only but, in addition, revenue is included provides dispatching and control orders. The software also provides information regarding average speeds, travel times, and total boarding counts for local services and trunk services. Additional collected data includes the number of stops, seat-km, boarding counts, as well as operating costs and revenues, and ICF and ICR indices. A correlation analysis was conducted as part of the design process to remove highly correlated variables from the data set (see Table 1 ). Boarding counts was discarded because it is correlated to other input variables and because it is related to demand which cannot be necessarily controlled by the operator. The same happened to traveled kms, which was also correlated to operating costs and thus removed from the input variables. 

As the data included several months, we only included the most representative months. That is, the months that were not affected by seasonality of other factors that are exogenous to the operation itself. The final used data included the average of three months of information for each bus service. The final selected input variables are: -Available seat-km (input). This variable is used as a proxy for capacity. It is computed as the number of seats available multiplied by the number of effective kilometers run per service. -Speed (input). This variable is the average speed per service [km/h]. -Operating costs (input). Comprises all operating costs incurred to run the operation per service. It includes variable costs, such as fuel, depreciation, conductor's salaries, maintenance, as well as the penalties (in Chilean pesos). On the output side, the analysis includes both the ICR and ICF service compliance indices, as well as the revenues for each service. This includes direct revenues obtained from passenger trips as well as subsidies received from the government. The latter is related to a subsidy received by the bus operators to compensate the high level of fare evasion (over 20% according to Guarda et al. [23] ) that affects Santiago bus operators. 

To measure efficiency, we used the SBM DEA model described in Section 3. For comparison, we also included the results of the model with the CCR model. All services for which a complete data on the inputs Notice also that 'revenue' has a clear effect on the as an output. For that reason, this model can resemble a model with a welfare objective given that the indices represent a social objective while revenues capture the private objective of the operation. Similar to Figure 1, Figure 2 shows the efficiency. The results are similar to those obtained in Model 1. However, the including revenues as output (see Table 2 ). However, the number of efficient DMUs doubles for cases in which either the ICR or ICF is less than 0. 9. This finding is additional evidence of the impact of including 'revenue' in the outputs of the models, pointed out in Section 5. 1. Revenue impacts the efficiency mainly in those services with low ICR or ICF compared to the case in which only the compliance indices are used as outputs. Notice also that for the efficient and inefficient cases the average ICR and ICF are very similar across models, regardless of whether revenue is used or not.
section_name: Results using ICR/ICF and revenues as outputs

section_content: A correlation analysis between the variables used both as inputs and outputs was performed (see Table 4 ). All correlations were small or moderate (between -0. 745 and 0. 391). It can be noticed also that both speed and seat-km have a negative correlation with efficiency. Interestingly, operating costs have a negative effect on efficiency when only ICR and ICF are included as outputs. Such effect is positive when revenue is included in the outputs. The analysis also included boarding counts and traveled kms, which follow the same pattern as operating costs, given that they are all correlated (see Section 4. 2). In terms of the outputs, the ICR and ICF have a higher correlation with efficiency when they are the only outputs included in the model. Their effect reduces to almost a third when revenue is included in the outputs. In general, it can be noticed that revenue has a strong effect on the efficiency when it is included in the model. In fact, it shows a negative correlation with efficiency when it is efficiency of DMUs as now the number of efficient DMUs increases. The SMB VRS now shows 21 efficient units compared to 16 identified by the SBM CRS and CCR.
section_name: Correlation analysis between variables and efficiency

section_content: Regarding efficient units, the CCR and SBM models deliver a similar number. Moreover, they choose the same services. However, it can be observed that, again, when 'revenue' is added as output, the models select a higher number of efficient units. To evaluate whether the ranking of the efficiency of units across models varies statistically, the Kendall rank correlation test [24] was used to evaluate consistency in the rankings of the DMUs across models. The Kendall rank correlation is a nonparametric test used to assess the correlation between two different rankings using the same evaluated units (DMUs). So, the Kendall test was used for all 99 services, regardless of the unit being an efficient/inefficient or effective/ineffective. Table 3 presents the τ coefficient, which measures the rank correlation between the models (SBM VRS, SBM CRS, and CCR). Both SBM models show a strong correlation (0. 84 with revenue and 0. 93 without revenue) while the CCR is slightly lower (between 0. 70 and 0. 80). The majority of the differences appear between using or not using revenues in the models (between 0. 53 and 0. 65)
section_name: Consistency of the models

section_content: The right upper corner in both Figure 1 and Figure 2 represents the areas in which ICF and ICR do not impose a penalty (ICF≥0. 9, and ICR≥0. 90). The number of DMUs in these areas differs only by two when is the optimal weight of ICR. Similar reasoning can be applied when using the SBM VRS or the CCR model to obtain the shadow prices under these models. 

To measure to what extent can the ICR be improved in inefficient services, we propose the following procedure. Suppose the shadow price is kept constant in the DEA model, then the amount of change in the ICR Dy ICR0 required to make an inefficient DMU achieve the efficient frontier would be proportional to the inefficiency. That is,

Using Equation 19, it is possible to find changes in the output that would make the DMU unit efficient. In that case, the new ICR value that makes a DMU efficient is (y ICR0 + Dy ICR0 ). However, notice that some DMUs will need a value (y ICR0 + Dy ICR0 )>1, which is not possible in practice. 

Using this approach, we evaluated how many of the inefficient units would become efficient after increasing only the ICR values to the required standard (ICR=0. 9). 

Table 5 presents the results from the inefficient units from all models tested including or not revenue as output. For the SBM VRS model (with revenue as output), from the 99 services, 78 inefficient services can achieve the status of efficient improving only ICR. Among them, 71 services require improving their ICR over 1. 00 to be efficient, which is practically infeasible. Only 4 inefficient services could achieve an ICR of 0. 9, while 3 could be efficient without necessarily reaching an ICR greater than 0. 9. Similar results are found for the SBM CRS model as well as for the CCR. The only difference is that with the CCR a few more services can become efficient. This is because the CCR provides higher values of efficiency, as shown in Section 5. 3, making it easier to become efficient. Another important aspect is that if revenues are not included, no service can become efficient even augmenting the ICR over 1 in the SBM models. This implies that operators would not have incentives to improve the ICR as this would have no effect on the efficiency of the service, from a private perspective. not included as output. This means that using only ICF and ICR as outputs would select as efficient DMUs that do not generate sufficient revenue for the operators. On the contrary, if revenue is included, it shows that firms are prioritizing revenues over the ICR, given that the correlation between efficiency and revenue of the efficient DMUs doubles the correlation between efficiency and ICR. It seems that ICF does not correlate with efficiency when revenue is included in the outputs of the models.
section_name: Efficiency vs. service compliance indices

section_content: As observed, the ICR is the most problematic index. Effectiveness or the ability to attain the desired value required by the agency is discussed in this section using the concept of shadow price. In mathematical optimization, the shadow price measures the marginal cost of improving the objective function. In linear programming (LP), which is the case for DEA models, the shadow price of a constraint is equal to a variable of the dual formulation of the same model. In the SBM CRS model (Equations 1-6), the objective function of the dual model also measures the efficiency x, and it is equal to the weighted summation of the outputs y i0 and x j0 inputs :

where u i0 and v j0 are the weights. Then the shadow price p i0 of the i-th output (Equation 4 in the primal) for each of the k-th DMU0 under analysis is the derivative of the efficiency x (Equation 8 ) of the k-th DMU under analysis with respect to the i-th output y i0. This is equal to the optimal weight u * i of the i-th output using the dual model in the multiplier form:

Using the previous equation, the shadow price of the ICR p ICR0 is: Este trabajo usa un modelo de Envoltura de Datos basado en holguras ("Slack-Based Measure Data Envelopment Analysis" o SBM en inglés) para investigar si es posible para un operador ser eficiente (desde una perspectiva privada) y a la vez alcanzar los estándares de cumplimiento de frecuencia y regularidad. Para esto, se usaron datos de 99 servicios de dos de los principales operadores de buses de Santiago, Chile. Los resultados muestran que cuando en el análisis se consideran los objetivos privados, como por ejemplo ingresos, los operadores no necesariamente mejoran la regularidad del servicio puesto que se encontraron servicios que, estando en la frontera de eficiencia, mantenían índices de cumplimiento por debajo del estándar exigido. Adicionalmente y usando los precios sombra se pudo encontrar que alcanzar los estándares de cumplimiento no es posible a menos que haya un cambio en factores que no están bajo control de los operadores (ej. : número de paraderos, longitud de la ruta, etc. ). Esto muestra la dificultad de alinear correctamente los objetivos privados con los objetivos de las agencias.
section_name: Feasibility of ICR improvements using shadow prices

section_content: eficiencia de sistema de buses; transporte público; índices de cumplimiento de servicio de buses; envoltura de datos basado en holguras; precios sombra;
section_name: PALABRAS CLAVE

section_content: In this paper, SBM DEA models with both VRS and CRS have been used to account for the compliances indices in bus services efficiency. The models were used to evaluate 99 services of the Transantiago transit system in Santiago, Chile. Using different input and output variables significantly influences the results. Two different models were constructed, using the ICR and ICF indices as outputs in one, and both indices plus the revenue of the bus service in the other. While in both cases the most efficient services do not vary significantly, the services with low ICR differ significantly in the efficiency obtained in both models. In particular, many of the services which are far from achieving the minimum ICR goal have higher efficiency when revenue is considered as output. That is, some services are already in the efficient frontier and their ICR would be hard to improve, unless there is, in terms of the frontier analysis, a change in technology. This analysis is complemented by evaluating whether it is possible for a service to become efficient after increasing the ICR. The results show that there are very few services that can be privately efficient after reaching the ICR standards. That is, operators have no incentive to provide a regular service (ICR). This is clearly an example that private operators do not necessarily view efficiency with the eyes of the users. Public administrators should therefore be aware of this issue when designing their systems and take into account private operations in order to establish regulations that assure a good standard in public transport. Future studies can focus on finding the right values of the penalties to ensure the quality of the service. In particular, dual values of the improvements of the ICR or ICF in DEA models can be used to compute the penalties that are sufficient enough to achieve the standards.
section_name: CONCLUSIONS

section_name: ACKNOWLEDGEMENTS

section_content: The authors want to thank Nicolás Pávez and Felipe Sandoval from City Planning for providing the data used in this paper. The hard work of Ms. Consuelo Stanke and Ms. Isabel Donoso who worked as undergraduate research assistants to this project is deeply appreciated.
section_name: ACKNOWLEDGEMENTS