# Technical Noncompliance Evaluation Criteria for Sustainable Production of IT Support Services in the Ministry of Work and Employment at Brazil

Nilo Costa Serpa, Ivanir Costa, Antônio De Araújo Neto

► To cite this version:

Nilo Costa Serpa, Ivanir Costa, Antônio De Araújo Neto. Technical Noncompliance Evaluation Criteria for Sustainable Production of IT Support Services in the Ministry of Work and Employment at Brazil. 20th Advances in Production Management Systems (APMS), Sep 2013, State College, PA, United States. pp.326-333, 10.1007/978-3-642-41263-9_40. hal-01451774

HAL Id: hal-01451774

https://inria.hal.science/hal-01451774

Submitted on 1 Feb 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Technical Noncompliance Evaluation Criteria for
Sustainable Production of IT Support Services in
the Ministry of Work and Employment at Brazil

Nilo Serpa¹, Ivanir Costa¹, and Antônio Palmeira de Araújo Neto¹

¹UNIP - Universidade Paulista, Graduate Program in Production Engineering, Brazil
niloserpa@gmail.com, icosta11@live.com, antoniopalmeira@ig.com.br
http://www.unip.br

**Abstract.** The design and materialization of quality indicators for support services in information technology are still activities of difficult execution in Brazil. The very analysis of noncompliances is often done improperly, as in Ministry of Work and Employment, with no standardized classification routines to aid management tasks. In addition, the high number of mistakes in recognizing the nature of the problems to be solved lead to an increasing cost of displacement of technicians, since there are no teams in Regional Superintendencies of Work and Employment to meet local demands. Present work discusses and proposes a technique for sizing noncompliance during audit processes on support services in situations where a robust quality program was not deployed yet. The central idea is to establish indicators geographically adjusted to reflect reality with minimum distortions to guide the endeavor for cost containments as well as to give contributions to minimize environmental impacts due to the excessive transport of specialized people.

**Key words:** noncompliance, indicator, performance, sigma techniques

# 1 Introduction

The geographic map of the information technology (IT) in Brazil clearly overlaps the map of regional exclusion that still stratifies our society, especially in terms of access to technology. Constant efforts have been made in favor of changing this scenario, but progress is slow mainly by severe contractual restrictions of outsourcing upon human and material resources.

The country's cultural discrepancies associated with investment inequalities outline an irregular behavioral way, very difficult to correct, especially since one vainly insist on a counterproductive technological regionalism unflattering to the old ideals of FUST (the Brazilian Fund for Universalization of Telecommunication Services).

In some government institutions it is practiced an evaluation of noncompliances without any objective criteria, a fact that causes unsustainable conclusions about the outsourced's performance. The lack of understanding about the national reality as described above and the non-trivial task to determining plausible indicators lead to merely palliative fiduciary – sometimes unfair – techniques.
As we previously said, there are no teams in Regional Superintendencies with sufficient technical competence to meet local demands, so that the displacement of technicians from technological towns is expensive, even more in view of the occurrence of classification errors of calls, causing wrong displacements. This study shows how to establish preliminary rules for the control of technical support noncompliances in the ambit of the Work and Employment Regional Superintendencies at Brazil, assuming misclassification of support services, by means of data supplied by a historical series between October 2011 and February 2012. The monitoring of the classification of calls shall reduce the cost of displacements and the waste of time, contributing to the overall reduction of environmental impacts caused by excessive airborne, since distances in Brazil have continental dimensions.

## 2 A bit on performance indicators

Quality has been the *agenda* in services since well before the onset of globalization. Important contributions to the subject can be found in [5], [9] — one of the great works on quality impact —, [7], [8], [11], [1], [10], [6] and [4] (in chronological order). According to the best practices advocated in the market, the management of IT services, specifically the technical support services, should be treated as an opportunity to improve quality — adding value to the business of the organization or institution — and not as a bureaucratic barrier. Once the contract administration is an execution process as assumed since PMBOK 2000, and, being essential for the success of projects, it must include periodic evaluations about the outsourced's performance comparing to their deliveries, it is very important the identification of items of improvement by means of the so-called performance indicators, derived quantities of checklists applied to the contract and the enterprise under legal agreement. Such indicators are byproducts of contract administration and they are prevalent even in the selection of suppliers. Through the use of these indicators, the registration of any disputes and noncompliancies should lead to improvement actions. Thus, the indicators act as regulators of the performance of the outsourced, allowing advancement feedbacks and consequential adjustments or adaptations. In short, the use of checklists and their derivatives (performance indicators) is *sine qua non* for the implementation of an Integrated Management System (IMS) based on standards ISO 9001:2000, ISO 14001:2004 and OHSAS 18001:1999 (updated in 2007).

## 3 The sigma approach

Modern enterprises and some government institutions in initial levels of maturity, looking for their guarantees, attempt to set quality parameters using only pieces of large frameworks, never acting by effective programs of incremental true quality. In general, they seek to establish an SLA, the Service Level Agreement between customers and suppliers. Nonetheless, there are situations where no SLA was implemented, even less the principles that guide the terms of some
future SLA, since this agreement shall be also the result of a negotiation as a
non-cooperative sequential bargaining game with conflict among self-interested
players [14]. In all cases it is necessary to establish a "flooring" of quality from
which one can think of SLA or some planning based on sigma techniques for
gradual implementation of quality.

Considerable improvements in the service sector can be achieved by the ap-
plication of relatively simple management changes such as the introduction of
new concepts, the planning of a system over customer support services and the
definition of a measurable patterning of services, the latter directly related to
performance indicators.

Performance indicators have been proposed and used in IT Governance at Brazilian organizations, widely documented in the Information Technology Development Planning – ITDP – and its annual ammendments. Despite that, in many cases they are not able to promote satisfactory control, or even help the consolidation of information from a proactive interpretation of data to solve problems. It is also a fact that, by following market practices only, we do not make the necessary abstraction on the theme "service indicators" to adapt them to the characteristics and business rules of each organization.

In particular, with regard to the monitoring of noncompliancies nowadays,
we have some indicators, such that:

- ICAC (Index of Compliance in Application Checklists): indicator calculated dividing the number of possible points by the number of points obtained from the checklists, averaging the results of applied lists;

- IPEN (Index of Closed Pending): indicator calculated dividing the number of closed pending by the total backlog detected according to checklists;

- ICNC (Index of Closed Noncompliance): indicator calculated dividing the number of ended noncompliances by the total detected noncompliances according to checklists.

However, from the viewpoint of the geography outlined above, in order to
avoid the improper practice of assigning a generalized noncompliance to the
entire package of services rendered in a particular span of days, based on the log
of noncompliances in one or more types of customer services, but not all, such
static indicators do not show enough. It is necessary to take a dynamic approach,
where the indicator will provide, beneath regional conditions, acceptable ranges
of compliance, highlighting that "flooring" of quality we refer earlier. We call
this indicator simply "INC" (Indicator of Noncompliance).

Since each Brazilian State manifests proper characteristics of infrastructure
and human resources, we chose to work with an indicator originated from the
analysis of the time series of demands, adjusted to minimize the demographic
distortions and associated with the standard deviation σ calculated on all kinds
of customer services. The Greek letter σ (sigma) is used in statistics to indicate
how far a given process deviates from its goal. Thus, the higher the sigma value,
the smaller the deviation.

Sigma techniques are applicable both in production of consumer goods and
services production for continuous improvement of processes. The Six Sigma
<table><caption>Table 1: INC at Ministry of Work and Employment – Brazil for Amapá (AP) and São Paulo (SP) States since October 2011 until February 2012.</caption><thead><tr><th>State/Type</th><th>10/11</th><th>11/11</th><th>12/11</th><th>01/12</th><th>02/12</th></tr></thead><tbody><tr><td>APNetwork</td><td>2,73</td><td>1,82</td><td>4,55</td><td>0,30</td><td>0,61</td></tr><tr><td>APHardware</td><td>0,92</td><td>1,51</td><td>2,77</td><td>2,18</td><td>2,61</td></tr><tr><td>APSoftware</td><td>0,97</td><td>0,76</td><td>1,46</td><td>3,13</td><td>3,68</td></tr><tr><td>APApplication</td><td>0,67</td><td>0,83</td><td>4,00</td><td>3,50</td><td>1,00</td></tr><tr><td>SPNetwork</td><td>3,70</td><td>2,04</td><td>2,23</td><td>1,71</td><td>0,33</td></tr><tr><td>SPHardware</td><td>1,26</td><td>1,22</td><td>4,21</td><td>2,57</td><td>0,73</td></tr><tr><td>SPSoftware</td><td>1,27</td><td>0,62</td><td>1,88</td><td>3,37</td><td>2,87</td></tr><tr><td>SPApplication</td><td>1,93</td><td>2,41</td><td>2,41</td><td>1,81</td><td>1,45</td></tr></tbody></table>

framework, for instance, exalts the total satisfaction of the customer and excellent quantitative results in production, cost and profit, human resources and services [13], although there is no model for effectively guiding the implementation of Six Sigma programs [3]. In a highly competitive global scenario, these techniques are focused on globalization and its demands for standardization and quality. In this context, Campos understands sigma as a symbol designating the "distribution or dispersion around the average of a process or procedure" [2].

Perez Wilson defines Six Sigma as "an optimized level of performance that approaches zero defect in a process of making a product, service or transaction. It indicates the obtaining and maintaining of a high level of performance" [12]. Anyway, the use of a sigma technique seeks to reduce defects, errors or failures, and, as such, constitutes a management framework. The higher the sigma value, the less will be found procedural or productive flaws. In other words, sigma will measure the degree to which a particular process is able to operate without failure, craving the ideal state translated by the notion of zero defects, developed by Philip Crosby [5]. In general sense, sigma approaches must be scalable, i. e., they should be related to achievable goals accordingly the available resources, so that quality is incremental as far as the process improvements are implemented according to a planning of investments.

The approach of using sigma techniques is to decrease the variability of production processes, improving quality, reducing costs and deficiencies. The determination of initial levels of variability serves as a reference from which that variability shall be reduced by processual implementations of continuous improvement. In terms of customer services, sigma can express how far the service is from their quality goal. Normally, it is considered that 1 sigma represents 68 % of services which were completed with an acceptable margin of error of classification. As the proposed technique strives to increase the rigor of the search for quality, quantitative noncompliance is a function of an overgrowth in demands of a certain type measured with respect to the so-called "$\sigma$-range" (Figures 1 and 2). Quantities of demands out of the $\sigma$-range generate indications of non-compliance, starting from the premise that high amounts can be associated with
misclassification. The σ generated by the sample is given by the time series,

$$ \sigma = \sqrt{\frac{1}{4n-1} \left[ \sum_{j=1}^{4} \left( \sum_{i=1}^{n} x_i \right)_j - \bar{x} \right]^2}, \quad (1) $$

where j denotes the nature of the calls, n is the length of time series (in months), 4n is the sample size, $x_i$ are the i registered values of the indicator, and $\bar{x}$ is the average value of the indicator along the series. Reducing the bias by differences in demands among the States, the indicator of a given type j, in month i, in State e was defined by dividing the number of demands by the total number of demands at the i chosen month in all kinds j, in all States and by 10 requestors, i.e.,

$$ x_{(i)je} = \frac{a_{(i)je}}{\sum_{e=1}^{28} \left( \sum_{j=1}^{4} a_{(i)j} \right)_e / 10}. \quad (2) $$

By whatever manner, there is no quality control of support services in IT produced at the Ministry of Work and Employment. The number of wrongly classified calls is large due to lack of specialized technical professionals available in the Regional Superintendencies, so that it is just expected highest correlation between high value of the indicator defined by the managers board and taxonomic noncompliance. Thus, the delay time to closing calls has greatly exceeded the expected average, causing customer dissatisfaction, increased costs with displacement of technical teams and wearing of management.

A supply of IT support services subject to fluctuations in quality of attendance may be statistically well modeled with floating standard deviation. We assume that we need to establish a minimum acceptable standard deviation as a quantitative basis to initialize real quality, starting from this limit to achieve incremental improvements. So, the idea is to find the maximum range of fluctuation of the standard deviation. This is particularly advantageous when there is no wide time series, so that we try to compensate changes in the standard deviation resulting from the size of the series assuming the standard deviation variability. In other words, since there is no well-established control of the failures of attendance, featuring an erratic character in the dispersion of the flaws, the best way is to operate on the historical series and try to establish a pattern of variation of the standard deviation.

For an accurate analytical understanding, if we had a continuous sum, the expression of the standard deviation would be

$$ \sigma = \sqrt{\frac{1}{4n-1} \left[ \left( \int_{i=1}^{n;4} x_{ij} dx_{ij} \right) - \bar{x} \right]^2}, \quad (3) $$
from which we have

$$ \sigma' = \frac{2}{4n-1} \left[ \left( \int_{i=1; j=1}^{n;4} x_{ij} dx_{ij} \right) - \bar{x} \right] \frac{x_{ij}}{2} \left\{ \frac{1}{4n-1} \left[ \left( \int_{i=1; j=1}^{n;4} x_{ij} dx_{ij} \right) - \bar{x} \right]^2 \right\}^{-1/2} $$

$$ \sigma' = \frac{\frac{x_{ij}}{4n-1} \left[ \left( \int_{i=1; j=1}^{n;4} x_{ij} dx_{ij} \right) - \bar{x} \right]}{\left\{ \frac{1}{4n-1} \left[ \left( \int_{i=1; j=1}^{n;4} x_{ij} dx_{ij} \right) - \bar{x} \right]^2 \right\}^{1/2}}; $$

$$ \sigma' = \frac{\frac{x_{ij}}{4n-1} \left[ \left( \int_{i=1; j=1}^{n;4} x_{ij} dx_{ij} \right) - \bar{x} \right]}{\left( \frac{1}{4n-1} \right)^{1/2} \left[ \left( \int_{i=1; j=1}^{n;4} x_{ij} dx_{ij} \right) - \bar{x} \right]}; $$

$$ \sigma' = \frac{\frac{x_{ij}}{4n-1}}{\left(\frac{1}{4n-1}\right)^{1/2}} = \frac{x_{ij}}{4n-1} (4n-1)^{1/2} = \frac{x_{ij}}{\sqrt{4n-1}}. \quad (4) $$

We then establish that the σ-range is

$$ \bar{\sigma} = \bar{x} \pm \frac{x_{ij}}{\sqrt{4n-1}}. \quad (5) $$

# 4 Results

We calculate the values of the INC (Table 1) of all types, in all months studied, for each State. Figures 1 and 2 show the evolution of σ related to the INC for support services scenarios in Amapá and São Paulo. The curves map Equation 5 and were obtained by smoothing the evolution of the standard deviations, mathematically assuming that there would be a continuous variation of demands. The maximum amplitude of the standard deviation was 2.25 at São Paulo and 2.33 at Amapá against 1.29 at Brazil as a whole. Amapá state had its maximum in network demands, while São Paulo in hardware demands. The highest maximum amplitude of the standard deviation shall be taken as the starting point for both the development of a generic SLA (compliance for hardware, software, network and application) and an incremental sigma quality program to be developed further. Although the available data, organized on a per calendar-month series, is not so extensive in time, the consideration of a variable standard deviation previously establishes a floating pattern representative of the uncertainties inherent to IT services; this is so because they are in process of undergoing improvements from the "flooring" of quality to which we refer with the aid of Equation 5.
Evolution of the Standard Deviation for Amapá

<figure><img src="image_2.png" /><figcaption>Figure 1: Standard deviation of INC for Amapá, since October 2011 until February 2012.</figcaption></figure>

Evolution of the Standard Deviation for São Paulo

<figure><img src="image_3.png" /><figcaption>Figure 2: Standard deviation of INC for São Paulo, since October 2011 until February 2012.</figcaption></figure>
Clearly, the scenarios are distinct in accordance with local realities. The σ-range of Amapá proved to be different from São Paulo. This geographic differ-entiation points to the inadequacy of noncompliance overall accreditation to a whole package of services without accurate quantitative and qualitative criteria, and more: it shows that hard accreditation trivializes the work of the professional of quality, going in the opposite hand of the goals of IT Governance. Obviously, other considerations about quality can and should be made based on verified evidence.

**5 Conclusion**

This article presented a technique for initial measurement of noncompliances in IT support services based on the concept of sigma evaluation and considering Brazilian geographical variety. The proposal technique can be applied in all or-ganizations with low maturity level of IT support services. The technique was applied only to noncompliances by misclassification of support services at Min-istry of Work and Employment – Brazil, assuming the need for quality control with the introduction of an abstract monitoring zone called σ-range over the pro-posed Indicator of Noncompliance (INC). By establishing this control range, the technique has defined a quality "flooring" determined from the consideration of a floating standard deviation on INC as a preliminary approach to realistic qual-ity programs mainly in government institutions. The technique was conceived having in mind the reduction of technicians travel costs and environmental im-pact of unnecessary air traffic, but it may be considered as an option to analyze other features such as human resources, time of attendance, etc..

**Acknowledgments.** We thank Moisés Alencar for the integral support.

**References**

1. Barçante, L. Qualidade Total, uma Visão Brasileira: O Impacto Estratégico na Uni-versidade e na Empresa. Rio de Janeiro: Campus, 1998, 175 p.

2. Campos, M. Glossário Six Sigma. Banas Qualidade, São Paulo, v. 9, n. 96, 2000, p. 35

3. Chakravorty, S. Six Sigma Programs: An Implementation Model. Int. J. Production Economics 119, 2009, pp. 1-16

4. Colenghi, V. O & M e Qualidade Total: Uma Integração Perfeita. 2. ed. Rio de Janeiro: Qualitymark, xiv, 2003, 275 p.

5. Crosby, P. Quality is Free. New York: New American Library, 1979

6. Deligonul, S. As Armadilhas Ocultas na Hora da Decisão. Banas Qualidade, São Paulo, v. 9, n. 96, 2000, p. 146

7. Garvin, D. Managing Quality. New York: Free Press, 1988

8. Green, C. Os Caminhos da Qualidade. São Paulo: Makron Books, Ed. SENAC/SP, 1995, 203 p.

9. Juran, J. Quality Control Handbook. 4th ed. New York: McGraw-Hill, 1988
10. Medori, D. & Steeple, D. A Framework for Auditing and Enhancing Performance Measurement Systems. International Journal of Operations and Production Management, 20, n. 5, 2000, pp. 520-533

11. Paladini, E. Gestão da Qualidade no Processo: A Qualidade na Produção de Bens e Serviços. São Paulo: Atlas, 1995, 286 p.

12. Perez Wilson, M. Seis Sigma: Compreendendo o Conceito, as Implicações e os Desafios. Rio de Janeiro: Qualitymark, 1999, 283 p.

13. Shafer, S. & Moeller, S. The Effects of Six Sigma on Corporate Performance: An Empirical Investigation. Journal of Operations Management, 30, 2012, pp. 521-532

14. Silaghi, G. et al. A Time-Constrained SLA Negotiation Strategy in Competitive Computational Grids. Future Generation Computer Systems, 28, 2012, pp. 1303-1315