# Logistic Operator Selection with Capacity of Storage and Transport Frozen Product Using Multicriteria Decision

Geraldo De Oliveira Neto, André Henriques Librantz, Washington De Sousa

► To cite this version:

Geraldo De Oliveira Neto, André Henriques Librantz, Washington De Sousa. Logistic Operator Selection with Capacity of Storage and Transport Frozen Product Using Multicriteria Decision. IFIP International Conference on Advances in Production Management Systems (APMS), Sep 2014, Ajaccio, France. pp.379-386, 10.1007/978-3-662-44739-0_46. hal-01388517

HAL Id: hal-01388517

https://inria.hal.science/hal-01388517

Submitted on 27 Oct 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Logistic operator selection with capacity of storage and transport frozen product using multicriteria decision

Geraldo Cardoso de Oliveira Neto¹, André Henriques Librantz¹, Washington Carvalho de Sousa¹

¹ Nove de Julho University-UNINOVE, Industrial Engineering Post-graduation Program, Av. Francisco Matarazzo, 612, São Paulo, Brazil
{Geraldo Cardoso de Oliveira Neto, geraldo.prod@ig.com.br}

**Abstract.** The aim of this work consisted in applying the Analytic Hierarchy Process (AHP) by considering the generic criteria and sub-criteria to select logistic operator to store and transport frozen products. The outcome of the AHP analysis is a preference priority for each alternative operator describing the expected performance level. Results pointed that the proposed model could be considered as a good alternative to this problem, thus contributing to achieve the goals of the organization.

**Keywords:** Selection, Logistic Operator, Multicriteria decision, Analytic Hierarchy Process

## 1 Introduction

The analytic hierarchy process (AHP) is a structured technique for organizing and analyzing complex decisions. The AHP method was developed during the 70's by Thomas Saaty [1] to solve complex problems that involves multicriteria decisions [2] and has been extensively studied and refined since then. The applications of AHP to complex decision situations have produced extensive results in problems involving planning, resource allocation, priority setting, and selection of logistic operators, among alternatives.

AHP's use for logistic operator selection consists in an important aspect to identify and consider the criteria [3, 4, 5, 6], by knowing the contractor demands in order to achieve the main goal [7, 8], and evaluate alternatives and contract logistic operators that are more efficient [9] adding value to the supply chain [10], by using a evaluation method of analysis before the contractual formalization [11, 12, 13], to verify whether it is worth to outsource or internalize the activities [8].

Usually the focus is on selecting a service provider that has the relational capacity to facilitate the business integration, operation and quality treatment [14], that turns out more feasible to collaboration [15], including establishing common goals to control the performance and the strategic alignment of the partnership [16] to manage the service level [17].
Logistic Operator Selection with storage and frozen product transport capacities using
multicriteria decision2

Regarding the quality analysis, it was developed a model based on
SERVQUAL (quality dimensions), taking into account the criteria utilized for logistic
operator selection [18].

It has been found some works, in which the AHP method was used to select
logistic service providers to operate reverse logistic that could be considered a
complex decision, once it is a recent operation, that needs reliability [19, 20, 21, 22],
with contractual renegotiation guaranteed politics [22] and mainly, really contributes
with sustainability by reducing waste on transport and storage [15]. Other researchers
used the AHP to select logistic operators to act on manufacturing of contractor [23,
24].

In Table 1 it is shown the logistic operator selection generic criteria and sub-criteria developed by action research in logistic services outsourcing processes on Brazil in the last six years. But, with a deep revision in literature it was possible to identify the researches that corroborate with the criteria and sub-criteria established on practice, making it possible to assure that it is made of a 15 criteria and 67 sub-criteria basis consolidated on theoretical research field.

In the literature review it wasn't identified researches that utilize the AHP to
select logistic operators that storage and transport frozen refrigerated products and it
was noted that it misses a more complete criteria and sub-criteria basis to apply the
multicriteria analysis in outsourcing processes. Some researches produced fragmented
results [3, 8, 9, 14, 18, 22] that makes it impossible to make decision.

In this context, the aim of this work consists in applying the AHP with
support of the generic criteria and sub-criteria basis of logistic operator selection of
storage and transport of frozen products.

## 2 Methodology

At first, the literature revision [25] was performed in the following data
bases: Proquest, Ebsco, Science Direct, Emerald, Capes and academic Google, with
the following keywords: (i) "Analytic hierarchy process" AND "outsourcing" AND
"logistics"; (ii) "Analytic hierarchy process" AND "selection" AND "third party
logistics"; (iii) "Analytic hierarchy process" AND "selection" AND "third-party
logistics" e (iv) "Analytic hierarchy process" AND "selection" AND "service
provider" with the objective of electing the articles that utilized the AHP to select the
logistic operators, summarizing 24 researches, that were submitted to content analysis
to categorize the data in a spread sheet [26] of the main themes and criteria utilized.

The utilized method was action research, in which the researcher is involved,
along with the members of the analyzed organization, to deal with a problem,
specifically in the appropriate selection of a logistic operator of storage and transport
of frozen load [27], with that, the researcher participates in the process [28, 29, 30].

The process of action research involves five stages [28, 29, 30]:

(i) Action research planning: – in this step, it was identified along with
the contractor, the insecurity in selecting a logistic operator to move and transport
Logistic Operator Selection with storage and frozen product transport capacities using multicriteria decision3

frozen products, which brought up the following research's question: how to select a logistic operator appropriate to store and transport frozen products.

(ii) Instruments to collect field data: – active participation in meetings, focal groups and documents analysis. The focal groups or observation groups generate important lessons about the observed phenomenon [30].

(iii) Action Plans: - the decision making relative to logistic operator selection inevitably includes a consideration and evidence based in multicriteria for the decision making [3, 4, 5, 2, 6]. With that in mind, we adopted 16 criteria and 67 sub-criteria of logistic operator selection [31].

<table><caption>Table 1. Criteria and Sub-criteria of logistic operator selection.</caption><thead><tr><td>Criteria</td><td>Sub-criteria</td><td>Corroborating Researches</td></tr></thead><tbody><tr><td>Cost</td><td>Transport; Storage and handling; Delay on supplying and Devolutions.</td><td>[23, 21, 7, 17, 9, 8]</td></tr><tr><td>Financial stability</td><td>Investment Capacity; Financial Health and Indebtedness.</td><td>[3, 14, 22, 9, 8]</td></tr><tr><td>Adaptability</td><td>Logistic Operator in focus with the Contractor; Flexibility in the treatment, Cultural Compatibility and Common Planning.</td><td>[17, 8]</td></tr><tr><td>Operation Infrastructure</td><td>Distribution Centers Location; Facilities Adequacy; Work Security; Loading; Truck Conditions; Equipment required for loading; Receipt in JIT without fault in the Logistic Operator; Equipment required in receipt; Equipment required for Moving; Handling Pallets without fault; Storage and Stacking in the warehouse; Equipment required in the warehouse; Separation to dispatch without fault; Uniting and Own cars; Equipment required to transport and dispatch of the order without fault in JIT.</td><td>[18, 3, 22, 16, 17, 8, 20, 6]</td></tr><tr><td>Operation Management</td><td>Management of: people; loading; receipt; storage; transport; quality and information.</td><td>[14, 22, 19]</td></tr><tr><td>Flexibility</td><td>Treatment capacity; Negotiated nuances and Treatment Service to the Clients.</td><td>[23, 3, 21, 15, 9, 19, 8]</td></tr><tr><td>Credibility</td><td>Business Reputation; Logistic Partnership and amount of Contracts per year.</td><td>[3, 14, 22, 6]</td></tr><tr><td>Experience</td><td>Service efficiency; Product's amount of experience; Amount of time in the market and Clients range.</td><td>[14, 7, 20, 6]</td></tr><tr><td>Quality</td><td>Quality Certification; Physical Integrity Compatibility; Logistic Operator seeks improvement in the process and Traceability.</td><td>[23, 14, 21, 22, 9, 8]</td></tr><tr><td>Information Technology</td><td>Easy Communication (Electronic Data Interchange - EDI); (Enterprise Resource Planning - ERP); (Warehousing Management System- WMS); (Transport Management System TMS) and Routing (Global Positioning System - GPS).</td><td>[23, 9, 19, 17, 20]</td></tr><tr><td>Human Resources</td><td>Employees Qualification; Capacity of Working in Group; Capacitation and practical test, Clothing and neatness of the employees.</td><td>[14, 21, 22]</td></tr><tr><td>Reliability</td><td>Complete Order Dispatching; Classified Information Leaking; Security against load stealing and Fines due to no-treatment.</td><td>[18, 8]</td></tr><tr><td>Responsibility</td><td>Operation in JIT and Dispatch in short term.</td><td>[18, 23, 21, 22, 15, 16, 17, 8]</td></tr><tr><td>Empathy</td><td>After-Sell and Commitment</td><td>[18, 14]</td></tr><tr><td>Environmental</td><td>Certification ISO 14000; Cleaner Production; Reverse Logistic and Training in environmental education</td><td>[3, 22, 15, 16, 19, 20]</td></tr></tbody></table>

Source: [31, 32] corroborated with some researches

In fact the process of decision making is commonly complex. The multicriteria Decision Analysis could support the decision maker to solve problems in which there are many goals to be reached at the same time. Three sets are initially
Logistic Operator Selection with storage and frozen product transport capacities using
multicriteria decision4

considered in the formulation of multicriteria decision problems: the set of
alternatives, the set of criteria and the set of consequences [33]. The process consists
of the following stages: (1) Define the alternatives; (2) Define the relevant criteria to
the decision problem; (3) Evaluate the alternatives related to the criteria; (4) Evaluate
the relative importance of each criteria and (5) Determine the global evaluation of
each alternative.

Along the methods developed in the Multicriteria Decisions research, the
most well-known is the Analytic Hierarchy Process – AHP, developed by Thomas
Saaty [1] to identify the relevant criteria through binary correlation, which allows to
associate a priority value above other elements in a numerical scale, merging an
hierarchical structure of the problem of decision with support of the mathematical
modeling.

As an example, when using the decision matrix A, the AHP method
calculates the partial results of the A set inside each criteria vi(Aj), j = 1, ..., n, called
impact value of the j alternative in relation to the i alternative, in which these results
represent numerical values of the given data by the decision maker in each alternative
comparison. The results are standardized by the expression 1:

$$
\Sigma_i=1 \quad v_i(A_j) = 1, \text{ for } j = 1, \dots, n; \tag{1}
$$

where n is the number of alternatives or compared elements. Each part of the sum
consists in: $v_i(A_j) = a_{ij} / \Sigma_i=1 a_{ij}$, for $j = 1, ..., n$.

This expression relates the priority vector of the alternative *i* in relation to the C<sub>k</sub>
criteria be: $v_k(A_i) = \Sigma_j=1 vi(A_j) / n$, for $i = 1, ..., n$

After having the priority vector or the impact value of the alternatives under each C<sub>k</sub> criteria, the next step is the criteria levels. In this case, we adopt again the verbal scale to the classification pair-to-pair of the criteria, that are standardized by the expression 2:

$$
\mathbf{wi}(C_j) = C_{ij} / \Sigma_{i=1}^{k} C_{ij}, \text{ for } j = 1, \dots, m; \quad (2)
$$

where m is the number of criteria in the same level. The priority vector is obtained by:

$$
w_i(C_i) = \frac{\Sigma_i=1 w(C_i)}{m}, \text{ for } i = 1, \dots, m; \quad (3)
$$

Finally, the final values of the alternatives are obtained through an
aggregation process, standardized by the expression 4, in which:

$$
f(A_j) = \sum_{i=1}^{k} w(C_i) * v_i(A), \text{ for } j = 1, \dots, n; \quad (4)
$$

where n is the number of alternatives

(iv) Implanting Actions – scale selection for decision making to identify
the most relevant contractual criteria. The Table 2 shows the scale.

<table><caption>Tabela 2. Criteria Evaluation Scale</caption>
   <thead>
    <tr>
     <td>
      Value
     </td>
     <td>
      Definiton
     </td>
     <td>
      Explanation
     </td>
    </tr>
   </thead>
   <tbody>
    <tr>
     <td>
      1
     </td>
     <td>
      Same Importance
     </td>
     <td>
      Both criteria contribute in the same way to the objective.
     </td>
    </tr>
    <tr>
     <td>
      3
     </td>
     <td>
      Little more Important
     </td>
     <td>
      The analysis and experience prove that a criteria is a little more important than the other.
     </td>
    </tr>
    <tr>
     <td>
      5
     </td>
     <td>
      Much more Important
     </td>
     <td>
      The analysis and experience prove that a criteria is much more important than the other.
     </td>
    </tr>
    <tr>
     <td>
      7
     </td>
     <td>
      A lot more Important
     </td>
     <td>
      The analysis and experience prove that a criteria is a lot more important than the other.
     </td>
    </tr>
    <tr>
     <td>
      9
     </td>
     <td>
      Hugely more Important
     </td>
     <td>
      Without doubt, a criteria is predominant over the other one.
     </td>
    </tr>
   </tbody>
  </table>

Source: based on Saaty [1].
(v) Discuss Results – the final stage implies in learning through reflection among the results of the action of the participants [29] to verify which logistic operator can successfully fulfill the operational capacities (criteria) required as described in the following stage.

**3 Results and Discussion**

The main focus of the corporation is the product industrialization. Thus, they decided to outsource its storage and frozen product transport processes, taking into account the dispatching capacity of the operators. So, three logistic operators have been chosen to be analysed by AHP method.

***Numerical Application***

In this section, the multi-criteria group decision model is applied to a procedure for selecting logistic operators. In order to obtain the Multicriteria numerical application it was used the AHP method based on the 16 criteria and 67 sub-criteria, allowing an identification of seven more critical criteria. The main criterion to be observed in the logistic operators' evaluation is the transport responsiveness. However, to make the responsiveness capacity better to the market, it was required criteria of infrastructure (physical resources) to improve the storage capacity, that was missing physical space in the appropriate floor to allocate 134 tons a day – 4 freezing tunnels that allocate 26 docks and infrastructure criteria (physical resources) to improve the transport capacity – strategic localization near the beltway to deliver the products quickly.

This way, the main concern of the contractor was the complementation of storage and transport capacities, variables associated to the responsiveness and infrastructure criteria. It is important to note that the contractor was not only worried about analyzing the physical resources to store and transport, as well, but the managing capacity of the logistic operator within these activities, by considering the efficiency indicators implantation required integrated with fee applications in case of non-effectiveness.

The results pointed that the qualitative analysis should precede the AHP application with the objective of knowing the demands of the contractor regarding the capacities. In most published works, decision makers applied the multicriteria analysis [3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 21, and 32], neglecting the qualitative evaluation. Many authors pointed out the need of relational capacity without a deeper analysis of the other required capacities [14, 15, 17, and 28]. The relational capacity and the quality capacity can be considered generic, because this is a common necessity in any outsourcing process, being required a deeper analysis on the operational routine of the contractor to identify specific capacities, as shown in the results.

Table 3 shows the consequence matrix with critic criteria obtained after the model application. The numbers were normalized to obtain the relative importance (pair-wise comparison).
Logistic Operator Selection with storage and frozen product transport capacities using multicriteria decision6

<table><caption>Table 3. Consequence Matrix.</caption><thead><tr><th>Criteria</th><th>Operation<br>Infrastructure</th><th>Operation<br>Management</th><th>Quality</th><th>Technology</th><th>Human<br>Resources</th><th>Reliability</th><th>Responsiveness</th></tr></thead><tbody><tr><td>Operation<br>Infrastructure</td><td>1,000</td><td>1,000</td><td>0,111</td><td>0,111</td><td>0,111</td><td>0,111</td><td>0,111</td></tr><tr><td>Operation<br>Management</td><td>1,000</td><td>1,000</td><td>0,111</td><td>0,111</td><td>0,111</td><td>0,111</td><td>0,111</td></tr><tr><td>Quality</td><td>3,000</td><td>1,000</td><td>3,000</td><td>0,333</td><td>0,333</td><td>0,333</td><td>0,333</td></tr><tr><td>Technology</td><td>3,000</td><td>1,000</td><td>0,600</td><td>0,333</td><td>0,333</td><td>0,333</td><td>0,333</td></tr><tr><td>Human<br>Resources</td><td>3,000</td><td>1,000</td><td>0,600</td><td>0,333</td><td>0,333</td><td>0,333</td><td>0,333</td></tr><tr><td>Reliability</td><td>3,000</td><td>1,000</td><td>0,600</td><td>0,333</td><td>0,333</td><td>0,333</td><td>0,333</td></tr><tr><td>Responsiveness</td><td>9,000</td><td>1,000</td><td>1,800</td><td>1,000</td><td>1,000</td><td>1,000</td><td>1,000</td></tr></tbody></table>

Based on the seven critical criteria, the three logistic operators were analyzed. Table 4 presented the decision matrix and the results obtained for each alternative.

<table><caption>Table 4. Decision Matrix</caption><thead><tr><th>Alter-<br>native</th><th>Operation<br>Infrastructure</th><th>Operation<br>Management</th><th>Quality</th><th>Technology</th><th>Human<br>Resources</th><th>Relia-<br>bility</th><th>Respons-<br>siveness</th></tr></thead><tbody><tr><td>LO 1</td><td>0,195</td><td>0,210</td><td>0,397</td><td>0,162</td><td>0,302</td><td>0,250</td><td>0,203</td></tr><tr><td>LO 2</td><td>0,520</td><td>0,519</td><td>0,503</td><td>0,543</td><td>0,397</td><td>0,491</td><td>0,593</td></tr><tr><td>LO 3</td><td>0,284</td><td>0,271</td><td>0,101</td><td>0,295</td><td>0,302</td><td>0,259</td><td>0,203</td></tr></tbody></table>

Once obtained the decision matrix, it was possible to evaluate the score and classify the logistic operators, indicating which of them was considered more able to store and transport frozen products.

The logistic operator 2 (score 0.41) was considered more able to improve the dispatch responsiveness of the contractor in the metropolitan region of São Paulo, because it is in a strategic localization near the beltway, allowing better product outflowing, among having better consolidated infrastructure concerning equipment, as for an example, term kings to control the temperature, and freezing tunnels in the warehouse.

The second-best (score 0.23) was the logistic operator 1 that presented low capacity of infrastructure that could prejudice the dispatching responsiveness.

Finally, the logistic operator 3 (score 0.21), that presented lack of control in the efficiency of the provided services, making it impossible to analyze the punctuality indicators of dispatch.

Both logistic operators 1 and 3 were not localized near a beltway, which was considered, by the contractor, a crucial aspect.

This results corroborate with the researches of [19, 20] related to the development of a ranking to avoid the opportunist behavior of logistic operator regarding the possibility of offering sets of generic services that do not meet the contractor demands and could lead to the outsourcing failure.

## 7 Conclusions

In this work, the AHP technique was used in the selection of logistic operators to store and transport frozen products. The AHP is a structured technique
Logistic Operator Selection with storage and frozen product transport capacities using
multicriteria decision7

for organizing and analyzing complex decisions, and coub be useful, especially in
complex decision processes, once it enables people to make decisions involving many
kinds of concerns including planning, setting priorities, selecting the best among a
number of alternatives, and allocating resources. Complex problems or issues
involving value or subjective judgments are suitable applications of the AHP method.
Its approach of division in hierarchical criteria and alternative correlation with
criteria, allows an easy comprehension and better evaluation of the problem. Taking
into account that many strategic decisions, such as logistic operator selection, are
many times, made without any methodical support, the proposed model can be
considered as a good alternative to the investigated problem, since it can support the
decision process, which can contribute to increase efficiency of the supply chain of
the organization.

References

1. SAATY, T.L.: Decision Making with the Analytic Hierarchy Process. International Journal of Services Sciences. 1, 83-98 (2008)

2. ALMEIDA, A. T. O conhecimento e o uso de métodos multicritério de apoio à decisão. Editora Universitária UFPE (2011)

3. IÂNEZ, M.M., CUNHA, C.B.: Uma metodologia para a seleção de um provedor de serviços logísticos. Revista Produção Online. 16, 394-412 (2006)

4. KASTURE, S., QURESH, M.N., KUMAR, P., GUPTA, I.: FAHP Sensity Analysis for selection of Third party Logistics (3PL) service providers. Journal of Supply Chain Management. 4, 41-60 (2008)

5. CHEN, Y.M., GOAN, MENG-JONG., HUANG, PEI-NI.: Selection process in logistics management – a view from third party logistics provider. Production Planning and Control. 22, 308-324 (2011)

6. ASUQUO, M., COWARD, L., YANG, Z.: Modeling selection of third party ship management services. Case Studies on Transport Policy. Journal of Manufacturing Technology Management 2, 28-25 (2014).

7. DAIM, T.U., UDBYE, A., BALASUBRAMANIAN,A.: Use of analytic hierarchy process (AHP) for selection of 3PL providers. Journal of Manufacturing Technology Management. 24, 28-51 (2013)

8. REIS, L.P., LADEIRA, M.B., FERNANDES, J.M.: Contribution of the analytic hierarchy Process (AHP) method for supporting the decision to outsource or internalize activities in the context of technology-based company. Revista Produção Online. 13, 1325-1354 (2013)

9. HO, W., HE, T., LEE, C.K.M., EMROUZNEJAD, A.: Strategic logistics outsourcing: An integrated QFD and Fuzzy AHP approach. Expert Systems With Applications: An International Journal. 39, 10841-10850 (2012)

10. ROUTROY, S.: Selection of Third Party Logistics Provider in Supply Chain. International Journal of Services Technology and Management. 12, 23-23 (2009)

11. RAJESH, R., PUGAZHENDHI, S., MURALIDHARAN, C.: Development of a composite model for selection of third party logistics service provider. International Journal of Electronic Customer Relationship Management. 3, 375-375 (2009)

12. BANSAL, A., KUMAR, P.: 3PL selection using hybrid model of AHP-PROMETHEE. International Journal of Services and Operations Management. 14, 373-373 (2013)

13. BAYAZIT, S., KARPAK, B.: Selection of a third party logistics service provider for an aerospace company: an analytical decision aiding approach. International Journal of Logistics Systems and Management. 5, 382-404 (2013)
Logistic Operator Selection with storage and frozen product transport capacities using multicriteria decision8

14. GOL, H., ÇATAY, B.: Third-party logistics provider selection: insights from a Turkish automotive company. Supply Chain Management: An International Journal. 12, 379-384 (2007)

15. SASIKUMAR, P., HAQ, A.N.: A multi-criteria decision making methodology for the selection of reverse logistics operating modes. International Journal of Enterprise Network Management. 4, 68-79 (2010)

16. XIU, G., CHEN, X.: The Third Party Logistics Supplier Selection and Evaluation. Journal of Software. 7, 1783-1790 (2012)

17. PENG, J.: Selection of Logistics Outsourcing Service Suppliers Based on AHP. Energy Procedia. In: International Conference on Future Electrical Power and Energy Systems. 17, 595-601 (2012)

18. SO, SOON-HOO., KIM, J., CHEONG, K., CHO, G.: Evaluating the service quality of third party logistics service providers using the analytic hierarchy process. Journal of information Systems and technology Management. 3, 261-270 (2006)

19. SENTHIL, S., SRIRANGACHARYULU, B., RAMESH, A.: A decision making methodology for the selection of reverse logistics operating channels. Revista Produção Online. 38, 418-428 (2012).

20. SENTHIL, S., SRIRANGACHARYULU, B., RAMESH, A.: A robust hybrid multi-criteria decision making methodology for contractor evaluation and selection in third-party reverse logistics. Expert Systems with Applications: An International Journal. 41, 50-58 (2014)

21. CHIANG, Z., TZENG, GWO-HSHIUNG.: A third party logistics provider for the best selection in Fuzzy dynamic decision environments. International Journal of Fuzzy Systems. 11, 1-10 (2009)

22. BHATTI, R.S., KUMAR, P., KUMAR, D.: Analytical Modeling of third party service provider selection in lead logistics provider environments. Journal of Modelling in Management. 5, 275-286 (2010).

23. ROSA, E.P.S., SELLITO, M.A., MENDES, L.W.: Avaliação multicriterial de desempenho e separação em aglomerados de fornecedores críticos de uma manufatura OKP. Revista Produção Online. 16, 413-428 (2006).

24. GREWAL, C.S., SAREEN, K.K., GILL, SUMIT.: A multicriteria logistics-outsourcing decision making using the analytic hierarchy process. International Journal of Services Technology and Management. 9, 1-13 (2008)

25. COOPER, H.M, LINDSAY, J.L.L: Research synthesis and meta-analysis. Sage Publications, (1998)

26. BARDIN, L.: El análisis de contenido. Ediciones akal. Madrid (1986)

27. BRYMAN, A.: Research Methods and Organization studies. Unwin Hyman. London (1989)

28. WESTBROOK, R.: Action Research: a new paradigm for research in production and operations management. International Journal of Operations and Production Management. 15, 6-20 (1995)

29. COUGHLAN, P., COGHLAN, D.: Action research for operations management. International Journal of Operations & Production Management. 22, 220-240 (2002)

30. THIOLLENT, M.: Metodologia da pesquisa-ação. 15ªed. Cortez. São Paulo (2007)

31. OLIVEIRA NETO, G.C., GODINHO FILHO, M., MEMORIAN, M.A.G., COSTA, B.K., SILVA, D.: Estratégia em Terceirização para contratação de Operador Logístico: uma abordagem teórica sob a ótica da teoria baseada em recursos. Revista Gestão & Produção (2014)

32. OLIVEIRA NETO, G.C.: Integração complexa entre empresa contratante e operador logístico: Critérios para contratação. Dissertação. UNIP. São Paulo (2008)

33. DENG, Y., CHAN, F. T. S.: A new fuzzy dempster MCDM method and its application in supplier selection. Expert Systems with Applications: An International Journal. 38, 9854-9861 (2011)