# Scheduling Performance Evaluation of Logistics Service Supply Chain Based on the Dynamic Index Weight

Authors: Weihua Liu; Zhicheng Liang; Shuqing Wang; Yang Liu; Wenchen Xie
Journal: Mathematical Problems in Engineering
 (2014)

---

## Abstract

Scheduling is crucial to the operation of logistics service supply chain (LSSC), so scientific performance evaluation method is required to evaluate the scheduling performance. Different from general project performance evaluation, scheduling activities are usually continuous and multiperiod. Therefore, the weight of scheduling performance evaluation index is not unchanged, but dynamically varied. In this paper, the factors that influence the scheduling performance are analyzed in three levels which are strategic environment, operating process, and scheduling results. Based on these three levels, the scheduling performance evaluation index system of LSSC is established. In all, a new performance evaluation method proposed based on dynamic index weight will have three innovation points. Firstly, a multiphase dynamic interaction method is introduced to improve the quality of quantification. Secondly, due to the large quantity of second-level indexes and the requirements of dynamic weight adjustment, the maximum attribute deviation method is introduced to determine weight of second-level indexes, which can remove the uncertainty of subjective factors. Thirdly, an adjustment coefficient method based on set-valued statistics is introduced to determine the first-level indexes weight. In the end, an application example from a logistics company in China is given to illustrate the effectiveness of the proposed method.


---

## Body

## 1. Introduction

Service supply chain is a new trend in supply chain research [1], and logistics service supply chain (LSSC) is a type of service supply chain centered on the cooperation of logistics service capacity [2, 3]. The main structure of the LSSC is the mode where the functional logistics service providers (FLSPs) flow to the logistics service integrator (LSI) and then to the manufacturers or retailers [2, 3]. FLSPs consist of traditional functional logistics enterprises, such as transportation and storage enterprises, among others, whose service function is simple and standardized, and the business is limited within a certain area; they are integrated as the suppliers by the LSI when a domestic or international logistics service network is established. For instance, as a LSI, Baogong logistics company in China integrates over 500 warehousing companies and 1200 highway transport companies as their FLSPs to provide personalized logistics services for many world famous companies such as P&G and Unilever. Therefore, managing logistics service operation has become a core content of LSSC management.Scheduling is an important part of service operations management, which is the activity of allocating and coordinating the enterprise’s resources, such as the workforces, machines, vehicles, and material, and stipulates the tasks to be executed by employees within a certain period [4, 5]. Scheduling has an important influence on enterprise performance by determining the order lead time, delivery flexibility, machine and material usage, and staff workload [4, 5]. Reference [6] pointed out that a process can be managed only when its performance can be measured. Therefore, studying the evaluation method of scheduling performance can not only compare the performance within each period, but also identify the problems during scheduling so as to improve the scheduling process. There have already been studies focusing on performance evaluation of LSSC. Reference [7] studied the effect of time scheduling on LSSC scheduling. But existing articles only consider the results evaluation [8], using KPIs, for instance, rather than consider the three phases: before scheduling, during scheduling, and after scheduling.In terms of scheduling performance evaluation method, much scheduling research concentrates on improving existing methods for specific classes of scheduling situations using techniques from operations research or mathematical programming and, lately, also from artificial intelligence [4, 5, 9, 10]. The majority of these studies pay attention to the performance evaluation of scheduling in a single enterprise or sector. However, different from the scheduling performance evaluation of an individual enterprise, for LSSC, the following problems should be paid more attention to and should be solved.(1)
LSSC is composed of many members (e.g., one LSI and many FLSPs), and the impact of scheduling activities from these members on the performance should be considered. So, how can we give full consideration about the different demands of these members when the LSSC performance indexes are designed?(2)
Different from solid product, service contains the characteristics of customer influence, intangibility, and inseparability; the scheduling indexes of LSSC may include both quantitative index and qualitative index ones. How to consider the qualitative indexes and quantitative indexes as a whole?(3)
Different from general project performance evaluation, scheduling activities are usually continuous and multiperiods. The activities of the previous period will have an impact on those of the next one. For example, along with the increasing of scheduling times, the understanding of decision-maker for each index weight may change [11], which leads to the adjustment of index weight. Therefore, index weight has the feature of dynamic change, so how to select a proper dynamic evaluation method is an important issue.In this paper, the performance evaluation index system of LSSC scheduling is proposed, and then an evaluation method based on dynamic index weight is adopted. In this method, firstly a multiphase dynamic interaction method is introduced by considering the difficulty of converting qualitative data into quantitative data. As a result, the quantitative results are more objective. Second, as there are many second-level indexes and dynamic variation is required, the maximum attribute deviation method is introduced to determine weight of second-level indexes. Thirdly, as there are few first-level indexes, an adjustment coefficient method based on set-valued statistics is introduced to determine the weight index. An application example from a logistics company in China illustrates that the method proposed is more suitable for scheduling performance evaluation of LSSC.This paper is organized as follows. In Section2, the literature review on evaluation methods of scheduling performance is presented. Section 3 proposes the performance evaluation index system of LSSC scheduling. Section 4 gives the evaluation method of dynamic index weight. Section 5 provides the innovations and advantages of the model proposed. The application in a logistics company to prove the effectiveness of this method is provided in Section 6. The last two sections put forward the main conclusions and future insights.


## 2. The Literature Review

Research about scheduling performance evaluation on LSSC mainly includes two aspects: one is the design of evaluation index system; the other is the evaluation method. Previous studies of supply chain performance mostly focused on manufacturing supply chain, but the research on SSC is an emerging field now. Thus, this section will review the performance evaluation research not only on manufacturing supply chain but also on SSC. The research on performance evaluation methods about scheduling management and supply chain management (SCM) will be discussed as well.


### 2.1. Performance Evaluation System of Supply Chain

Since the 1990s, the research about manufacturing supply chain performance evaluation has begun and gained many achievements [12]. The authoritative research institute of supply chain, Supply Chain Council (SCC), in 1996 presented 13 indexes about performance evaluation in supply chain operations reference (SCOR). Reference [13] established an evaluation system based on resources, output, and flexibility. Reference [14] studied why and how the SCM influenced the organization’s time performance. Reference [15] put forth the performance evaluation system based on four main processes of supply chain, planning, purchasing, assembling, and delivering. On the basis of the SCOR model, [16] proposed the performance evaluation system from the perspective of customers, internal processes, and shareholders. Reference [17] built up the performance evaluation system from the view of financing, customers, operation procedure, organization learning, and growth by adopting the balanced scorecard.With the development of SSC, more and more studies focus on the performance evaluation of SSC. Reference [18] made the definition of SSC from the perspective of professional services outsourcing and established a comprehensive performance evaluation system. Reference [19] explored the integration SCOR model with service business and developed a reference model of performance evaluation for service organizations. Reference [20] made a review on performance evaluation issues of SSC and constructed the performance evaluation system from the aspects of the strategic, tactical, and operational levels.In the field of LSSC, based on the characteristics of LSSC, [21] designed the performance evaluation system on the procedure joint process of LSSC and used an ANP method. Reference [7] proposed a time scheduling model and explored the effect of time delay coefficient and cost coefficient on the performance of LSSC.When considering the evaluation index, the qualitative indexes and quantitative indexes are different. Reference [13] stated that qualitative evaluations were vague and difficult to utilize in any meaningful way. Thus, quantitative performance measures were often preferred over such qualitative evaluations. However, the chosen numerical performance measure may not adequately describe the system's performance [22]. Therefore, many studies use both qualitative and quantitative indicators [23–25]. In these papers, the qualitative indexes are evaluated by questionnaire and they ignore the stability and the consistency of the respondents.


### 2.2. Scheduling Performance Evaluation Method

Moreover, the methods of scheduling performance evaluation are investigated by many scholars [26–28], but these studies are mostly emphasized by using the method of operation research or optimization algorithm to improve the performance [4, 5, 10] rather than performance evaluation. Reference [29] optimized the performance evaluation method of production scheduling within a workshop, but the method was not extended to supply chain environment. Reference [8] conducted an empirical study about performance evaluation indexes of scheduling with a three-part questionnaire.Dynamic evaluation method used in this paper is also applied in many fields. References [30, 31] examined air quality model’s changes in pollutant concentrations with dynamic evaluation method. Furthermore, the performance evaluation methods of supply chain are enriching with the development of the evaluation theories. Basically, there are several methods including balanced scorecard [17, 32], data envelopment analysis [33, 34], analytic hierarchy process [35, 36], analytic network process [3], and fuzzy evaluation method [37]. Different approaches can be applied to different environments.From the literature review, it can be found that performance evaluation research of supply chain has gained abundant achievements, but the research on scheduling performance evaluation of LSSC has not been reported before. Therefore, it is necessary to analyze the characteristics of the scheduling performance evaluation in depth and explore the evaluation index system and evaluation methods, which will provide scientific reference for improving the effectiveness and efficiency of LSSC scheduling.


## 2.1. Performance Evaluation System of Supply Chain

Since the 1990s, the research about manufacturing supply chain performance evaluation has begun and gained many achievements [12]. The authoritative research institute of supply chain, Supply Chain Council (SCC), in 1996 presented 13 indexes about performance evaluation in supply chain operations reference (SCOR). Reference [13] established an evaluation system based on resources, output, and flexibility. Reference [14] studied why and how the SCM influenced the organization’s time performance. Reference [15] put forth the performance evaluation system based on four main processes of supply chain, planning, purchasing, assembling, and delivering. On the basis of the SCOR model, [16] proposed the performance evaluation system from the perspective of customers, internal processes, and shareholders. Reference [17] built up the performance evaluation system from the view of financing, customers, operation procedure, organization learning, and growth by adopting the balanced scorecard.With the development of SSC, more and more studies focus on the performance evaluation of SSC. Reference [18] made the definition of SSC from the perspective of professional services outsourcing and established a comprehensive performance evaluation system. Reference [19] explored the integration SCOR model with service business and developed a reference model of performance evaluation for service organizations. Reference [20] made a review on performance evaluation issues of SSC and constructed the performance evaluation system from the aspects of the strategic, tactical, and operational levels.In the field of LSSC, based on the characteristics of LSSC, [21] designed the performance evaluation system on the procedure joint process of LSSC and used an ANP method. Reference [7] proposed a time scheduling model and explored the effect of time delay coefficient and cost coefficient on the performance of LSSC.When considering the evaluation index, the qualitative indexes and quantitative indexes are different. Reference [13] stated that qualitative evaluations were vague and difficult to utilize in any meaningful way. Thus, quantitative performance measures were often preferred over such qualitative evaluations. However, the chosen numerical performance measure may not adequately describe the system's performance [22]. Therefore, many studies use both qualitative and quantitative indicators [23–25]. In these papers, the qualitative indexes are evaluated by questionnaire and they ignore the stability and the consistency of the respondents.


## 2.2. Scheduling Performance Evaluation Method

Moreover, the methods of scheduling performance evaluation are investigated by many scholars [26–28], but these studies are mostly emphasized by using the method of operation research or optimization algorithm to improve the performance [4, 5, 10] rather than performance evaluation. Reference [29] optimized the performance evaluation method of production scheduling within a workshop, but the method was not extended to supply chain environment. Reference [8] conducted an empirical study about performance evaluation indexes of scheduling with a three-part questionnaire.Dynamic evaluation method used in this paper is also applied in many fields. References [30, 31] examined air quality model’s changes in pollutant concentrations with dynamic evaluation method. Furthermore, the performance evaluation methods of supply chain are enriching with the development of the evaluation theories. Basically, there are several methods including balanced scorecard [17, 32], data envelopment analysis [33, 34], analytic hierarchy process [35, 36], analytic network process [3], and fuzzy evaluation method [37]. Different approaches can be applied to different environments.From the literature review, it can be found that performance evaluation research of supply chain has gained abundant achievements, but the research on scheduling performance evaluation of LSSC has not been reported before. Therefore, it is necessary to analyze the characteristics of the scheduling performance evaluation in depth and explore the evaluation index system and evaluation methods, which will provide scientific reference for improving the effectiveness and efficiency of LSSC scheduling.


## 3. The Scheduling Performance Evaluation Index System of LSSC

The goal of LSSC operation management is to provide the customers with the best logistics service through the cooperation of LSI and FLSPs. LSSC scheduling means that LSI manages the logistics service capacities of different FLSPs uniformly and makes various service capacities coordinate with each other so as to meet customers' logistics service demand. LSSC scheduling involves many aspects, including order allocation scheduling, time flexibility scheduling, and process scheduling. In order to complete the scheduling, suitable organization and coordination are required through the information communication, human resource allocation, relationship coordination, and other activities.To evaluate the scheduling performance of LSSC, the evaluation index system should be given, because it provides significant value to users, which can tell the users how valuable the evaluated objects are [38, 39]. Some scholars have already proposed several performance evaluation systems of manufacturing supply chain, but these systems cannot be completely duplicated into service supply chain [40]. Therefore, the corresponding evaluation index system should be presented based on the characteristics of LSSC scheduling.First of all, the evaluation index system of LSSC scheduling should take the hierarchy into consideration. Reference [41] pointed out that supply chain management issues covered a wide range of enterprise activities from the strategic through the tactical to the operational levels. Therefore, evaluation index system in this study will be divided into three levels, strategic environment level, operating process level, and scheduling results level, corresponding to strategic level, tactical level, and operational level, respectively.Second, the index system should reflect the characteristics of supply chain and service. The essence of SCM is integration and coordination among members, while scheduling is the bridge of connecting upstream and downstream in SSC, whose performance evaluation should involve the coordination ability and cooperation effect [42–44]. Additionally, the service characteristics of SSC should be considered. References [45, 46] verified that the dimensions of performance evaluation for manufacturing supply chain are insufficient to evaluate the performance of SSC. Service is intangible and service delivery is quite different from product delivery, which should be considered in the index system design.Third, the index system should take into account the different members’ demand. According to [15], the main reason why few enterprises succeed to maximize the overall supply chain profit through integration and coordination is that they fail to be integrated completely and do not share performance evaluation systems among them. Thus, measurements should be shared and manipulated by all supply chain members. There are three principal members in LSSC, LSI, FLSP, and customers. The LSI and FLSP focus on all the three levels, while the customers pay more attention to scheduling results level.According to the consideration above, the scheduling performance evaluation index system of LSSC is proposed as in Table1. It must be noted that for the indexes needed by LSI and FLSP, both LSI and FLSP should be joint during evaluation, and the final index value should be determined by LSI and FLSP jointly.Table 1
The scheduling performance evaluation index system of LSSC.
First-level indexes
Evaluation subject
Factors
Second-level indexes
Description
Scheduling environmentC1
LSIFLSP
EnvironmentC11
Organization structureC111
Reasonable setting of scheduler responsibility and power
Information technologyC112
The usage of modern logistics information technology and MIS
Information collecting abilityC113
The availability and accuracy of information
Cooperation relationshipC114
Trust and coordinated operation between LSI and FLSP
Scheduling processC2
LSIFLSP
OperationsC21
Adaptability of schedulingC211
Adaptability for uncertain external environment
Timeliness of schedulingC212
Timeliness of initial release and schedule adjustment
Communication qualityC213
Understandability and efficiency of communication
Flexibility of schedulingC214
Time and cost needed to release or change a schedule
Scheduling resultsC3
Customer
Customer satisfactionC31
Customer complaints rateC311
Customer complaint about the delivered service
Fulfillment rate of customer orderC312
The rate of order fulfillment
On-time deliveryC313
Orders completed within the prescribed time.
Response time to demandC314
Speed for response to customer demand
LSIFLSP
Service qualityC32
Wrong scheduling rateC321
The number of schedule errors
Scheduling costC322
Cost of all the scheduling process
FLSP satisfactionC323
Satisfaction of FLSPs
Resource efficiencyC324
Efficiency of resource utilization


## 4. Scheduling Performance Evaluation Method Based on Dynamic Index Weight

In the section of introduction, there are problems in LSSC scheduling performance evaluation that should be solved. The first is how to quantify the qualitative indexes; the second is how to deal with the continuous and multiperiod characteristics of LSSC scheduling activities; and the third is how to implement the dynamic change of index weight. Thus, the scheduling performance evaluation method of LSSC based on the dynamic index weight is proposed. Section4.1 proposes an improved method of multiphase dynamic interaction to quantify the qualitative indexes. Section 4.2 uses an improved attribute deviation maximization method to determine the weight of second-level indexes. In Section 4.3, we propose an adjustment coefficient method based on set-valued statistics to adjust the first-level indexes’ weight. Section 4.4 shows the main application process of this method.The notations involved in the method are shown in Table2.Table 2
Notations used in the method.
Notations
Description
x
i
Theith index
s
j
Thejth evaluator
p
i
j
The score of theith index evaluated by the jth evaluator
P
t
Score matrix of thetth round
a
j
t
The difference of similarity of thejth evaluator from other group members in the tth round
u
j
t
The interaction coefficient of thejth evaluator in the tth round
p
i
t
*
The final score of theith index in the tth round
υ
t
Index stability for thetth round
ω
t
Index consistency for thetth round
ε
Stability threshold of the group information
η
Consistency threshold of the group information
p
i
*
Final score of theith index
A
(
a
i
j
)
Decision matrix
R
(
r
i
j
)
Normalized decision matrix
Φ
i
Set of the intervals of second-level indexes’ weights under theith first-level index
w
j
i
The weight of thejth second-level index under the ith first-level index
[
d
i
1
k
,
d
i
2
k
]
The adjustment range of theith first-level index evaluated by the kth evaluator
d
i
Aggregation point of theith first-level index
g
i
Ramification degree of the expert scoring for theith first-level index
e
i
Consistency of experts scoring of theith first-level index
w
i
t
The weight of theith first-level index in the tth round


### 4.1. Quantify the Qualitative Indexes Using the Method of Multiphase Dynamic Interaction

This method is applicable to quantify the qualitative index by multiphase dynamic interactive expert scoring. The thought is described as in the following: let some experts rate the qualitative indexes with several rounds. Every round can be regarded as a revision for last round till it reaches a relatively stable and consistent level. Suppose that there arem experts scoring for n qualitative indexes, and the higher the score, the better the performance of the index for both cost-type indexes and benefit-type indexes. The index set evaluated is X={x1,x2,…,xn} and the evaluator group set is S={s1,s2,…,sm}. The interval of scoring is [0,10] and the score matrix is P=(pij)n×m.


#### 4.1.1. Multiphase Dynamic Interaction and the Decision of Interaction Coefficient

The purpose of multiphase dynamic interaction is to give the evaluators a chance of revising their grading according to the group information of last round.Pt(t=1,2,…,l) is assumed as the score matrix of the tth round. In the process of multiphase dynamic interaction, the variation of an evaluator’s score between two continuous rounds can reflect the influence by the group information of last round. ajt can be used to indicate the similarity between the jth evaluator with other group members in the tth round,
(1)ajt=1m-1∑k=1,k≠jm(cos⁡θjkt-cos⁡θjkt-1),
where cos⁡θjkt=(pjt,pkt)/|pjt||pkt|=∑i=1npijtpikt/∑i=1n(pijt)2∑i=1n(pikt)2.During the course of different interaction phases, the influence of a group member is varied with score matrix. Thus,ujt is used to indicate the interaction coefficient of the jth evaluator in the tth round as follows:
(2)ujt=(1+ajt)∑j=1m(1+ajt).The interaction coefficient vector of thetth round is ut=(u1t,u2t,…,umt).


#### 4.1.2. The Aggregation of Scoring Information

The purpose of the aggregation is to obtain the comprehensive score of each index after thetth round scoring. In order to balance every evaluator’s opinion, the minimal deviation with the members’ scoring is used as the final score. By solving the following programming model, the final score of the ith index in the tth round can be reached as pit*(i=1,2,…,n;  t=1,2,…,l), where
(3)min⁡π(pit*)=∑i=1n∑j=1mujt(pijt*-pijt)2s.t.l{min⁡jpijt≤pijt*≤max⁡jpijtmmmlmmi=1,2,…,n,mlmmmmj=1,2,…,m.


#### 4.1.3. The Stability and Consistency of Group Information

After several rounds of interscoring, the opinions of group members are known to each other. So the group information tends to be relatively stable and consistent. The stability means the invariance of group information in consecutive rounds and the consistency represents the invariance between different members within the same round. According to the two indexes, the scoring termination condition can be determined.The indexes stability for thetth round is calculated as follows:
(4)υt=1-1mn∑i=1n∑k=1m(pikt-pikt-1)2.The stability vector is labeled asυ=(υ1,υ2,…,υl).The index consistency for thetth round is calculated as follows:
(5)ωt=1-1mn∑i=1n∑h=1m(piht-pi-t)2,
where pi-t=(1/m)∑k=1mpikt and the consistency vector is labeled as ω=(ω1,ω2,…,ωl).In order to determine the termination conditions of interscoring, two thresholds for stability (ε) and consistency (η) are given firstly. If there exists
(6)|1-υt|<ε,
we think that it passes the stability test. Meanwhile, according to the indexes consistency in the tth round, if there exists
(7)|1-ωt|<η
we believe that it passes the consistency test. When both the stability and the consistency tests are satisfied, the multiphase dynamic interaction can be stopped. In practical application, the stability is more emphasized, so the threshold of consistency can be looser than that of stability.


#### 4.1.4. The Aggregation of the Final Score Result

The comprehensive score of each index in different roundspit*(i=1,2,…,n;  t=1,2,…,l) will have an impact on the final score result. For these impacts, the induced ordered weighted averaging (IOWA) [47] operators are adopted to aggregate the index score in different rounds.Definition 1.
Set〈v1,f1〉,〈v2,f2〉,…,〈vl,fl〉 as l two-dimensional arrays. Let hw(〈v1,f1〉,〈v2,f2〉,…,〈vl,fl〉)=∑i=1lwiav-index(i); then the function hw is regarded as l dimensional IOWA operator, and vi is induced component of fi, where v-index(i) stands for the subscript of the ith large number among v1,v2,…,vl and w=(w1,w2,…,wl) is weighted vector, which is satisfied with ∑i=1lwi=1,  wi≥0,  i=1,2,…,l.Definition1 indicates that the value of IOWA is obtained from the ordered weighted average of fi, which is ranked in order of size of vi. The values and location of wi are not associated with those of fi, but associated with the location of the induced values.In this paper, because the index stability is more important, it could be used as the induced components of IOWA operator. The final score of each qualitative index isP*=pi*(i=1,2,…,n).


### 4.2. Determining the Weight of Second-Level Indexes

Using the method proposed in Section4.1, we can convert the qualitative indexes into benefit-type quantitative ones. Now, decision matrix A=(aij)c×b of whole index system within b period is built up. However, as for the quantitative index aij, the two types of index, cost-type index and benefit-type index, need to be normalized. The method is described as in the following.For cost-type index(8)rij=min⁡j(aij)aij,j=1,2,…,b.For benefit-type index(9)rij=aijmax⁡j(aij),j=1,2,…,b.It is important to note that because of the assumption that the higher the score, the better the performance of the index for benefit-type indexes, so all the qualitative indexes are changed into benefit-type index.With (8) and (9), the normalized decision matrix R=(rij)c×b of the whole index evaluation system within b period can be obtained. Let Φ be the set of possible weight intervals for known indexes. The single-objective linear programming model can be established by using maximum deviation method based on deviation degree, as shown in
(10)max⁡D(w)=∑i=1b∑j=1c∑k=1b(|rij-rkj|)wjs.t.wj∈Φ.The weight of each index can be achieved by solving the linear programming model.


### 4.3. Method for Adjusting the Weight of First-Level Indexes

By considering that in practice the decision-makers may have different demands of scheduling performance management in different periods, the first-level indexes of the evaluation system should be adjusted. Since there are only three indexes in the first level, the range of the adjustment will not change frequently. Thus, an adjustment coefficient method based on set-valued statistics is used to adjust the weight. And set-valued statistics have been used in many articles [48–51].


#### 4.3.1. Expert Evaluation

In accordance with the requirements of different performance management, 3 to 5 experts are invited to estimate the proper adjustment ranges of three first-level index weights based on the past data. Because the experts may not hold the specific adjustment range accurately and objectively, they can give their opinions with an interval value[di1k,di2k](di1k<di2k,-1<di1k<1,-1<di2k<1), where di1k and di2k should be consistent with positive number or negative number. The values indicate the percentages of increase or decrease in the weight of original index. The matrix of adjustment range is B:
(11)B=[(d111,d121)(d112,d122)⋯(d11k,d12k)(d211,d221)(d212,d222)⋯(d21k,d22k)(d311,d321)(d312,d322)⋯(d31k,d32k)].


#### 4.3.2. Processing the Data with Set-Valued Statistics

Set-valued statistics is an extension of classical statistics and fuzzy statistics. In classical statistics, a certain point in the phase space can be obtained in each test, while in set-valued statistics a fuzzy subset can be achieved. Set-valued statistics can deal with uncertain judgment so as to concentrate various opinions conveniently and reduce the random error. In this study, the method is adopted to handle the interval numbers of weight adjustment range, and the detailed algorithm is described as follows.Letdi be the aggregation point of the ith index;
(12)di=∑k=1n|(di2k)2-(di1k)2|2∑k=1n(di2k-di1k).
In order to measure whether the evaluations of experts are uniform, set gi as the degree of ramification:
(13)gi=∑k=1n[(di2k-di)3-(di1k-di)3]3∑k=1n(di2k-di1k).
So the consistency of experts’ evaluation (ei) is measured as
(14)ei=11+gi.
The closer the value of ei to 1 is, the higher the consistency is. A certain threshold is set to testify. If the consistency is too low, the new round evaluation is required.


#### 4.3.3. Weight of Adjusted First-Level Indexes

After the consistency test reaches the acceptable range, we can use the results to adjust the weight and then normalize the adjusted weight with(15)wit=wit′∑i=13wit′,
where wit′=(1+di)wit-1 and wit is the weight of ith first-level index for the tth period.


### 4.4. Application Procedure of Performance Evaluation Method Based on the Dynamic Index Weight

The evaluation method of scheduling performance presented here is adopted to evaluate the performance data of the(b+1)th period. The application procedure is described as in the following.Step 1.
Determine the weight of first-level indexes. According to the requirements of dynamic adjustment, the weight can be achieved by the method proposed in Section4.4.Step 2.
Process the qualitative data. After the data of previousb periods are collected, separate them into qualitative and quantitative data. For each period’s qualitative data, quantify them with the multiphase dynamic interactive scoring method. Then the decision matrix A=(aij)c×b with fixed value can be obtained.Step 3.
NormalizeA=(aij)c×b and obtain the normalized decision matrix R=(rij)c×b in b periods.Step 4.
Determine the weight of second-level indexes using the maximum deviation method based on deviation degree proposed in Section4.3.Step 5.
Collect the data of the(b+1)th period and quantify the qualitative data with the multiphase dynamic interactive scoring method proposed in Section 4.2 and then normalize all the data in this period.Step 6.
Obtain the final score with the weight of two-level indexes and the fixed value of data in the(b+1)th period.The flow diagram is shown in Figure1.Figure 1
Flow diagram of performance evaluation method based on the dynamic index weight.


## 4.1. Quantify the Qualitative Indexes Using the Method of Multiphase Dynamic Interaction

This method is applicable to quantify the qualitative index by multiphase dynamic interactive expert scoring. The thought is described as in the following: let some experts rate the qualitative indexes with several rounds. Every round can be regarded as a revision for last round till it reaches a relatively stable and consistent level. Suppose that there arem experts scoring for n qualitative indexes, and the higher the score, the better the performance of the index for both cost-type indexes and benefit-type indexes. The index set evaluated is X={x1,x2,…,xn} and the evaluator group set is S={s1,s2,…,sm}. The interval of scoring is [0,10] and the score matrix is P=(pij)n×m.


### 4.1.1. Multiphase Dynamic Interaction and the Decision of Interaction Coefficient

The purpose of multiphase dynamic interaction is to give the evaluators a chance of revising their grading according to the group information of last round.Pt(t=1,2,…,l) is assumed as the score matrix of the tth round. In the process of multiphase dynamic interaction, the variation of an evaluator’s score between two continuous rounds can reflect the influence by the group information of last round. ajt can be used to indicate the similarity between the jth evaluator with other group members in the tth round,
(1)ajt=1m-1∑k=1,k≠jm(cos⁡θjkt-cos⁡θjkt-1),
where cos⁡θjkt=(pjt,pkt)/|pjt||pkt|=∑i=1npijtpikt/∑i=1n(pijt)2∑i=1n(pikt)2.During the course of different interaction phases, the influence of a group member is varied with score matrix. Thus,ujt is used to indicate the interaction coefficient of the jth evaluator in the tth round as follows:
(2)ujt=(1+ajt)∑j=1m(1+ajt).The interaction coefficient vector of thetth round is ut=(u1t,u2t,…,umt).


### 4.1.2. The Aggregation of Scoring Information

The purpose of the aggregation is to obtain the comprehensive score of each index after thetth round scoring. In order to balance every evaluator’s opinion, the minimal deviation with the members’ scoring is used as the final score. By solving the following programming model, the final score of the ith index in the tth round can be reached as pit*(i=1,2,…,n;  t=1,2,…,l), where
(3)min⁡π(pit*)=∑i=1n∑j=1mujt(pijt*-pijt)2s.t.l{min⁡jpijt≤pijt*≤max⁡jpijtmmmlmmi=1,2,…,n,mlmmmmj=1,2,…,m.


### 4.1.3. The Stability and Consistency of Group Information

After several rounds of interscoring, the opinions of group members are known to each other. So the group information tends to be relatively stable and consistent. The stability means the invariance of group information in consecutive rounds and the consistency represents the invariance between different members within the same round. According to the two indexes, the scoring termination condition can be determined.The indexes stability for thetth round is calculated as follows:
(4)υt=1-1mn∑i=1n∑k=1m(pikt-pikt-1)2.The stability vector is labeled asυ=(υ1,υ2,…,υl).The index consistency for thetth round is calculated as follows:
(5)ωt=1-1mn∑i=1n∑h=1m(piht-pi-t)2,
where pi-t=(1/m)∑k=1mpikt and the consistency vector is labeled as ω=(ω1,ω2,…,ωl).In order to determine the termination conditions of interscoring, two thresholds for stability (ε) and consistency (η) are given firstly. If there exists
(6)|1-υt|<ε,
we think that it passes the stability test. Meanwhile, according to the indexes consistency in the tth round, if there exists
(7)|1-ωt|<η
we believe that it passes the consistency test. When both the stability and the consistency tests are satisfied, the multiphase dynamic interaction can be stopped. In practical application, the stability is more emphasized, so the threshold of consistency can be looser than that of stability.


### 4.1.4. The Aggregation of the Final Score Result

The comprehensive score of each index in different roundspit*(i=1,2,…,n;  t=1,2,…,l) will have an impact on the final score result. For these impacts, the induced ordered weighted averaging (IOWA) [47] operators are adopted to aggregate the index score in different rounds.Definition 1.
Set〈v1,f1〉,〈v2,f2〉,…,〈vl,fl〉 as l two-dimensional arrays. Let hw(〈v1,f1〉,〈v2,f2〉,…,〈vl,fl〉)=∑i=1lwiav-index(i); then the function hw is regarded as l dimensional IOWA operator, and vi is induced component of fi, where v-index(i) stands for the subscript of the ith large number among v1,v2,…,vl and w=(w1,w2,…,wl) is weighted vector, which is satisfied with ∑i=1lwi=1,  wi≥0,  i=1,2,…,l.Definition1 indicates that the value of IOWA is obtained from the ordered weighted average of fi, which is ranked in order of size of vi. The values and location of wi are not associated with those of fi, but associated with the location of the induced values.In this paper, because the index stability is more important, it could be used as the induced components of IOWA operator. The final score of each qualitative index isP*=pi*(i=1,2,…,n).


## 4.1.1. Multiphase Dynamic Interaction and the Decision of Interaction Coefficient

The purpose of multiphase dynamic interaction is to give the evaluators a chance of revising their grading according to the group information of last round.Pt(t=1,2,…,l) is assumed as the score matrix of the tth round. In the process of multiphase dynamic interaction, the variation of an evaluator’s score between two continuous rounds can reflect the influence by the group information of last round. ajt can be used to indicate the similarity between the jth evaluator with other group members in the tth round,
(1)ajt=1m-1∑k=1,k≠jm(cos⁡θjkt-cos⁡θjkt-1),
where cos⁡θjkt=(pjt,pkt)/|pjt||pkt|=∑i=1npijtpikt/∑i=1n(pijt)2∑i=1n(pikt)2.During the course of different interaction phases, the influence of a group member is varied with score matrix. Thus,ujt is used to indicate the interaction coefficient of the jth evaluator in the tth round as follows:
(2)ujt=(1+ajt)∑j=1m(1+ajt).The interaction coefficient vector of thetth round is ut=(u1t,u2t,…,umt).


## 4.1.2. The Aggregation of Scoring Information

The purpose of the aggregation is to obtain the comprehensive score of each index after thetth round scoring. In order to balance every evaluator’s opinion, the minimal deviation with the members’ scoring is used as the final score. By solving the following programming model, the final score of the ith index in the tth round can be reached as pit*(i=1,2,…,n;  t=1,2,…,l), where
(3)min⁡π(pit*)=∑i=1n∑j=1mujt(pijt*-pijt)2s.t.l{min⁡jpijt≤pijt*≤max⁡jpijtmmmlmmi=1,2,…,n,mlmmmmj=1,2,…,m.


## 4.1.3. The Stability and Consistency of Group Information

After several rounds of interscoring, the opinions of group members are known to each other. So the group information tends to be relatively stable and consistent. The stability means the invariance of group information in consecutive rounds and the consistency represents the invariance between different members within the same round. According to the two indexes, the scoring termination condition can be determined.The indexes stability for thetth round is calculated as follows:
(4)υt=1-1mn∑i=1n∑k=1m(pikt-pikt-1)2.The stability vector is labeled asυ=(υ1,υ2,…,υl).The index consistency for thetth round is calculated as follows:
(5)ωt=1-1mn∑i=1n∑h=1m(piht-pi-t)2,
where pi-t=(1/m)∑k=1mpikt and the consistency vector is labeled as ω=(ω1,ω2,…,ωl).In order to determine the termination conditions of interscoring, two thresholds for stability (ε) and consistency (η) are given firstly. If there exists
(6)|1-υt|<ε,
we think that it passes the stability test. Meanwhile, according to the indexes consistency in the tth round, if there exists
(7)|1-ωt|<η
we believe that it passes the consistency test. When both the stability and the consistency tests are satisfied, the multiphase dynamic interaction can be stopped. In practical application, the stability is more emphasized, so the threshold of consistency can be looser than that of stability.


## 4.1.4. The Aggregation of the Final Score Result

The comprehensive score of each index in different roundspit*(i=1,2,…,n;  t=1,2,…,l) will have an impact on the final score result. For these impacts, the induced ordered weighted averaging (IOWA) [47] operators are adopted to aggregate the index score in different rounds.Definition 1.
Set〈v1,f1〉,〈v2,f2〉,…,〈vl,fl〉 as l two-dimensional arrays. Let hw(〈v1,f1〉,〈v2,f2〉,…,〈vl,fl〉)=∑i=1lwiav-index(i); then the function hw is regarded as l dimensional IOWA operator, and vi is induced component of fi, where v-index(i) stands for the subscript of the ith large number among v1,v2,…,vl and w=(w1,w2,…,wl) is weighted vector, which is satisfied with ∑i=1lwi=1,  wi≥0,  i=1,2,…,l.Definition1 indicates that the value of IOWA is obtained from the ordered weighted average of fi, which is ranked in order of size of vi. The values and location of wi are not associated with those of fi, but associated with the location of the induced values.In this paper, because the index stability is more important, it could be used as the induced components of IOWA operator. The final score of each qualitative index isP*=pi*(i=1,2,…,n).


## 4.2. Determining the Weight of Second-Level Indexes

Using the method proposed in Section4.1, we can convert the qualitative indexes into benefit-type quantitative ones. Now, decision matrix A=(aij)c×b of whole index system within b period is built up. However, as for the quantitative index aij, the two types of index, cost-type index and benefit-type index, need to be normalized. The method is described as in the following.For cost-type index(8)rij=min⁡j(aij)aij,j=1,2,…,b.For benefit-type index(9)rij=aijmax⁡j(aij),j=1,2,…,b.It is important to note that because of the assumption that the higher the score, the better the performance of the index for benefit-type indexes, so all the qualitative indexes are changed into benefit-type index.With (8) and (9), the normalized decision matrix R=(rij)c×b of the whole index evaluation system within b period can be obtained. Let Φ be the set of possible weight intervals for known indexes. The single-objective linear programming model can be established by using maximum deviation method based on deviation degree, as shown in
(10)max⁡D(w)=∑i=1b∑j=1c∑k=1b(|rij-rkj|)wjs.t.wj∈Φ.The weight of each index can be achieved by solving the linear programming model.


## 4.3. Method for Adjusting the Weight of First-Level Indexes

By considering that in practice the decision-makers may have different demands of scheduling performance management in different periods, the first-level indexes of the evaluation system should be adjusted. Since there are only three indexes in the first level, the range of the adjustment will not change frequently. Thus, an adjustment coefficient method based on set-valued statistics is used to adjust the weight. And set-valued statistics have been used in many articles [48–51].


### 4.3.1. Expert Evaluation

In accordance with the requirements of different performance management, 3 to 5 experts are invited to estimate the proper adjustment ranges of three first-level index weights based on the past data. Because the experts may not hold the specific adjustment range accurately and objectively, they can give their opinions with an interval value[di1k,di2k](di1k<di2k,-1<di1k<1,-1<di2k<1), where di1k and di2k should be consistent with positive number or negative number. The values indicate the percentages of increase or decrease in the weight of original index. The matrix of adjustment range is B:
(11)B=[(d111,d121)(d112,d122)⋯(d11k,d12k)(d211,d221)(d212,d222)⋯(d21k,d22k)(d311,d321)(d312,d322)⋯(d31k,d32k)].


### 4.3.2. Processing the Data with Set-Valued Statistics

Set-valued statistics is an extension of classical statistics and fuzzy statistics. In classical statistics, a certain point in the phase space can be obtained in each test, while in set-valued statistics a fuzzy subset can be achieved. Set-valued statistics can deal with uncertain judgment so as to concentrate various opinions conveniently and reduce the random error. In this study, the method is adopted to handle the interval numbers of weight adjustment range, and the detailed algorithm is described as follows.Letdi be the aggregation point of the ith index;
(12)di=∑k=1n|(di2k)2-(di1k)2|2∑k=1n(di2k-di1k).
In order to measure whether the evaluations of experts are uniform, set gi as the degree of ramification:
(13)gi=∑k=1n[(di2k-di)3-(di1k-di)3]3∑k=1n(di2k-di1k).
So the consistency of experts’ evaluation (ei) is measured as
(14)ei=11+gi.
The closer the value of ei to 1 is, the higher the consistency is. A certain threshold is set to testify. If the consistency is too low, the new round evaluation is required.


### 4.3.3. Weight of Adjusted First-Level Indexes

After the consistency test reaches the acceptable range, we can use the results to adjust the weight and then normalize the adjusted weight with(15)wit=wit′∑i=13wit′,
where wit′=(1+di)wit-1 and wit is the weight of ith first-level index for the tth period.


## 4.3.1. Expert Evaluation

In accordance with the requirements of different performance management, 3 to 5 experts are invited to estimate the proper adjustment ranges of three first-level index weights based on the past data. Because the experts may not hold the specific adjustment range accurately and objectively, they can give their opinions with an interval value[di1k,di2k](di1k<di2k,-1<di1k<1,-1<di2k<1), where di1k and di2k should be consistent with positive number or negative number. The values indicate the percentages of increase or decrease in the weight of original index. The matrix of adjustment range is B:
(11)B=[(d111,d121)(d112,d122)⋯(d11k,d12k)(d211,d221)(d212,d222)⋯(d21k,d22k)(d311,d321)(d312,d322)⋯(d31k,d32k)].


## 4.3.2. Processing the Data with Set-Valued Statistics

Set-valued statistics is an extension of classical statistics and fuzzy statistics. In classical statistics, a certain point in the phase space can be obtained in each test, while in set-valued statistics a fuzzy subset can be achieved. Set-valued statistics can deal with uncertain judgment so as to concentrate various opinions conveniently and reduce the random error. In this study, the method is adopted to handle the interval numbers of weight adjustment range, and the detailed algorithm is described as follows.Letdi be the aggregation point of the ith index;
(12)di=∑k=1n|(di2k)2-(di1k)2|2∑k=1n(di2k-di1k).
In order to measure whether the evaluations of experts are uniform, set gi as the degree of ramification:
(13)gi=∑k=1n[(di2k-di)3-(di1k-di)3]3∑k=1n(di2k-di1k).
So the consistency of experts’ evaluation (ei) is measured as
(14)ei=11+gi.
The closer the value of ei to 1 is, the higher the consistency is. A certain threshold is set to testify. If the consistency is too low, the new round evaluation is required.


## 4.3.3. Weight of Adjusted First-Level Indexes

After the consistency test reaches the acceptable range, we can use the results to adjust the weight and then normalize the adjusted weight with(15)wit=wit′∑i=13wit′,
where wit′=(1+di)wit-1 and wit is the weight of ith first-level index for the tth period.


## 4.4. Application Procedure of Performance Evaluation Method Based on the Dynamic Index Weight

The evaluation method of scheduling performance presented here is adopted to evaluate the performance data of the(b+1)th period. The application procedure is described as in the following.Step 1.
Determine the weight of first-level indexes. According to the requirements of dynamic adjustment, the weight can be achieved by the method proposed in Section4.4.Step 2.
Process the qualitative data. After the data of previousb periods are collected, separate them into qualitative and quantitative data. For each period’s qualitative data, quantify them with the multiphase dynamic interactive scoring method. Then the decision matrix A=(aij)c×b with fixed value can be obtained.Step 3.
NormalizeA=(aij)c×b and obtain the normalized decision matrix R=(rij)c×b in b periods.Step 4.
Determine the weight of second-level indexes using the maximum deviation method based on deviation degree proposed in Section4.3.Step 5.
Collect the data of the(b+1)th period and quantify the qualitative data with the multiphase dynamic interactive scoring method proposed in Section 4.2 and then normalize all the data in this period.Step 6.
Obtain the final score with the weight of two-level indexes and the fixed value of data in the(b+1)th period.The flow diagram is shown in Figure1.Figure 1
Flow diagram of performance evaluation method based on the dynamic index weight.


## 5. Innovations and Advantages of This New Performance Evaluation Method

### 5.1. The Main Innovations and Advantages of This New Method

In the implementation of the performance evaluation methods, the most important two aspects are the accuracy of the data and the accuracy of the weight. Compared with other papers, this paper has made much effort on these two aspects.Firstly, the new evaluation method proposed has advantages in quantifying the qualitative index. In general, there are two types of traditional quantifying methods. One is a simple method which needs one-off scoring and removing the singular points according to some rule [52, 53]; the other is a complex method, such as analytic hierarchy process [35, 36], analytic network process [3], and fuzzy evaluation method [37]. These methods are all one-off scoring and solving with judging criteria [23–25]. The above two types of traditional methods may be faced with the problem that the selected experts have different understanding with the indexes measure and only have once evaluation opportunity, which will lead to inaccurate results. Therefore, this paper uses the multiperiod interaction method. The core idea of this method is through several rounds interacting evaluations to make the group’s opinions reach certain stability and consistency so as to reduce the subjectivity and improve the accuracy in expert scoring.Secondly, in previous evaluation methods, the weight of indexes was given, single-period and constant [7, 8, 15–17] while in this paper we consider the problem of indexes weight dynamic adjustment in the multiperiod context. We draw the multiperiod interaction method presented by [54] and improve it. The method of [54] is suitable for direct evaluation and ranking for several objects, but not for the quantified assessment of a single object. In this paper, the method is improved properly and applied to realize the quantified assessment for LSSC scheduling which is single object.Thirdly, because there are only three first-level indexes (scheduling environment, scheduling process, and scheduling results) and, in multiperiod dynamic adjustment, their weight will not change frequently in practice, we adopt an adjustment coefficient method based on set-valued statistics to adjust first-level indexes weight. Obviously, set-valued statistics is a group decision and has better effect than general one-off scoring in integrating the opinions of experts group [48–51].


### 5.2. Other Merits of This New Performance Evaluation Method

Compared with other performance evaluation methods, this method has some other merits of the following aspects.(1)
The evaluation index system used in this new method overcomes the shortcoming of traditional methods that only focus on a single enterprise or sector. From the perspective of supply chain cooperation, the evaluation index system emphasizes the whole scheduling process, namely, before scheduling, during scheduling, and after scheduling.(2)
The maximum attribute deviation method is introduced to determine the weight of the second-level indexes, which can remove the uncertainty of subjective factors. This improves the method proposed by [55].(3)
An adjustment coefficient method based on set-valued statistics is introduced to determine the weight of the first-level indexes, which can make the adjustment range more precise and decrease the difference among experts more significantly.


## 5.1. The Main Innovations and Advantages of This New Method

In the implementation of the performance evaluation methods, the most important two aspects are the accuracy of the data and the accuracy of the weight. Compared with other papers, this paper has made much effort on these two aspects.Firstly, the new evaluation method proposed has advantages in quantifying the qualitative index. In general, there are two types of traditional quantifying methods. One is a simple method which needs one-off scoring and removing the singular points according to some rule [52, 53]; the other is a complex method, such as analytic hierarchy process [35, 36], analytic network process [3], and fuzzy evaluation method [37]. These methods are all one-off scoring and solving with judging criteria [23–25]. The above two types of traditional methods may be faced with the problem that the selected experts have different understanding with the indexes measure and only have once evaluation opportunity, which will lead to inaccurate results. Therefore, this paper uses the multiperiod interaction method. The core idea of this method is through several rounds interacting evaluations to make the group’s opinions reach certain stability and consistency so as to reduce the subjectivity and improve the accuracy in expert scoring.Secondly, in previous evaluation methods, the weight of indexes was given, single-period and constant [7, 8, 15–17] while in this paper we consider the problem of indexes weight dynamic adjustment in the multiperiod context. We draw the multiperiod interaction method presented by [54] and improve it. The method of [54] is suitable for direct evaluation and ranking for several objects, but not for the quantified assessment of a single object. In this paper, the method is improved properly and applied to realize the quantified assessment for LSSC scheduling which is single object.Thirdly, because there are only three first-level indexes (scheduling environment, scheduling process, and scheduling results) and, in multiperiod dynamic adjustment, their weight will not change frequently in practice, we adopt an adjustment coefficient method based on set-valued statistics to adjust first-level indexes weight. Obviously, set-valued statistics is a group decision and has better effect than general one-off scoring in integrating the opinions of experts group [48–51].


## 5.2. Other Merits of This New Performance Evaluation Method

Compared with other performance evaluation methods, this method has some other merits of the following aspects.(1)
The evaluation index system used in this new method overcomes the shortcoming of traditional methods that only focus on a single enterprise or sector. From the perspective of supply chain cooperation, the evaluation index system emphasizes the whole scheduling process, namely, before scheduling, during scheduling, and after scheduling.(2)
The maximum attribute deviation method is introduced to determine the weight of the second-level indexes, which can remove the uncertainty of subjective factors. This improves the method proposed by [55].(3)
An adjustment coefficient method based on set-valued statistics is introduced to determine the weight of the first-level indexes, which can make the adjustment range more precise and decrease the difference among experts more significantly.


## 6. Method Application: A Case from Tianjin Baoyun Logistics Company, China

### 6.1. Case Description

#### 6.1.1. Company Introduction

In order to verify the usability of the method, the data are collected from Baoyun Logistics Company in Tianjin, China. The company is a professional third-part logistics enterprise and an excellent logistics service integrator listed as the top 100 logistics enterprises of China in three consecutive years from 2005 to 2007. Currently, the company has 28 branches across the country and builds up a good relationship of cooperation with 32 large-scale warehousing enterprises, 20 transportation enterprises, and more than 15 professional logistics enterprises. By integrating these functional logistics service providers, Baoyun has established wide business connection with over 20 multinational customers such as P&G, Siemens, and Delphi Corporation and offered customized logistics service in accordance with logistics demands.During the cooperation with P&G, Baoyun Logistics Company provides integrated service, including railway transportation service, warehousing service, and road distribution and delivery service. Generally speaking, Baoyun evaluates the scheduling performance of these services provided to P&G monthly. When the evaluation is conducted, P&G, logistics service providers, and Baoyun jointly participated, and the indexes are evaluated individually to get the original data. If some indexes involve multiple parts, they need to be negotiated with each other to determine the final values.In this case, the original data were collected from January 2012 to May 2012. Then the method proposed in the paper is used to evaluate the scheduling performance of June according to the data of the previous 5 months.


#### 6.1.2. Selection of Evaluation Experts

In the scheduling performance evaluation of Baoyun, the biggest challenge is how to quantify the qualitative data. For this, Baoyun selects the managers who directly take charge of scheduling activities as evaluation experts. Usually, the three members which are P&G, Baoyun, and FLSP of LSSC assign their managers to participate in the performance evaluation. The experts group was composed as in Tables3 and 4.Table 3
Experts group for evaluating the first-level indexes.
Number of experts
Composition
Evaluation task
Baoyun (LSI)
2
General manager, operation manager
Weight ofC1, C2, C3
FLSP
2
General manager, operation manager
Weight ofC1, C2, C3
P&G
1
Logistics manager
Weight ofC1, C2, C3Table 4
Experts group for evaluating the second-level qualitative indexes.
Number of experts
Composition
Evaluation task
Baoyun (LSI)
2
Marketing manager, operation manager
Giving score for indexes inC11 and C21
FLSP
3
General manager, marketing manager, operation manager
Giving score for indexes inC11 and C21It is important to note that, according to the task of evaluation subject in Table1, the first-level indexes are evaluated by all the three evaluation subjects, P&G, Baoyun, and FLSP. There are 5 experts in total shown in Table 3. While for the second-level indexes which are closely related to practical scheduling process, they are evaluated by Baoyun and FLSP shown in Table 4. When evaluating, all experts give their scores independently first and then start multistage interactive grading.


#### 6.1.3. Challenges and Solutions in Evaluation

There are three aspects of challenges. The first is that the experts are from different companies and they may have different preferences. The second is that it is hard for these experts to get together doing evaluation work. The third is that sometimes the divergences among experts are too big to obtain a consistent result. So we need more grading rounds.To overcome these challenges, P&G, Baoyun, and FLSP cooperate a lot. First, before the evaluating, the experts studied and discussed the indexes to make a consistent understanding. Second, Baoyun Company assigns a secretary to coordinate the time of these experts. She would inform the meeting time ten days before. Third, to reduce the evaluation round, every expert would explain his evaluation in the first round, so that other experts can understand his considerations. This can improve the evaluation efficiency and reach an agreement faster.


### 6.2. Application of Scheduling Performance Evaluation Method

#### 6.2.1. Step1: Determine the Weight of First-Level Indexes

According to the evaluation index system, there are three first-level indexes, scheduling environment, scheduling process, and scheduling result. The index of scheduling result is composed of two factors, which are customer satisfaction and service quality. In this paper, it is assumed that the two factors are equally important. The weights of the three indexes for the previous 5 months are 0.3, 0.3, and 0.4, respectively, and need to readjust in the sixth month. The matrix of weight adjustment range obtained from 5 experts is as follows and the consistency test threshold is 0.95:(16)B=[(0.0,0.1)(0.1,0.2)(0.1,0.3)(0.2,0.4)(0.1,0.2)(0.1,0.2)(0.3,0.4)(0.2,0.4)(0.0,0.1)(0.2,0.3)(-0.3,-0.2)(-0.2,0.0)(-0.2,-0.1)(-0.2,-0.1)(-0.3,-0.2)].According to the matrixB, adjustment range di could be calculated by (12); then consistency index ei could be gotten by (13) and (14). Original weight wit-1 and adjusted weight wit′ could be obtained by (15). The weight of first-level indexes is calculated as in Table 5.Table 5
The results of adjusted weight of first-level indexes.
Scheduling environment
Scheduling process
Scheduling outcome
Adjustment rangedi
0.193
0.233
−0.167
Consistency indexei
0.991
0.988
0.990
Original weightwit-1
0.3
0.3
0.4
Adjusted weightwit′
0.358
0.370
0.333
Normalized weightwit
0.34
0.35
0.31See from Table5 that the consistency index values of the experts evaluation are all over 0.95, and the adjustment of index weight is acceptable, so the first-level weights of the 6th period are 0.34, 0.35, and 0.31, respectively.


#### 6.2.2. Step2: Quantify the Qualitative Indexes Using the Method of Multiphase Dynamic Interaction

According to the original data, by using the method of multiphase dynamic interaction, the qualitative indexes of previous five periods can be converted into benefit-type quantitative ones, and the decision matrixA=(aij)c×b can be obtained as in Table 6.Table 6
Decision matrix.
January
February
March
April
May
C
11
C111
5.6
5.8
6.1
4.5
5.1
C112
7
4.6
4.6
4.6
4.4
C113
6.5
6.6
5.7
6.8
5.2
C114
4.3
6.2
4.5
4.3
5.2
C
21
C211
6.3
6.2
6.3
5.1
4.2
C212
4.5
4.5
6.2
6
6.5
C213
5.1
4.3
6.7
4.5
4.1
C214
6.9
5.1
6.5
6.9
6.8
C
31
C311
0.051
0.038
0.067
0.041
0.045
C312
0.731
0.91
0.909
0.711
0.85
C313
0.727
0.932
0.75
0.903
0.897
C314
1.22
2.16
1.36
1.75
2.83
C
32
C321
0.014
0.048
0.035
0.057
0.024
C322
17400
19100
18700
18000
15700
C323
0.801
0.685
0.829
0.727
0.842
C324
0.927
0.825
0.926
0.72
0.75


#### 6.2.3. Step3: Normalize Decision Matrix

Customer complaints rate (C311), response time for demand (C314), wrong scheduling rate (C321), and scheduling cost (C322) are cost-type indexes and the rest are benefit-type indexes. After the decision matrix is normalized, the normalized decision matrix can be obtained as in Table 7.Table 7
Normalized decision matrix.
January
February
March
April
May
C
11
C111
0.92
0.95
1
0.74
0.84
C112
1
0.66
0.66
0.66
0.63
C113
0.96
0.97
0.84
1
0.76
C114
0.69
1
0.73
0.69
0.84
C
21
C211
1
0.98
1
0.81
0.67
C212
0.69
0.69
0.95
0.92
1
C213
0.76
0.64
1
0.67
0.61
C214
1
0.74
0.94
1
0.99
C
31
C311
0.75
1
0.57
0.93
0.84
C312
0.8
1
1
0.78
0.93
C313
0.78
1
0.8
0.97
0.96
C314
1
0.56
0.9
0.7
0.43
C
32
C321
1
0.29
0.4
0.25
0.58
C322
0.9
0.82
0.84
0.87
1
C323
0.95
0.81
0.98
0.86
1
C324
1
0.89
1
0.78
0.81


#### 6.2.4. Step4: Determine the weight of second-level indexes

The maximum deviation method based on deviation degree is applied to calculate the weight of scheduling environment, scheduling process, and scheduling result respectively.The possible index weight of each index for scheduling environment isΦ1={w1=(w11,w21,w31,w41)0.25≤w11≤0.27,0.21≤w21≤0.23,0.24≤w31≤0.28,0.20≤w41≤0.32}.And that for scheduling process isΦ2={w2=(w12,w22,w32,w42)0.22≤w12≤0.25,0.24≤w22≤0.31,0.18≤w32≤0.22,0.28≤w42≤0.32}.And that for scheduling result isΦ3={w3=(w13,w23,w33,w43,w53,w63,w73,w83)∣0.120≤w13≤0.135,0.118≤w23≤0.132,0.109≤w33≤0.141,0.101≤w43≤0.141,0.114≤w53≤0.147,0.119≤w63≤0.132,0.106≤w73≤0.134,0.113≤w83≤0.134}.Three single-objective linear programming models are established as in (17). Software MATLAB 7.0 is adopted and the weight of each index is obtained as shown in Table 8.Table 8
Weight of second-level indexes.
Organization structureC111
0.25
Information technologyC112
0.21
Information collecting abilityC113
0.24
Cooperation relationshipC114
0.3
Adaptability of schedulingC211
0.22
Timeliness of schedulingC212
0.28
Communication qualityC213
0.22
Flexibility of schedulingC214
0.28
Customer complaints rateC311
0.135
Fulfillment rateC312
0.13
On-time deliveryC313
0.109
Response time for demandC314
0.141
Wrong scheduling rateC321
0.147
Scheduling costC322
0.119
Provider satisfactionC323
0.106
Resource efficiencyC324
0.113Consider(17)max⁡D(w)=∑i=15∑j=14∑k=15(|rij-rkj|)wjs.t.wj∈Φ1max⁡D(w)=∑i=15∑j=14∑k=15(|rij-rkj|)wjs.t.wj∈Φ2max⁡D(w)=∑i=15∑j=18∑k=15(|rij-rkj|)wjs.t.wj∈Φ3.


#### 6.2.5. Step5: Process the Data of the Sixth Month Based on Multiphase Dynamic Interaction

In this step, the application example of the method based on multiphase dynamic interaction is given. The original data of the sixth month is shown in Table9.Table 9
Original data of the sixth month.
Index value
Nature
C
11
C111
Fair
Qualitative
C112
Very poor
Qualitative
C113
Fair
Qualitative
C114
Fair
Qualitative
C
21
C211
Good
Qualitative
C212
Poor
Qualitative
C213
Very good
Qualitative
C214
Fair
Qualitative
C
31
C311
0.055
Quantitative
C312
0.849
Quantitative
C313
0.913
Quantitative
C314
1.64
Quantitative
C
32
C321
0.022
Quantitative
C322
16900
Quantitative
C323
0.823
Quantitative
C324
0.896
QuantitativeAccording to the original data, 5 experts are invited to score the 8 qualitative indexes. After discussion, 4 times dynamic interaction are conducted. Som=5, n=8, and l=4. The scores are distributed between 0 and 10, and the matrixes are shown as follows:
(18)P0=C111C112C113C114C211C212C213C214[8436555832682448253763657528637737473662]P1=C111C112C113C114C211C212C213C214[8636744355547637853645973694889837657622]P2=C111C112C113C114C211C212C213C214[7563425546458638347257835454676468486624]P3=C111C112C113C114C211C212C213C214[6664426435657537457356846253648558557636]P4=C111C112C113C114C211C212C213C214[6664436445657547557356746453648558557646].Calculate the interaction coefficient with (1) and (2) as in the following:
(19)u1=(0.194,0.214,0.194,0.196,0.203)u2=(0.192,0.201,0.219,0.190,0.198)u3=(0.202,0.196,0.197,0.202,0.202)u4=(0.202,0.199,0.198,0.201,0.199).Aggregate the score of each round with (3):
(20)P1*=(6.009,4.205,4.964,5.840,5.567,7.051,6.623,4.426)P2*=(5.025,4.432,5.259,4.471,5.679,5.175,5.582,5.228)P3*=(5.191,3.988,5.192,5.208,5.788,3.999,6.213,5.387)P4*=(5.199,4.395,5.399,5.408,5.594,4.403,6.210,5.595).Set the stability threshold as0.1(ε=0.1) and the consistency threshold as 0.25(η=0.25), and then calculate the stability index and the consistency index of each round as follows:
(21)υ=(0.56,0.641,0.813,0.921)ω=(0.685,0.7,0.744,0.771).The index stability and index consistency of the fourth round are 0.921 and 0.771, respectively. The test value of stability index is 0.079 less than 0.1, and the test value of consistency index is 0.229 less than 0.25. So it can be concluded that the results pass the tests. Finally, the final score of each index with IOWA is aggregated:(22)P*=(5.243,4.261,5.265,5.258,5.667,4.701,6.127,5.342).The data of the sixth month are normalized by combining the data in previous 5 months, and the results are shown in Table10.Table 10
Normalized value of data in the sixth month.
Index
Data
Normalized data
Scheduling environment
C
111
5.2428
0.86
C
112
4.2613
0.61
C
113
5.2654
0.8
C
114
5.2578
0.85
Scheduling process
C
211
5.6665
0.9
C
212
4.701
0.72
C
213
6.1266
0.91
C
214
5.3423
0.77
Scheduling result
C
311
0.055
0.69
C
312
0.849
0.93
C
313
0.913
0.98
C
314
1.64
0.74
C
321
0.022
0.64
C
322
16900
0.93
C
323
0.823
0.98
C
324
0.896
0.97


#### 6.2.6. Step6: Obtain the Final Score

Obtain the final score by the fixed data as shown in Table8 and the index weights determined in Steps 1 and 4. The expressions are shown as in the following:
(23)(0.86×0.25+0.61×0.21+0.80×0.24+0.85×0.30)×0.34+(0.90×0.22+0.72×0.28+0.91×0.22+0.77×0.28)×0.35+(0.69×0.135+0.93×0.130+0.98×0.109mmmmmm+0.74×0.141+0.64×0.147+0.93×0.119mmllmmml+0.98×0.106+0.97×0.113)×0.31=0.815.Therefore, the final scheduling performance of the sixth month (June 2012) is 0.815. According to the performance ratings standardization of Baoyun Logistics Company, 0.815 reaches the good level, but it also has great potential to increase the scheduling performance in the future.


## 6.1. Case Description

### 6.1.1. Company Introduction

In order to verify the usability of the method, the data are collected from Baoyun Logistics Company in Tianjin, China. The company is a professional third-part logistics enterprise and an excellent logistics service integrator listed as the top 100 logistics enterprises of China in three consecutive years from 2005 to 2007. Currently, the company has 28 branches across the country and builds up a good relationship of cooperation with 32 large-scale warehousing enterprises, 20 transportation enterprises, and more than 15 professional logistics enterprises. By integrating these functional logistics service providers, Baoyun has established wide business connection with over 20 multinational customers such as P&G, Siemens, and Delphi Corporation and offered customized logistics service in accordance with logistics demands.During the cooperation with P&G, Baoyun Logistics Company provides integrated service, including railway transportation service, warehousing service, and road distribution and delivery service. Generally speaking, Baoyun evaluates the scheduling performance of these services provided to P&G monthly. When the evaluation is conducted, P&G, logistics service providers, and Baoyun jointly participated, and the indexes are evaluated individually to get the original data. If some indexes involve multiple parts, they need to be negotiated with each other to determine the final values.In this case, the original data were collected from January 2012 to May 2012. Then the method proposed in the paper is used to evaluate the scheduling performance of June according to the data of the previous 5 months.


### 6.1.2. Selection of Evaluation Experts

In the scheduling performance evaluation of Baoyun, the biggest challenge is how to quantify the qualitative data. For this, Baoyun selects the managers who directly take charge of scheduling activities as evaluation experts. Usually, the three members which are P&G, Baoyun, and FLSP of LSSC assign their managers to participate in the performance evaluation. The experts group was composed as in Tables3 and 4.Table 3
Experts group for evaluating the first-level indexes.
Number of experts
Composition
Evaluation task
Baoyun (LSI)
2
General manager, operation manager
Weight ofC1, C2, C3
FLSP
2
General manager, operation manager
Weight ofC1, C2, C3
P&G
1
Logistics manager
Weight ofC1, C2, C3Table 4
Experts group for evaluating the second-level qualitative indexes.
Number of experts
Composition
Evaluation task
Baoyun (LSI)
2
Marketing manager, operation manager
Giving score for indexes inC11 and C21
FLSP
3
General manager, marketing manager, operation manager
Giving score for indexes inC11 and C21It is important to note that, according to the task of evaluation subject in Table1, the first-level indexes are evaluated by all the three evaluation subjects, P&G, Baoyun, and FLSP. There are 5 experts in total shown in Table 3. While for the second-level indexes which are closely related to practical scheduling process, they are evaluated by Baoyun and FLSP shown in Table 4. When evaluating, all experts give their scores independently first and then start multistage interactive grading.


### 6.1.3. Challenges and Solutions in Evaluation

There are three aspects of challenges. The first is that the experts are from different companies and they may have different preferences. The second is that it is hard for these experts to get together doing evaluation work. The third is that sometimes the divergences among experts are too big to obtain a consistent result. So we need more grading rounds.To overcome these challenges, P&G, Baoyun, and FLSP cooperate a lot. First, before the evaluating, the experts studied and discussed the indexes to make a consistent understanding. Second, Baoyun Company assigns a secretary to coordinate the time of these experts. She would inform the meeting time ten days before. Third, to reduce the evaluation round, every expert would explain his evaluation in the first round, so that other experts can understand his considerations. This can improve the evaluation efficiency and reach an agreement faster.


## 6.1.1. Company Introduction

In order to verify the usability of the method, the data are collected from Baoyun Logistics Company in Tianjin, China. The company is a professional third-part logistics enterprise and an excellent logistics service integrator listed as the top 100 logistics enterprises of China in three consecutive years from 2005 to 2007. Currently, the company has 28 branches across the country and builds up a good relationship of cooperation with 32 large-scale warehousing enterprises, 20 transportation enterprises, and more than 15 professional logistics enterprises. By integrating these functional logistics service providers, Baoyun has established wide business connection with over 20 multinational customers such as P&G, Siemens, and Delphi Corporation and offered customized logistics service in accordance with logistics demands.During the cooperation with P&G, Baoyun Logistics Company provides integrated service, including railway transportation service, warehousing service, and road distribution and delivery service. Generally speaking, Baoyun evaluates the scheduling performance of these services provided to P&G monthly. When the evaluation is conducted, P&G, logistics service providers, and Baoyun jointly participated, and the indexes are evaluated individually to get the original data. If some indexes involve multiple parts, they need to be negotiated with each other to determine the final values.In this case, the original data were collected from January 2012 to May 2012. Then the method proposed in the paper is used to evaluate the scheduling performance of June according to the data of the previous 5 months.


## 6.1.2. Selection of Evaluation Experts

In the scheduling performance evaluation of Baoyun, the biggest challenge is how to quantify the qualitative data. For this, Baoyun selects the managers who directly take charge of scheduling activities as evaluation experts. Usually, the three members which are P&G, Baoyun, and FLSP of LSSC assign their managers to participate in the performance evaluation. The experts group was composed as in Tables3 and 4.Table 3
Experts group for evaluating the first-level indexes.
Number of experts
Composition
Evaluation task
Baoyun (LSI)
2
General manager, operation manager
Weight ofC1, C2, C3
FLSP
2
General manager, operation manager
Weight ofC1, C2, C3
P&G
1
Logistics manager
Weight ofC1, C2, C3Table 4
Experts group for evaluating the second-level qualitative indexes.
Number of experts
Composition
Evaluation task
Baoyun (LSI)
2
Marketing manager, operation manager
Giving score for indexes inC11 and C21
FLSP
3
General manager, marketing manager, operation manager
Giving score for indexes inC11 and C21It is important to note that, according to the task of evaluation subject in Table1, the first-level indexes are evaluated by all the three evaluation subjects, P&G, Baoyun, and FLSP. There are 5 experts in total shown in Table 3. While for the second-level indexes which are closely related to practical scheduling process, they are evaluated by Baoyun and FLSP shown in Table 4. When evaluating, all experts give their scores independently first and then start multistage interactive grading.


## 6.1.3. Challenges and Solutions in Evaluation

There are three aspects of challenges. The first is that the experts are from different companies and they may have different preferences. The second is that it is hard for these experts to get together doing evaluation work. The third is that sometimes the divergences among experts are too big to obtain a consistent result. So we need more grading rounds.To overcome these challenges, P&G, Baoyun, and FLSP cooperate a lot. First, before the evaluating, the experts studied and discussed the indexes to make a consistent understanding. Second, Baoyun Company assigns a secretary to coordinate the time of these experts. She would inform the meeting time ten days before. Third, to reduce the evaluation round, every expert would explain his evaluation in the first round, so that other experts can understand his considerations. This can improve the evaluation efficiency and reach an agreement faster.


## 6.2. Application of Scheduling Performance Evaluation Method

### 6.2.1. Step1: Determine the Weight of First-Level Indexes

According to the evaluation index system, there are three first-level indexes, scheduling environment, scheduling process, and scheduling result. The index of scheduling result is composed of two factors, which are customer satisfaction and service quality. In this paper, it is assumed that the two factors are equally important. The weights of the three indexes for the previous 5 months are 0.3, 0.3, and 0.4, respectively, and need to readjust in the sixth month. The matrix of weight adjustment range obtained from 5 experts is as follows and the consistency test threshold is 0.95:(16)B=[(0.0,0.1)(0.1,0.2)(0.1,0.3)(0.2,0.4)(0.1,0.2)(0.1,0.2)(0.3,0.4)(0.2,0.4)(0.0,0.1)(0.2,0.3)(-0.3,-0.2)(-0.2,0.0)(-0.2,-0.1)(-0.2,-0.1)(-0.3,-0.2)].According to the matrixB, adjustment range di could be calculated by (12); then consistency index ei could be gotten by (13) and (14). Original weight wit-1 and adjusted weight wit′ could be obtained by (15). The weight of first-level indexes is calculated as in Table 5.Table 5
The results of adjusted weight of first-level indexes.
Scheduling environment
Scheduling process
Scheduling outcome
Adjustment rangedi
0.193
0.233
−0.167
Consistency indexei
0.991
0.988
0.990
Original weightwit-1
0.3
0.3
0.4
Adjusted weightwit′
0.358
0.370
0.333
Normalized weightwit
0.34
0.35
0.31See from Table5 that the consistency index values of the experts evaluation are all over 0.95, and the adjustment of index weight is acceptable, so the first-level weights of the 6th period are 0.34, 0.35, and 0.31, respectively.


### 6.2.2. Step2: Quantify the Qualitative Indexes Using the Method of Multiphase Dynamic Interaction

According to the original data, by using the method of multiphase dynamic interaction, the qualitative indexes of previous five periods can be converted into benefit-type quantitative ones, and the decision matrixA=(aij)c×b can be obtained as in Table 6.Table 6
Decision matrix.
January
February
March
April
May
C
11
C111
5.6
5.8
6.1
4.5
5.1
C112
7
4.6
4.6
4.6
4.4
C113
6.5
6.6
5.7
6.8
5.2
C114
4.3
6.2
4.5
4.3
5.2
C
21
C211
6.3
6.2
6.3
5.1
4.2
C212
4.5
4.5
6.2
6
6.5
C213
5.1
4.3
6.7
4.5
4.1
C214
6.9
5.1
6.5
6.9
6.8
C
31
C311
0.051
0.038
0.067
0.041
0.045
C312
0.731
0.91
0.909
0.711
0.85
C313
0.727
0.932
0.75
0.903
0.897
C314
1.22
2.16
1.36
1.75
2.83
C
32
C321
0.014
0.048
0.035
0.057
0.024
C322
17400
19100
18700
18000
15700
C323
0.801
0.685
0.829
0.727
0.842
C324
0.927
0.825
0.926
0.72
0.75


### 6.2.3. Step3: Normalize Decision Matrix

Customer complaints rate (C311), response time for demand (C314), wrong scheduling rate (C321), and scheduling cost (C322) are cost-type indexes and the rest are benefit-type indexes. After the decision matrix is normalized, the normalized decision matrix can be obtained as in Table 7.Table 7
Normalized decision matrix.
January
February
March
April
May
C
11
C111
0.92
0.95
1
0.74
0.84
C112
1
0.66
0.66
0.66
0.63
C113
0.96
0.97
0.84
1
0.76
C114
0.69
1
0.73
0.69
0.84
C
21
C211
1
0.98
1
0.81
0.67
C212
0.69
0.69
0.95
0.92
1
C213
0.76
0.64
1
0.67
0.61
C214
1
0.74
0.94
1
0.99
C
31
C311
0.75
1
0.57
0.93
0.84
C312
0.8
1
1
0.78
0.93
C313
0.78
1
0.8
0.97
0.96
C314
1
0.56
0.9
0.7
0.43
C
32
C321
1
0.29
0.4
0.25
0.58
C322
0.9
0.82
0.84
0.87
1
C323
0.95
0.81
0.98
0.86
1
C324
1
0.89
1
0.78
0.81


### 6.2.4. Step4: Determine the weight of second-level indexes

The maximum deviation method based on deviation degree is applied to calculate the weight of scheduling environment, scheduling process, and scheduling result respectively.The possible index weight of each index for scheduling environment isΦ1={w1=(w11,w21,w31,w41)0.25≤w11≤0.27,0.21≤w21≤0.23,0.24≤w31≤0.28,0.20≤w41≤0.32}.And that for scheduling process isΦ2={w2=(w12,w22,w32,w42)0.22≤w12≤0.25,0.24≤w22≤0.31,0.18≤w32≤0.22,0.28≤w42≤0.32}.And that for scheduling result isΦ3={w3=(w13,w23,w33,w43,w53,w63,w73,w83)∣0.120≤w13≤0.135,0.118≤w23≤0.132,0.109≤w33≤0.141,0.101≤w43≤0.141,0.114≤w53≤0.147,0.119≤w63≤0.132,0.106≤w73≤0.134,0.113≤w83≤0.134}.Three single-objective linear programming models are established as in (17). Software MATLAB 7.0 is adopted and the weight of each index is obtained as shown in Table 8.Table 8
Weight of second-level indexes.
Organization structureC111
0.25
Information technologyC112
0.21
Information collecting abilityC113
0.24
Cooperation relationshipC114
0.3
Adaptability of schedulingC211
0.22
Timeliness of schedulingC212
0.28
Communication qualityC213
0.22
Flexibility of schedulingC214
0.28
Customer complaints rateC311
0.135
Fulfillment rateC312
0.13
On-time deliveryC313
0.109
Response time for demandC314
0.141
Wrong scheduling rateC321
0.147
Scheduling costC322
0.119
Provider satisfactionC323
0.106
Resource efficiencyC324
0.113Consider(17)max⁡D(w)=∑i=15∑j=14∑k=15(|rij-rkj|)wjs.t.wj∈Φ1max⁡D(w)=∑i=15∑j=14∑k=15(|rij-rkj|)wjs.t.wj∈Φ2max⁡D(w)=∑i=15∑j=18∑k=15(|rij-rkj|)wjs.t.wj∈Φ3.


### 6.2.5. Step5: Process the Data of the Sixth Month Based on Multiphase Dynamic Interaction

In this step, the application example of the method based on multiphase dynamic interaction is given. The original data of the sixth month is shown in Table9.Table 9
Original data of the sixth month.
Index value
Nature
C
11
C111
Fair
Qualitative
C112
Very poor
Qualitative
C113
Fair
Qualitative
C114
Fair
Qualitative
C
21
C211
Good
Qualitative
C212
Poor
Qualitative
C213
Very good
Qualitative
C214
Fair
Qualitative
C
31
C311
0.055
Quantitative
C312
0.849
Quantitative
C313
0.913
Quantitative
C314
1.64
Quantitative
C
32
C321
0.022
Quantitative
C322
16900
Quantitative
C323
0.823
Quantitative
C324
0.896
QuantitativeAccording to the original data, 5 experts are invited to score the 8 qualitative indexes. After discussion, 4 times dynamic interaction are conducted. Som=5, n=8, and l=4. The scores are distributed between 0 and 10, and the matrixes are shown as follows:
(18)P0=C111C112C113C114C211C212C213C214[8436555832682448253763657528637737473662]P1=C111C112C113C114C211C212C213C214[8636744355547637853645973694889837657622]P2=C111C112C113C114C211C212C213C214[7563425546458638347257835454676468486624]P3=C111C112C113C114C211C212C213C214[6664426435657537457356846253648558557636]P4=C111C112C113C114C211C212C213C214[6664436445657547557356746453648558557646].Calculate the interaction coefficient with (1) and (2) as in the following:
(19)u1=(0.194,0.214,0.194,0.196,0.203)u2=(0.192,0.201,0.219,0.190,0.198)u3=(0.202,0.196,0.197,0.202,0.202)u4=(0.202,0.199,0.198,0.201,0.199).Aggregate the score of each round with (3):
(20)P1*=(6.009,4.205,4.964,5.840,5.567,7.051,6.623,4.426)P2*=(5.025,4.432,5.259,4.471,5.679,5.175,5.582,5.228)P3*=(5.191,3.988,5.192,5.208,5.788,3.999,6.213,5.387)P4*=(5.199,4.395,5.399,5.408,5.594,4.403,6.210,5.595).Set the stability threshold as0.1(ε=0.1) and the consistency threshold as 0.25(η=0.25), and then calculate the stability index and the consistency index of each round as follows:
(21)υ=(0.56,0.641,0.813,0.921)ω=(0.685,0.7,0.744,0.771).The index stability and index consistency of the fourth round are 0.921 and 0.771, respectively. The test value of stability index is 0.079 less than 0.1, and the test value of consistency index is 0.229 less than 0.25. So it can be concluded that the results pass the tests. Finally, the final score of each index with IOWA is aggregated:(22)P*=(5.243,4.261,5.265,5.258,5.667,4.701,6.127,5.342).The data of the sixth month are normalized by combining the data in previous 5 months, and the results are shown in Table10.Table 10
Normalized value of data in the sixth month.
Index
Data
Normalized data
Scheduling environment
C
111
5.2428
0.86
C
112
4.2613
0.61
C
113
5.2654
0.8
C
114
5.2578
0.85
Scheduling process
C
211
5.6665
0.9
C
212
4.701
0.72
C
213
6.1266
0.91
C
214
5.3423
0.77
Scheduling result
C
311
0.055
0.69
C
312
0.849
0.93
C
313
0.913
0.98
C
314
1.64
0.74
C
321
0.022
0.64
C
322
16900
0.93
C
323
0.823
0.98
C
324
0.896
0.97


### 6.2.6. Step6: Obtain the Final Score

Obtain the final score by the fixed data as shown in Table8 and the index weights determined in Steps 1 and 4. The expressions are shown as in the following:
(23)(0.86×0.25+0.61×0.21+0.80×0.24+0.85×0.30)×0.34+(0.90×0.22+0.72×0.28+0.91×0.22+0.77×0.28)×0.35+(0.69×0.135+0.93×0.130+0.98×0.109mmmmmm+0.74×0.141+0.64×0.147+0.93×0.119mmllmmml+0.98×0.106+0.97×0.113)×0.31=0.815.Therefore, the final scheduling performance of the sixth month (June 2012) is 0.815. According to the performance ratings standardization of Baoyun Logistics Company, 0.815 reaches the good level, but it also has great potential to increase the scheduling performance in the future.


## 6.2.1. Step1: Determine the Weight of First-Level Indexes

According to the evaluation index system, there are three first-level indexes, scheduling environment, scheduling process, and scheduling result. The index of scheduling result is composed of two factors, which are customer satisfaction and service quality. In this paper, it is assumed that the two factors are equally important. The weights of the three indexes for the previous 5 months are 0.3, 0.3, and 0.4, respectively, and need to readjust in the sixth month. The matrix of weight adjustment range obtained from 5 experts is as follows and the consistency test threshold is 0.95:(16)B=[(0.0,0.1)(0.1,0.2)(0.1,0.3)(0.2,0.4)(0.1,0.2)(0.1,0.2)(0.3,0.4)(0.2,0.4)(0.0,0.1)(0.2,0.3)(-0.3,-0.2)(-0.2,0.0)(-0.2,-0.1)(-0.2,-0.1)(-0.3,-0.2)].According to the matrixB, adjustment range di could be calculated by (12); then consistency index ei could be gotten by (13) and (14). Original weight wit-1 and adjusted weight wit′ could be obtained by (15). The weight of first-level indexes is calculated as in Table 5.Table 5
The results of adjusted weight of first-level indexes.
Scheduling environment
Scheduling process
Scheduling outcome
Adjustment rangedi
0.193
0.233
−0.167
Consistency indexei
0.991
0.988
0.990
Original weightwit-1
0.3
0.3
0.4
Adjusted weightwit′
0.358
0.370
0.333
Normalized weightwit
0.34
0.35
0.31See from Table5 that the consistency index values of the experts evaluation are all over 0.95, and the adjustment of index weight is acceptable, so the first-level weights of the 6th period are 0.34, 0.35, and 0.31, respectively.


## 6.2.2. Step2: Quantify the Qualitative Indexes Using the Method of Multiphase Dynamic Interaction

According to the original data, by using the method of multiphase dynamic interaction, the qualitative indexes of previous five periods can be converted into benefit-type quantitative ones, and the decision matrixA=(aij)c×b can be obtained as in Table 6.Table 6
Decision matrix.
January
February
March
April
May
C
11
C111
5.6
5.8
6.1
4.5
5.1
C112
7
4.6
4.6
4.6
4.4
C113
6.5
6.6
5.7
6.8
5.2
C114
4.3
6.2
4.5
4.3
5.2
C
21
C211
6.3
6.2
6.3
5.1
4.2
C212
4.5
4.5
6.2
6
6.5
C213
5.1
4.3
6.7
4.5
4.1
C214
6.9
5.1
6.5
6.9
6.8
C
31
C311
0.051
0.038
0.067
0.041
0.045
C312
0.731
0.91
0.909
0.711
0.85
C313
0.727
0.932
0.75
0.903
0.897
C314
1.22
2.16
1.36
1.75
2.83
C
32
C321
0.014
0.048
0.035
0.057
0.024
C322
17400
19100
18700
18000
15700
C323
0.801
0.685
0.829
0.727
0.842
C324
0.927
0.825
0.926
0.72
0.75


## 6.2.3. Step3: Normalize Decision Matrix

Customer complaints rate (C311), response time for demand (C314), wrong scheduling rate (C321), and scheduling cost (C322) are cost-type indexes and the rest are benefit-type indexes. After the decision matrix is normalized, the normalized decision matrix can be obtained as in Table 7.Table 7
Normalized decision matrix.
January
February
March
April
May
C
11
C111
0.92
0.95
1
0.74
0.84
C112
1
0.66
0.66
0.66
0.63
C113
0.96
0.97
0.84
1
0.76
C114
0.69
1
0.73
0.69
0.84
C
21
C211
1
0.98
1
0.81
0.67
C212
0.69
0.69
0.95
0.92
1
C213
0.76
0.64
1
0.67
0.61
C214
1
0.74
0.94
1
0.99
C
31
C311
0.75
1
0.57
0.93
0.84
C312
0.8
1
1
0.78
0.93
C313
0.78
1
0.8
0.97
0.96
C314
1
0.56
0.9
0.7
0.43
C
32
C321
1
0.29
0.4
0.25
0.58
C322
0.9
0.82
0.84
0.87
1
C323
0.95
0.81
0.98
0.86
1
C324
1
0.89
1
0.78
0.81


## 6.2.4. Step4: Determine the weight of second-level indexes

The maximum deviation method based on deviation degree is applied to calculate the weight of scheduling environment, scheduling process, and scheduling result respectively.The possible index weight of each index for scheduling environment isΦ1={w1=(w11,w21,w31,w41)0.25≤w11≤0.27,0.21≤w21≤0.23,0.24≤w31≤0.28,0.20≤w41≤0.32}.And that for scheduling process isΦ2={w2=(w12,w22,w32,w42)0.22≤w12≤0.25,0.24≤w22≤0.31,0.18≤w32≤0.22,0.28≤w42≤0.32}.And that for scheduling result isΦ3={w3=(w13,w23,w33,w43,w53,w63,w73,w83)∣0.120≤w13≤0.135,0.118≤w23≤0.132,0.109≤w33≤0.141,0.101≤w43≤0.141,0.114≤w53≤0.147,0.119≤w63≤0.132,0.106≤w73≤0.134,0.113≤w83≤0.134}.Three single-objective linear programming models are established as in (17). Software MATLAB 7.0 is adopted and the weight of each index is obtained as shown in Table 8.Table 8
Weight of second-level indexes.
Organization structureC111
0.25
Information technologyC112
0.21
Information collecting abilityC113
0.24
Cooperation relationshipC114
0.3
Adaptability of schedulingC211
0.22
Timeliness of schedulingC212
0.28
Communication qualityC213
0.22
Flexibility of schedulingC214
0.28
Customer complaints rateC311
0.135
Fulfillment rateC312
0.13
On-time deliveryC313
0.109
Response time for demandC314
0.141
Wrong scheduling rateC321
0.147
Scheduling costC322
0.119
Provider satisfactionC323
0.106
Resource efficiencyC324
0.113Consider(17)max⁡D(w)=∑i=15∑j=14∑k=15(|rij-rkj|)wjs.t.wj∈Φ1max⁡D(w)=∑i=15∑j=14∑k=15(|rij-rkj|)wjs.t.wj∈Φ2max⁡D(w)=∑i=15∑j=18∑k=15(|rij-rkj|)wjs.t.wj∈Φ3.


## 6.2.5. Step5: Process the Data of the Sixth Month Based on Multiphase Dynamic Interaction

In this step, the application example of the method based on multiphase dynamic interaction is given. The original data of the sixth month is shown in Table9.Table 9
Original data of the sixth month.
Index value
Nature
C
11
C111
Fair
Qualitative
C112
Very poor
Qualitative
C113
Fair
Qualitative
C114
Fair
Qualitative
C
21
C211
Good
Qualitative
C212
Poor
Qualitative
C213
Very good
Qualitative
C214
Fair
Qualitative
C
31
C311
0.055
Quantitative
C312
0.849
Quantitative
C313
0.913
Quantitative
C314
1.64
Quantitative
C
32
C321
0.022
Quantitative
C322
16900
Quantitative
C323
0.823
Quantitative
C324
0.896
QuantitativeAccording to the original data, 5 experts are invited to score the 8 qualitative indexes. After discussion, 4 times dynamic interaction are conducted. Som=5, n=8, and l=4. The scores are distributed between 0 and 10, and the matrixes are shown as follows:
(18)P0=C111C112C113C114C211C212C213C214[8436555832682448253763657528637737473662]P1=C111C112C113C114C211C212C213C214[8636744355547637853645973694889837657622]P2=C111C112C113C114C211C212C213C214[7563425546458638347257835454676468486624]P3=C111C112C113C114C211C212C213C214[6664426435657537457356846253648558557636]P4=C111C112C113C114C211C212C213C214[6664436445657547557356746453648558557646].Calculate the interaction coefficient with (1) and (2) as in the following:
(19)u1=(0.194,0.214,0.194,0.196,0.203)u2=(0.192,0.201,0.219,0.190,0.198)u3=(0.202,0.196,0.197,0.202,0.202)u4=(0.202,0.199,0.198,0.201,0.199).Aggregate the score of each round with (3):
(20)P1*=(6.009,4.205,4.964,5.840,5.567,7.051,6.623,4.426)P2*=(5.025,4.432,5.259,4.471,5.679,5.175,5.582,5.228)P3*=(5.191,3.988,5.192,5.208,5.788,3.999,6.213,5.387)P4*=(5.199,4.395,5.399,5.408,5.594,4.403,6.210,5.595).Set the stability threshold as0.1(ε=0.1) and the consistency threshold as 0.25(η=0.25), and then calculate the stability index and the consistency index of each round as follows:
(21)υ=(0.56,0.641,0.813,0.921)ω=(0.685,0.7,0.744,0.771).The index stability and index consistency of the fourth round are 0.921 and 0.771, respectively. The test value of stability index is 0.079 less than 0.1, and the test value of consistency index is 0.229 less than 0.25. So it can be concluded that the results pass the tests. Finally, the final score of each index with IOWA is aggregated:(22)P*=(5.243,4.261,5.265,5.258,5.667,4.701,6.127,5.342).The data of the sixth month are normalized by combining the data in previous 5 months, and the results are shown in Table10.Table 10
Normalized value of data in the sixth month.
Index
Data
Normalized data
Scheduling environment
C
111
5.2428
0.86
C
112
4.2613
0.61
C
113
5.2654
0.8
C
114
5.2578
0.85
Scheduling process
C
211
5.6665
0.9
C
212
4.701
0.72
C
213
6.1266
0.91
C
214
5.3423
0.77
Scheduling result
C
311
0.055
0.69
C
312
0.849
0.93
C
313
0.913
0.98
C
314
1.64
0.74
C
321
0.022
0.64
C
322
16900
0.93
C
323
0.823
0.98
C
324
0.896
0.97


## 6.2.6. Step6: Obtain the Final Score

Obtain the final score by the fixed data as shown in Table8 and the index weights determined in Steps 1 and 4. The expressions are shown as in the following:
(23)(0.86×0.25+0.61×0.21+0.80×0.24+0.85×0.30)×0.34+(0.90×0.22+0.72×0.28+0.91×0.22+0.77×0.28)×0.35+(0.69×0.135+0.93×0.130+0.98×0.109mmmmmm+0.74×0.141+0.64×0.147+0.93×0.119mmllmmml+0.98×0.106+0.97×0.113)×0.31=0.815.Therefore, the final scheduling performance of the sixth month (June 2012) is 0.815. According to the performance ratings standardization of Baoyun Logistics Company, 0.815 reaches the good level, but it also has great potential to increase the scheduling performance in the future.


## 7. Conclusions

Evaluating the LSSC scheduling performance is beneficial to make better decision for LSI. In this paper, the scheduling performance evaluation of LSSC is explored in depth and the evaluation index system is established. Meanwhile, an evaluation method based on dynamic index weight is proposed. According to the study, the following conclusions are reached.(1)
The evaluation index system of LSSC scheduling performance should be designed from the perspective of strategic level, tactical level, and operational level. The system proposed in this context considers not only the different demands of different members in LSSC, but also the requirement of coordination and characteristics of service, which is scientific and can be used for reference to scheduling performance management.(2)
Considering the multiperiod feature of scheduling activities, the difficulty in measuring the qualitative indexes, and the needs of index weight adjustment, we propose a method based on dynamic index weight. In this method, firstly, for the difficulty in measuring the qualitative indexes, an improved method of multiphase dynamic interaction is adopted to improve the accuracy. Secondly, as for the need of index weight adjustment, an improved maximum deviation method based on deviation degree is utilized to determine the weight of second-level indexes, which can remove the uncertainty of human decision. Thirdly, an adjustment coefficient method based on set-valued statistics is developed to adjust the weight of first-level indexes reasonably. Thus, this method is more suitable than the traditional evaluation ones in LSSC scheduling performance evaluation.(3)
Actual data from Baoyun Logistics Company were collected to exemplify this new scheduling performance evaluation method. The application example illustrates that the method can evaluate the scheduling performance scientifically and provide good basis to LSI for improving the performance of LSSC scheduling.In this paper, the problem of LSSC scheduling performance evaluation is investigated and a new evaluation method is proposed, but there are still some limitations. For instance, the evaluation index system reflects the factors of before, during, and after the schedule, while the more detailed factors, such as order allocation, process selection, and time flexibility scheduling, are not considered. As for the evaluation method proposed, the possible intervals of second-level indexes weights are obtained by experience, which contains some subjectivity. How to improve the objectivity of index weight intervals is one of future directions. Moreover, with the evaluation value obtained from the method proposed, how to use it to forecast the performance of next period and present improvement program is also the focus of future researches.



---

*Source: 280741-2014-05-19.xml*