# Service Selection and Ranking: A Framework Proposal and Prototype Implementation

Firmino Silva, Claudia-Melania Chituc, Paul Grefen

► To cite this version:

Firmino Silva, Claudia-Melania Chituc, Paul Grefen. Service Selection and Ranking: A Framework Proposal and Prototype Implementation. 18th Working Conference on Virtual Enterprises (PROVE), Sep 2017, Vicenza, Italy. pp.395-403, 10.1007/978-3-319-65151-4_36 . hal-01674900

HAL Id: hal-01674900

https://inria.hal.science/hal-01674900

Submitted on 3 Jan 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Service Selection and Ranking: A Framework Proposal
and Prototype Implementation

Firmino Oliveira da Silva, Claudia-Melania Chituc and Paul Grefen

School of Industrial Engineering, Eindhoven University of Technology,
5612 AZ Eindhoven, The Netherlands
{F.Oliveira.da.Silva, C.M.Chituc, P.W.P.J.Grefen}@tue.nl

**Abstract.** Organizations that are part of collaborative service networks need to handle increasing amounts of data in their information systems to construct complex customer-oriented solutions from dynamically selected service elements. This brings numerous challenges in today's highly competitive markets, where companies need to provide customers with services that have a high level of quality, in a time and cost effective manner. Having prior knowledge of the performance associated with specific choreographies of services allows companies to provide customers with services tailored to their specific requests, while maintaining a high quality of the services. This paper presents an approach for service selection and ranking considering customers' requirements, knowledge of the historic behavior of services, business process constraints and the characteristics of the execution environment. Following the specifications of the proposed framework, a prototype has been implemented targeting the automotive sector. Tests of this prototype illustrate the usability and validity of the proposed approach.

**Keywords:** collaborative network, service selection, service ranking, prototype.

# 1 Introduction

The increasing digitization of the business world creates new opportunities for companies [1]. More and more devices are connected and new services are available, generating a huge traffic and volume of data, creating challenges for enterprises that have to constantly adapt to new demands. Customers' demands for high quality services tailored to their specific needs are increasingly high, and companies need to rethink their business strategy and focus on their core activities. Offering tools to customers that allow to configure the requested services (that may be provided by different partners that are part of a collaborative network), and enabling an accurate estimate of the results based on their expectations (e.g., estimates of service cost, duration, and quality level) is nowadays of great importance to face competition. In today's dynamic business environment, organizations increasingly form partnerships in order to improve their performance, increase their competitive advantage to share knowledge, skills and resources to seize market opportunities and reduce operating costs [2]. Assessment and monitoring of cross-organizational business processes plays
an important role in this dynamic environment to evaluate the performance of the services of the involved partners [3]. The development of methods of service selection and ranking that most appropriately apply to complex customer's requests is thus a relevant matter of concern.

The aim of this paper is to present an approach for service selection and ranking, considering customers' requirements, knowledge of the behavior of services, business process constraints and the characteristics of the execution environment. Following the specifications of the proposed framework advanced in [4-5], a prototype has been implemented targeting the automotive sector. We illustrate in this paper how the prototype can be used to obtain the most appropriate services considering customers' requirements. Tests of this prototype illustrate the usability and validity of the proposed approach.

The rest of this paper is organized as follows. The proposed framework for service selection and ranking is briefly presented next. The implementation of the prototype and results of its tests are discussed in Section 3. Related work is analyzed in Section 4. The article concludes with a section addressing future research work.

## 2 Framework for Service Selection and Ranking

The framework developed for service selection and ranking using service choreography advanced in [4-5] follows an adaptive control system approach, as described in [6-7], and integrates the basis of the hierarchical model of [8]. The hierarchical control model was built to allow a self-adaptation of the system by obtaining information from the monitoring and assessment mechanism for a learning process in order to ensure its evolution and adjust deviations.

Fig. 1 illustrates the dynamic cycles of the proposed framework. Following specifications of a complex adaptive system (as in [9]) and adaptive control system [10], the proposed framework allows to achieve and maintain a desired level of control when variables dynamically change in time (e.g., product cost, availability). The framework is built upon four main modules that interact with two levels of repositories [4-5]: the *Basic Application Setup module* contains elements that depend on the customer criteria and preferences, allowing them to enter the data required to fulfill the characteristics of the service; *Core module* elements support the definition of the business and scoring rules for the provider, the identification of the services that will build up the specific business, the pools where all the services with similar functions will compete, and the definition of metrics to evaluate the performance of services; *Choreography Engine Setup module* contains elements that assemble and instantiate the choreography; *Monitoring and Assessment System module* whose elements support the instantiation of the monitoring and assessment mechanism; *Central Operational Repository* which stores operational information; and *Knowledge repository* which stores historical information from metrics assessments and choreography execution results.

The modules of the framework are organized in hierarchical levels so that each role can be applied in each center of competences. The first functional group (*Decisional: 1*) integrates elements for setting strategic and tactical approaches, i.e., the provider
business rules, types of metrics definition to assess service behaviors. The second group of elements (Operational: 2) is responsible for the instantiation of elements that fulfill requirements of a specific instance requested by the customer and retrieve data from the assessment mechanism.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Framework dynamic improvement cycles (following the specifications in [4-5])</figcaption></figure>

Following these specifications, the prototype that supports the service selection and ranking method is described next.

# 3 Prototype Implementation and Discussion of Results

The prototype implemented follows the specifications of the framework for service selection and ranking using service choreography advanced in [4-5] and its dynamic cycles illustrated in Fig. 1. This prototype targets the automotive business sector, in particular, the aftermarket automotive environment, that is: a manufacturer independent Car Garage that develops Car Maintenance Operations (CMOs).

## 3.1 Introduction

A high percentage of CMO interventions are basically summarized to a standard set of operations that depend on vehicle mileage or a contract formalized between the customer and the brand. These standard interventions are likely to predict which car parts are targeted by a CMO intervention, reflected in the services that will use the respective car parts. The customer (who is the car owner) can change or decline a suggested car part indicated by the car manufacturer and select another one if it is compliant. A customer may also request to replace other car parts not listed in the CMO mapping of the car manufacturer. Each brand, model, and year of manufacturing car series maps all the operations according to the characteristics of the vehicle, and this allows the Car Garage to perform CMOs based on an advanced knowledge of the operations required to meet the maintenance objectives. The Car
Garage does not need to have a conventional warehouse to stock parts (with significant earnings in space, building, electricity, air conditioning, information system, people) which reduces the operating costs. The main activities as follows: a customer fills in a form and sets the request for the CMOs (e.g., by choosing the car parts), books and pays the car parts in advance taking into account the CMO intervention date. The provider's garage offers skilled labor in mechanics, electricity and electronics to assemble, replace and install car parts to perform the agreed CMOs. When all the CMOs are completed, the provider pays all the partners involved in the CMO.

## 3.2 Customer Front-End

For the client side, the front-end interface has been designed to allow a wide variety of service choices in order to proceed with the CMO, such as: different alternatives for the car parts to be included in the CMO, the quantity and the part brand availability; the criteria of the brand price (i.e., highest price / best price / normal), the quality criteria (highest / medium / normal), the intervention date and the prioritization of the criteria (where the default mode is: 1st-Quality and Brand Availability; 2nd - Price; 3rd - Quality). The weights assigned to each metric by the customer (and controlled by the provider through scoring and metrics tables) are automatically calculated based on the choices selected from this *Front-end* form. Each time a new request is submitted, different alternatives (considering the customer weights) are calculated by the Service Selection Services and Services Ranking Matrix elements into the matrix of calculus, as explained in [5]. The matrix is associated to a pool that stores in the database all the customer requests. After a service is selected, the values of the metrics associated with the respective service are updated.

## 3.3 Simulation Builder

After a customer submits its requirements and preferences for the CMO, the *Generator of the Pools of Services* (Fig. 2) builds a number of service pools that the provider may test, and the services to compete in each pool. It also produces, at the provider's request, a number of metrics by service pool, needed to monitor and assess each service. In the example illustrated in Fig. 2, ten service pools and ten metrics for each pool were chosen for the simulation. The number of services by each pool is randomly generated in a range of [300..500] in order to simulate a market service database. Each metric varies in predefined ranges and all the data types are Numeric and Boolean. The provider configures tables with scoring rules that implement a more aggressive strategy, or gives the customer more decision power to influence this behavior. For each customer request a matrix to perform the service selection and ranking is generated (as explained in [4-5]). The definition of metrics<fnref n="1" /> for the monitoring system is identified at each layer (e.g., service's composition layer),

<footnote n="1">¹ An example of a metric considered in this selection is the *Global Service Cost* referring to all the costs of all the operations and parts, including the required labor effort and a fee that is applied in order to support the operational business cost.</footnote>
considering a specific dimension (e.g., technical infrastructure), data types, range of
values, and impact on other metrics. Assigning the customer's weight to metrics is
automatic, according to the tables managed by the provider.

<figure><img src="image_3.png" /><figcaption>Fig. 2. Generator of pools of services.</figcaption></figure>

3.4 Service Proposal and Service Request Results

The customer needs to submit a service request in order to get the estimated cost of
the CMO. After the customer requests a "New Proposal", the method for selecting
and ranking the services is executed. The algorithm consists of (1) selecting the
services whose priority is defined in terms of "Prioritize Criteria", and (2) obtaining
as a result the first sub-set of services whose pool of performance range values are in
accordance with what the customer selected as the first criterion. In the next iteration,
the first subset of services is used to evaluate the second priority and the resulting
subset matches the performance range values for the second criterion. Another
iteration is run (for the third criterion) until a reduced set of services is achieved,
responding with an estimate to the request submitted. On this latter subset, an
algorithm for ranking is run that allows obtaining the best service positions in the
pool. This classification depends on:

- the criteria and preferences of the customer and the weight assigned to each
metric that analyzes the performance of each service;

- the business rules that the provider imposes which allow to control the
performance of the system;

- the number of entries in the best rated choreographies of services;

- the evaluation of the service performance resulting from the comparison between
the estimated value and the actual value (with direct attribution of a penalty or a
bonus);

- and the ratio between the number of participations in the best choreographies and
in the lower rated ones.

From this processing we obtain the identification of the services that will participate
in the choreography. The expected values for the performance of the services are
recorded taking into account the existing knowledge in the pools.
<figure><img src="image_4.png" /><figcaption>Fig. 3. Service Proposal partial layout.</figcaption></figure>

The results obtained are displayed to the client, as illustrated in Fig. 3. Different colors may be used to differentiate between the results (e.g., green for the best choreography). Considering the result of “Pool 4”, for example, the service identified as the best service has the ID = 327 (with a score of 3,906) cannot assure the Best price for the brand, given the percentage obtained (at yellow signage: 50%). Additional information is given at the bottom of the form, such as: the availability of the Car Garage on the chosen date, estimated global cost and duration.

Taking the example of Pool 3 (Fig. 4), the part of the matrix responsible for calculating the scoring rules is visible in *a*) and reflects the ranking of the top three services in that pool. The application of "wAct" described in [5] makes the service ID=365 pass from the 8th to the 4th (in rank2 column of *a*)), precisely because it obtains superior performances for all the metrics having weights in *b*). SC1 [5] determines that the service ID=65 moves from 3rd to 1st position since it has benefited in the last participation in a choreography with *b*) 0,5. SC3 [5] allows the service with the ID=365, which has the highest number of entries in the most valued choreographies, to move from the 2nd to the 1st position and to remain in this position since the ratio between the number of participations in best choreographies and in the normal rated ones SC4 [5] is higher (0.729) than the other services (0,403: Service ID=65) and (0,063: service ID=85) respectively. In case the client does not accept some of these values, he/she can re-submit a request for another proposal by changing the service parameter that is below expectations or by filling in other values (i.e., choose other “Brand” in the "Front-end" form), and repeat the execution for specific service pools. If the customer accepts the proposal presented, the next step is to obtain the order of the global service and proceed with the payment.

The last step of this process is to receive the feedback information from the monitoring and assessment system to update the performance values of the services of the different pools from the submitted request. For various market circumstances, some of the performance values may differ. This aspect is managed by the method implemented by the parameter "penalty/benefit" explained in [5] which is related to the comparison between the expected performance of each service and its actual
performance. Differences resulting from this comparison allow to classify the degree of (in)success of the choreography, compared to its expected performance.

<figure><img src="image_5.png" /><figcaption>Fig. 4. Pool 3: partial view of (a)ranked services and (b)the metrics services behavior</figcaption></figure>

## 4 Related Work

Several works are concerned with decoupling the data flow from the control flow [11][15]. Monitoring and assessing service compositions are fundamental to allow the provisioning of a valid business process. Several approaches for service monitoring exist. Some approaches are concerned with identifying erroneous situations after they occur and early detection of faults [12-13]. Different from these works, this approach takes into account the performance results of services executed and its benefits or penalties according to the estimated values of the performance metrics. Other approaches are concerned with discovering the main factors of influence of process performance [3]. In this research work the control of the performance of the framework is determined by the provider strategy through business scoring rules. The present work relates to the monitoring of choreographies in a cross-organizational setting, similar to [3]. The added value of the approach presented in this article lies in the strong responsiveness addressing the customer's request, in which the service that is selected as the most suitable is the one that is executed. It does not focus on the analysis during the execution process related to predicting failures, adaptations and corrections of service violations [12-14]. The analysis of the most appropriate service is carried out before being submitted to the market, based on previous behavior recorded in pools of historical data. The framework learns and enhances results iteratively, iterating each time a customer submits a request.

## 5 Conclusions and Future Work

This work presents a prototype implemented following the framework specifications presented in [4-5], supporting the service selection and ranking. The analysis of data shows that the weights that the customer assigns to the selection criteria, and which are automatically channeled to the respective metrics, provide clients with service
proposals according to their expectations. This approach also allows the supplier to adjust the parameters of the scoring rules such that the framework behaves as expected. The results of the tests performed showed that the framework evolves and self adjusts with the execution of customer requests, and improved services are offered at each new customer iteration. The novelty of the proposed approach is related to the fact that it allows the selection of the services that best fit the customer requests as a result of a refined processing of several variables (i.e., the criteria defined by the customer). This approach can be easily applied to other business sectors that involve the composition of several services to fulfill the customers' requests. Future work will focus on the implementation of the instantiation modules of the services choreography.

## References

1. Weill, P. et al., 2015. Thriving in an increasingly digital ecosystem. MIT Sloan Management Review 56(4) pp. 27-34.

2. Afsarmanesh, H., Camarinha-Matos, L., 2005. A framework for management of virtual organization breeding environments. In: Collaborative Networks and Their Breeding Environments. IFIP, vol. 186, pp. 35-49. Springer, Heidelberg.

3. Wetzstein, B., et al., 2010. Cross-organizational process monitoring based on service choreographies. In proceedings 2010 ACM Symposium on Applied Computing.

4. Silva, F., Chituc, C.-M., 2013. Towards the Definition of a Framework Supporting High Level Reliability of Services. In: Ghose A. et al. (Eds.) Service-Oriented Computing - ICSOC 2012 Workshops. ICSOC 2012. LNCS vol. 7759, Springer, Berlin, Heidelberg.

5. Silva, F., Chituc, C.-M., Grefen, P., 2015. An Approach for Automated Service Selection and Ranking using Services Choreography. in Proceedings WEBIST-2015, ISBN: 978-989-758-106-9, pp. 259-266.

6. Landau, I. D. et al., 2011. Adaptive Control - Algorithms, Analysis and Applications. Communications and Control Engineering, Springer, ISBN 978-0-85729-664-1.

7. Hellerstein, J.L. et al., 2004. Feedback Ctrl of Comp. Systems. IEEE P/Wiley Interscience.

8. Kaplan, R. S.; D.P. Norton, 2008. Mastering the Management System. Harvard Business Review, pp. 62-77.

9. Holland, J.H. 1992. Complex Adaptive Systems. The MIT Press - Daedalus, Vol. 121, A New Era in Computation, pp. 17-30.

10. Feng, G., R. Lozano, 1999. Adaptive Control Systems. B. H., Oxford, U.K.: Newnes.

11. Hahn, M. et al., 2016. Data-Aware Service Choreographies through Transparent Data Exchange. In Proceedings of the 16th Int. Conf. on Web Engineering, ICWE 2016, pp.6-9.

12. Wetzstein, B. et al., 2012. Preventing KPI violations in business processes based on decision tree learning and proactive runtime adaptation. J. of Systems Integration 3(1), pp. 3-18.

13. Leitner, P. et al., 2010. Monitoring, Prediction and Prevention of SLA Violations in Composite Services. In Proc. IEEE ICWS'10. IEEE Computer Society, pp. 369-376.

14. Rajaram, K. et al., 2015. Monitoring Flow of Web Services in Dynamic Composition Using Event Calculus Rules. EEE Int. Conf. on Computer Comm. and Control (IC4).

15. Hahn, M. et al., 2016. A Management Life Cycle for Data-Aware Service Choreographies. Proceedings of the 23rd IEEE ICWS 2016, pp. 364-371.