# A Multi-agent Approach to Implement a Reverse Production Virtual Market in Green Supply Chains

Adriana Giret, Miguel A. Salido

► To cite this version:

Adriana Giret, Miguel A. Salido. A Multi-agent Approach to Implement a Reverse Production Virtual Market in Green Supply Chains. IFIP International Conference on Advances in Production Management Systems (APMS), Sep 2017, Hamburg, Germany. pp.399-407, 10.1007/978-3-319-66926-7_46 hal-01707285

HAL Id: hal-01707285

https://inria.hal.science/hal-01707285

Submitted on 12 Feb 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A multi-agent approach to implement a Reverse Production Virtual Market in Green Supply Chains

Adriana Giret\* and Miguel A. Salido

Dpto. Sistemas Informáticos y Computación.
Universitat Politècnica de Valencia, Spain.
Valencia, Spain.
{agiret,msalido}@dsic.upv.es

**Abstract.** The implementation of internal reverse production process programs often involves significant allocations of capital and resources for the construction and implementation of all the steps in the process. But, what if we think of reverse production process as a service-based manufacturing network, in which all the activities are outsourced and the only thing that a manufacturing company needs in order to participate is an interface/service to "play" in that ecosystem. In this work we present an approach to implement reverse production process following a Service-Oriented Manufacturing paradigm by means of a virtual market supported by intelligent software agents.

## 1 Introduction

Over the last few years there has been a well-recognized need for achieving over-all sustainability in manufacturing activities [6, 8], due to several established and emerging causes: environmental concerns, diminishing non-renewable resources, stricter legislation and inflated energy costs, increasing consumer preference for environmentally friendly products, etc. In this way corporate environmental responsibility has gradually and more consistently extended beyond complying with increasingly stringent environmental regulation and also beyond the taking up of proactive initiatives. Many research works have demonstrated the positive effects of internal measures towards sustainability [9]. Lately the goal has begun to shift to integrating all of the business value-adding operations, including purchasing and in-bound logistics, production and manufacturing, distribution and out-bound logistics, in such a way that activities associated with these functions have the least harmful environmental impact [13]. Greening the supply chain is one such innovative idea that is fast gaining attention in the industry [3].

There are a number of definitions and philosophies on what a green supply chain may be. These vary from something as small as buying green products from a supplier to the much broader context of an industrial ecosystem [5]. Sustainable

<footnote>\* Corresponding Author</footnote>
supply chain is an active research field in which new techniques are continuously proposed to reduce negative environmental impacts while pursuing production economy [13]. Sustainable supply chain network consists of forward production processes and reverse (recycling) production processes (reverse logistics). In the specialized literature, most of the attention is targeted to the forward action of the supply chain [5], whereas few manufacturers have considered how this supply chain can or should work in reverse to reclaim products at the end of their life-cycle and return them through the supply chain for decomposition, disposal or re-use of key components. Systems in which the two approaches (forward and reverse) are combined can drastically reduce the negative impacts to the environment [17]. These systems are called closed-loop production systems<fnref n="1" />.

The implementation of internal reverse production process programs often involves significant allocations of capital and resources for the construction and implementation of all the steps in the process. But, what if we think of reverse production process as a service-based manufacturing network [7], in which all the activities are outsourced and the only thing that a manufacturing company needs in order to participate is an interface/service to “play” in that ecosystem.

In this work we present an approach to implement reverse production process following a Service-Oriented Manufacturing paradigm by means of a virtual market supported by intelligent software agents [18]. In a nutshell our approach consists of an ecosystem of services (i.e. web services) that implements a virtual market for buying and selling items subject to be recycled (i.e. old materials, wastes, used products, etc.). A company will participate in the market: (i) when the company gets economic profit from buying or selling those items, or; (ii) when the company is trying to reduce its "cuota" of negative environmental impact (i.e. reducing CO2 emissions, waste of materials, pollutants, etc.).

## 2 Green Supply Chains

Translating the definition of sustainable development to supply chains impose the need to migrate from a field that myopically addressed only operational and economic matters to one that comprehensively considers the broader environmental and social issues that face organizations of today [5]. A consensus definition of green and sustainable supply chains does not exist (for a large review on definitions see [1]). It is identified as a sub-discipline of supply chain managing expanding the work in a variety of areas. An in-depth state-of-the-art review on green supply chains can be found in [5]. In this work we consider a manufacturing supply chain as a system that consists of 5 layers, including raw material supply, manufacturing, wholesaling, retailing, and end-customers.

<footnote n="1">¹ A closed-loop production system is defined as the “taking back of products from customers and recovering added value by reusing the entire product, and/or some of its modules, components and parts” [16]. The closed-loop construct consists of the common forward supply chain and the so-called reverse supply chain which closes the loop. In summary, there exist three different options to close the loop: reusing the product as a whole, reusing the components or reusing the materials.</footnote>
Whereas, a used-product and/or waste materials reverse logistics chain includes collecting points, recycling plants, disassembly plants, secondary material markets, and final disposal locations of wastes. All these activities are executed with the purpose of capturing value, or proper disposal.

The keys to the successful design and use of reverse logistics systems include costs, overall quality, customer service, environmental concerns and legislative concerns. Other factors to consider are: cost-benefit analysis, transportation, warehousing, supply management, remanufacturing and recycling, and packaging. All these factors require high economic costs for a company to implement a reverse production process. In this way financially attractive approaches are urgently required to facilitate the participation and commitment of manufacturing companies and customers in green supply chains. Bearing this issue in mind and trying to contribute to the state-of-the-art on research works related with reverse production systems, recognized as still low [5], we propose a virtual market of used-products and/or waste materials supported by artificial intelligent techniques and service oriented manufacturing approaches.

## 3 MAS and Intelligent Manufacturing Systems

New technologies are revolutionizing the way manufacturing and supply chain management are implemented. Initiatives such as Industrie 4.0, Smart Industry, Industrie du Futur, among others from Europe; Industrial Internet Consortium from USA; Industrial Value Chain Initiative from Japan; Made in China 2025 from China, are trying to integrate Internet and manufacturing systems in order to fully digitalize the factories of the future. This new production paradigm is based on concepts as autonomy and co-operation because both are necessary to create flexible behavior and thus to adapt to the changing production conditions. One such approach is the Intelligent Manufacturing Systems (IMS) paradigm. In IMS the manufacturing system is conceived as a distributed system in which its constituent components, such as machines, resources, products, and staff, have intelligent capabilities for acting in its environment pursuing global system goals; and have autonomous execution for decision making, social interaction with other intelligent entities, and collaboration for achieving the system goals.

Agent based Manufacturing Systems, are based on Multi-Agent System (MAS) technology [2]. MAS is focused on the co-ordination of entities, called agents, with intelligent behaviors that interact in a group of (possibly pre-existing) agents. A closely related approach to MAS is Holonic Manufacturing System (HMS). HMS is based on the concept of "holonic systems" to describe autonomous manufacturing modules (holons) with distributed control. In a HMS, a holon can be used for transforming, transporting, storing and/or validating information and physical objects. A holon has an information processing part, in charge of the logical and intelligent computation; and often a physical processing part (optional), in charge of controlling a physical component: a machine, a tool, etc. In this way holons are a powerful approach to implement Cyber Physical Systems for the factories of the future in Industrie 4.0.
<figure><img src="image_2.png" /><figcaption>Fig. 1. Reverse Logistic Virtual Market Ecosystem</figcaption></figure>

The specialized literature in IMS, focused on MAS and HMS, offers successful approaches for achieving sustainability in manufacturing systems. The concept of Go-green Holon proposed in [15], is a pre-built development artifact that includes efficiency-oriented mechanisms (optimizing sustainability means), in addition to classical effectiveness-oriented mechanisms, to make a decision and/or execute an operation in IMS. Go-green ANEMONA [10] is a complete software engineering method for developing sustainable IMS. The method helps the designer to specify and implement sustainable optimization functions in IMS providing development guidelines and modeling templates. On the other hand, in [11] it is proposed the application of agent-based systems for supply-chain synchronized production planning including management of raw materials flow as well as flow of returned by customer obsolete products and defected semi-products and products that are refused by quality control within the factory. Whereas, in [12] a multi-agent system framework to achieve coherent and consistent workflows that can meet order requirements is proposed.

## 4 A Virtual Market approach for Reverse Production Systems

The overall idea behind the proposed approach is to facilitate the participation of stakeholders in the activities of green supply chains by means of Service Oriented Manufacturing Systems [7], providing an easy to use interface for the players of a wider reverse logistic ecosystem as a step towards virtual outsourcing of reverse manufacturing processes.

The focus of the work presented in this paper is the market of products subject to be recycle. Figure 1 shows the structure of the virtual market. In the market we can find different roles: Buyers of items (old materials, wastes, used
products, etc.) that want to further process the items or just want to re-sell them
in other markets; Sellers that want to get economic profit (or even "green" profit)
from the disposable items (i.e. if the seller is a manufacturing company, the items
may usually come from their production activities, being wastes, old materials
or used products. etc.); Market Staff which is a special role that is in charge of
ensuring the supporting activities in the market (i.e. execution control activities,
enforcement activities for the market rules, etc.). The virtual interaction among
these roles are implemented by means of web services, that are published in the
Cloud and managed by a Service Oriented Architecture (SOA) framework. The
services provide the basic functions to play in the market ecosystem such as:
Negotiation Protocol Service, Call for Bid Service, Bid Service, etc.

The three roles of the market are implemented by means of agents. In our
approach we follow the guidelines from Go-green ANEMONA [10] in order to
design the agents and the negotiation interactions in the market. The Buyer
and Seller roles are specified as Go-green Holons [15]. Whereas, the interaction
functions of the market are encapsulated as web services following the process
definition of Service Oriented Manufacturing System approach [7].

Picture a market, where customers (Buyers or Sellers) are involved in face-
to-face negotiation, participate in auctions or convene to consolidate an offer
to other large customer. A market with several *negotiation dialogs*, each with
a specific negotiation protocol and each with its own access and termination
conventions (and other rules). Moreover, a market that also allows information
exchange among the participants, request to open or enter a *negotiation dialog*,
invite players to *negotiation dialogs*, and participants to reconvene after leaving a
*negotiation dialog*. We capture this global arrangement with the MAS approach
described in next sub-sections.

**4.1 The agents**

The three main roles mentioned above are, in turn, specialized into five roles that
may interact. The customer (or party agent) of the market that can participate
as **Buyer** (b) and/or **Seller** (s). Finally, there are three types of **Market Staff**
roles. The **Mediator** role (m) represents framework agents who run standard
negotiation activities for example managing the users data, the structure, the
specific parameters of the negotiation protocols, etc. The **Negotiation Dialog
Manager** role (dm) represents framework agents who execute activities that are
specific of a given negotiation protocol, for example accept valid negotiators, tune
negotiation parameters of the dialog, mediate in the negotiation process, expel
negotiators, etc. The **Legal Authority** role (la) represents framework agents
who are in charge of activities for agreement enactments that are executed as a
result of a successful negotiation process.

The ontology structure of the negotiation framework is defined by the fol-
lowing key entities in the market which constitute the core concepts that the
framework recognizes:

An **item** is defined as $i = \langle \tau_i, O, \rho_i \rangle$, where $\tau_i$ is the item type, $O$ is the ontology where the item $i$ is defined, and $\rho_i$ is a set of property values that define
the particular attributes that characterize the item itself. The list $\rho_t$ is domain-dependent, and is defined using the approach presented in [14]. For example the following is a fragment description of a waste ItemX that also specifies its component materials.

Individual: ItemX
Types: GenericProducts, Waste
and (contains some((material-value some Glass) and (material-percentage value 35)))
and (contains some((material-value some Plastic) and (material-percentage value 62)))
and (has price((price-value 40) and (price-currency Euros))) ...

This ontology description is used by the agents in the market to reason about the items and evaluate the profit they can get from the item negotiation.

A deal $\delta = \{\iota\}$ is a sequence of items $\iota$ that can be negotiated.

An agreement is a full instantiated deal among two or more parties. It is defined by a tuple $\alpha = (P, \delta', d, st)$, where $P$ is the set of parties that enact the deal; $\delta'$ is an agreed deal from a previous negotiation process; $d$ is the agreement date; $st$ represents the stage where the agreement currently holds —it can take a value from an enumeration, e.g. pending, executed, cancelled, etc.

## 4.2 The Negotiation Structures

The virtual market provides a set of services that are organized in two different groups: informational, and negotiation (see Tables 1). The informational set of services lists ongoing deals and active agreements, and also provides invitations or call for negotiations. Whereas, the negotiation set allows to create a negotiation dialog, to participate into a negotiation dialog, and once an agreement on a sequence of items has been successfully reached, it allows to settled the agreement according to the given conventions.

In order to assure a regulated execution of the virtual market, special attention is made in order to enact an agreement. First of all, the Mediator checks whether or not the agreement satisfies some formal conditions. If the agreement complies with these, a transfer contract is agreed upon and signed by the customer agents involved, and then the agreement becomes active. Once an agreement is active it may be executed and, consequently, other customer agents may initiate a grievance procedure that may overturn or modify the agreement. Even if there are no grievances that modify a contract, customers may not fulfill the contract properly and there might be some contract reparation actions. If things proceed smoothly, the agreement subsists until maturity.

# 5 Conclusion and Future Works

In this paper a MAS supported Virtual Market approach for reverse production process was presented. The goal of the proposed approach is to facilitate the participation of stakeholders in the activities of green supply chains by means of Service Oriented Manufacturing Systems [7], providing an easy to use interface for the players of a wider reverse logistic ecosystem as a step towards virtual outsourcing of reverse manufacturing processes. The proposed framework define
<table><caption>**Table 1. Negotiation Services**</caption><thead><tr><th>Inform. Services</th><th>Specification</th></tr></thead><tbody><tr><td>QueryNegotiation</td><td><i>informNegotiation(m, p<sub>x</sub>, dialogID, error)</i>, where <i>m</i> is the manager, <i>p<sub>x</sub></i> is the customer, <i>dialogID</i> is the ID of the negotiation dialog, or a <i>nil</i> value when there is an <i>error</i>.</td></tr><tr><td>QueryAgreement</td><td><i>informAgreement(m, p<sub>x</sub>, α<sub>ID</sub>, error)</i>, where <i>α<sub>ID</sub></i> is the ID of the agreement.</td></tr><tr><td>CallForBid</td><td>When the negotiation dialog has a <i>Public</i> type of access, <i>m</i> broadcasts an invitation message to all the participants: <i> inviteAll(m, dialogID, protocol, δ, C)</i>, where <i>dialogID</i> is the negotiation dialog that is receiving players; the negotiation protocol <i>protocol</i> used in that dialog; the set of issues, <i>δ</i>, that is being negotiated; and the set of constraints, <i>C</i>, to participate in are also made public. When the negotiation dialog has a <i>Private</i> type of access the invitation message is <i> invite(m, p<sub>y</sub>, dialogID, protocol, δ, C)</i>, where each candidate <i>p<sub>y</sub></i> ∈ <i>P<sub>dialogID</sub></i>.</td></tr></tbody></table>

<table><thead><tr><th>Negotiation Services</th><th>Specification</th></tr></thead><tbody><tr><td>CreateNegotiationDialog</td><td><i>request(p<sub>x</sub>, m, open, protocol(params), δ, pt, at, C)</i>, where the semantic is as follows; Party agent <i>p<sub>x</sub></i> requests to the Mediator, <i>m</i>, to <i>open</i> a negotiation dialog with the negotiation protocol <i>protocol</i>. The negotiation protocol is instantiated with the set of values for the parameters <i>params</i>. The dialog is created to negotiate about a deal <i>δ</i>. The requesting party, <i>p<sub>x</sub></i>, will participate as <i>pt</i> that can take one of these values: <i>p</i>, that is an observer Party; a Buyer party <i>b</i>; or a Seller party <i>s</i>. <i>at</i> is the access type that can be: <i>Public</i>, any body can be invited; or <i>Private</i>, only Party agents that fulfill the set of constraints <i>C</i> can participate in the negotiation dialog.</td></tr><tr><td>EnterNegotiationDialog</td><td><i>informRole(p<sub>x</sub>, dm, dialogID, pt)</i>, where <i>p<sub>x</sub></i> is the participant; <i>dm</i> is the negotiation dialog manager; <i>dialogID</i> is the dialog ID; <i>pt</i> is the participation type, <i>b</i> or <i>s</i>.</td></tr><tr><td>LeaveNegotiationDialog</td><td><i>informLeave(p<sub>x</sub>, dm, dialogID, pt)</i>.</td></tr><tr><td>Bid</td><td><i>sendBid(p<sub>x</sub>, dm, dialogID, pt, π<sub>x</sub>, δ)</i>, where <i>p<sub>x</sub></i> is the participant; <i>dm</i> is the negotiation dialog manager; <i>dialogID</i> is the negotiation dialog ID; <i>pt</i> is the participation type, <i>b</i> (the bid is a <i>Put</i>) or <i>s</i> (the bid is a <i>Call</i>); <i>π<sub>x</sub></i> is the offered price; <i>δ</i> is the item set that is being negotiated.</td></tr><tr><td>AcceptAgreement</td><td><i>acceptBid(p<sub>x</sub>, dm, dialogID, pt, π<sub>x</sub>, δ)</i>.</td></tr><tr><td>RejectAgreement</td><td><i>rejectBid(p<sub>x</sub>, dm, dialogID, pt, π<sub>x</sub>, δ)</i>.</td></tr></tbody></table>

three different players: Buyer, Seller and Market Staff. The virtual interaction among these roles are implemented by means of web services, that are published in the Cloud and managed by a SOA framework. The services provide the basic functions to play in the market ecosystem such as: Negotiation Protocol Service, Call for Bid Service, Bid Service, etc. The framework is currently under development in order to implement a first prototype to perform validation tests. Moreover, the authors are interested in enhancing the work in order to "close-the-loop" allowing to retro-feed the internal sustainable manufacturing operations scheduling functions (such as [4]), that are running in the manufacturing companies, in order to fine-tune the values of the sustainable variables used by these internal algorithms for better optimizing the sustainable constraints.

**Acknowledgement.** This research is supported by research projects TIN2015-65515-C4-1-R and TIN2016-80856-R from the Spanish government.
References

1. Payman Ahi and Cory Searcy. A comparative literature analysis of definitions for green and sustainable supply chain management. *Journal of Cleaner Production*, 52:329–341, 2013.

2. Cristiano Castelfranchi and Yves Lesperance. *Intelligent Agents VII. Agent Theories Architectures and Languages: 7th International Workshop, ATAL 2000, Boston, MA, USA, July 7-9, 2000. Proceedings*. Springer, July 2003.

3. Frank Ebinger, Maria Goldbach, and Uwe Schneidewind. Greening Supply Chains: A Competence-based Perspective. In Joseph Sarkis, editor, *Greening the Supply Chain*, pages 251–269. Springer London, 2006. DOI: 10.1007/1-84628-299-3_14.

4. Joan Escamilla, Miguel A. Salido, Adriana Giret, and Federico Barber. A meta-heuristic technique for energy-efficiency in job-shop scheduling. *The Knowledge Engineering Review*, 31(5):475–485, November 2016.

5. Behnam Fahimnia, Joseph Sarkis, and Hoda Davarzani. Green supply chain management: A review and bibliometric analysis. *International Journal of Production Economics*, 162:101–114, 2015.

6. Marco Garetti and Marco Taisch. Sustainable manufacturing: trends and research challenges. *Production Planning & Control*, 23(2-3):83–104, February 2012.

7. Adriana Giret, Emilia Garcia, and Vicente Botti. An engineering framework for Service-Oriented Intelligent Manufacturing Systems. *Computers in Industry*, 81:116–127, September 2016.

8. Adriana Giret and Damien Trentesaux. Software engineering methods for Intelligent Manufacturing Systems: A comparative survey. *Industrial Applications of Holonic and Multi-Agent Systems. HoloMAS 2015. LNAI 9266*, pages 11–21, 2015.

9. Adriana Giret, Damien Trentesaux, and Vittal Prabhu. Sustainability in manufacturing operations scheduling: A state of the art review. *Journal of Manufacturing Systems*, 37, Part 1:126–140, October 2015.

10. Adriana Giret, Damien Trentesaux, Miguel A. Salido, Emilia Garcia, and Emmanuel Adam. A holonic multi-agent methodology to design sustainable intelligent manufacturing control systems. *Journal of Cleaner Production*, pages -, 2017.

11. Paulina Golinska, Marek Fertsch, Jorge Marx Gmez, and Joanna Oleskow. The Concept of Closed-loop Supply Chain Integration Through Agents-based System. In *Information Technologies in Environmental Engineering*, Environmental Science and Engineering, pages 189–202. Springer Berlin Heidelberg, 2007.

12. F. S. Hsieh. Scheduling Sustainable Supply Chains Based on Multi-agent Systems and Workflow Models. In *2015 10th International Conference on Intelligent Systems and Knowledge Engineering (ISKE)*, pages 252–259, November 2015.

13. Joseph Sarkis. A strategic decision framework for green supply chain management. *Journal of Cleaner Production*, 11(4):397–409, 2003.

14. A. Sinha and P. Couderc. Using owl ontologies for selective waste sorting and recycling. In *in OWLED, ser. CEUR Workshop Proc.*, vol. 849, 2012.

15. Damien Trentesaux and Adriana Giret. Go-green manufacturing holons: A step towards sustainable manufacturing operations control. *Manuf. Letters*, 5:29–33, 2015.

16. Jr. V. Daniel R. Guide and Luk N. Van Wassenhove. Or forumthe evolution of closed-loop supply chain research. *Operations Research*, 57(1):10–18, 2009.

17. H. Winkler. Closed-loop production systems - a sustainable supply chain approach. {CIRP} *Journal of Manufacturing Science and Technology*, 4(3):243–246, 2011.

18. M. Wooldridge and N. R. Jennings. *Intelligent Agents - Theories, Architectures, and Languages*, volume LNCS 890. Springer-Verlag, Berlin, 1995.