## 6.4 Norms and Policies for Agents on the Web

Timotheus Kampik (University of Umeå, SE), Olivier Boissier (Ecole des Mines – St. Etienne, FR), Sabrina Kirrane (Wirtschaftsuniversität Wien, AT), Adnane Mansour (Mines Saint-Étienne, FR), Julian Padget (University of Bath, GB), Terry Payne (University of Liverpool, GB), Munindar P. Singh (North Carolina State University – Raleigh, US), Valentina Tamma (University of Liverpool, GB), and Antoine Zimmermann (Ecole des Mines – St. Etienne, FR)

**License** © Creative Commons BY 4.0 International license

© Timotheus Kampik, Olivier Boissier, Sabrina Kirrane, Adnane Mansour, Julian Padget, Terry Payne, Munindar P. Singh, Valentina Tamma, and Antoine Zimmermann

The study of policies and norms has a long tradition in the Semantic Web & Linked Data and Multi-Agent Systems communities, with applications ranging from business processes, and legal reasoning, to information systems governance. Although representing and reasoning about norms is crucial in ensuring that autonomous agents act in a manner to satisfy stakeholder requirements, normative concepts have yet to be considered as first-class abstractions in Web-based software systems. The chapter motivates the practical need to apply research on policies and norms to autonomous agents on the Web, and highlights research challenges and opportunities at the intersection of the Semantic Web & Linked Data, Multi-Agent Systems, and Web Architecture & Web of Things communities.

### 6.4.1 Introduction

When envisioning autonomous agents on the Web, it is crucial to consider a *governance* perspective that defines how an agent should act in a given situation, including the normative consequences of its potential actions and how frameworks that govern groups of agents are to be designed, interoperate, and evolve. This perspective is of particular importance for Web-based contexts, where usage may cross different social and legal jurisdictions, and where there is no centralised control over the provenance of the different agents.

This chapter makes two main contributions. First, it motivates norms and policies for *autonomous agents on the Web* using a hypothetical use case (Section 6.4.2) and provides a concise overview of the state of the art on norms and policies for autonomous agents and web-based systems to underline the need for alignment and joint research across communities (Section 6.4.3). Second, the research challenges and opportunities are identified for the Semantic Web & Linked Data, Multi-Agent Systems (MAS), and Web Architecture & Web of Things (WoT) communities (Section 6.4.4).

### 6.4.2 Use Case

Consider a hospital that is responsible for COVID-19 vaccine administration, supported by a range of IT systems and sub-systems. Some of these sub-systems may be considered as *autonomous agents*, which are assigned different roles and work towards fulfilling different goals. One agent manages the access and inventory of different COVID-19 vaccine doses stored within a specialised freezer, via a robotic arm, and dispenses the vials to other lab equipment or healthcare staff when required. The freezer itself is digitally represented by a WoT *Thing* (based on the W3C Web of Things *thing description* [42]), which in turn defines its *properties, actions* (and action handlers), and *events* (and event handlers).

Individual patients are represented by *patient agents* that manage and provide access to the patient's medical data; such access is determined by negotiation with *collector agents*. These agents obtain information about each patient whose agent requests a vaccination
appointment and verify the patient’s eligibility for receiving the vaccine. Furthermore, collector agents have the capacity to solicit patients through dissemination channels on the Web. An *organiser* seeks to compile lists of eligible patients based on both the determined criteria and information provided by the collector agents.

For this scenario, there are a number of “permissions” that an agent acting on behalf of a medical practitioner may have to consider when being asked to withdraw a vaccine vial:

- The agent must *authenticate* itself as legitimately acting on behalf of a medical practitioner.

- The agent should also validate the association the medical practitioner has with a particular *organisation*, i.e., with the healthcare organisation that is in charge of administering the vaccine or any sub-organisation to which the vaccine administration has been delegated.

- Most likely, the medical practitioner has a specific organisational *role* that may or may not authorise their agent to withdraw the vaccine, potentially depending on the current *context*. For example, the practitioner must be assigned to the *vaccination staff* role, and their current context must be set to *on duty*.

The issues listed above merely deal with the “permission” to access a vaccine dose from the freezer. There are also implied “obligations”:

- The medical practitioner that the agent represents must ensure the dose is not wasted, i.e. it must be administered to a valid patient (i.e. one in need of the vaccine that satisfies the current eligibility criteria) before the expiry date<fnref n="33" />.

- This practitioner must respect the priority order for vaccine administration (for example, the old and vulnerable population must be vaccinated first), unless disrespecting the priority order implies wasting the dose.

- This practitioner must not allow friends and relatives to “jump the line” and violate the priority order for vaccine administration (unless a valid exception exists; see below for details).

The practitioner may rely on IT systems (i.e. “agents”, see above) to make decisions; these agents need to consider these obligations as constraints. Further complications may arise if any of the agents attempt to negotiate a relaxation of the obligations, either in anticipation of or after a (potential or factual) violation.

- A patient agent can try to negotiate an exception for a potential obligation violation (through argumentation, for example, where the patient agent believes that the data it has for a patient satisfies the eligibility criteria but the same conclusion is not held by the collector agent). In such cases, the practitioner may have to choose between either administering a vaccine dose where eligibility is uncertain, or letting the dose expire.

- On behalf of the organisation, another agent may respond by granting or denying the request for an exception, or even by updating the organisations norms to generally permit the request.

These issues merely relate to the permissions needed for the obligations implied by opening the fridge. Additional challenges arise when considering the complete sociotechnical system, including electronic health record access [43] and supply chain integration [66]. This underlines the need for systematic and scalable approaches for the governance of autonomous agents on the Web that interact with other agents or with physical devices.

<footnote n="33">³³ In a practical scenario, we would not expect 100% compliance with this constraint, but the number of excess (wasted) doses relative to administered doses does not exceed a specified threshold.</footnote>
6.4.3 Background

6.4.3.1 Web Architecture and the Web of Things

Bringing agents to the Web requires more than simply exploiting Web protocols (HTTP [31]) and data formats (e.g., XML [10], RDF [18]); the communication infrastructure used by agents and MAS should also comply with an architectural style based on well-defined principles, such as Representational State Transfer (REST [30]) as instantiated in the Architecture of the World Wide Web [40]. The Web of Things is an attempt to take advantage of the Web architecture [51] in the context of the *Internet of Things*: “things”, which can be physical objects or virtual entities such as Web services, become resources that can be acted upon or queried via APIs (WoT scripting API [49]). To ensure that such “things” can be used without human intervention, they have to be formally described. To this end, the W3C standardised the *Thing Description* [42] which is a specification that defines how to provide a JSON-LD representation of the affordances (properties or actions) of a “thing” via Web APIs. On top of this, *WoT Discovery* [16] provides a mechanism for automatically discovering “thing” descriptions without hard-coding the location of the descriptions. These standards provide more autonomy for agents that make use of “things” connected to the Internet via Web standards. However, in terms of norms and policies, these standards offer only limited support [62], and are currently described in a set of guidelines targeted at human developers rather than as declarative, machine-readable statements usable by autonomous agents.

6.4.3.2 Semantic Web and Linked Data

[5] originally envisioned a system in which intelligent agents act on behalf of humans. A key component of such a system is a policy language capable of capturing the goals and constraints under which the agents operate. During the early days of the Semantic Web, the development of general policy languages that leverage semantic technologies (such as KAoS [9], Rei [44] and Protune [8]) was an active area of research. Indeed, much of the early work on policies coincided with the active development of rule languages such as SWRL [38] and RIF [46]. The study by [48] provides a detailed survey of the various access control models, standards and policy languages, and the different access control enforcement strategies for RDF. In addition to access control, there has also been work on usage control in the form of licensing [11, 71, 33, 34, 35], and more recently, policy languages have been used as a means to represent regulatory constraints [60, 21].

Some of the early Semantic Web research addressed decentralisation through P2P archi-
tectures, where bespoke protocols were defined to support the decentralised management and
exchange of knowledge and information amongst networks of agents [67]. Although peers
typically adhered to those protocols they could satisfy, many such protocols, in addition to
defining the modalities of interaction, often defined illegal moves and penalties for violations;
thus becoming de-facto policies [50]. Furthermore, Semantic Web approaches often assumed
autonomous and rational agents as first-class citizens; thus there has been a focus more on
how to support knowledge sharing in a centralised or decentralised fashion [26, 41], and on
the type of explicit knowledge needed to support interoperability [53, 70, 24, 36].

From the service perspective, the use of ontologically grounded annotations for web
services, or those services provided by autonomous agents dates back to the early days of
the Semantic Web [39, 28]. The formalisms used to provide an ontological grounding and
description of different kinds of services included both F-Logic [47] as used by WSMO [27],
and DAML-S [19] (based on the DARPA Agent Markup Language) which later evolved as
OWL-S [55]). Other approaches to support service utilisation were developed using OWL,
e.g. the OWL ontology for protocols, OWL-P [22], or federated service discovery mechanisms
such as the semantically annotated version of UDDI [61]. Although such services could be
provided by autonomous agents, their modelling assumed functional service calls (through
the invocation of endpoints with relevant arguments) rather than through the type of speech
acts more familiar to autonomous agents.

6.4.3.3 Multiagent Systems

The study of norms is a long-running and active line of research in the Multi-Agent Systems (MAS) community, as evidenced by Dagstuhl seminars [20, 3], and a handbook on the topic [13]. Normative MAS [6] are realised in a number of ways, which are characterised on the one hand by the reasoning capabilities of the agents themselves, and on the other by whether norms are implicit or explicit with the support of monitoring and enforcement mechanisms. Agent capabilities vis-á-vis norms typically fall into three categories:

(i) *norm unaware*, agents do what they want, but may be regimented by external agencies to enforce norm compliance;

(ii) *norm aware*, agents may choose to comply or not with norms, depending on the alignment of their goals with those norms, the penalties for non-compliance, and the likelihood of enforcement; and

(iii) *value aware*, where agents, in addition to being norm aware, may be able to participate in norm creation and norm revision by reasoning about the value supported (or not) by particular norms.

Implicit norms that reside in the agents themselves are expressed through agent behaviour, and are not otherwise externally discernible. Explicit or referenceable norms may have an abstract representation involving variables, and a grounded (detached) representation in an entity such as a contract [65], institution [58, 25, 23], or organisation [69, 7]. Agents that are norm or value aware should be able to:

(i) recognise the norms;

(ii) decide whether they want to follow them; and

(iii) adapt their behaviour according to the norms, if they decide to do so.

Compliance with normative systems depends on how individual agents can reason and adapt to norms both at design and at run time [63, 68, 52].

6.4.4 Challenges and Opportunities

In this Section we present a range of research challenges and opportunities related to normative
agents on the Web. The four horizontal challenges that characterise the contributions of
norm-based Multi Agent Systems for the Web are presented below (Figure 7) in addition to
a further two orthogonal challenges that need to be tackled to some extent to address the
horizontal ones.

■ Norms and Interaction Protocols

Although the W3C provides Web-based standards for retrieving and querying machine
readable data, they should be extended to cater to usage constraints, such as access
policies, intellectual property rights, and privacy preferences. From a service-provision
perspective, there is a need to develop policy-aware querying and data retrieval protocols,
whereas from an agent interaction perspective, norms should be considered, both in terms
of the agent platform and the environment. This raises a number of questions, including:
<figure><img src="image_11.png" /><figcaption>Figure 7 Overview: research challenges for normative MAS and the Web</figcaption></figure>

(i) How can we design norm-aware dynamic interaction protocols?

(ii) Can existing querying and data retrieval protocols be extended, such that they are policy aware?

(iii) What new protocols are needed to facilitate norm governance?

Existing work [32] largely focuses on request-response interactions and imposes restrictions on computation for scenarios involving the interaction of three or more parties [29, 15]. In particular, traditional approaches entwine the control flow details into the protocol, thereby making it difficult to tease apart the content for which a declarative meaning can be specified. Prior work on specifying protocols in terms of norms (commitments) [12, 54] was stymied by the lack of declarative specification of constraints on messages. Recent approaches describe causality and integrity constraints on messages declaratively [64].

## Distributed normative and open organisations on the web

Organisations, institutions or contracts are useful abstractions to structure norms and make them accessible to agents. Although agents have the choice of joining such structures, they may be controlled and be subject to conditions that regulate their admission (and exit), as well as there being an expectation to comply with the organisation's norms. Due to the scale of the Web, several of such permanent, ephemeral or evolving structures may exist, which raises several challenges. First, an agent needs to be able to discover and reason about such organisations and the corresponding norms. The use of ontologies has already been used to facilitate the discovery of services [61] and thus their use in representing organisations is promising. A second challenge is the distributed management of such structures [7], i.e., to monitor and enforce norm compliance. Finally, since the Web is an evolving sociotechnical system, agents require proper abstractions and mechanisms to build and adapt organisations [14].

## Internal state of an agent and norms

Several languages express and support automated reasoning about agent internals, such as beliefs, desires, and intentions. However, challenges exist when it comes to reasoning not only with respect to goals, but also privacy preferences, regulatory constraints, and norms. From a norms perspective, there are several open questions in terms of both specification and enforcement, including:

(i) How can we ensure consistent representation of, and adherence to norms?

(ii) How should a governance architecture be designed in which “rational” agents are incentivised to comply with norms?
(iii) Can a norm violation be excused, based on explanation or beliefs?

(iv) Could transparency facilitate the persuasiveness of the explanation or argument?

(v) How do we ensure an agent is aware of the implications of violating a norm?

(vi) How do we cater for non “rational” (e.g. poorly designed) agents?

## Governing norm emergence

Approaches to the governance of norm emergence are dependent on the capabilities of the agents in a MAS, bearing in mind that population properties may not be homogeneous. We identify three means to govern norm emergence [57] which we classify by adapting the oversight terminology put forward by the [37, §B.II.1.1]:

(i) an external agency observes the behaviour of the population to identify patterns of behaviour and revise the norms to optimise for the system goals (external agent on-the-loop)

(ii) agents propose norm revisions to an external agency, which then implements them subject to an assessment of how those revisions contribute towards system goals (external agent in-command)

(iii) agents propose norm revisions and use an internal decision-making mechanism to establish which changes will be implemented (internal agents in-the-loop).

In an alternative, decentralized, approach, the norms emerge through the interactions of the agents without an external agency being involved [1]. The question of emergence stems from its role in the norm life cycle. Norm emergence is widely accepted to have happened when a predetermined percentage of the population observes the norm or chooses the same action, and most experimental studies put that percentage at approximately 90%.

## Heterogeneous and inconsistent norms and beliefs

In heterogeneous information system landscapes, one cannot assume that norms and policies are managed and implemented based on a single global specification synchronised across different subsystems, as agents may hold inconsistent beliefs about the norms and policies that apply in any given context. Thus, reaching (partial) agreements in the face of conflicting beliefs regarding norms and policies is an important challenge that needs to be tackled to enable normative distributed MAS on the Web, using, for example, long-running lines of research on agreement technologies [59] and formal argumentation approaches [4].

## Technological integration

To facilitate the practical applicability of research on norms and policies for autonomous agents on the web, it is crucial to build bridges between the technology ecosystems of the different communities. In the Web of Things community, engineering-oriented work is conducted in a highly practice-oriented manner, in close alignment with industry practitioners as well as standardisation bodies such as the W3C. An example of a practice-oriented work is the line of W3C IoT standards that features an abstract architecture description [51] and an interface specification (W3C WoT Scripting API) [49], supported by a JavaScript reference implementation<fnref n="34" />.

Research on engineering autonomous agents and MAS has primarily gained traction within the academic community [56], and standardisation attempts such as FIPA<fnref n="35" /> have lacked significant adoption. Adjusting agent-oriented programming and software engineering approaches to better serve the Semantic Web & Linked Data and Web Architecture &

<footnote n="34">³⁴ https://github.com/eclipse/thingweb.node-wot</footnote>

<footnote n="35">³⁵ http://www.fipa.org/.</footnote>
WoT communities are an opportunity for the MAS community to move their engineering research closer to practice. This can be achieved by:

(i) building general-purpose interfaces to powerful multi-agent programming frameworks like JaCaMo [7] (starting points are, e.g., hypermedia-mas [17], jacamo-rest [2]), and/or

(ii) implementing minimally viable agent-oriented abstractions for mainstream programming language ecosystems (e.g., JS-son [45]).

### 6.4.5 Conclusion

In this chapter we discussed the relevance of norms and policies for governing complex sociotechnical multi-agent systems on the Web. The key challenge – the conceptual and technological integration of normative concepts with WoT abstractions and systematic evaluation of the practical usefulness of the integration results – is aligned with the general challenge for autonomous agents on the Web to transfer the rich theoretical achievements of the broader MAS community to the practical and engineering-oriented WoT community, and to facilitate real-world applications at scale.

#### References

1. N. Ajmeri, H. Guo, P. K. Murukannaiah, and M. P. Singh. Robust norm emergence by revealing and reasoning about context: Socially intelligent agents for enhancing privacy. In *Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI)*, pages 28–34, Stockholm, July 2018. IJCAI.

2. C. J. Amaral, J. F. Hübner, and T. Kampik. Towards Jacamo-rest: A Resource-Oriented Abstraction for Managing Multi-Agent Systems. In G. Vaz Alves, G. Guiménez Lug, A. Pinz Borgeso, and C. E. Pantoja, editors, *Proceedings of the 14th Workshop-School on Agents, Environments, and Applications*, pages 140–151, 2020.

3. G. Andrighetto, G. Governatori, P. Noriega, and L. van der Torre. Normative multi-agent systems (Dagstuhl Seminar 12111). *Dagstuhl Reports*, 2(3), 2012.

4. P. Baroni, D. M. Gabbay, M. Giacomini, and L. van der Torre. *Handbook of formal argumentation*. College Publications, 2018.

5. T. Berners-Lee, J. Hendler, O. Lassila, et al. The semantic web. *Scientific American*, 2001.

6. G. Boella, L. Van Der Torre, and H. Verhagen. Introduction to normative multiagent systems. *Computational & Mathematical Organization Theory*, 12(2-3):71–79, 2006.

7. O. Boissier, R. H. Bordini, J. Hübner, and A. Ricci. *Multi-agent oriented programming: programming multi-agent systems using JaCaMo*. MIT Press, 2020.

8. P. A. Bonatti and D. Olmedilla. Rule-based policy representation and reasoning for the semantic web. In *Proceedings of the Third International Summer School Conference on Reasoning Web*. Springer-Verlag, 2007.

9. J. M. Bradshaw, editor. *Software agents*. MIT press, 1997.

10. T. Bray, J. Paoli, C. M. Sperberg-McQueen, E. Maler, and F. Yergeau. Extensible Markup Language (XML) 1.0 (Fifth Edition), W3C Recommendation 26 November 2008. W3C Recommendation, World Wide Web Consortium (W3C), Nov. 26 2008. URL https://www.w3.org/TR/2008/REC-xml-20081126/.

11. E. Cabrio, A. Palmero Aprosio, and S. Villata. These Are Your Rights. In *The Semantic Web: Trends and Challenges*. Springer International Publishing, 2014.

12. A. Chopra and M. P. Singh. Nonmonotonic commitment machines. In F. Dignum, editor, *Advances in Agent Communication: Proceedings of the International Workshop on Agent Communication Languages (ACL 2003)*, number 2922 in Lecture Notes in Artificial Intelligence, pages 183–200, Melbourne, 2004. Springer.
13 A. Chopra, L. van der Torre, H. Verhagen, and S. Villata. *Handbook of Normative Multiagent Systems*. College Publications, 2018. ISBN 9781848902855.

14 A. K. Chopra and M. P. Singh. From social machines to social protocols: Software engineering foundations for sociotechnical systems. In *Proceedings of the 25th International World Wide Web Conference*, pages 903–914, Montréal, Apr. 2016. ACM.

15 A. K. Chopra, S. H. Christie V, and M. P. Singh. An evaluation of communication protocol languages for engineering multiagent systems. *Journal of Artificial Intelligence Research* (JAIR), 69:1351–1393, Dec. 2020.

16 A. Cimmino, M. McCool, F. Tavakolizadeh, and K. Toumura. Web of Things (WoT) Discovery, W3C First Public Working Draft 24 November 2020. W3C First Public Working Draft, World Wide Web Consortium (W3C), Nov. 24 2020. URL http://www.w3.org/TR/2020/WD-wot-discovery-20201124/.

17 A. Ciortea, O. Boissier, and A. Ricci. Engineering world-wide multi-agent systems with hypermedia. In D. Weyns, V. Mascardi, and A. Ricci, editors, *Engineering Multi-Agent Systems – 6th International Workshop, EMAS 2018, Stockholm, Sweden, July 14-15, 2018, Revised Selected Papers*, volume 11375 of *Lecture Notes in Computer Science*, pages 285–301. Springer, 2018. doi: 10.1007/978-3-030-25693-7\_15. URL https://doi.org/10.1007/978-3-030-25693-7_15.

18 R. Cyganiak, D. Wood, and M. Lanthaler. RDF 1.1 Concepts and Abstract Syntax, W3C Recommendation 25 February 2014. W3C Recommendation, World Wide Web Consortium (W3C), Feb. 25 2014. URL http://www.w3.org/TR/2014/REC-rdf11-concepts-20140225/.

19 DAML-S Coalition: A. Ankolekar, M. Burstein, J. Hobbs, O. Lassila, D. McDermott, D. Martin, S. McIlraith, S. Narayanan, M. Paolucci, T. Payne, and K. Sycara. DAML-S: Web Service Description for the Semantic Web. In *First International Semantic Web Conference (ISWC) Proceedings*, pages 348–363, 2002.

20 M. Dastani, J. Dix, H. Verhagen, and S. Villata. Normative multi-agent systems (dagstuhl seminar 18171). *Dagstuhl Reports*, 8(4):72–103, 2018.

21 M. De Vos, S. Kirrane, J. Padget, and K. Satoh. ODRL policy modelling and compliance checking. In P. Fodor, M. Montali, D. Calvanese, and D. Roman, editors, *Rules and Reasoning*, pages 36–51, Cham, 2019. Springer International Publishing. ISBN 978-3-030-31095-0.

22 N. Desai, A. U. Mallya, A. K. Chopra, and M. P. Singh. OWL-P: A methodology for business process development. In *Agent-Oriented Information Systems III, 7th International Bi-Conference Workshop, AOIS2005, Utrecht, Netherlands, July 26, 2005, and Klagenfurt, Austria, October 27, 2005, Revised Selected Papers*, number 3529 in Lecture Notes in Computer Science, pages 79–94, Berlin, 2006. Springer.

23 M. d'Inverno, M. Luck, P. Noriega, J. A. Rodríguez-Aguilar, and C. Sierra. Communicating open systems. *Artif. Intell.*, 186:38–94, 2012. doi: 10.1016/j.artint.2012.03.004.

24 P. Doran, V. A. M. Tamma, T. R. Payne, and I. Palmisano. Dynamic selection of ontological alignments: A space reduction mechanism. In C. Boutilier, editor, *IJCAI 2009, Proceedings of the 21st International Joint Conference on Artificial Intelligence, Pasadena, California, USA, July 11-17, 2009*, pages 2028–2033, 2009.

25 M. Esteva, J. Padget, and C. Sierra. Formalizing a language for institutions and norms. In J.-J. C. Meyer and M. Tambe, editors, *Intelligent Agents VIII*, pages 348–366, Berlin, Heidelberg, 2002. Springer Berlin Heidelberg. ISBN 978-3-540-45448-9.

26 J. Euzenat and P. Shvaiko. *Ontology Matching, Second Edition*. Springer, 2013. ISBN 978-3-642-38720-3.
27 C. Feier, A. Polleres, R. Dumitru, J. Domingue, M. Stollberg, and D. Fensel. Towards intelligent web services: the web service modeling ontology (WSMO). In *2005 International Conference on Intelligent Computing (ICIC'05)*, 2005.

28 D. Fensel and E. Motta. Structured development of problem solving methods. *IEEE Trans. Knowl. Data Eng.*, 13(6):913–932, 2001. doi: 10.1109/69.971187.

29 A. Ferrando, M. Winikoff, S. Cranefield, F. Dignum, and V. Mascardi. On the enactability of agent interaction protocols: Toward a unified approach. *CoRR*, abs/1902.01131v4:1–13, Feb. 2019.

30 R. T. Fielding. *Representational State Transfer (REST)*. PhD thesis, University of California, Irvine, 2000. Chapter 5 of Roy Fielding's thesis.

31 R. T. Fielding and J. Reschke. Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content. Technical report, IETF, June 2014. URL http://tools.ietf.org/html/rfc7231.

32 FIPA. FIPA interaction protocol specifications, 2003. URL http://www.fipa.org/repository/ips.html. FIPA: The Foundation for Intelligent Physical Agents.

33 G. Governatori, L. Ho-Pun, A. Rotolo, S. Villata, and F. Gandon. Heuristics for Licenses Composition. *Frontiers in Artificial Intelligence and Applications*, 2013a.

34 G. Governatori, A. Rotolo, S. Villata, and F. Gandon. One license to compose them all. In *International Semantic Web Conference*, 2013b.

35 G. Governatori, H.-P. Lam, A. Rotolo, S. Villata, G. A. Atemezing, and F. L. Gandon. Live: a tool for checking licenses compatibility between vocabularies and data. In *International Semantic Web Conference*, 2014.

36 B. C. Grau and B. Motik. Reasoning over ontologies with hidden content: The import-by-query approach. *J. Artif. Int. Res.*, 45(1):197–255, Sept. 2012. ISSN 1076-9757.

37 High-Level Expert Group on Artificial Intelligence (AI HLEG). Ethics Guidelines for Trustworthy AI, 2019. URL https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai.

38 I. Horrocks, P. F. Patel-Schneider, H. Boley, S. Tabet, B. Grosof, and M. Dean. Swrl: A semantic web rule language combining owl and ruleml. W3C member submission, World Wide Web Consortium, 2004. URL http://www.w3.org/Submission/SWRL.

39 M. N. Huhns and M. P. Singh. Ontologies for agents. *IEEE Internet Computing (IC)*, 1(6): 81–83, Nov. 1997. Instance of the column *Agents on the Web*.

40 I. Jacobs and N. Walsh. Architecture of the World Wide Web, Volume One, W3C Recommendation 15 December 2004. W3C Recommendation, World Wide Web Consortium (W3C), Dec. 15 2004. URL http://www.w3.org/TR/2004/REC-webarch-20041215/.

41 E. Jimenez-Ruiz, T. R. Payne, A. Solimando, and V. Tamma. Limiting logical violations in ontology alignment through negotiation. In *Proceedings of the 15th International Conference on Principles of Knowledge Representation and Reasoning (KR)*. AAAI Press, April 2016.

42 S. Kaebisch, T. Kamiya, M. McCool, V. Charpenay, and M. Kovatsch. Web of Things (WoT) Thing Description, W3C Recommendation 9 April 2020. W3C Recommendation, World Wide Web Consortium (W3C), Apr. 09 2020. URL https://www.w3.org/TR/2020/REC-wot-thing-description-20200409/.

43 O. Kafali, N. Ajmeri, and M. P. Singh. DESEN: Specification of sociotechnical systems via patterns of regulation and control. *ACM Transactions on Software Engineering and Methodology (TOSEM)*, 29(1), Feb. 2020. ISSN 1049-331X.

44 L. Kagal and T. Finin. A policy language for a pervasive computing environment. In *Proceedings POLICY 2003. IEEE 4th International Workshop on Policies for Distributed Systems and Networks*, 2003.
45 T. Kampik and J. C. Nieves. Js-son – a lean, extensible javascript agent programming library. In L. A. Dennis, R. H. Bordini, and Y. Lespérance, editors, *Engineering Multi-Agent Systems*, pages 215–234, Cham, 2020. Springer International Publishing. ISBN 978-3-030-51417-4.

46 M. Kifer and H. Boley. Rif overview (second edition). W3c working group note 5 february 2013, World Wide Web Consortium, 2013. URL https://www.w3.org/TR/rif-overview/.

47 M. Kifer, G. Lausen, and J. Wu. Logical foundations of object-oriented and frame-based languages. *J. ACM*, 42(4):741–843, July 1995. ISSN 0004-5411.

48 S. Kirrane, A. Mileo, and S. Decker. Access control and the resource description framework: A survey. *Semantic Web*, 2017.

49 Z. Kis, D. Peinter, C. Aguzzi, J. Hund, and K. Nimura. Web of Things (WoT) Scripting API, W3C Working Group Note 24 November 2020. W3C Working Group Note, World Wide Web Consortium (W3C), Nov. 24 2020. URL https://www.w3.org/TR/2020/NOTE-wot-scripting-api-20201124/.

50 M. Koubarakis. Multi-agent systems and peer-to-peer computing: Methods, systems, and challenges. In M. Klusch, A. Omicini, S. Ossowski, and H. Laamanen, editors, *Cooperative Information Agents VII*, pages 46–61, Berlin, Heidelberg, 2003. Springer Berlin Heidelberg. ISBN 978-3-540-45217-1.

51 M. Kovatsch, R. Matsukura, M. Lagally, T. Kawaguchi, K. Toumura, and K. Kajimoto. Web of Things (WoT) Architecture, W3C Recommendation 9 April 2020. W3C Recommendation, World Wide Web Consortium (W3C), Apr. 9 2020. URL https://www.w3.org/TR/2020/REC-wot-architecture-20200409/.

52 J. Lee, J. Padget, B. Logan, D. Dybalova, and N. Alechina. N-Jason: Run-time norm compliance in AgentSpeak(L). In F. Dalpiaz, J. Dix, and M. B. van Riemsdijk, editors, *Engineering Multi-Agent Systems – Second International Workshop, EMAS 2014, Paris, France, May 5-6, 2014, Revised Selected Papers*, volume 8758 of *Lecture Notes in Computer Science*, pages 367–387. Springer, 2014. ISBN 978-3-319-14483-2.

53 S. Luke, L. Spector, D. Rager, and J. Hendler. Ontology-based web agents. In *Proceedings of the First International Conference on Autonomous Agents*, AGENTS ’97, page 59–66, New York, NY, USA, 1997. Association for Computing Machinery. ISBN 0897918770.

54 A. U. Mallya and M. P. Singh. An algebra for commitment protocols. *Journal of Autonomous Agents and Multi-Agent Systems (JAAMAS)*, 14(2):143–163, Apr. 2007.

55 D. Martin, M. Paolucci, S. McIlraith, M. Burstein, D. McDermott, D. McGuinness, B. Parsia, T. Payne, M. Sabou, M. Solanki, N. Srinivasan, and K. Sycara. Bringing semantics to web services: The owl-s approach. In J. Cardoso and A. Sheth, editors, *Semantic Web Services and Web Process Composition*, pages 26–42, Berlin, Heidelberg, 2005. Springer Berlin Heidelberg. ISBN 978-3-540-30581-1.

56 V. Mascardi, D. Weyns, A. Ricci, C. B. Earle, A. Casals, M. Challenger, A. Chopra, A. Ciortea, L. A. Dennis, A. F. Díaz, A. El Fallah-Seghrouchni, A. Ferrando, L.-r. Fredlund, E. Giunchiglia, Z. Guessoum, A. Günay, K. Hindriks, C. A. Iglesias, B. Logan, T. Kampik, G. Kardas, V. J. Koeman, J. B. Larsen, S. Mayer, T. Méndez, J. C. Nieves, V. Seidita, B. T. Teze, L. Z. Varga, and M. Winikoff. Engineering multi-agent systems: State of affairs and the road ahead. *SIGSOFT Softw. Eng. Notes*, 44(1):18–28, Mar. 2019. ISSN 0163-5948.

57 A. Morris-Martin, M. De Vos, and J. Padget. Norm emergence in multiagent systems: A viewpoint paper. *Autonomous Agents and Multi-Agent Systems*, 33(6):706–749, 2019.

58 P. Noriega. *Agent Mediated Auctions: The Fishmarket Metaphor*. PhD thesis, Universitat Autònoma de Barcelona, 1997. URL https://www.iia.csic.es/research/thesis-details?pastphd_id=85.

59 S. Ossowski. *Agreement Technologies*. Springer Publishing Company, Incorporated, 2012. ISBN 9400755821.
60 M. Palmirani, G. Governatori, A. Rotolo, S. Tabet, H. Boley, and A. Paschke. Legalruleml:Xml-based rules and norms. In *International Workshop on Rules and Rule Markup Languages for the Semantic Web*, pages 298–312. Springer, 2011.

61 M. Paolucci, T. Kawamura, T. R. Payne, and K. Sycara. Semantic matching of web services capabilities. In I. Horrocks and J. Hendler, editors, *The Semantic Web – ISWC 2002*, pages 333–347, Berlin, Heidelberg, 2002. Springer Berlin Heidelberg. ISBN 978-3-540-48005-1.

62 E. Reshetova and M. McCool. Web of Things (WoT) Security and Privacy Guidelines, W3C Working Group Note 6 November 2019. W3C Working Group Note, World Wide Web Consortium (W3C), Nov. 6 2019. URL https://www.w3.org/TR/2019/NOTE-wot-security-20191106/.

63 M. Riemsdijk, L. Dennis, M. Fisher, and K. Hindriks. Agent reasoning for norm compliance: A semantic approach. In *12th International Conference on Autonomous Agents and Multiagent Systems 2013, AAMAS 2013*, volume 1, pages 499–506, 05 2013.

64 M. P. Singh. Information-driven interaction-oriented programming: BSPL, the Blindingly Simple Protocol Language. In *Proceedings of the 10th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS)*, pages 491–498, Taipei, May 2011. IFAAMAS.

65 M. P. Singh. Norms as a basis for governing sociotechnical systems. *ACM Transactions on Intelligent Systems and Technology (TIST)*, 5(1), Dec. 2013. ISSN 2157-6904.

66 M. P. Singh, A. K. Chopra, and N. Desai. Commitment-based service-oriented architecture. *Computer*, 42(11):72–79, 2009. doi: 10.1109/MC.2009.347.

67 S. Staab and H. Stuckenschmidt, editors. *Semantic Web and Peer-to-Peer – Decentralized Management and Exchange of Knowledge and Information*. Springer, 2006. ISBN 978-3-540-28346-1.

68 P. R. Telang, M. P. Singh, and N. Yorke-Smith. A coupled operational semantics for goals and commitments. *Journal of Artificial Intelligence Research (JAIR)*, 65:31–85, May 2019.

69 Y. B. Udupi and M. P. Singh. Contract enactment in virtual organizations: A commitment-based approach. In *Proceedings of the 21st National Conference on Artificial Intelligence (AAAI)*, pages 722–727, Boston, July 2006. AAAI Press.

70 C. D. Vescovo, M. Horridge, B. Parsia, U. Sattler, T. Schneider, and H. Zhao. Modular structures and atomic decomposition in ontologies. *J. Artif. Intell. Res.*, 69:963–1021, 2020.

71 S. Villata and F. Gandon. Licenses compatibility and composition in the web of data. In *Third International Workshop on Consuming Linked Data (COLD2012)*, 2012.
<figure><img src="image_12.png" /></figure>

Participants

Cleber Jorge Amaral
Universidade Federal de Santa Catarina, BR

Mike Amundsen
MuleSoft LLC -
San Francisco, US

Olivier Boissier
Ecole des Mines - St. Etienne, FR

Rafael H. Bordini
PUCRS - Porto Alegre, BR

Arne Bröring
Siemens AG - München, DE

Samuele Burattini
University of Bologna, IT

Jean-Paul Calbimonte
HES-SO Valais - Sierre, CH

Victor Charpenay
Mines Saint-Étienne, FR

Samuel Christie
Lancaster University, GB

Andrei Ciortea
Universität St. Gallen, CH

Olivier Corby
INRIA - Sophia Antipolis, FR

Angelo Croatti
Università di Bologna, IT

Jérôme Euzenat
INRIA - Grenoble, FR

Matthias Farnbauer-Schmidt
Universität Erlangen-Nürnberg, DE

Fabien Gandon
INRIA - Sophia Antipolis, FR

Dominique Guinard
EVRYTNG - London, GB

Andreas Harth
Fraunhofer IIS - Nürnberg, DE

James A. Hendler
Rensselaer Polytechnic Institute -
Troy, US

Jomi Fred Hübner
Federal University of Santa Catarina, BR

Tobias Käfer
KIT - Karlsruher Institut für Technologie, DE

Lalana Kagal
MIT - Cambridge, US

Timotheus Kampik
University of Umeå, SE

Sabrina Kirrane
Wirtschaftsuniversität Wien, AT

Ege Korkan
TU München, DE

Matthias Kovatsch
Huawei Technologies -
München, DE

Maxime Lefrançois
Mines Saint-Étienne, FR

Adnane Mansour
Mines Saint-Étienne, FR

Simon Mayer
Universität St. Gallen, CH

Mahda Noura
TU Chemnitz, DE

Julian Padget
University of Bath, GB

Cesare Pautasso
University of Lugano, CH

Terry Payne
University of Liverpool, GB

María Poveda-Villalón
Technical University of Madrid, ES

Dave Raggett
W3C - United Kingdom, GB

Alessandro Ricci
Università di Bologna, IT

Sebastian Schmid
Universität Erlangen-Nürnberg, DE

Daniel Schraudner
Universität Erlangen-Nürnberg, DE

Ronny Seiger
Universität St. Gallen, CH

Munindar P. Singh
North Carolina State University -
Raleigh, US

Ruben Taelman
Ghent University, BE

Valentina Tamma
University of Liverpool, GB

Danai Vachtsevanou
Universität St. Gallen, CH

Ruben Verborgh
Ghent University, BE

Erik Wilde
Axway - Zürich, CH

Antoine Zimmermann
Ecole des Mines -
St. Etienne, FR

<figure><img src="image_13.png" /></figure>
<figure><img src="image_14.png" /></figure>

<figure><img src="image_15.png" /></figure>