<figure><img src="image_1.png" /></figure>

HAL
open science

# Combination between Multi-Agent System and Tangigets for DUI Design on several Tabletops

Yoann Lebrun, Sophie Lepreux, Christophe Kolski, René Mandiau

► To cite this version:

Yoann Lebrun, Sophie Lepreux, Christophe Kolski, René Mandiau. Combination between Multi-Agent System and Tangigets for DUI Design on several Tabletops. DUI 2013: 3rd Workshop on Distributed User Interfaces: Models, Methods and Tools. In conjunction with ACM EICS 2013, Jun 2013, Londres, United Kingdom. pp.54-57. hal-03408326

HAL Id: hal-03408326

https://uphf.hal.science/hal-03408326

Submitted on 10 Sep 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Combination between Multi-Agent System and Tangigets for DUI Design on several Tabletops

**Yoann Lebrun**

UVHC,
LAMIH-UMR CNRS 8201,
F-59313 Valenciennes, France.
CCI Grand Hainaut - PRL
360 rue Marc Lefrancq,
F-59300 Valenciennes, France
lebrun.yoann@gmail.com

**Sophie Lepreux**

UVHC,
LAMIH-UMR CNRS 8201,
F-59313 Valenciennes, France,
sophie.lepreux@univ-valenciennes.fr

**Christophe Kolski**

UVHC,
LAMIH-UMR CNRS 8201,
F-59313 Valenciennes, France,
christophe.kolski@univ-valenciennes.fr

**René Mandiau**

UVHC,
LAMIH-UMR CNRS 8201, F-59313 Valenciennes, France,
rene.mandiau@univ-valenciennes.fr

## ABSTRACT

The paper concerns the design of DUI composed with interactive tabletops allowing users to manipulate virtual and tangible objects around these several surfaces. We propose a model dedicated to the management of distributed interactive surfaces with Multi-Agent platforms. A case study illustrates the approach used: this case study implies a traffic management simulator distributed on two *TangiSense* tabletops equipped with RFID technology.

## Author Keywords

Distributed User Interfaces, Multi-Agent System (MAS),
Tabletop, Tangible Object, Tangigets, RFID.

## ACM Classification Keywords

H.5.m. Information interfaces and presentation (e.g., HCI):
Miscellaneous.

## General Terms

Design; Human Factors.

## INTRODUCTION

Through the use of connected tabletops, it is possible to interact with several people located in remote places [11]. Indeed, the emergence of computer networks (increase data transfer, wireless network, etc.) enables the design of new distributed applications on these interactive supports.

In consequence adapted design methods and models have to be proposed. This paper describes a generic agent-based model of a Multi-Agent System (MAS) for DUI design

dedicated to connected tabletops. In this model, we suppose that generic tangible objects, called Tangigets [2, 12] are the interaction supports, manipulated by the users. This model aims at taking into consideration the located and remote interactions. The located interactions do not have incidence on connected supports whereas the remote interactions allow modify the display on the remote connected platforms. The criteria used in the model to adapt the HCI are the number of connected platforms. Furthermore the MAS model allows connecting the remote platforms.

A case study is presented; it illustrates an innovative simulator allowing the distributed management of road traffic. This simulator is distributed on two *TangiSense* interactive tabletops equipped with RFID technology. These tabletops have the particularly to be tangible and not multitouch. The illustrations show these connected tabletops and the advantages of the proposed model in order to facilitate the interactions realized by two types of *Tangigets*.

## STATE OF THE ART

### Design methods for DUI domain

In the HCI field, the idea of UI plasticity was adapted to distributed interfaces [1, 4, 6, 9, 14]. One suppose that the users want to be able to move from one platform to another one without loss of coherence in the use of the application, without loss of data (e.g. they want to continue to deal with their e-mails or surf on Internet while being mobile). The Cameleon model became a framework for the modeling /transformations of the HCI [3]. Nevertheless, this model concerns only the user interfaces.

Otherwise, the technological progress implies the multiplication of interacting devices such as smartphones (Ubiquitous Computing [17]), tablets, tabletops, etc.

DUI 2013: 3rd Workshop on Distributed User Interfaces: Models, Methods and Tools. In conjunction with ACM EICS 2013
June 24th, 2013. London, UK.
ISBN-10: 84-616-4792-0
ISBN-13: 978-84-616-4792-7
Among researches, the aim is to allow the users interact more naturally, which includes the tangible interaction.

In [5], and also in a recent Special Issue on "Distributed User Interfaces" of IJHCI (vol. 28, issue 11, 2012) a set of approaches for DUI design are envisaged.

## Design methods for TUI domain

TUI (Tangible User Interface) has been proven effectively to allow users to control and comprehend digital information as it supports direct manipulation and the utilization of spatial knowledge [7]. Since, numerous researches are focused on this type of interaction (see for instance the recent editions of the TEI and ITS conferences).

For example, Ullmer [16] introduced the concept of token+constraint systems, which considers tokens as physical objects representing digital information or operations, and constraints confining regions in which tokens are placed and manipulated. The TAC paradigm uses Ullmer's token+constraint approach as its basis. It then extends the concept of constraint, stressing that a TUI may be described as a set of relationships between a token, a set of constraints, and a variable [15].

The tangible objects have to be clearly defined in order to have a sense to the users. The user has to understand the functionality associated to each tangible object (in the GUI, the problematic was the same which led to the solution of widget concept). With the same logic, we proposed the Tangiget concept [2, 12].

Zoil [18] aims at proposing design principles dedicated to post-wimp distributed interfaces. But this article shows also that it is difficult to maintain the usability in these systems.

Provide intelligence in new post-WIMP and distributed systems in order to allow designing applications more complex than office software but also to ensure a use comfort, lead us to combine MAS with design principles of DUI.

## PROPOSITION

This section presents our model for managing entities on several tabletops. These entities may be virtual and / or tangible (depending on the interactive surfaces concerned) and are evolving either locally (e.g. on only one interactive surface) or in a distributed way (e.g. on a network of interactive surfaces).

The proposed interaction model is based on Multi-Agent System (MAS) concepts. These concepts are used to design and implement distributed applications for interactive tabletops. They take into consideration the heterogeneousness of the entities, and also the distribution of the information between entities. In addition, it considers the interaction between several interactive surfaces.

In Figure 1, we present a model for the design of applications distributed on several interactive surfaces. For

ease of reading, we represent only two interactive surfaces but the number of surfaces is not limited. To define the relationships between agents evolving on a MAS platform, we associate one MAS platform by interactive surface.

The MAS platform is FIPA-compliant (cf. http://www.fipa.org/) to follow criteria of compatibility. The MAS platform is composed of a set of functionalities like the agents management (e.g. to create, to delete, to research, etc., an agent on the platform) and messages protocols (MSG) to exchange information between local agents and between agents in a remote platform (e.g. to inform, to request, to confirm, etc.).

To represent the entities evolving on interactive surface, we distinguish and associate three types of agents: virtual, Tangiget and clone agent. **The Virtual Agent (VA)** is associated to virtual objects with a graphical representation on the interactive support. **The Tangiget Agent (TA)** is connected to a Tangiget object. The agent association to Tangiget object depends on the object proprieties. We distinguish local and distributed properties. A Tangiget object with local property can be manipulated by users and have local actions in only one interactive surface. A Tangiget object with distributed property can be manipulated by users to interact remotely with other tabletops. **The clone agent (CA)** is linked to a Tangiget agent with distributed property. The number of clone agents for one Tangiget agent depends mainly on the number of connected interactive tables and the concerned distributed application.

In Figure 1, we show that the number of agents is not limited and that an agent is necessarily associated to a platform (we consider also in this figure two platforms; but the number of platforms is not restricted). For example, $VA^2$ is associated to the virtual agent number 1 evolving on the interactive surface number 2. We note **i** and **k**, the integer variables representing the Tangiget agents (with local properties) matched to interactive surface 1 and 2. We note **j** and **l**, the integer variables representing the Tangiget agents (with distributed properties) and the clone agents associated with these agents. For example on the interactive surface 1, when one of these agents is created ($TA_j^2$), it is automatically cloned to all other connected interactive surfaces. In the case of two connected platforms, we have only a single clone created on the interactive surface 2 ($CA_j^2$). If an interactive surface is not initially connected, the MAS platform analyzes dependencies to create all the useful clone agents during the initialization.

## CASE STUDY

In this section, we show a case study related to the distributed simulation of road traffic management to implement the model presented in the previous section.

Initially, the tool [8] has been designed on one interactive *TangiSense* tabletop. This tabletop exploits the RFID technology for the detection of objects equipped with tags.
This tabletop is designed and produced by the RFIidées Company. We have extended our approach to a distributed approach for two interactive *TangiSense* tabletops (these tabletops are connected on a network). The simulator aims to test different hypotheses concerning the traffic [10]: waiting time reduction in crossroads, crisis situations management, infrastructure changes, and so on.

<figure><img src="image_3.png" /><figcaption>**Figure 1.** Model dedicated to the management of distributed interactive surfaces with MAS Platforms (legend: TA = Tangiget Agent; VA = Virtual Agent; CA = Clone Agent, MSG = messages protocols)</figcaption></figure>

The road traffic simulator is implemented with the JADE multi-agent platform that respects the specifications provided by the FIPA standard.

The Figure 2 illustrates two examples of implementation of Tangigets in the road traffic simulation. Virtual agents are represented on each tabletop by vehicles moving randomly or according to a set of goals on the entire road network. These agents have behaviors that allow or not for example to respect the Highway Code. Tangigets (with local properties) are used for manipulating the map. These objects are manipulated by the users and they interact with other virtual objects. These objects are equipped with RFID tags to be able to modify the network structure.

For example, to move the map, to view information about the name or the speed of the road, to zoom in or out and change the scale of the map, they may use different Tangigets. We note that these Tangigets do not affect the

other tables. This kind of Tangiget allows the users to have an independent view of the map. For example, as showed in Figure 2(a), a user at the tabletop 2 zooms by turning a Tangiget and changes locally the scale of the map.

Figure 2(b) shows another example: a Tangiget with distributed properties is used on the tabletop 1. These properties allow it to be cloned using agent located in another interactive surface. The advantage of this Tangiget is to coordinate display tabletops to work together on the same area. When this object is put down on the tabletop 1, the tabletop 2 will generate a clone agent (these two agents exchange messages in order to keep the data coherence). Messages contain the position and the scale of the map. This message is used by the tabletop 2 to obtain the same vision as the tabletop 1.

<figure><img src="image_4.png" /><figcaption>**Figure 2.** Road traffic management on two *TangiSense* with Tangigets (a) without effect on the other tabletop, (b) with effect on it</figcaption></figure>

Accordingly such representative examples, we validated the principles of MAS to Tangiget, virtual and clone entities operating at the surface of interactive tables. We showed the interactions between two interactive surfaces enable the collaboration and the information exchange between different users during the simulation.
## CONCLUSION

Distributed applications on several tabletops lead to new design problematic, particularly when the objective is to manage different types of virtual and tangible objects in these applications. Each tabletop is in fact considered as an interactive surface usable by several users. We have proposed a model for the management of distributed interactive surfaces with MAS platforms. In this case a tangible object used on a surface may be cloned and represented virtually in another surface, by the use of so-called Tangiget objects proposed in previous works [2, 12].

The application concerns road traffic management simulation on several tabletops. This innovative distributed application allows us to validate the model proposed and to implement (a) different types of functions, but also (b) tangible objects able to manage them locally or remotely.

Our research perspectives are the following: (a) to implement different types of Tangigets usable with several connected tabletops, (b) to study conflict management between the different Tangigets. More the works of [13] which propose so-called GaussBits to interact with objects on different types of surfaces offer perspectives to test our model with other surfaces and technologies.

## ACKNOWLEDGMENTS

This research was partially financed by the French Ministry of Education, Research & Technology, the Nord/Pas-de-Calais Region, the CNRS, the FEDER program, the International Campus on Safety and Intermodality in Transportation (Plaïimob project), and the French National Research Agency (ANR TTT and IMAGIT projects, financial IMAGIT support: ANR-10-CORD-017). The authors would like to thank the partners with whom we collaborated on the TTT and IMAGIT projects: LIG, RFIdees and the CEA.

## REFERENCES

1. Bandelloni, R. and Paternò, F. Migratory user interfaces able to adapt to various interaction platforms. *Int. J. Human-Computer Studies* 60, 5-6 (2004), 621-639.

2. Caelen, J. and Perrot, C. Bibliothèques d'objets. Rapport de projet ANR IMAGIT [Object libraries. Report of the ANR IMAGIT project] (2011), Grenoble, France: LIG.

3. Calvary, G., Coutaz, J., Thevenin, D., Limbourg, Q., Bouillon, L. and Vanderdonckt, J. A Unifying Reference Framework for Multi-Target User Interfaces. *Interacting with Computer* 15, 3 (2003), 289-308.

4. Demeure, A., Sottet, J-S., Calvary, G., Coutaz, J., Ganneau, V. and Vanderdonckt J. The 4C Reference Model for Distributed User Interfaces. In *Proc. ICAS 2008*, D. Greenwood, M. Grottke, H. Lutfiyya, M. Popescu (eds.), IEEE Computer Society Press (2008), Los Alamitos, Gosier, 61-69.

5. Gallud, J.A., Tesoriero, R. and Penichet, V.M.R. (Eds.), *Distributed User Interfaces, Designing Interfaces for the Distributed Ecosystem*, Springer (2011).

6. Grolaux, D., Van Roy, P. and Vanderdonckt, J. Migratable User Interfaces: Beyond Migratory User Interfaces. In *Proc. of MOBIQUITOUS'04*, IEEE Computer Society Press (2004), Los Alamitos, 422-430.

7. Ishii, H. and Ullmer, B. Tangible bits: towards seamless interfaces between people, bits and atoms. In *Proc. ACM CHI'97* (1997), 234-241.

8. Kubicki, S., Lebrun Y., Lepreux, S., Adam, E., Kolski, C. and Mandiau, R. Simulation in Contexts Involving an Interactive Table and Tangible Objects. *Simulation Modelling Practice and Theory* 31 (2013), 116-131.

9. Larsson, A., Ingmarsson, M. and Sun, B. A Development Platform for Distributed User Interfaces. In *Proc. SEKE07* (2007), Boston, U.S.A.

10. Lebrun, Y., Adam, E., Kubicki, S., Mandiau, R. A Multi-Agent System Approach for Interactive Table Using RFID. In *Proc. PAAMS* (2010), 125-134

11. Lepreux, S., Kubicki, S., Kolski, C. and Caelen, J. Distributed Interactive Surfaces: A step towards the distribution of tangible and virtual objects. In J.A. Gallud, R. Tesoriero, V.M.R. Penichet (Eds.), *Distributed User Interfaces, Designing Interfaces for the Distributed Ecosystem*, Springer (2011), 133-143.

12. Lepreux, S., Kubicki S., Kolski C. and Caelen J. From Centralized interactive tabletops to Distributed surfaces: the Tangiget concept. *International Journal of Human-Computer Interaction* 28, 11 (2012), 709-721.

13. Liang, R.H., Cheng, K.Y., Chan, L.W., Peng, C.X., Chen, M.Y., Liang, R.H., Yang, D.N., Chen, B.Y. GaussBits: magnetic tangible bits for portable and occlusion-free near-surface interactions. In *Proc. CHI* (2013), ACM, 1391-1400

14. Luyten, K., Van den Bergh, J., Vandervelpen, C. and Coninx K. Designing distributed user interfaces for ambient intelligent environments using models and simulations, *Computers & Graphics* 30, 5 (2006), 702-713.

15. Shaer, O., Leland, N., Calvillo-Gamez, E. H. and Jacob, R. J. K. The TAC paradigm: Specifying tangible user interfaces. *Personal and Ubiquitous Computing* 8 (2004), 359-369.

16. Ullmer, B. Tangible interfaces for manipulating aggregates of digital information. *PhD thesis*, Massachusetts Institute of Technology (2002).

17. Weiser, M. Some computer science issues in ubiquitous computing. *Communications ACM* 36, 7 (1993), 75-84.

18. Zöllner, M., Jetter, H.C. and Reiterer, H. ZOIL: A Design Paradigm and Software Framework for Post-WIMP Distributed User Interfaces. In *Distributed User Interfaces: Designing Interfaces for the Distributed Ecosystem*, Springer-Verlag (2011), 87-94