# SyMAPse: Augmented Interactive Maps for Subjective Expression

Julia Chatain

► To cite this version:

Julia Chatain. SyMAPse: Augmented Interactive Maps for Subjective Expression. Human-Computer Interaction [cs.HC]. 2015. hal-01191429v2

HAL Id: hal-01191429

https://inria.hal.science/hal-01191429v2

Submitted on 24 Sep 2015

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

Public Domain
# SyMAPse:
Augmented Interactive Maps for
Subjective Expression

MASTER THESIS

September 24, 2015

<figure><img src="image_1.png" /></figure>

**Author:**

Julia CHATAIN

julia.chatain@polytechnique.org

**Supervisors:**

Anke BROCK

Martin HACHET

Didier LAVAL

Pierre DILLENBOURG

<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>
Abstract

Interactive geographic maps are widely available nowadays. Yet, these maps remain mostly limited to standard interaction contexts and only allow a small panel of expression means to novice users. For example, most available inter- active maps only enable users to explore content but not to contribute to the content. On the other hand, the "Smart city" process enlightens the importance of providing ways of expression to the citizens as they are directly affected by urban issues. We introduce a spatial augmented reality map, in which a virtual map is projected on a physical piece of paper. This system, developed for a museum, has two main uses for the users: expressing themselves about their city, and exploring the multiple visions of that city. In a preliminary study we compared interaction techniques for this system based on multi-touch, tangible and spatial modalities for three common map functions: zooming, repositioning the map excerpt, and changing the basemap. Based on the results from this study we designed a new version of this prototype and present future working directions.

**Keywords:** interactive maps; tangible interaction; projection mapping; multi-touch; interactive paper; augmented reality; smart city; museum

Résumé

Les cartes géographiques interactives sont largement disponibles de nos jours. Cependant, ces cartes sont pourtant limitées à des contextes d'interaction standards et n'autorisent qu'un panel limité de moyens d'expression pour les utilisateurs novices. Par exemple, la plupart des cartes interactives disponibles permettent seulement aux utilisateurs d'explorer du contenu mais rarement d'en ajouter. À côté de cela, le processus "Villes Intelligentes" met l'accent sur l'importance de fournir des façons de s'exprimer aux citoyens car ils sont directement concernés par les questions urbaines. Nous introduisons ici un système de réalité augmentée grâce auquel une carte virtuelle est projetée sur une feuille de papier physique. Cet système, développé pour un musée, a deux utilisations pour les utilisateurs : s'exprimer sur leur ville et explorer les multiples visions de cette même ville. Dans une étude préliminaire, nous avons comparé plusieurs techniques d'interactions pour ce système, basées sur des modalités multi-touch, tangibles et spatiales, pour trois fonctionnalités communes pour les cartes : le zoom, le repositionnement du centre de la carte, and le changement de base de carte. En nous appuyant sur cette étude, nous avons développé une nouvelle version de ce prototype et nous présentons plusieurs directions pour de futurs travaux.

**Mots-clés** : cartes interactives ; interaction tangible ; mappage de projection ; multi-touch ; papier interactif ; réalité augmentée ; ville intelligente ; musée
# Contents

<table><tr><td>1</td><td>Introduction</td><td>5</td></tr><tr><td>2</td><td>Context</td><td>6</td></tr><tr><td>2.1</td><td>Inria - Potioc</td><td>6</td></tr><tr><td>2.2</td><td>Cap Sciences</td><td>7</td></tr><tr><td>2.2.1</td><td>Exhibitions</td><td>7</td></tr><tr><td>2.2.2</td><td>127°</td><td>8</td></tr><tr><td>2.2.3</td><td>Smart City</td><td>9</td></tr><tr><td>2.3</td><td>EPFL - CHILI lab</td><td>10</td></tr><tr><td>3</td><td>State of the art</td><td>11</td></tr><tr><td>3.1</td><td>Interactive Maps</td><td>11</td></tr><tr><td>3.1.1</td><td>Visualization</td><td>11</td></tr><tr><td>3.1.2</td><td>Navigation</td><td>14</td></tr><tr><td>3.1.3</td><td>Simulation</td><td>14</td></tr><tr><td>3.2</td><td>Interactive devices in museums</td><td>16</td></tr><tr><td>3.3</td><td>Participatory design and co-design</td><td>20</td></tr><tr><td>3.3.1</td><td>Evolution of design</td><td>20</td></tr><tr><td>3.3.2</td><td>Participation</td><td>22</td></tr><tr><td>3.3.3</td><td>Tools and techniques</td><td>23</td></tr><tr><td>3.4</td><td>Smart City</td><td>24</td></tr><tr><td>4</td><td>Definition of the subject</td><td>27</td></tr><tr><td>4.1</td><td>Propositions</td><td>27</td></tr><tr><td>4.1.1</td><td>Astronomical exploration</td><td>27</td></tr><tr><td>4.1.2</td><td>Gravitation</td><td>28</td></tr><tr><td>4.1.3</td><td>Planet simulation</td><td>28</td></tr><tr><td>4.1.4</td><td>Subjective maps creation</td><td>28</td></tr><tr><td>4.1.5</td><td>Historical and spatial exploration</td><td>28</td></tr><tr><td>4.2</td><td>Decision and motivation</td><td>29</td></tr><tr><td>5</td><td>Development</td><td>30</td></tr><tr><td>5.1</td><td>Technological choices</td><td>30</td></tr><tr><td>5.1.1</td><td>Augmented reality framework</td><td>30</td></tr><tr><td>5.1.2</td><td>Geographic Information System</td><td>34</td></tr><tr><td>5.2</td><td>Architecture</td><td>35</td></tr><tr><td>5.3</td><td>Software development</td><td>35</td></tr><tr><td>6</td><td>Design of the prototype</td><td>38</td></tr><tr><td>6.1</td><td>Guessability study</td><td>38</td></tr><tr><td>6.1.1</td><td>Protocole</td><td>39</td></tr><tr><td>6.1.2</td><td>Results</td><td>40</td></tr><tr><td>6.2</td><td>Visual elements questionnaire</td><td>41</td></tr><tr><td>6.3</td><td>Interaction techniques experiment</td><td>41</td></tr></table>
<table><tr><td>6.3.1</td><td>Interaction techniques</td><td>42</td></tr><tr><td>6.3.2</td><td>Protocol</td><td>44</td></tr><tr><td>6.3.3</td><td>Results</td><td>45</td></tr><tr><td>6.3.4</td><td>Discussion</td><td>46</td></tr><tr><td>6.4</td><td>Map themes brainstorming</td><td>48</td></tr><tr><td>6.4.1</td><td>What can be expressed on a map?</td><td>48</td></tr><tr><td>6.4.2</td><td>How can it be expressed?</td><td>48</td></tr><tr><td>6.4.3</td><td>What do you want to influence?</td><td>49</td></tr><tr><td>7</td><td>Final System</td><td>50</td></tr><tr><td>7.1</td><td>Final Prototype</td><td>51</td></tr><tr><td>7.2</td><td>Engagement pack</td><td>51</td></tr><tr><td>8</td><td>Conclusion</td><td>54</td></tr><tr><td>9</td><td>Acknowledgments</td><td>55</td></tr></table>
List of Figures

<table><tr><td>1</td><td>Potioc projects (Potioc website)</td><td>6</td></tr><tr><td>2</td><td>Cap Sciences (Cap Sciences website)</td><td>7</td></tr><tr><td>3</td><td>Examples of devices</td><td>8</td></tr><tr><td>4</td><td>Bracelets</td><td>8</td></tr><tr><td>5</td><td>Architecture of the 127 ◦ (Cap Sciences)</td><td>9</td></tr><tr><td>6</td><td>Example of Chili projects (Chili website)</td><td>10</td></tr><tr><td>7</td><td>Tangible 3D tabletops (Paper [5])</td><td>12</td></tr><tr><td>8</td><td>SkyHunter (Paper [30])</td><td>12</td></tr><tr><td>9</td><td>Bancada (Paper [27])</td><td>13</td></tr><tr><td>10</td><td>Tangible Rings (Paper [9])</td><td>13</td></tr><tr><td>11</td><td>Tangible Augmented Street Map (Paper [21])</td><td>14</td></tr><tr><td>12</td><td>Augmented map of Inria</td><td>15</td></tr><tr><td>13</td><td>Cityscope (Medialab website)</td><td>15</td></tr><tr><td>14</td><td>Augmented sandbox (O. Kreylos website)</td><td>16</td></tr><tr><td>15</td><td>Interaction systems for a Lascaux Caves exhibition (Paper [12])</td><td>17</td></tr><tr><td>16</td><td>Sintra landscape (Parques de Sintra website)</td><td>18</td></tr><tr><td>17</td><td>Map exploration with tangible rings (Paper [17])</td><td>18</td></tr><tr><td>18</td><td>Art explorer (Paper [1])</td><td>19</td></tr><tr><td>19</td><td>Different design processes (Paper [29])</td><td>21</td></tr><tr><td>20</td><td>Four levels of creativity (Paper [29])</td><td>22</td></tr><tr><td>21</td><td>Different design techniques (Paper [18])</td><td>24</td></tr><tr><td>22</td><td>Sensor CitizenWatt (CitizenWatt website)</td><td>27</td></tr><tr><td>23</td><td>Subjective map of Rennes (Geographie Subjective website)</td><td>29</td></tr><tr><td>24</td><td>PapARt for drawing (Potioc website)</td><td>31</td></tr><tr><td>25</td><td>PapARt's features (Potioc website)</td><td>32</td></tr><tr><td>26</td><td>Basemap styles (Generated using Unfolding maps library)</td><td>35</td></tr><tr><td>27</td><td>Architecture of SyMAPse</td><td>36</td></tr><tr><td>28</td><td>System for guessability study</td><td>39</td></tr><tr><td>29</td><td>Visual elements for default basemap (Various sources)</td><td>41</td></tr><tr><td>30</td><td>System for interaction techniques experiment</td><td>42</td></tr><tr><td>31</td><td>Interaction techniques for changing the position of the map center</td><td>43</td></tr><tr><td>32</td><td>Interaction techniques for zooming in and out</td><td>43</td></tr><tr><td>33</td><td>Interaction techniques for changing the basemap</td><td>44</td></tr><tr><td>34</td><td>Two positions for the lock</td><td>45</td></tr><tr><td>35</td><td>Example of a drawing the user has to find on the map and copy</td><td>46</td></tr><tr><td>36</td><td>Boxplots of the results of the interaction techniques experiment</td><td>47</td></tr><tr><td>37</td><td>Brainstorming on the subject "What can be expressed on a map?"</td><td>49</td></tr><tr><td>38</td><td>Drawings for "Souvenir"</td><td>50</td></tr><tr><td>39</td><td>Brainstorming on the subject "What do you want to influence?"</td><td>50</td></tr><tr><td>40</td><td>Final prototype</td><td>52</td></tr><tr><td>41</td><td>AEIOU card game concept (O. Azibou)</td><td>52</td></tr><tr><td>42</td><td>Test of the AEIOU card game</td><td>53</td></tr><tr><td>43</td><td>Use of the AEIOU card game</td><td>53</td></tr><tr><td>44</td><td>Examples of emoticons (O. Azibou)</td><td>54</td></tr></table>
# 1 Introduction

Digital maps are very common and have various applications in several domains: itinerary searching, self localization, traffic jam visualization, urban planning, restaurant suggestion, etc. They also provide many interesting features such as zooming, switching between different styles of views (satellite, vector or even water color view) or linking to websites corresponding to places. Moreover, they enable several persons to contribute to a same map and thus allow aggregation of several opinions, advice, information and many other kinds of data. Some of those maps can even be personalized and enable users to add their own markers or itineraries. Unfortunately, this flexibility remains quite limited for a novice user and thus limits the ways of expression.

On the other hand, paper maps can accept any kind of annotations or drawings and can completely be personalized by their owner. They are often used for personal and expressive purposes: telling the story of a trip, of a memory, indicating how to go to a wedding, helping a friend to reach a place... Paper maps can also be folded, transported, sent with a letter, cut, pasted. Moreover, the use of paper materials and tangible objects also has positive effects on the attractiveness of a display [17].

Both digital maps and paper maps have interesting features and properties, but going from one medium to the other is not often possible. Moreover, the current existing digital tools do not provide a framework that enables the expression of several personal views while still remaining easy to use. This implies that it is not currently possible to have a crowd sourced map that keeps the intimate aspects of paper maps.

In our project, we use augmented reality to display a virtual map on paper, keeping most of the features of both the mediums. Thanks to the digital map base, the user can navigate on the map, zoom-in or zoom-out and even change the style of the basemap. Thanks to the paper base, the users can express themselves using regular pens without any technological constraint on their contribution.

In this project, we first explore through a guessability study the different interaction techniques that can be used to navigate on such a map and present a user study to compare three of them (interaction based on tactile contact, interaction based on manipulation of tangible objects, and interaction based on physical displacement of the map) for three different features (zooming, reposi- tioning, and changing the style of the map). We also performed a study to analyze the kind of topics the users want to express on. Using the results of those studies, we developed a prototype fitting the user requirements and the technical constraints of the system.
# 2 Context

This project was conducted between three actors with various resources, constraints and goals. The three of them followed the internship and advised its directions.

## 2.1 Inria - Potioc

The Potioc team<fnref n="1" /> is part of Inria<fnref n="2" />, the French national institute for computer science and applied mathematics. The main objective of the Potioc team is to design, develop, and evaluate new ways of interacting with the digital world while stimulating the user's curiosity, creativity, pleasure or learning abilities. The team explores various fields such as interactive 3D graphics, augmented and virtual reality, tangible interaction, brain-computer interfaces (BCI) or physio-logical interfaces.

For example, the PapARt project (Figure 1a) is a drawing assistant, projecting an image on a sheet of regular paper so the user can be inspired from the drawing lines. The projection follows the paper as the user moves it. Another project, Teegi (Figure 1b), enables users to visualize their brain activity on a puppet, thus understanding on their own how their brain works.

<figure><img src="image_5.png" /><figcaption>**Figure 1: Examples of Potioc projects**</figcaption></figure>

The team projects are interesting for mediation, learning, and creation in various contexts such as classrooms or museums. Potioc also provided inputs regarding the technical aspects and knowledge regarding interaction techniques during this project.

<footnote n="1">1. https://team.inria.fr/potioc/</footnote>

<footnote n="2">2. https://www.inria.fr/en/</footnote>
## 2.2 Cap Sciences

Cap Sciences<fnref n="3" /> is a Science museum based in Bordeaux (France) and welcoming over 500.000 visitors every year. The center has different missions such as making scientific culture accessible to a broader number of persons, sensitizing the public about social issues, or encouraging vocations among the young audience. It is composed of two main exhibits: a large yearly one on a specific scientific topic ("Dinosaurs" in 2014, "Space" in 2015, and "Light" in 2016), and a smaller one related with current cultural trends ("Video games" in 2015). The museum also includes a space for a very young audience and organizes several group activities such as cooking courses or crime investigation.

<figure><img src="image_6.png" /><figcaption>Figure 2: Cap Sciences</figcaption></figure>

### 2.2.1 Exhibitions

Cap Sciences' exhibitions have several interesting aspects that can be seen with the example of the video games exhibition <fnref n="4" />. First of all, the exhibition mixes a large amount of different novel and original technologies. For example, the visitors could play a first person shooter game using a half-sphere screen (Figure 3a), they could play the game "Pong" on a mechanical device (Figure 3b), or discover the history of video games on an augmented book.

Another interesting point is how the exhibition handles collaboration. Some of the stands only allow one player to try out the system while the other visitors would watch. Some other stands enable visitors to play against each other or with each other. One interesting example was a tactile table (Figure 3c) where four visitors could collaborate through various puzzles in order to unlock a map of "Video Games Land". Even though each player had a personal task to complete, the system would implicitly enable them to help their neighbours if they are stucked.

<footnote n="3">3. http://www.cap-sciences.net/</footnote>

<footnote n="4">4. http://www.cap-sciences.net/pageseditos, 483, left_2B109AE8.html</footnote>
<figure><img src="image_7.png" /><figcaption>**Figure 3: Examples of devices**</figcaption></figure>

One other interesting thing to note is that at the entrance of the museum, each visitor gets a bracelet (Figure 4) that is linked to a personal account. Those bracelets can then be used to unlock some features in the exhibitions such as saving a game or recording a test and let visitors pursue their experience from home.

<figure><img src="image_8.png" /><figcaption>**Figure 4: Bracelets**</figcaption></figure>

### 2.2.2 127°

The first floor is currently in construction and will host a space called "127°", that will include a FabLab, a LivingLab and a MediaLab.

The goal of the FabLab is to allow the visitors to access machines such as laser cutters and 3D printers and to create their own devices with the museum mediators. The LivingLab aims at introducing new technologies to the visitors and let them work with researchers in order to build a project together. Finally, the MediaLab enables the visitors to record their creations and reflections and share them outside of the museum.

Most of the visitors of the museum are children or adults (coming with their children). This is why the whole "127°" space is built in order to attract visitors between 15 and 25 years old and help them build new projects and ideas. The design of the space is based on a study of the target audience. For example, as the study showed that people between 15 and 25 years old tend to sit in nontraditional positions, the space will include activities on the floor, on
<figure><img src="image_9.png" /><figcaption>Figure 5: *Architecture of the 127°*</figcaption></figure>

pillows, in lounge chairs... The name "127°" comes from a study directed by the NASA that shows that 128°± 7 is the angle where a human being carries the least of his body weight <fnref n="5" />.

### 2.2.3 Smart City

The museum is also involved in a "Smart city" project with the city of Bordeaux for 2016-2018. This project includes four participants: Cap Sciences, Bordeaux Métropole <fnref n="6" /> (the urban community composed of Bordeaux and 27 neighbouring communes), and the Forum Urbain de Bordeaux (a project developing social science research on the city and urban issues).

The goal of Smart City projects such as this one is to mobilize citizens on urban issues [24]. In particular, this project is decomposed in different themes: the general study of mobility in the city, the study of the effect of the other citizens on ourselves, and the study of the effect of smartphones and other mobile devices on our mobility. The prototype developed during this thesis, SyMAPse, is interesting in this context and can be a mean for museum visitors to express their view on the city.

Finally, Cap Sciences will host the prototype as part of its Living Lab.

<footnote n="5">5. https://spinoff.nasa.gov/Spinoff2013/t_4.html</footnote>

<footnote n="6">6. http://www.bordeaux-metropole.fr/</footnote>
## 2.3 EPFL - CHILI lab

This thesis was also supervised by Pr. Pierre Dillenbourg, from CHILI lab<fnref n="7" /> in École Polytechnique Fédérale de Lausanne<fnref n="8" />. The lab studies, designs, develops and evaluates collaboration technology for learning and problem solving. The focus is both on the student side, on the teacher side, and on the technological side.

For example, the Ladybug project (Figure 6a) presents a paper-based game to manipulate and learn to compare fractions. The goal of the game is to help a ladybug avoid obstacles and return to her home. To do so, the participants have to attract it with different fractions of food. Those fractions can be described in various ways, continuous or discrete, as presented on Figure 6a. This project insists on the importance of paper-based activities as they are easy to manipulate and enable the students to bring them home and keep practicing. Moreover, the use of a tabletop enables collaboration between the different students and can create interesting group dynamics.

Another project of the lab uses a tabletop to enable logistics students to build a small-scale model of a warehouse (Figure 6b) and simulate the behaviour of the delivery trucks and the different products. The tangible interface enables the students to easily modify the warehouse and the simulation can help them visualize the impact of different storing strategies. This project is particularly interesting as the students spend half of the week in a real warehouse and can directly link what they learn on the small-scale warehouse to the real one.

<figure><img src="image_10.png" /><figcaption>**Figure 6: Examples of Chili projects**</figcaption></figure>

Overall, those three actors have a special interest in involving users and using new technologies to improve learning, engagement and enjoyment.

<footnote n="7">7. http://chili.epfl.ch/</footnote>

<footnote n="8">8. http://www.epfl.ch/</footnote>
# 3 State of the art

## 3.1 Interactive Maps

Maps are an efficient way to explore and understand the environment we live in. They can be of various types, from topological to political [2]. Maps can also focus on various scales, from a room to the universe. Finally, they can serve different purposes such as navigation, information, education, or simulation [14]. Interactive maps enable users to interact with the information displayed on the map in order to modify the visualization, to navigate in the geography, or to simulate some space-related or space-dependent behaviour.

### 3.1.1 Visualization

A first interesting aspect of interactive maps is that they enable users to display extra information and to choose the information they want to display.

For example, [5] explores different ways of including tangible objects in tabletops systems for map interaction. Two types of tangible objects are presented: 3-dimensions objects (cubes), and 2-dimensions ones (planes). Those objects can be used in two different ways: integrated (i.e. the tangible object becomes part of the map), and not integrated (i.e. the tangible object presents information about the map without being a part of the map itself). For all the possible combinations, the paper presents an example of application (Figure 7):

- **Integrated 3-dimensions tangible objects:** Placed on a map of a city, the cube can be augmented as buildings.

- **Not integrated 3-dimensions tangible objects:** Placed on a map of a country, the cubes can provide information about the place they are put on (density of population, size of the city, speed limit, etc)

- **Integrated 2-dimensions tangible objects:** Placed on a map of a building, those objects can present the 3-dimensions view of a room.

- **Not integrated 2-dimensions tangible objects:** Placed on a map of a city, those objects can present information about a building (for example pictures from Google Image or informations from Wikipedia).

Another way to interact with maps for visualization is presented in the Sky-Hunter project [30]. This project was initiated by the need of a collaboration tool for a multi-disciplinary team responsible of oil and gas exploration. The main concern of this paper was to find a way to enable different professions that require different type, amount and resolution of data to work together. To address this problem, Seyed et al. suggested a solution involving a tabletop with zoom-in and zoom-out features in order to enable every member of the team to access to the required resolution for the map. Moreover, as some members of the team needed a way to access the underground information, they provided a solution involving a tablet: as a user puts the tablet on the table in a perpendicular way, he or she can display the corresponding underground cut (Figure 8). Finally, the system is also completed by gestural interaction techniques that enable each participant to load/save information to/from the collaborative task.
<figure><img src="image_11.png" /><figcaption>**Figure 7:** Tangible 3D tabletops. **Left side:** examples of different uses for 3D and 2D tangible objects on a map. **Right side:** tabletop system using two top projectors (in order to be able to project on 3D objects) and a bottom projector to display the map on the table. A camera under the table detects the positions of the tangible objects using the markers beneath them.</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>**Figure 8:** Skyhunter. **Left side:** the system includes a tabletop (b) for displaying the map, a depth camera (a) to detect the gestures, and a tablet (c) for the user to interact. **Right side:** a tablet is used to display the underground information of the corresponding cut.</figcaption></figure>

Thanks to the different functions (zoom, transversal cut) the system enables people of different professions to access the different data they need and thus collaborate instantly. Moreover, as all the users use a personal tablet, they can decide to save the elements they need, to take notes when they want to, to view the underground data that they are interested in. Thus, they have both a global (tabletop) and local (tablet) view of the problem, and are able to collaborate while still saving the information that is important for their specific profession.

In order to select the visible information, filtering techniques can also be used. The Bancada project for example uses tablets to perform filtering operations [27]. Each tablet has a different functionality (represented by the color of the tablet of the Figure 9) and can be used to select a zone of the overall map on the table and to activate the corresponding functionality on this section. The
functionality of the tablet can be changed by the user.

<figure><img src="image_13.png" /><figcaption>**Figure 9: Bancada**</figcaption></figure>

The filtering can also be done using tangible objects such as the tangible rings [9]. This system was created to enable users to access different types of information on a same map. It is based on physical rings (Figure 10) that the user can manipulate and put on a tabletop displaying a map. The ring is then augmented with a menu that provides different features to the user (filtering, zooming, selecting, etc). The rings can be combined in order to combine several filters. Moreover, the rotation of the disks is detected and can be affected to an extra functionality such as adjusting the opacity of the associated filter. The filters can be of various types: satellite view, density of population, names of the buildings, etc.

<figure><img src="image_14.png" /><figcaption>**Figure 10: Tangible Rings**</figcaption></figure>

For this project, the design of the rings is particularly important, and led to rings with asperities in order to make them more manipulable by the user (this avoids having to slide the ring from one place to another). The ring also have to be thin enough in order to avoid occluding too much of the map. The diameter of the ring has to be investigated according to the size of the table top and the size of the map patches that users might want to investigate. Moreover, the number of rings is particularly interesting as it limits the number of users
interacting with the map at the same time and avoid the interaction of more
users than what the system can carry.

**3.1.2 Navigation**

Interactive maps can also enable users to navigate on the map. This navigation can be done using augmented reality as we will see in this section.

The Tangible Augmented Street Map (TASM) presented in [21] is based on a cube with markers that is then augmented using a camera (Figure 11). The cube is filmed using a camera and the resulting video is augmented in order to display a map on the cube. This cube can then be manipulated to navigate in the map (a rotation in one direction will present the next patch of the map in that direction).

<figure><img src="image_15.png" /><figcaption>**Figure 11: Tangible Augmented Street Map**</figcaption></figure>

The cube can be manipulated to navigate in a map without boundaries with
simple gestures. It can be especially interesting for an exploration task. On
the other hand, the cube has a fixed size that makes it hard to have a global
view of the location. Even with a zooming feature the size of the map remains
restricted by the physical boundaries of the cube.

A system presented at Inria also exploits augmented reality to enable users to navigate within a 3-dimensional map. This system is based on a 2-dimensions top view of the Inria building and an application for tablets or smartphones that enables a user to visualize a 3-dimensions version of the building using the camera of his device (Figure 12). The 3-dimensions view is integrated in the real environment surrounding the user. Moreover, users can enter the building and visit the inside by moving the tablet in the direction they want to explore. The purpose of this installation was to enable the users to visit the building before its actual opening, with the constraint of making it as realistic as possible.

**3.1.3 Simulation**

Interactive maps can also enable users to set and generate simulations.

The Cityscope<fnref n="9" /> project is based on a 3D representation of a city made of
construction blocks (Figure 13) in order to fit as much as possible the real

<footnote n="9">9. http://cp.media.mit.edu/city-simulation</footnote>
<figure><img src="image_16.png" /><figcaption>**Figure 12: Augmented map of Inria**</figcaption></figure>

geometry of the city. This model is then augmented using projection to display extra information (the common transports lines, the density of population, the number of tweets emitted, etc) according to the choice of the user. The user can also complete the city map by adding colored construction blocks (red for “houses” and blue for “jobs” for example). The system will then simulate the evolution of the situation with this distribution of the tokens.

The model of the city can be either a precise representation of a city, in scale with the real city (necessary for the visualization component) but it can also be an imaginary one (possible for the simulation component) for design or understanding purposes.

<figure><img src="image_17.png" /><figcaption>**Figure 13: Cityscope**</figcaption></figure>

The simulation is done in (almost) real time. This enables users to try different combinations quite fast and learn from the process (causality of their actions). Because of the physical 3D model of the landscape, it is also easier for the user to understand the consequences of the topology of the region on the simulation. It avoids having to make an extra effort to analyze this aspect. Moreover, as there are many tokens, several users can interact on the same map and place them together. They can then analyze together the result of their collaboration by checking the validity of the simulated result.

Another way to design a simulation is to use an augmented sandbox<fnref n="10" /> where

<footnote n="10">10. http://idav.ucdavis.edu/~okreylos/ResDev/SARndbox/</footnote>
users can sculpt and create a certain topology for the terrain (Figure 14). The system will then detect the depth of the current situation and project topology lines and colors over it. Once users are satisfied of their creation, they can use their hand to generate some virtual water. A fluid simulation algorithm simulates the behavior of the water on the terrain [25].

<figure><img src="image_18.png" /><figcaption>Figure 14: Augmented sandbox</figcaption></figure>

As for the previous system, this augmented sandbox enables users to interact directly with the simulation by modifying the terrain and lets them explore the consequence of their actions. Moreover, the system takes advantage of augmented reality by displaying a virtual liquid that does not alter the sand as a real liquid would and thus enables immediate usability of the sandbox.

As we saw, interactive maps can be particularly interesting. They let users decide what they want to display according to their goals and motivations. For example, using filtering tangibles or tablets, it is possible for several users to display at the same time the information they need on a particular location. Using such maps it is also possible to let users modify the map themselves and try out different configurations for setting a simulation for example and understand by themselves various phenomenons. We also notice that tangible objects are often use and can serve several purposes such as filtering, displaying information, exploring hidden places, completing the map in the missing dimension or even navigating on the map. As we saw, several techniques also involve tablets: this enables each user to keep a personal space while still being able to collaborate on the general map. However, in a museum context this might not always be possible as it requires each participant to come with a tablet or the museum to provide tablets to every visitor.

## 3.2 Interactive devices in museums

Interaction techniques need to be adapted according to the context where the display is used and exposed. In particular, the context of the museum requires specific features such as attractiveness to make the visitors try the system, entertainment to make the visitors stay on the system long enough to fulfill its purpose, or mediation to let the visitors learn things from their experience with the system.

Cap Sciences already hosted several research projects done in collaboration with Inria. Among them, a research project regarding interaction techniques
in museums [12] during an exhibition about Lascaux Caves. The authors investigated three different interaction techniques. First, as the objects in the caves are old and precious, it is not possible to allow users to manipulate them directly. Thus, the team provided a physical cube (CubTile, Figure 15a, developed by the company "Immersion" [7]) to manipulate 3-dimensions models of the objects in a more natural way: by performing rotation or translation gestures on the different faces of the cube, the users could rotate or translate the 3-dimensions model of the object. Second, it was possible to reconstruct objects from different pieces in the manner of an archaeologist. In order to manipulate and assemble those pieces, a system called "Coucheo" (Figure 15b) and based on 2D multi-touch interaction and head-tracked 3-dimensions stereoscopic visualization was used [11]. Finally, the last step was to help people draw from the 3-dimensions views of the objects. This task was performed using PapARt (Figure 1a), an augmented reality system combining physical paper and digital information [16].

<figure><img src="image_19.png" /><figcaption>**Figure 15: Interaction systems for a Lascaux Caves exhibition**</figcaption></figure>

Maps have also been used in museums. For example, a museum in Sintra (Portugal)<fnref n="11" />, hosts a display to present information about the region. Based on a physical 3-dimensional model and the local landscape, the system projects extra information on it. Users can then select what information they are interested in, and have the projection reacting accordingly to their desire (Figure 16).

Moreover, Ma et al. [17] compared the effect of tangible and virtual objects on the users in a context of a map exploration task involving “magnifying glass” lenses (Figure 17). Two installations were available: one with tangible rings as lenses and one with virtual rings as lenses. The installations were designed in a way that the virtual rings would mimic as much as possible the real ones. The study concludes that there is no significant difference in the demography (gender, age) of users attracted by both of the techniques. However, the users

<footnote n="11">11. https://www.parquesdesintra.pt/en/event/sintra-3d-landscape/</footnote>
<figure><img src="image_20.png" /><figcaption>**Figure 16: Sintra 3-dimensional landscape**</figcaption></figure>

were more attracted by the tangible interface, probably because the tangible 3-dimensional rings are more visible from further away. The tangible interface also enabled a broader exploration of the map and tended to attract more groups of visitors than single visitors. On the other hand, there is no significant difference in the think aloud data and none of the interfaces raises more questions than the other. There is also no difference in the way visitors share the rings. Thus, the major difference between the two interfaces is regarding the attractiveness and the exploration of the map where tangible interfaces prove to be better. Those factors are very important in a museum context and would thus push in the direction of tangible interfaces.

<figure><img src="image_21.png" /><figcaption>**Figure 17: Map exploration with tangible rings**</figcaption></figure>

Albarelli et al. [1], presented a study based on an installation that was designed for an exhibition at Ca'Foscari university in Venice. The goal of the installation was to enable users to browse art pieces from different epochs. Glass tables with maps of Venice printed on them are installed. Each of the maps corresponds to an era (satellite view for modern era, city plan of the Napoleonic cadastral map for the XVIII Century). Then, projectors under the tables display light patterns on the points of interest of the maps. If a point of interest is
selected, a projector displays the corresponding artwork on a wall next to the installation (Figure 18). Three interaction techniques were compared for selecting the artworks: finger touch, passive cursors and active cursors with personal screen. First, users could select a point of interest using their fingers but this technique had several problems. There was no limit on the number of users selecting points of interest, however, the number of projectors was limited and they could not display all the selections for a reasonable time. Moreover, the selection events were also launched when another part of the body was put on a point of interest: in particular, if users were putting their arm on the table to be able to look at the map more precisely, they would also select the underneath points of interest by mistake. Second, passive objects with infrared reflective material on the bottom face were used to select the points of interest. The number of cursors was thus directly limiting the number of users interacting with the device and avoiding false selecting events. When the user selected a point of interest, the color of the light pattern would change, providing visual feedback to his action. Moreover, if the cursor was misplaced (outside of a site), a special animation was played to indicate the closest site, as an indication for the user instead of forcing the choice. This technique solved the detection of false selection events and reduced the crowding effect, but still forced a high rhythm on the change of the paintings displayed by the projectors. The last technique tried involved active cursors. Those cursors were based on the same principle as the passive one, except that this time, the cursors contained a screen where the user could decide to lock the current point of interest and then look at the painting on this personal screen instead of committing a projector.

<figure><img src="image_22.png" /><figcaption>**Figure 18: Art explorer**</figcaption></figure>

Museums bring specific constraints on the system such as a need for built-in crowd management by limited the number of interacting tools for example. Moreover, we saw that tangible objects are particularly attractive and a good solution in many cases. However, the interaction techniques should also be designed in accordance with the task they will be used for in order to strongly limit the steepness of the learning curve for the visitors.
## 3.3 Participatory design and co-design

Participatory design is a concept that has been discussed since the 1960s. It is based on two concepts: design and participation. Reich et al. [26] describe **design** as any purposeful activity aimed at creating a product or process that changes an environment or organization. Then they describe **participation** as a legitimate right for any person affected by a design.

Sander et al. [29] explain that this idea of participation and its importance is the result of two major changes in our society. First of all, there is a shift in the focus of design processes. Initially, the creation of a product would focus on improving its quality and the technology involved. However, as we reached a point where those factors are already meeting the requirements and cannot be improved considerably, we tend to focus more on the way the product will be used, how the user will interact with it. This is also correlated with a slight change in the attitude of the users that are not looking for consumption anymore but for an experience. Secondly, the fast and broad expansion of the Internet got a strong impact on the expectations of the users. The Internet enables any connected person to express his or her opinion on a subject independently from his or her location, culture, expertise, or position in the local hierarchy. As a consequence, users want their voices to be heard, they want to collaborate in a non-hierarchical manner, directly with the experts. In the end, users are experts themselves, they are experts of their experience with a product. Sander et al. suggest that there is a wish to participate, both from the industrial side and from the users side.

Moreover, Reich et al. [26] explains that “knowledge is essentially social and maintained through being shared and contested by many different consciousnesses and perspectives. Knowledge is thereby instituted in the face of many different perspectives and for this reason is always subject to change, even radical change. The acceptance of this new ideal encourages increasing participation as much as feasible but also requires organizational, and may benefit from computational, support to reconcile these contesting perspectives.” Thus, the participation is also required to improve our knowledge and needs to be monitored by organizational and computational structures.

### 3.3.1 Evolution of design

As we saw before, there is both a wish and a need for the participation of the users in design activities. This creates a distinction between two kinds of processes [29]: the user-centered design process, and the co-design process (Figure 19).

Initially, the design process was user-centered. In this model, the **user** is considered as a passive object to study. Then the **researcher** combines his or her knowledge from theories with interviews and observation to create more knowledge. This knowledge, on the form of a report, is then transferred to a **designer** that adds an understanding of technology and the creative thinking required to generate ideas and concepts.
<figure><img src="image_23.png" /><figcaption>**Figure 19: Different design processes**</figcaption></figure>

On the other hand, a co-design process describes the application of an act of shared creativity to a design process. In such a process, the **user** is given the position of “expert of his or her experience” and plays a large role in the generation and development of knowledge, ideas, and concepts. In the meantime, the **researcher** and the **designer** work in pair to support the user: the former suggests tools for ideation and expression, while the latter gives a form to those tools. This new way of doing design is one step further participatory design: the users are not just providing ideas, they are also acting at the same level as the researcher and the designer.

Moreover, this process implies directly the user and avoids extra layers of communication: no one knows better the opinion and needs of the user than the user him/herself. Moreover, there is no hierarchy, everybody can create and be creative. This idea can even be pushed further by including the user in the conception of the communication tools.

An intermediate alternative is participatory design where the users are allowed to participate in ideation sessions such as brainstorming but they do not build the product until the end at the same level as the other members of the project.

As we can see, such approaches fit the new goals of design processes that we described earlier, but require extra tools for communication and creativity. Moreover it requires more coordination from the participants and a continuous communication effort from all of them.

Because of those constraints, this new process decreases the efficiency as “time spent from the beginning to the end of the project”. However, if the tools employed are correct and if the process is done properly, it can increase the satisfaction of the final users while decreasing the risk of creating a product that does not convince the target market. With this approach, “efficiency” needs to be redefined according to the problem that is addressed.
### 3.3.2 Participation

In order to ensure participation, different techniques have to be designed. Those techniques depend on many parameters and have to be adapted or modified for each new problem.

First of all, co-design approaches assume that users are “creative”. This assumption is strong and needs to be detailed. E. Sanders [29] presents four levels of creativity (Figure 20).

<table><caption>Figure 20: Four levels of creativity</caption><thead><tr><td>Level</td><td>Type</td><td>Motivated by</td><td>Purpose</td><td>Example</td></tr></thead><tbody><tr><td>4</td><td>Creating</td><td>Inspiration</td><td>'express my creativity'</td><td>Dreaming up a new dish</td></tr><tr><td>3</td><td>Making</td><td>Asserting my ability or skill</td><td>'make with my own hands'</td><td>Cooking with a recipe</td></tr><tr><td>2</td><td>Adapting</td><td>Appropriation</td><td>'make things my own'</td><td>Embellishing a ready-made meal</td></tr><tr><td>1</td><td>Doing</td><td>Productivity</td><td>'getting something done'</td><td>Organising my herbs and spices</td></tr></tbody></table>

Each level of creativity then requires different level of monitoring from the researcher in order to enable the participants to express themselves in the best possible way. People in the **doing** creative state will require leading, people in the **adapting** state will require guiding, people in the **making** state should be provided scaffolds, and people in the **creating** state needs to offer a clean state.

The process also needs to be adapted to the characteristics of the artifact that is being designed. For example, the design of a building and the design of a consumer product will not require the same procedures. Y. Reich suggests a list of characteristics that should be taken into account [26]: the lifetime of the artifact (short to long term), its risk and cost (low to high), its trace (the mark it leaves on a community), the user-artifact interaction (the way the user interacts with the artifact), its salience (the important or meaning the participants assign to the artifact), and the nature of design required (formal to informal technical knowledge).

Once the characteristics of the artifact are taken into account, the process still has to be adapted according to the characteristics of the participation itself. Y. Reich also suggests the following list of characteristics for this aspect [26]: the duration of the projects (from months to years), the duration of the participation (duration and frequency of the participation evens, depending on the resources each participant wishes to expand), the epistemic limits on participation (from a team of experts and non-experts to a team of experts only), the form of participation (who participates and who is affected by the trace, limited by the number of persons that can physically participate), and the role or responsibility of customers participants (from subject of study to participants).

All those parameters have to be taken into account in the organization of the design process and vary from a problem to another. Moreover, this list is not exhaustive and should be completed according to the knowledge of the problem that is available.
### 3.3.3 Tools and techniques

According to all the parameters described in the previous sections, the next step is to provide ways to enable and suggest the participation.

First of all, let us present the different definitions given by E. Sanders [28]:

- **Tools:** material components that are used in participatory design activities.

- **Toolkit:** collection of tools that are used in combination to serve a specific purpose.

- **Technique:** description of how the tools and toolkits are put into action.

- **Method:** combination of tools, toolkits, techniques that are put together to address defined goals within the research plan.

- **Approach:** describes the overall mindset with which the research plan is to be conducted (for example the assumption that everybody is creative).

Using those definitions, E. Sanders then defines a three dimensional framework to order the different tools and techniques.

First of all, an important axis is the **form**. This axis describes the kind of actions that are taking place between the participants. It can be a combination of three types of actions: **making**, **telling** and **enacting**.

The second axis is the **purpose**. It addresses why the tools and techniques are being used. There are four kinds of purposes in this framework, **probing** the participants, **priming** the participants in order to immerse them in the domain of interest, **getting a better understanding** of the current experience of the participants, and **generating** ideas or design for the future.

Finally, the last axis is the **context**: where and how the tools are used. This axis can also be decomposed into four categories: the group size and composition, the **interaction** (face-to-face or online), the **venue**, and the shareholders' **relationships**.

A particular case of participatory design is the case of multi-disciplinary teams. Different disciplines have a different required state of mind, a different vocabulary, a different point of focus. Thus, many communication problems might arise, requiring a particular effort on the choice and design of the approach.

W. Mackay suggests a highly interactive and iterative process to address this kind of case [18]. The process is decomposed in a loop over five steps (presented here with examples):

- **Finding out about the use:** the “Critical incident technique” uses the fact that it is easier to get relevant information from a user by asking a specific question and then generalize than by directly asking a general question.

- **Analyzing user data:** this step could be the analysis of a survey or a protocol.

- **Generation ideas:** the “brainstorming” technique invites participants to write down all their ideas during a certain amount of time and then reflect on them.

- **Designing systems:** the “Wizard of Oz” technique consists of creating
a paper prototype of the product and test it with users while mimicking its behavior.

— Evaluating systems: a usability study can be realized by asking two users to speak out loud about the product.

A list of the techniques available for the different steps is presented on Figure 21.

<figure><img src="image_24.png" /><figcaption>Figure 21: Different design techniques</figcaption></figure>

We saw that participatory design and co-design are more popular and more required, but also quite complicated to put into action. It is very important to study the problem and all its parameters in order to build an efficient process. Many techniques already exist, even for the case of multi-disciplinary teams, and can be modified and adapted carefully to new problems.

## 3.4 Smart City

Smart city projects aim at placing the citizen at the heart of the city, allowing a higher participation of citizens in the urban issues and improving their well-being by making use of new technologies [24]. Several definitions exist [24]. First, a definition from Jean-Louis Missika says that the term "Smart City" is actually a pleonasm as cities are always smart. Second, the definition given by the "Smart City Council" 12 is the omnipresence of the technology and the integration of digital to all the issues of the city. Finally, the definition of the British government states that a Smart City is a non-static concept, a process aiming at making the cities liveable and able to answer quickly to any challenge.

<footnote n="12">12. http://smartcitiescouncil.com/</footnote>
Carlos Moreno explains [31] that a Smart City process mixes three key con-
cepts: social inclusion, technological revolution, and urban reinvention.

Regarding **social inclusion**, the participants of [31] explain that citizen
participation should count. To do so, this participation has to be enabled re-
gardless of the timidity or the ease of speech of the different involved citizens.
Moreover, this participation has to be listened to and satisfied in order to bring
back the interest of citizens for public concerns. Even more, the citizens should
be integrated in the design of public goods. Virginie Calmels insists on the
importance of social innovation: the fact of encouraging the citizens into pro-
ducing wealth instead of redistributing it. In relation with this, she also states
that this cannot be done without societal innovation: giving to every citizen the
possibility of innovating.

The second point is **technological revolution**. Francis Pisani [31] explains that many urban issues can be solved using technology: thanks to the cloud, most of the citizens are connected and all the information created from this network could be exploited to solve major problems such as ecological hazards. In order to make this work, it is important to ensure that everybody can have access to this technology, starting with an internet connection. It is also important to adapt the technology to its users: for example, many people understand better a satellite representation of a map or a 3D model than a standard schematic map. Finally, this technological revolution is also a lot about connection. Emmanuelle Durand-Rodriguez [31] explains that the Internet can now connect people that are far away from each other in a new way: when a phone enables users to contact a distant person that they know, the Internet provides a place where they can meet distant persons they do not know. In a smart city context, this technological connection can be done at different levels. For example, using smart-phones, it is possible to connect users to their houses and enable them to visualize and control their energy consumption. It is also interesting to connect those houses to the rest of the neighbourhood in order to reallocate resources or organize transports.

Finally, a smart city project is also about **urban reinvention**. Regarding this aspect, Stéphan De Faÿ insists on the importance of creating structures to enable innovation such as Fablabs or Livinglabs [31]. Gilles Babinet explains that in this matter it is important to think about rupture innovation instead of incremental innovation.

Different interesting examples follow those key directions and can illustrate
this Smart City process. For example, the city Carbon-blanc provides on its
website a "Write to the mayor" feature<fnref n="13" /> that enable the citizen to express
problems or satisfactions regarding the city. This feature is really appreciated
as their messages are answered and taken into account for public decisions. In
parallel, the mayor of Carbon-blanc maintains a similar interaction based on
paper letter and ensured that both digital and paper requests receive the same
attention.

This idea of providing tools to the citizens for expressing themselves even

<footnote n="13">13. http://www.carbon-blanc.fr/41-ecrivez-nous.html</footnote>
gave birth to start-ups such as Jaidemaville<sup>14</sup>. This start-up lets citizens report problems regarding the city such as a broken light, a flooded gutter, cumbersome garbage, etc. Those problems are described using pictures, location and text. Anyone can rate up the problem in order to make it more visible. The administration of the city can then process to solve the problem. This system involves the citizens in the management of public goods and reduces the need of patrolling agents.

Some projects also try to handle urban problems with technological approaches. For example, the start-up Qucit<sup>15</sup> proposes quantifying solutions for cities that aims at enlightening resources management. For example, bike stations keep track of the number of bikes available but also the flux of entering and exiting bikes. This information could be used to generate a visualization of the bikes flux in the city and help the decision of building a new bike station.

Another important aspect of Smart Cities is the connection of the people. This connection can be done online only with crowdfunding platforms such as Kisskissbankbank<sup>16</sup> or KickStarter<sup>17</sup> that enforces social connections. But this can also be done in other various ways. For example, Belgium hosts many places called "Repair Café"<sup>18</sup> where citizens can come with some of their broken items and use available tools to repair them with the other participants. This kind of place centralizes tools, knowledge and competences in order to reduce wasting and connect the citizens. The CitizenWatt project [20] is at the interface between those distant and close connections. The goal of this project is to provide totally open source captors (Figure 22) for energy consumption measurement. The software code and the hardware blueprints are available online to anyone. Moreover, the association organizes special events called "Soudathons" where they bring all the required tools to a place where they think people could be interested in such captors (for example, a Soudathon took place in front of a building where many inhabitants have low income and could make life changing savings by controlling their energy consumption). The interested persons can then come and build the captors together. They are also invited to provide ideas and can work with the members of the association in order to improve the captors according to their needs.

The Smart City effort is, in the end, a never ending process making use of technology to tackle urban issues and work for a more ecological city. This process is done for and with the citizens, providing them new ways of expressing themselves, giving them the will and the means to participate together in building the city of tomorrow.

<footnote n="14">14. http://jaidemaville.com/</footnote>

<footnote n="15">15. http://qucit.com/</footnote>

<footnote n="16">16. http://www.kisskissbankbank.com/</footnote>

<footnote n="17">17. https://www.kickstarter.com/</footnote>

<footnote n="18">18. http://www.repaircafe.be/</footnote>
<figure><img src="image_25.png" /><figcaption>Figure 22: *Sensor Citizen Watt*</figcaption></figure>

# 4 Definition of the subject

This project was the first collaboration between the different researchers and stakeholders from the museum. Consequently, an important step of the project was to define possible directions that fit both the research interests of the laboratories involved and the Living Lab objectives of the museum. The approach of collaboratively designing the subject was also part of the co-design process that we employed during this project. In this section, we present some of the propositions we made and motivate our final decision.

## 4.1 Propositions

We presented different propositions according to two main axis. The first one was the exploration of the universe and understanding of the astronomical phenomena following the trend of the current main exhibit of Cap Sciences<fnref n="19" />. The second one was more focused on the city of Bordeaux, where the museum is, as Cap Sciences currently focuses on "Smart city" approaches. The propositions presented in this section were selected during a participatory design session with 5 participants (three researchers from Inria, one participant from Cap Sciences, and one invited researcher).

### 4.1.1 Astronomical exploration

Space involves various different scales: the size of a satellite, the size of a planet, the size of a galaxy, the distance between galaxies, etc. Interesting datasets also present Human activity in Space<fnref n="20" />. The goal of this project would be to enable visitors to understand the huge scale differences in Space and also acquire some insights regarding Human activity. The challenge would there be to provide a visualization that takes into account data of very different sizes while still giving the possibility to focus on some specific areas of interest such

<footnote n="19">19. http://www.cap-sciences.net/pageseditos, 486, left_25E41C81, odyssee.html</footnote>

<footnote n="20">20. http://qz.com/296941/interactive-graphic-every-active-satellite-orbiting-earth/</footnote>
as the geostationary orbit. Another challenge is to provide a way to interact with such a visualization.

### 4.1.2 Gravitation

Gravitation is a phenomenon that can be sometimes hard to understand for younger audiences. A study already proved that using augmented reality for teaching astronomy in classrooms can improve the results of the students [10]. In this project we would investigate this idea in a museum context regarding gravitation. We would also focus on the impact of different kinds of feedback (visual, haptic, acoustic,...) on attractiveness and learning.

### 4.1.3 Planet simulation

One last idea regarding the "Space" topic would be to enable visitors to create their own planet with different kind of terrains and resources. A simulation could present the evolution of life forms on the planet and the different planets could even interact with each other using a gravitational model. Using the bracelet of the museum (Figure 4) the visitors could save their planet with an account and continue visualizing it from their personal device. The main challenge with this project would be to find interaction techniques for modeling geographic information on a 3-dimensional sphere.

### 4.1.4 Subjective maps creation

Various projects in different cities focus on subjective maps. For example, in Rennes (France), a group of kids from Maison des Squares<fnref n="21" /> – helped by animators and designers – built a map of the city according to their perception of their everyday environment (Figure 23). Those children went to interview their friends in order to collect their testimonials (drawn or oral) regarding different places of the city. Then, they aggregated all those elements in order to create a map made of various drawings and childish legends ("The shop where we can buy things to eat", "The frogs' choir area", etc).

The map can then be used to learn interesting information regarding the perception the children have of the city and the changes they would like to make. For example, we can see a street renamed "Traffic jams street", a building commented by "We would like to change the buildings because sometimes they burn and are dirty" or even a park captioned as "Here the children are being stolen by thieves".

In this project, we would use augmented reality to enable people to annotate a map with subjective representations and anecdotes.

### 4.1.5 Historical and spatial exploration

Maps are also used for exploration and could here be exploited using augmented reality to enable visitors to explore their environment in time and space.

<footnote n="21">21. http://www.geographiesubjective.org/Geographie_subjective/Projet_Rennes.html</footnote>
<figure><img src="image_26.png" /><figcaption>**Figure 23:** *Collaborative subjective map of Rennes (the complete map can be found on http://www.geographiesubjective.org/Geographie_subjective/Carte_Rennes.html)*</figcaption></figure>

For example, each visitor could have a personal paper map that could be augmented to present the evolution of the corresponding area (topology, population, infrastructures, etc). Visitors could annotate their maps with the information they judge useful and keep them.

Regarding spatial exploration, an idea could be to provide a physical 3-dimensional model of the city (like the one presented in the Sintra museum<sup>22</sup>) and let visitors visit it by placing tangible objects on it. Similar to Albarelli et al.[1], the position of the tangible object would launch the projection of images and information related to this place.

Regarding those topics, many challenges arise such as the aggregation of information of various types and epochs.

## 4.2 Decision and motivation

Regarding those propositions and the objectives of the different actors of the project, we took the decision to focus on the creation of interactive augmented map for subjective expression (Section 4.1.4).

<footnote n="22">22. https://www.parquesdesintra.pt/en/event/sintra-3d-landscape/</footnote>
The idea of the project is to display a map on a sheet of paper using aug-
mented reality. Users could then select a location they want to express them-
selves on and draw or write directly on the paper using regular felt pens. The
contribution could be captured, saved, and reused on a digital map for example.

This subject falls in the living lab philosophy as it invites users to partici-
pate in the construction of a subjective representation of the city and be active
regarding the technology. As we also saw before, Cap Sciences is also part of the
Smart City process that has as objective to bring citizens back at the heart of
the urban issues using technology. With such a project we can investigate new
ways of integrating the citizen. Moreover, the museum was also interested in
letting the visitors go back home with the paper version of their contribution,
a request that augmented reality can meet as the visitors will directly draw
on the paper. In a co-design perspective, this project can also be an interest-
ing direction to enable creative thinking for urban issues. Finally, this is also
an interesting research direction as our project would mix both paper maps
and digital maps, providing the advantages of both modalities [3], a topic that
remains little explored.

5 Development

Once the subject was chosen, we implemented a system to answer the re-
quirements of the project.

5.1 Technological choices

First of all, we made the inventory of all the requirements we need for our
system in order to chose the technologies to use.

5.1.1 Augmented reality framework

We want to build a system that enables users to pick a location and annotate
it with drawings or writings. In order to let the user pick any location we need
a map that enables repositioning of the map center. Also, as we want to let
the users express at various scales (city, neighbourhood, building,...), a zooming
feature is requested. This pushes us towards the use of digital maps.

On the other hand, paper maps can be better for gathering more sensible and
subjective information [3] and are thus particularly interesting for our project.
In a general manner, paper interfaces offer several possibilities for interaction
and representation [8]. Moreover, from the museum point of view, paper is
cheaper and less fragile than tablets and thus can be manipulated by the visitors
without any risk. Paper also enables the visitors to keep a physical souvenir of
their experience.

Finally, we would like the users to be able to draw with tools they are used to
such as regular pens of felt pens in order to be comfortable in their contributing
process.
In the end, we want a paper-based map that would enable to users to interact with it and draw on it with regular tools. In order to satisfy those desires, we decided to use a technology developed by the Potioc team (Inria): PapARt [16], a Processing<sup>23</sup> library for paper-based augmented reality. At the beginning, the technology was used as a drawing assistant, based on a camera, a projector, and a depth camera (Figure 24a). The system projects a drawing on a blank sheet of paper (with ARToolKitPlus markers [6]) and enables the user to trace it (Figures 24b and 24c). The image follows the moves of the sheet of paper and lets users pick the position that is the most comfortable for drawing. After this, the team completed the system in order to make it available for other purposes than drawing assistant.

<figure><img src="image_27.png" /><figcaption>**Figure 24: PapARt for drawing**</figcaption></figure>

The current version of the system includes various features. First of all, the support of the drawing is a sheet of paper. This means that the user can **draw**, but also **take notes** for example. Moreover, the user can **keep the sheet of paper** for further use or as a souvenir. However, the drawings and notes of the user are not interpreted by the system.

The system is also capable of **detecting small objects** like small color cubes for example (Figure 25a). It is then possible to use those objects in order to launch events or adjust parameters. At the moment, the system is only able to differentiate simple objects of very different colors. It would be possible to add an object detection algorithm to enable detection of more complex objects but this could slow down the system.

Moreover, the system is able to **detect the contact of a finger** (Figure 25b). Thanks to this feature we can detect when users touch the piece of paper and thus enable them to interact directly with some virtual elements. The contact is detected within a small threshold above the sheet of paper. The height of this layer can be adjusted according to the goal of the application.

It is also possible to **handle several paper sheets in parallel**. Those sheets of paper can display different applications or different panels of the same application, such as menus. For example, one sheet can also be used as color or object reference for the detection algorithm: if a blue cube is put on the

<footnote n="23">23. https://www.processing.org/</footnote>
menu, the system can apply a function to all the cubes of the same color (blue) used on the main sheet of paper. This avoids calibrating the color detection algorithm with the environmental light.

<figure><img src="image_28.png" /><figcaption>**Figure 25: PapARt's features**</figcaption></figure>

This system has various advantages that are particularly interesting in our context. Indeed, the augmented reality aspect of the system brings many advantages. First of all, the system can provide assistance during a task, as seen for the drawing task where users can visualize several renderings of an object in order to get inspired and/or guided in their creative process. The system can also be used to provide complementary information to the user: for example, in a task where the user is asked to draw angles, the system could indicate the value of the angle and provide a score. This kind of systems have already been explored and proved interesting regarding the learning aspect [4]. Moreover, the system adds some dynamism: the physical static drawing of the user can be augmented by a virtual mobile element (projected). For example, the users could indicate a street light on their drawing and the system would change the lightning of the area according to the moment of the day.

The system is also paper-based and any kind of paper can be used as long as the right markers are printed on it. This can be very interesting in a museum: as the users manipulate paper there is no risk of damage as there would be with a tablet for example. They are also used to interact with paper and are not stopped by a potential learning curve. Moreover, the user can leave a trace on this support and keep it afterwards as a souvenir, as notes for remembering a taught concept or as an art creation. Those marks could be color marks, where each color represents a theme, a concept, a feeling, etc. The user could also use colors to modify the city (add trees, lakes, bridges, roads, buildings, etc). This can be done with tampons or felt pens for example, and color detection.

The system also allows several ways of interacting with it. First of all, the users can use parts of their body such as their **fingers or hands**. The interaction can be done through the contacts between the user's fingers [32] and the map or simply using hovering, but gestural interactions are not implemented yet. There are several limitations due to the current technology and the users' behavior. First of all, because of the incertitude of the depth camera and of the calibration, the zone where a contact is detected between the user's finger and
the table is a layer (about one centimeter thick) around the surface. Because of this layer, it can be complicated to accurately distinguish the contact area from the hovering one. Moreover, when users interact with the sheet of paper, they have a tendency to try to hold it with one hand or to support their fingers by putting their palm in contact with the paper. Those events should not be interpreted in the same way as a finger tip touching the paper, and the distinction can be imprecise.

The next interaction possibility is the **position and translation of the sheet of paper**. The position can be 2-dimensional — in the plane of the table where the sheet of paper is — or 3-dimensional if a vertical translation is added. We believe that the 3-dimensional position is not convenient: it requires to either slide the sheet of paper to the border of the table in order to be able to grab it and move it vertically, or to bend it. Moreover, the translations according to the vertical axis bring the sheet of paper out of the focus of the display (projector, depth camera, and camera) resulting in visual and functional artifacts. In the continuity of this technique, the **angle and rotation of the sheet of paper** can be interesting. However, this has to be taken into account carefully if some written input is asked from the user: in this case, the user might want to rotate the piece of paper in order to feel more comfortable for drawing or writing and might be frustrated if it changes the picture that is displayed.

Another possibility is the use of **small tangible objects** that the user can put on the map. We saw in our state of the art that the use of tangible objects is more attractive to the user and is particularly appreciated in a museum context [17]. The main limitation to this technique is related to the complexity of object recognition, especially with this display that allows objects to come with different orientations but also different lightnings due to the projection and the environment. A solution could be to have objects of different colors and thus to reduce the problem to a color recognition problem, or to add markers to the objects that we want to detect.

Finally, the user can also interact by **drawing or writing**. This approach can be complicated as it might require drawing or writing understanding. Moreover, the capture of the drawn or written elements has to be corrected according to the overlaying projection. It might also have to be filtered in order to be able to be aggregated with other drawings or writings.

If we enable drawn and/or written inputs, three levels of interpretation are inferred: the **no interpretation** level where the inputs are just captures and displayed without any interpretation of their meaning, the **color interpretation** level where the colors are extracted from the inputs and interpreted according to a predefined code; and finally the **input interpretation level** where the drawings and/or the writing is interpreted and the system has to be able to recognize the shape that is drawn (a table, a tree, a building, etc) and/or the text that is written. The last level of interpretation is still an unsolved problem and can be reduced using similarity with expected shapes.

The system also has some limitations, mostly technical constraints.

First of all, there is a short delay in the tracking, creating a small shift between the projected image and the position of the sheet of paper during
a movement. Moreover, in order to make the tracking robust enough a large amount of markers is necessary on the sheet of paper. Those markers have to be big enough to be detected by the camera and numerous enough to let the users hide some of them while drawing or manipulating the sheet of paper. However, in certain lightning conditions, the tracking remains quite uncertain. Then, as the image is projected, it can be occluded by the users, reducing their possibility of actions. This also reduces the amount of persons that can manipulate a sheet of paper at the same time. Finally, the detection of touch and object is done using one single depth camera. Thus, if two elements are touching the sheet of paper but are aligned in the field of view of the depth camera, only one will be detected.

As the PapARt framework is still in development, our implementation highlighted some required features or fixes and contributed to the overall technology.

## 5.1.2 Geographic Information System

Our system is based on a digital map. However, this map can be of various kinds and different dimensions can be taken into account.

First of all, **space** is an important factor. For example, the map can be in two dimensions, as a common paper map, but it can also be in three dimensions, displaying underground information or height information of the landscape and infrastructures. Moreover, the scale of the map can play a role here, going from the whole city to a single building.

Secondly, **time** should be addressed in our project. There are two ways of considering this dimension: the time can represent the time of the day (does the input of the user refer to the morning? to the evening?) but also the time when the input was provided. Thus, when defining a time evolution of the map, it is important to distinguish the evolution regarding the time of the day in the constructed city from the evolution regarding the time when the users added the inputs.

Moreover, the map can present different **basemaps**. Three examples are presented in Figure 26: the first one is a water color version of the map with no extra information, the second one is a vector map with extra information such as the names of the streets, the transport lines, etc, and the last one is a satellite view of the city.

In the end, we want a map that allows zooming and repositioning. We also want to be able to handle different contents and visual aspects for the map. The time feature, on the other hand, can be handled externally quite easily. The last constraint is to be able to interface our geographic information system with the PapARt framework, ie to be compatible with Java or directly with Processing. To serve those purposes, we found the Unfolding<sup>24</sup> library, for creation of interactive maps and visualization of geographic information with Processing or Java. This library in particularly relevant as it handles several types of map providers such as the ones presented Figure 26 but also allows

<footnote n="24">24. http://unfoldingmaps.org/</footnote>
<figure><img src="image_29.png" /><figcaption>Figure 26: Examples of basemap styles</figcaption></figure>

multi-touch interaction and addition of personal content [22].

Finally, we integrated the Unfolding library in the PapARt toolkit and added
other features in order to build our final system.

## 5.2 Architecture

For this implementation, we aimed at being very modular, in particular regarding the interaction techniques, in order to enable simple change of interaction techniques for example. The architecture is detailed on Figure 27. The system is based on a main map, implemented using the PapARt library<fnref n="25" />. This main map exchanges information with the drawings manager that is responsible both for saving the drawings and their meta-data and for displaying those drawings on the map when requested. The drawing manager also applies filter-based image processing algorithms in order to remove the background of the drawings, smooth the contours and adjust the colors. The main map also exchanges with the map factory, based on the unfolding maps library<fnref n="26" />, that handles the different map providers. Finally, the map factory receives information from the different interaction techniques modules. Those modules can be completely modified independently from the rest in order to change the interaction techniques.

## 5.3 Software development

This project involved a lot of development with various technologies. In this section, we detail the different modules and our implementation for some of them.

The first step was to build a main sheet of paper to carry our application. For this, we used the PapARt framework, as described above. For the main sheet of paper, we implemented a class *DrawingMap*, extending the *PaperTouchScreen* class of the framework that provides tracking, display, and touch detection on a

<footnote n="25">25. http://papart.gforge.inria.fr/</footnote>

<footnote n="26">26. http://www.unfoldingmaps.org</footnote>
<figure><img src="image_30.png" /><figcaption>Figure 27: *Architecture of SyMAPse*</figcaption></figure>
sheet of paper. We improved the class by providing it methods to communicate with the other modules of the application.

We then included a geographic information system library, Unfolding. To do so, we created a class `MapFactory` that is able to interact with several map providers at the same time (in our case `Google.GoogleMapProvider()`, `Microsoft.AerialProvider()`, and `StamenMapProvider.WaterColor()` as presented in Figure 26). Our class implements several methods to modify the map, such as `void panBy(float dx, float dy)` that translates the map center of a specific distance. Our class also provides a mini-map of the main map.

We also implemented several modules to handle interaction techniques. We wanted interactions techniques for three features: zoom, repositioning, and changing the basemap. We also wanted to be able to switch easily from one technique to another in order to try out several combinations of the different techniques. Thus, we implemented a handler for the interaction techniques that lets the owner of the application indicate the names of the desired techniques and activates the corresponding ones. Regarding the interaction techniques, we implemented three different types.

First, using the touch recognition provided in the PapARt framework, we implement different classes for touch-based techniques. Here, we implemented a class that detects the first contact of the user's finger with the sheet of paper and tracks its displacement in order to update some parameters. For example, we implemented a `ScaleMenu` class that displays a scale of the sheet of paper and zooms the map in if users move their finger in the direction of the "+" sign, or zooms out if they move their finger in the direction of the "-" sign.

We also implemented some classes for tangible interaction techniques. The idea of this kind of techniques is to enable users to interact with the map using physical objects. For those techniques, we do not want the interaction to take place on the main sheet of paper in order to save some place for the drawing. Thus, the first step was to implement new classes for new augmented sheet of paper where the tangible object would take place. To do so, we extended again the `PaperTouchScreen` class of the paper framework. Then, we implemented two ways of detecting the tangible objects. First, we implemented a technique based on touch detection that detects the object according to its 3-dimensional position on the sheet of paper. Second, we completed this technique with a color detection algorithm in order to be able to distinguish objects of different colors. This part was complicated as our system was used in a space where the lightning was not controlled: thus, the colors of the object would change easily. To solve this problem, we implemented a software to calibrate the object recognition algorithm according to the colors of the objects in the current lightning conditions.

Finally, we implemented interaction techniques based on the positions and displacement of the main sheet of paper. In order to do so, we tracked the corners of the sheet of paper at any moment. Using this information, it was then possible to access three variables: the position of the sheet of paper, the displacement of the sheet of paper from an initial position, but also the rotation of the sheet of paper.
For all those interaction techniques, we implemented tools to let them com-
municate with the MapFactory class and edit the zoom level, the GPS coordi-
nates of the center of the displayed map, and the map provider used.

Another important part of our project was the possibility to scan and store
the drawings of the users. First of all, we implemented a void capture() method that shuts down the projection for a short time and takes a picture of the sheet of paper with the camera before sending this picture to the module handling user drawings. The drawing handler creates a UserDrawing object that contains the picture of the drawing, the time when that picture was taken, the GPS coordinates of the center of the map when the user drew the picture, and the zoom level of the map at that moment. We also implemented a filtering algorithm using Gaussian filters for this handler that removes the background of the picture in order to only keep the drawing. After the capture, we display the processed drawing and let the user validate or not the captured drawing. If the user validates it, we copy the drawing and its meta-data to the hard-drive. When the application is launched, the drawings and their data are loaded from the hard-drive and displayed on the map.

# 6 Design of the prototype

The elaboration of the prototype was composed of several preliminary steps
described in this section. This part was done in a participatory design ap-
proach: we performed experiments and user studies with the users, improved
our prototype with the results and their remarks, and iterated this process.

## 6.1 Guessability study

First of all, we wanted to define interaction techniques for the system. Those
interaction techniques need to answer various constraints of different types.
First, **technical constraints** linked to the precision of the system: we suggest
that the techniques should function in most cases in order to avoid generation
frustration for the users. Second, **efficacity and efficiency constraints** are
taken into account, accrued with the learning time of the technique that should
be very short in a museum context. Finally, **engagement constraints**, mea-
sured using users' feedback, are important in order to indicate if the users enjoy
using the technique and are not limited in their experience.

In order to define the interaction technique we decided to perform a guess-
ability study focusing on two features: zooming and positioning. This study in-
volved 14 participants: 6 women and 8 men, 4 left-handed and one ambidextrous
(the other participants being right-handed), with an averaged age of 25.9 [σ =
4.29] years old. Half of the participants already knew the PapARt technology.
One participant registered as novice with new technologies, four participants
registered as medium, and the rest of the participants registered as experts. 11
participants were from Bordeaux, and 3 participants had lived in Bordeaux for
just a few months.
### 6.1.1 Protocole

For this experiment we used a small version of the system, composed of a projector, a RGB-camera and a depth camera (Figure 28). This system projected a map of Bordeaux on a sheet of paper. The map followed the sheet of paper as it moved. Moreover, the finger of the user was detected and the detected area was represented by a pink circle displayed on the map. However, no interaction technique was active.

<figure><img src="image_31.png" /><figcaption>**Figure 28: System for the guessability study (Inria)**</figcaption></figure>

The users received different instructions according to their order of participation.

For the first condition participants had only one instruction "Go to Inria". They were then free to use any gesture or action in order to try to do so. Moreover, as Inria was at the South of the map and invisible at the default zoom level, it would have been necessary to zoom and move the location of the map center in order to reach Inria. As the study was performed at Inria, all the participants knew its geographical location.

For the second condition, the participants had to do several tasks. First, they were asked to zoom in. Second, they were asked to zoom on the "Jardin Public" (indicated on the map by a circle and its name). Third, they had to zoom out. Finally, they had to move to the west of the map.

Of course, as no interaction technique was provided, the system did not react to any of the tries. The participants were filmed in order to make the inventory of all the techniques they used and to note at which moment they started to get discouraged or angry. We also recorded their remarks.
After this experiment, the participants were asked to fill a form with several questions such as their age, their knowledge of the city, or their laterality.

## 6.1.2 Results

We made the inventory of the techniques tried by the participants and sorted them according to the goal they were trying to satisfy. All the participants commented aloud their actions, thus enabling us to understand better their intentions and reflections (in particular for the participants of the *first condition*).

Regarding the **zoom in**, 11 of the 14 participants tried first a "pinch" technique based on the spacing of two fingers of one hand (thumb/index finger, index finger/middle finger, or thumb/middle finger) or between two fingers, one from each hand (index finger of the left hand with index finger of the right hand). 8 persons tried to tap twice quickly the sheet of paper in the manner of a double-click. 7 persons tried to touch the sheet of paper for a long time on the location they wanted to zoom on. A few other techniques appeared: rotation of the sheet of paper, rubbing of the finger around the point of interest, vertical translation of the sheet of paper, or translation of the sheet of paper in the plane of the table.

Only the 7 participants of the *second condition* were explicitly asked to **zoom on a precise location**, and the participants of the *first condition* had no reason to try it. This time, the finger-spacing technique was less used (only 4 participants tried it), and was only used once the first technique used. All the participants tried to touch the sheet of paper for a long time and 4 participants even tried this technique first. 3 users tried to double tap the sheet of paper.

10 participants tried to **zoom out**. The first technique used was a pinching technique between two fingers of one hand (thumb/index finger, index finger/middle finger, or thumb/middle finger) or between two fingers, one from each hand (index finger/index finger). Only two participants tried other techniques (triple-tap with a finger, rotation of the sheet of paper, or translation of the sheet of paper).

Finally, 13 participants tried to **change the location of the map**. 10 of them starting by touching the sheet of paper with their index finger and move their finger in the direction opposed to the direction where they were willing to go (for example they would move the finger from the bottom of the map to the top of the map in order to go South). One person did not have the idea of holding the paper with the other hand and was bothered by the fact that the paper was moving with the finger. 3 persons tried to move the sheet of paper in the direction where they wanted to go.

In conclusion, many persons tried techniques that are common on tablets and smartphones. However, most of the participants did not only try one technique. Nonetheless, most of the participants gave up or got angry after two or three tries. Some tasks inspired less the participants (most of the participants only tried one technique for zooming out). Some special cases are interesting to note: several participants tried to slide their fingers on the side of the sheet of paper
for zooming as if they tried to used a scrolling bar; moreover, some participants complained about the lack of buttons (for example: "I would like arrows to move"). The almost total absence of feedback from the system (visual, acoustic, haptic,...) regarding the efforts of the participants was really frustrating for them and raised many complaints. Finally, many participants specified the importance of gestures over a system based on buttons.

## 6.2 Visual elements questionnaire

Before defining the interaction techniques for our map prototype, we prepared a questionnaire in order to help our choices for the different visual aids involved. Visual aids are important as users complained about their absence in the guessability study. In the end, 33 persons filled the form, 17 men and 16 women. 8 of them used paper maps often and 29 used digital maps on a regular basis.

The participants were asked to pick between different representations which one made them think the most about a certain features (an example is provided Figure 29). The questions focused on the satellite view, the default digital map, the watercolor map, the zooming feature, a feature to go back to the main view, a scale changing feature, a geographical displacement feature, and a locking feature.

Some questions regarding the participants were also asked, such as their gender, their age, or their habits regarding maps.

<figure><img src="image_32.png" /><figcaption>**Figure 29: Possible visual elements for the default basemap**</figcaption></figure>

The results from this study helped us design the visual aids involved in the next sections such as the scale for the zoom, the representations of the different basemaps, or the arrows for repositioning.

## 6.3 Interaction techniques experiment

For the following experiment, we built a bigger setup in the museum, composed of a projector, an RGB-camera and a depth camera (Figure 30). This system offers three different ways to interact with the map: tangible interaction using physical objects, touch-based interaction, and spatial interaction, i.e. moving the paper sheet. In order to explore interaction possibilities for specific map tasks, we implemented interaction techniques each based on one of the modalities for three map functions (repositioning, zooming and changing the basemap). We chose those techniques for different reasons. First, touch-based interaction is today a standard interaction and was used by all the participants
of our guessability study (Section 6.1). Then, we added tangible object-based
interaction as it has raised a lot of interest in research, has proven to be very
interesting in a museum context [17], but has rarely been used for geographic
map. The last interaction technique based on spatial movements of the sheet
of paper was inspired by our guessability study (Section 6.1) and has not been
explored a lot in the literature. Regarding the map functions, we focused on
repositioning in order to enable the users to choose a place they are interested
in for expressing themselves. We also included a zooming feature as it can be
useful combined with repositioning, but also because we want to let the users
chose the scale they want to express on (a whole neighbourhood or a building).
Finally, we added a feature to change the basemap between three styles: de-
fault, watercolor and satellite view (Figure 26). The satellite view was added
as it is sometimes easier for some users to recognize the environment [24]. We
also added the watercolor view as it is very light of information and might be
interesting for expression and creativity.

<figure><img src="image_33.png" /><figcaption>**Figure 30: System for the interaction techniques experiment (Museum)**</figcaption></figure>

We performed an experiment with visitors of the museum in order to compare
the performance and enjoyment of the different interaction techniques.<fnref n="27" />

**6.3.1 Interaction techniques**

First of all, we implemented three interaction techniques for changing the
position of the map center. Here, the touch-based technique involves a fixed

<footnote n="27">27. This section of the report has been submitted as a TeC to the IHM conference http://ihm2015.afihm.org/</footnote>
menu with eight arrows for different geographic directions (N, S, E, W, NE, NW, SE, SW) that is displayed on the map (Figure 31a). Users can touch an arrow to move the map excerpt in this direction. Bringing back the finger to the center of the menu or removing the finger stops the navigation. The object-based interaction technique for positioning uses the same arrows printed on an external sheet of paper (Figure 31b). Users can then put a small object on the desired arrow in order to move in the corresponding direction. Placing the object on the center or removing it stops the navigation. Finally, The spatial interaction technique for positioning is based on the movement of the sheet of paper. The arrows are, this time, fixed on the table (Figure 31c). The user can then move the sheet of paper in the direction of an arrow to navigate. Placing the sheet of paper back in the center stops the navigation.

<figure><img src="image_34.png" /><figcaption>**Figure 31: Interaction techniques for changing the position of the map center**</figcaption></figure>

We then implemented three interaction techniques for the zoom feature. Here, the touch-based technique involves a scale that appears on the map image when the user touches the sheet of paper (Figure 32a). By moving the finger along the scale the user zooms in or out. We have chosen this implementation as it proved technically more stable than the pinch gesture that is usually used in multi-touch applications for zooming. The object-based technique involves a printed scale on which the user can slide a bar object (Figure 32b). The position of the bar on the scale defines the zoom level of the map. For the spatial zoom, a scale is placed on the left of the table (Figure 32c), and the position of the sheet of paper regarding this scale changes the zoom level.

<figure><img src="image_35.png" /><figcaption>**Figure 32: Interaction techniques for zooming in and out**</figcaption></figure>

Finally, for changing the basemap, we implemented a touch-based technique based on a pie menu that is displayed when the user touches the paper (Figure
33a). By moving the finger on the piece of pie corresponding to the desired basemap style, the user can select a new basemap. The current basemap is represented with a red border. For the object-based technique, we created three small-scale objects, one for each basemap (Figure 33b): a miniature city for the basic basemap, a satellite for the satellite view, and an easel for the watercolor style. Putting one object in a dedicated area of the table changes the current style of the map to the corresponding basemap. In the absence of objects in this area, the basemap remains unchanged. For the spatial technique, we defined three vertical areas on the table corresponding to the different basemaps (Figure 33c). Putting the sheet of paper below one of those areas changes the basemap correspondingly. The design of the tangible objects and the drawings representing the spatial areas were based on the questionnaires for the visual elements (Section 6.2).

<figure><img src="image_36.png" /><figcaption>**Figure 33: Interaction techniques for changing the basemap**</figcaption></figure>

Regarding the touch-based techniques, we explained that the menu for positioning is fixed (Figure 31a) while the menus for zooming (Figure 32a) and changing the basemap (Figure 33a) appear where the users touch the sheet of paper. The last two are interesting as the map is completely visible when the user does not interact with the system. However, it is currently not possible to distinguish a touching even from a drawing event and this can create undesirable behaviours. For example, the system could interpret a contact of the felt pen with the paper as a zooming interaction and change the zoom level while the user is drawing on a specific location. In order to avoid this, we provided to the users a lock that they could press to lock or unlock the touch detection (Figure 34).

### 6.3.2 Protocol

In order to compare the different interaction techniques, we performed an experiment at “Cap Sciences” science center with 36 users that were recruited from the visitors (see Table 1). Three groups of 12 users were formed and each group was assigned to one function (repositioning, zooming or basemap).

Each user had to perform the same task three times using touch-based, object-based, and spatial interaction in counterbalanced order. After each technique, we asked users to fill out a questionnaire regarding their satisfaction with the technique. After the three experiments, a final questionnaire invited users
<figure><img src="image_37.png" /><figcaption>Figure 34: Two positions for the lock</figcaption></figure>

<table><caption>**Table 1:** User population according to gender (f/m), laterality (right-handed/left-handed/ambidextrous), and age average</caption><thead><tr><td>Feature</td><td>Gender</td><td>Laterality</td><td>Age [σ]</td></tr></thead><tbody><tr><td>Repositioning</td><td>5 / 7</td><td>11 / 1 / 0</td><td>24.75 [3.02]</td></tr><tr><td>Zoom</td><td>5 / 7</td><td>8 / 4 / 0</td><td>29.42 [13.26]</td></tr><tr><td>Basemap</td><td>3 / 9</td><td>9 / 2 / 1</td><td>27.25 [4.29]</td></tr><tr><td>Total</td><td>13 / 23</td><td>28 / 7 / 1</td><td>27.14 [8.22]</td></tr></tbody></table>

to grade each technique (Likert scale from 1 "very bad" to 10 "very good") and provide qualitative feedback. In order to motivate users to carefully test the techniques, we introduced a game. The maps contained hidden drawings that appeared when interaction techniques were used (e.g., for zooming, the drawings were hidden in the different zoom levels and could be found only by zooming in or out, see Figure 35). Users were given 3 minutes to find a maximum of those drawings. Users had to trace each drawing they found with a felt pen on the paper sheet. Once the drawing completed, the users were free to search for a new drawing and so forth until the end of the three minutes. The order of application of the drawings was defined randomly in order to avoid learning effects. To sum up, each function was evaluated in a 3-condition within-participants study. We measured satisfaction with a questionnaire and efficiency as the number of found drawings (maximum 13) within 3 minutes.

### 6.3.3 Results

Statistical analysis were performed for each map function (zoom, repositioning, basemap change) regarding the satisfaction scores and the number of figures found and drawn by the user. Shapiro-Wilk tests revealed that distributions were not normal, thus we performed Friedman tests for all analysis.

We observed a statistically significant result regarding the users' grades for the zoom ($X^2 = 18.14, p < .001$). Post-hoc Wilcoxon rank sum tests with FDR correction revealed that satisfaction with the object-based interaction was significantly higher than for touch-based interaction ($p = .001$), as well as for spatial than for touch-based interaction ($p = .001$). Regarding the number of
<figure><img src="image_38.png" /><figcaption>Figure 35: Example of a drawing the user has to find on the map and copy</figcaption></figure>

reproduced figures, the Friedman test was not statistically significant ($X^2 = 5.78, p = 0.06$). However, generally more figures were detected for object-based interaction ($M = 6.3, \sigma = 3.58$) and spatial interaction ($M = 6.16, \sigma = 2.44$) than for touch-based interaction ($M = 3.8, \sigma = 2.04$).

For the repositioning task, there was a significant difference concerning user satisfaction ($X^2 = 6.39, p = 0.04$). Post-hoc Wilcoxon rank sum tests with FDR correction revealed significantly higher scores for object-based ($M = 7.54, \sigma = 1.03$) than for touch-based interaction ($M = 5.54, \sigma = 2.6$). There was no statistically significant difference regarding number of detected figures ($X^2 = 1.9, p = 0.39$) for the repositioning task.

Finally, with regard to changing the basemap there was a statistically significant difference regarding number of detected figures ($X^2 = 6.67, p = 0.04$). Post-hoc Wilcoxon rank sum tests with FDR correction revealed that the detected number of figures was significantly higher for spatial interaction ($M = 12.08, \sigma = 2.11$) than for touch-based interaction ($M = 10.16, \sigma = 2.55$). There was no statistically significant difference regarding user satisfaction ($X^2 = 1.5, p = 0.47$).

### 6.3.4 Discussion

This preliminary study has provided few statistically significant results, which might be due to the low sample size. However, it allowed us to identify advantages and inconveniences of all techniques.

In general, we noticed that touch-based interaction technique scored less in performance and appreciation than the other interaction techniques. This is supposedly due to the fact that the touch detection is limited by the precision of the kinect and is not as good as the detection on a smartphone or a tablet.
<figure><img src="image_39.png" /><figcaption>Figure 36: Boxplots of the results of the interaction techniques experiment</figcaption></figure>

Many users commented that they are used to handle those technologies and feel frustrated with the inferior precision of our system. We therefore intend to compare our prototype with a standard multi-touch device in our future studies.

On the other hand, object-based techniques were usually appreciated and performant. This is in line with prior studies [15]. The spatial techniques were also appreciated, even by some participants who expected this technique to be the most complicated. Consequently we suggest that both interaction techniques are interesting approaches that should be studied more.

All interaction techniques for repositioning received bad satisfaction scores (none above 5/10), while the performance was above average. We conclude that none of the techniques was pleasant to use for achieving the task: indeed, the users expressed that they felt lost and often had troubles visiting the whole map. As a solution, some of them suggested the use of a mini map as an overview. They were also missing the possibility of adjusting the speed of movement. Those problems are related to the fact that we used a relative navigation (no absolute relation between the position of the interactive element and the position of the map excerpt). We believe that using an absolute navigation (position of the map excerpt in correspondence with the position of the interactive element) could solve those problems. This will be investigated in our future work.
Finally, efficiency was limited for all zooming techniques. This maybe be
due to technical problems which made it difficult to stabilize a zoom level and
resulted in jumping from one to another. Users also lost time trying to go
further than the maximum zoom level and complained about the lack of visual
aids regarding the zoom limits.

In the end, this preliminary study allowed us to gain first inputs and ideas for the realization of a prototype. Indeed, tangible object-based and spatial interaction seem to be promising for this kind of augmented-reality map system. As we compared each function separately, we now need to investigate how to combine those techniques in one prototype. For example, it could be cumbersome and require a lot of space to integrate three tangible objects on a same system. Furthermore, this study allowed us to gain some ideas for improving the proposed interaction techniques. For instance, we will investigate using a tangible mini map as overview for choosing the map excerpt (repositioning and zooming).

## 6.4 Map themes brainstorming

Before implementing the final prototype, we made one last experiment regarding the topics that could be treated with our project. The focus here was double: first, the users of our system should be attracted and inspired by the suggested task. Second, the results should be interesting for understanding the perception of the city by its inhabitants.

We organized a multi-disciplinary brainstorming session between two participants, a designer and an engineer [18]. This session was composed of three different parts.

### 6.4.1 What can be expressed on a map?

The first part focused on listing the different things that can be expressed on a map (Figure 37). The results can be sorted between different categories (not disjoint). First, there is geographic information linked to data and phenomena. This information can be linked to positions (location of all the restaurants of the city), to flux (commutes of the citizens), to densities (traffic jams), or to routes (access maps). Second, some topics are linked to perception and what the city stimulates with sounds, smells, lights, etc. Finally, the experience of the city is also represented regarding the way citizens appreciate a city, what do they remember from it and what do they get from it (souvenirs, feelings, landmarks, etc).

### 6.4.2 How can it be expressed?

The participants then selected several themes among the ones expressed previously and, without discussing with each other, they drew on a paper map what the themes inspired them (an example for the "Souvenir" topic is presented in Figure 38).
<figure><img src="image_40.png" /><figcaption>Figure 37: Brainstorming on the subject "What can be expressed on a map?"</figcaption></figure>

Several conclusions came out of this study. For example, some themes such as "Places of interest", "Ecosystem" or "Surface" bring out similar places for each participant (for example, the work location is often represented such as the favorite restaurants). Moreover, we can notice that without explicit indication, the participants expressed in different ways: the participant A used a lot of drawings to express herself when the participant B wrote more, linked his drawings to specific locations and even used color-based legend. The example of the "souvenir" (Figure 38) can illustrate this difference: when the participant A drew one souvenir as a story, the participant B expressed the locations where his memories took place. We also observed that drawings linked with commutes turn unreadable really fast. Moreover, we concluded that the zooming feature is very important: indeed, even if some drawings were coherent with the provided scale, some others were really difficult to read as they were grouped at the same place. Finally, during the experiment, the two participants exchanged a lot, asking each other about the meaning of their drawings, asking the addresses of the restaurants that the other participant represented, etc. The participants also drew conclusions regarding their personal use of the city (for example, they realized that all their activities were located along a specific streetcar line, and then that they only use this transport).

### 6.4.3 What do you want to influence?

The last part of the session aimed at listing all the elements that the participants would like to be able to have influence on in the city. The results are presented Figure 39 and are very various, from prices to climate. This proves that there are many topics that the participants would like to be able to change.
<figure><img src="image_41.png" /><figcaption>**Figure 38:** *Drawings for "Souvenir"*</figcaption></figure>

Our system could be explored as a mean to gather data regarding those topics.

<figure><img src="image_42.png" /><figcaption>**Figure 39:** *Brainstorming on the subject "What do you want to influence?"*</figcaption></figure>

All those studies and experiments provided many insights for designing the final prototype.

# 7 Final System

Base on the results of our studies and technical constraints, we implemented a final system for the museum.
## 7.1 Final Prototype

As described in Section 6.3, we performed an experiment to compare three modalities (touch-based, object-based and spatial) for three map functions (repositioning, zooming, and basemap selection). This preliminary study allowed us to gain first inputs and ideas for the next steps of the project. Indeed, tangible object-based and spatial interaction seem to be promising for this kind of augmented-reality map system. As we compared each function separately, we now need to investigate how to combine those techniques in one prototype. For example, it could be cumbersome and require a lot of space to integrate three tangible objects on a same system. Furthermore, this study allowed us to gain some ideas for improving the proposed interaction techniques.

We implemented a prototype for the museum based on the results from our studies (Figure 40). First of all, we decided to include a minimap as the users of our experiment were often feeling lost in the repositioning task. The minimap displays the location of the main map but from further away, in order to give a global point of view to the user. There are 8 arrows on the minimap that the user can activate using a tangible object in order to move in the following direction. We chose a tangible object for this technique as the users from our experiment found more comfortable to leave the object while letting the map scroll in the desired direction than keeping a finger on the arrow. With a touch-based interaction, they were forced to keep their finger on the arrows and found that tiring. Moreover, as the spatial interaction technique was appreciated for the zooming feature, we created a zooming technique based on the displacement of the minimap along the length of the table. Here, we chose a displacement along the length of the table instead of a displacement along the width of the table in order to enable larger steps between two zoom levels and limit the imprecision observed during the experiment. We also chose to keep the sheet of paper for the minimap a little bigger than the map displayed on it in order to let the users manipulate it without interacting with it or hiding the markers. Finally, regarding the feature for changing the basemap, we decided to keep the spatial interaction technique based on three areas of the table corresponding to the three possible basemaps. This technique was really appreciated and proved more robust than the tangible one.

## 7.2 Engagement pack

Finally, with the designers of the museum, we worked on the user engagement towards the use of the system for expressing themselves. For this part, we proposed the use of external frameworks in order to help the users in their creativity step if they need it. These tools are external to the system (software) as they should be changed easily by the museum designers in order to let them try new approaches. They could then be either provided by a guide from the museum or by other means. In order to pick those tools, we used our preliminary study presented in Section 6.4. As noticed before, some users used a color code in order to organize their drawing. This can be also interesting for drawing
<figure><img src="image_43.png" /><figcaption>**Figure 40: Final prototype**</figcaption></figure>

interpretation as it is possible to count the histogram of the colors in the image.

<figure><img src="image_44.png" /><figcaption>**Figure 41: AEIOU card game concept**</figcaption></figure>

In order to explore this direction, we designed a card game of five cards based on the AEIOU design technique, a technique usually used to structure observational research. The technique provides five axis to observe a situation: Activities, Environment, Interactions, Objects, and Users. In our card game, each card has a word written on it and a color. The users pick a card and a felt pen of the corresponding color and draw the element of their memories corresponding to the topic of the card (Figure 41). The first card is about the environment of the memory: the physical place and the context of the memory. The second card is about the objects that attracted the attention of the user. The third card focuses on the users involved, the persons that were present during the memory. The fourth card asks to represent the activities that were involved in that memory. And finally, the last card is about interactions with
other users or with objects, vocal or physical.

We performed a fast study with this technique. We asked one user to choose a place and draw a memory about this place (Figure 42a). We then ask the user to redraw the same memory, using our card game (Figure 42b). The difference between the two drawings is quite impressive: when the first one is very difficult to understand, the second one is much detailed and complete. The user himself also had an interesting moments while drawing with the card game as it reminded him of some forgotten details (for example, the user commented "Those balconies were really beautiful, how were they already?" before drawing the balconies).

<figure><img src="image_45.png" /><figcaption>**Figure 42: Test of the AEIOU card game**</figcaption></figure>

We also used the card game with four other participants in order to see the kind of results we could get. Two examples are presented Figure 43. As we can see, the card game is used differently by the different participants: for some it just affects the colors of the drawing (Figure 43a) when for others it splits the drawing in different components corresponding to each card (Figure 43b).

Of course, the study should be pursued further to have more significant results regarding the possible use of this kind of technique.

<figure><img src="image_46.png" /><figcaption>**Figure 43: Use of the AEIOU card game**</figcaption></figure>

The prototype combined with the engagement techniques will be exposed in the museum and used to gather inputs regarding the way citizens live their city.
# 8 Conclusion

In this thesis, we introduced a new approach that combines paper maps and digital maps in order to explore the gap between the different uses of those two kinds of map. We performed different studies in order to pick interaction techniques for this system, meeting both the user preferences and the technical constraints of the system. We developed a first functional prototype and proposed an engagement technique based on a card game.

We believe that our project has two temporalities. The first one aims at gathering sensible data about the city, both helping citizens to express themselves, but also to assist them and opinion making and decision taking. The second one, not explored here, is the investigation of the use of our system as a tool for co-design and find new uses for it. As the prototype is included in the Living Lab of the museum, the second temporality will be explored as part of their workshops. Regarding the first temporality, there are still points to investigate. For example, it can be interesting to define ways to classify the users' participation in order to enable easier exploration of the resulting maps. In this matter, we propose a classification using emotions (Figure 44), time (morning, afternoon, evening, night) and keywords (based of the pyramid of Maslow [19]), but those approaches have to be tested and can be completed. Other interaction techniques can also still be studied. For example, we developed interaction techniques based on positioning a tangible or a virtual frame on a minimap in order to select an area of focus. Those techniques, and many others, still have to be tested. Moreover, another future direction can be the use of printed paper maps. We are currently investigating the possibility of interacting with such a map as an overview, using the virtual map as a focus point. Finally, the system should be tested with creative tasks and compared with other technologies (such as tablets) in order to evaluate its impact on the users' creative process.

<figure><img src="image_47.png" /><figcaption>**Figure 44: Examples of emoticons available for labeling the content**</figcaption></figure>
# 9 Acknowledgments

This adventure counted many companions who made it better and happier. First on all, I would like to thank my main advisor, Anke Brock (Inria - Potioc), for helping and supporting me, for giving me some of her time even when she did not have any, for always being enthusiastic and for believing in me.

I also want to thank Martin Hachet (Inria - Potioc) for following the project and improving it with his advice and ideas. I want to thank Didier Laval (Cap Sciences - Living Lab) for taking part in the adventure and being the main mediator with Cap Sciences. I also want to thank Pierre Dillenbourg (EPFL - Chili lab) for his great support every week and his different inputs during those six months.

I want to thank Stéphanie Fleck for providing her expertise, her ideas, and her support for this project.

Moreover, I want to address special thanks to my colleagues from Potioc team (Inria), starting with Jérémy Laviole for all the support and advice on the PapARt technology, Marie Demangeat for assisting me on all the experimental process with her happiness, her energy, and her never ending enthusiasm, Joan Sol Roo for always supporting the project and enlightening it with his complementary advice, and finally, Jérémy Frey for his availability and unconditional help.

I also want to thank my colleagues of the Cap Sciences museum, in particular Oussama Azibou for teaching me so many things about design, believing in the project, making most of the artwork (including the SyMAPse logo and part of the Figures 40, 41, and 44), and accompanying me in all the process while also being a great friend. I want to thank Pauline Clark for introducing us to the Smart City project, being the mediator in the middle of this very multidisciplinary team, and defending the interests of all its members. I also want to thank Clément, Manon, Arnaud, Romain, and Mathias for their help and support all along the project.

# References

[1] Andrea Albarelli and Augusto Celentano. Sandboxed Interaction for Casual Users in Shared Spaces. pages 97–104.

[2] Amanda Briney. Types of maps: The many different kinds of map. http://geography.about.com/od/understandmaps/a/map-types.htm. [Online; accessed 13-February-2015].

[3] A. Brock, J. Schoening, B. Hetch, and B. Signer. Mapping with MS Paint: Informing the Design of Reference Map Customization Technologies. *Sigspatial (submitted, under revision)*, 2015.

[4] Sébastien Cuendet, Quentin Bonnard, Son Do-Lenh, and Pierre Dillenbourg. Designing augmented reality for the classroom. *Computers and Education*, 68:557–569, 2013.
[5] Peter Dalsgaard and Kim Halskov. Tangible 3D tabletops. *Proceedings of the 7th Nordic Conference on Human-Computer Interaction Making Sense Through Design - NordiCHI '12*, page 109, 2012.

[6] Dieter Schmalstieg Daniel Wagner. ARToolKitPlus for Pose Tracking on Mobile Devices. In *Computer Vision Winter Workshop*, pages 6–8, 2007.

[7] Jean-Baptiste de la Rivière, Cédric Kervégant, Emmanuel Orvain, and Nicolas Dittlo. Cubtile: a multi-touch cubic interface. In *Proceedings of the 2008 ACM symposium on Virtual reality software and technology*, pages 69–72. ACM, 2008.

[8] Pierre Dillenbourg and Michael Evans. Interactive tabletops in education. *International Journal of Computer-Supported Collaborative Learning*, 6(July):491–514, 2011.

[9] Achim Ebert, C Weber, D Cernea, and S Petsch. TangibleRings: nestable circular tangibles. *CHI'13 Extended Abstracts on ...*, pages 1617–1622, 2013.

[10] Stéphanie Fleck and G Simon. An Augmented Reality Environment for Astronomy Learning in Elementary Grades: An Exploratory Study. *Proceedings of the 25ième conférence francophone on l'Interaction Homme-Machine*, pages 14–22, 2013.

[11] Martin Hachet, Benoit Bossavit, Aurélie Cohé, and Jean-Baptiste de la Rivière. Toucheo: multitouch and stereo combined in a seamless workspace. In *Proceedings of the 24th annual ACM symposium on User interface software and technology*, pages 587–592. ACM, 2011.

[12] Martin Hachet, Jean Baptiste De La Rivière, Jérémy Laviole, Aurélie Cohé, and Sébastien Cursan. Touch-based interfaces for interacting with 3D content in public exhibitions. *IEEE Computer Graphics and Applications*, 33:80–85, 2013.

[13] Kim Halskov and Nicolai Brodersen Hansen. The diversity of participatory design research practice. *Journal of Human Computer Studies*, 74:81–92, 2014.

[14] Brent Hecht. Geographic Human-Computer Interaction. *CHI*, pages 3163–3166, 2013.

[15] Eva Hornecker and Matthias Stifter. Learning from interactive museum installations about interaction design for public settings. In *Proc. of OZCHI '06*, pages 135–142, New York, USA, November 2006. ACM Press.

[16] Jeremy Laviole and Martin Hachet. PapARt: Interactive 3D graphics and multi-touch augmented paper for artistic creation. *IEEE Symposium on 3D User Interfaces 2012, 3DUI 2012 - Proceedings*, pages 3–6, 2012.

[17] Joyce Ma, Lisa Sindorf, Isaac Liao, and Jennifer Frazier. Using a Tangible Versus a Multi-touch Graphical User Interface to Support Data Exploration at a Museum Exhibit. pages 33–40, 2015.

[18] Wendy E. Mackay. Educating Multi-disciplinary Design Teams. *Proc. of Tales of the Disappearing Computer*, pages 105–118, 2003.
[19] Abraham Harold Maslow. A theory of human motivation. *Psychological review*, 50(4):370, 1943.

[20] David Menga and Anne-Cécile Lamballe. Séminaire aristote, la ville intelligente au service du citoyen. http://association-aristote.fr/doku.php/public:seminaires:seminaire-2014-03-15. [Online; accessed 21-July-2015].

[21] Antoni Moore and Holger Regenbrecht. The tangible augmented street map. *Proceedings of the 2005 international conference on Augmented teleexistence - ICAT '05*, page 249, 2005.

[22] Till Nagel, Joris Klerkx, Andrew Vande Moere, and Erik Duval. Unfolding - A library for interactive maps. In *Human Factors in Computing and Informatics*, volume 7946 LNCS, pages 497-513. Springer, 2013.

[23] Sheng-Ying Pao and Kent Larson. Augmented participatory design. *ACM SIGGRAPH 2013 Posters on - SIGGRAPH '13*, page 1, 2013.

[24] Francis Pisani. *Voyage dans les villes intelligentes : entre datapolis et participolis*. 2015.

[25] S. E. Reed, O. Kreylos, S. Hsi, L. H. Kellogg, G. Schladow, M. B. Yikilmaz, H. Segale, J. Silverman, S. Yalowitz, and E. Sato. Shaping Watersheds Exhibit: An Interactive, Augmented Reality Sandbox for Advancing Earth Science Education. *AGU Fall Meeting Abstracts*, page A1, December 2014.

[26] Yoram Reich, Suresh L. Konda, Ira a. Monarch, Sean N. Levy, and Eswaran Subrahmanian. Varieties and issues of participation and design. *Design Studies*, 17:165-179, 1996.

[27] Francisco Marinho Rodrigues. Bancada · Using Mobile Zoomable Lenses for Geospatial Exploration. *ITS*, pages 409-414, 2014.

[28] Elizabeth B.-N. Sanders, Eva Brandt, and Thomas Binder. A Framework for Organizing the Tools and Techniques of Participatory Design. *PDC 2010 - Proceedings of the 11th Biennial Participatory Design Conference*, pages 195-198, 2010.

[29] Elizabeth B.-N. Sanders and Pieter Jan Stappers. Co-creation and the new landscapes of design. *CoDesign*, 4(May 2012):5-18, 2008.

[30] Teddy Seyed and Mario Costa Sousa. SkyHunter: a multi-surface environment for supporting oil and gas exploration. *Proc. ITS*, pages 15-22, 2013.

[31] La tribune. 1er forum smart city de bordeaux. http://smartcitybordeaux.latribune.fr/conferences/. [Online; accessed 21-July-2015].

[32] A Wilson. Using a depth camera as a touch sensor. *Proceedings of ITS 2010*, pages 69-72, 2010.