# First Steps in Developing Tangible Artifacts for All: Enabling Ideation and Discussion Processes

Vanessa Maike, M. Baranauskas

► To cite this version:

Vanessa Maike, M. Baranauskas. First Steps in Developing Tangible Artifacts for All: Enabling Ideation and Discussion Processes. 18th International Conference on Informatics and Semiotics in Organisations (ICISO), Jul 2018, Reading, United Kingdom. pp.238-247, 10.1007/978-3-319-94541-5_24. hal-01920733

HAL Id: hal-01920733

https://inria.hal.science/hal-01920733

Submitted on 13 Nov 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# First Steps in Developing Tangible Artifacts for All: Enabling Ideation and Discussion Processes

Vanessa R. M. L. Maike and M. Cecília C. Baranauskas

Institute of Computing, University of Campinas (UNICAMP), Campinas-SP, Brazil
vanessa.maike, cecilia{@ic.unicamp.br}

**Abstract.** Including everyone in the process of designing information systems is a challenge, especially considering that techniques and tools traditionally used in this process are written documents. This can make them non-accessible to people in special conditions, e.g., the visually impaired. Therefore, in this paper we present the first steps we took towards redesigning some of these techniques and tools, turning them into tangible digital artifacts. These initiatives are presented as two case studies. One intends to tackle the problem of materializing an idea discussion tool that has an existing graphical representation. The other, takes on the challenge of re-interpreting an ideation technique with well-defined dynamics. In the end, they point towards how to redesign other artifacts.

**Keywords:** Universal Design, Tangible User Interface, TUI, Organizational Semiotics, Participatory Design, Human-Computer Interaction.

## 1 Introduction

The design of information systems that consider the uniqueness of each person represents a challenge. However, it is also the principle behind *Design for All* or *Universal Design (UD)*. Here considered synonyms, both represent an approach to the design of environments or products that suit the broadest possible range of people, regardless of age or ability [13]. This has major importance in our current information society [12], which is based on the production and exchange of information, and a place for a variety of computer-mediated activities that everyone should have access to.

Universal Design is the core philosophy behind this work. In particular, we propose to take ideation and discussion tools and techniques which are usually based on written documents, and make them accessible to all, by turning them into digital artifacts. This way, we are creating technology that is inclusive and, at the same time, making more accessible the process of designing technology.

The technological approach we chose for such task was *Tangible User Interface (TUI)*, which is the coupling of digital information with tangible objects, with the goal of materializing the digital. Ishii [4] proposed the concept of TUI to push beyond the paradigm of \ac{GUI} and its restriction to flat displays, mouse and keyboard. Inspiration came from the work of Weiser [15], who described ubiquitous computing as a new paradigm where computers were everywhere, invisible. In essence, TUI provides a new
interaction paradigm, that intends to create a closer relationship between the physical
and the virtual worlds. For the purpose of our work, i.e., bringing artifacts and tech-
niques from the paper to the digital, this paradigm provides a good middle-term.

In this paper, we present the first steps we took in this direction. We do so in the
form of two case studies. The first takes on the challenge of turning a paper-based dis-
cussion tool, the *Stakeholders Identification Diagram (SID)*, into a tangible, electronic
artifact. The second case study is about reinterpreting a writing ideation technique,
called BrainWriting, to make it accessible. Hence, this paper is organized as follows:
Section 2 describes the theoretical foundation for this work, Section 3 presents the tech-
niques and tools we intend to redesign, Section 4 contains the two case studies, and,
finally, Section 5 highlights our concluding remarks and future steps.

## 2 Theoretical Foundation

We built this work upon two main pillars: **Participatory Design (PD)** and **Organizational Semiotics (OS)**.

On one hand, **PD** is an approach to the design of computer systems that brings people who are prospective users of the system into the design process, as active participants. PD started out in Scandinavia and the main principle behind it is that the people who will use the system are the experts in their domain [9]; as so, they have much to contribute to the design of products that will affect them. In other words, they hold the knowledge about their tasks and how to perform or improve them. Designers, then, should act as technical consultants. Hence, there is cooperation in the sense of a “mutual learning” [5], so that users can learn about the technical possibilities, and designers can learn about the application domain.

While working with participatory approaches, it is necessary to accept that they involve dealing with conflict [5], since different groups within the organization have their own interests and visions of what the system should do, and how it should do it [9]. Ignoring this fact can lead to a poor understanding of the organizational reality, or to systems that are for a few group of users [5]. To properly deal with this, [5] suggests applying different activities, ranging from individual ones to others that involve group negotiation and cooperation. In this sense, [8] describes sixty-one PD practices, applicable within different moments in the design process, and adequate for varied objectives and group sizes. These practices, however, are not meant to be linear step-by-step guides towards predictable and safe outcomes. Instead, they should be seen as scaffolds for complex, non-linear group processes.

On the other hand, **OS** involves the application of **Semiotics** to the study of organizations [6]. In this context, **organization** refers to when a group of people work together to accomplish a task [10]. However, the more elaborate its product, the more difficult is for all the involved to fully realize the secondary consequences of decisions they make. For example, a road is a product that reduces travel time, but at the same time it might increase air pollution and noise.
In this sense, organizations can be seen as **information systems**, i.e., they can create and convey information, as well as define and change meanings [6]. Here enters **Semiotics**, the study of *signs*, i.e., the triad of *representamen* – concrete form of the sign, *object* – what the sign represents, and *interpretant* – who is doing the sense-making. Therefore, looking at organizations as information systems and using the lens of Semiotics to their design, might allow that quality and accurate information gets to the right people, improving the decision-making process.

One useful way of looking at information systems is the organizational onion representation [6]. First proposed by [11], it shows an understanding of organizations as three sequential layers, one containing the other, like an onion. The outermost layer is called “informal information system”, and it refers to the values, beliefs, intentions, habits, meanings and other cultural manifestations that can be established or changed. The intermediate layer is called “formal information system”, where are the well-established aspects of culture such as rules, laws and social conventions. The innermost layer is called “technical information system”, and it contains the parts of the formal system that can be automated by machines. Based on the organizational onion, [1] extended the idea to think about a design process. The basic idea is that there are three main stages of design – Analysis, Synthesis and Evaluation – and on each design iteration, these stages cut through the three layers – informal, formal and technical – in a non-linear order. The iterations persist until there is an acceptable product. In addition, for each stage there are different artifacts, techniques and tools to support the design process. This, in part, comes from a strong relationship between this design process and PD.

In fact, the common grounds PD and OS share show the importance of communication and of quality of information in the decision-making process. Hence, while PD emphasizes the role of *egalitarian participation* in system design, OS brings out the importance of *information*, not just the meanings it carries, but also the channels it travels through.

# 3 The Objects of Redesign

The techniques and tools we intend to redesign to become tangible (concrete) digital artifacts are separated into two categories: those for *ideation* (i.e. generation of ideas), and those for supporting *discussion* in three artefacts of OS (Stakeholders Identification Diagram, Evaluation Frame and Semiotic Framework).

## 3.1 Tangible Ideation Tools

We will focus our attention in two ideation methods, the *BrainWriting* [14] and the *BrainDraw* [8]. They are alternatives for *brainstorming*, the method for oral raising of ideas by a group of people. BrainWriting and BrainDraw follow a round-robin dynamics, where each participant starts with their own idea on a piece of paper and, after a
predetermined amount of time, they pass the paper to another person and receive some-
one else's sheet. Then, they must complement the other person's idea until time runs
out. The procedure is repeated until at least one cycle is completed, i.e., until everyone
receives their sheet back. This style of brainstorming solves some problems of having
an influencer among the participants, and leads to a more distributed discussion and
proposition of ideas.

Both BrainWriting and BrainDraw are usually done in a silent manner, using pen
and paper. The difference between them is that BrainWriting is the generation of written
ideas, while BrainDraw consists of drawings of interface design ideas. They both are
good options for groups of people where status differences are evident, or where con-
flict is likely to happen. Since everyone has an equal chance to show their own ideas
and also to contribute with the ideas of all others, the democratic aspect of Participatory
Design is strongly present in both techniques.

## 3.2 Tangible OS Tools

We chose to work on three artifacts from Organizational Semiotics. The first is the *Stakeholders Identification Diagram (SID)*, and it is used for pointing out all who are involved in a design process [7], which can be either groups of people (e.g. developers, users), or entities (e.g. universities, corporations). The traditional graphical representation of the artifact organizes the stakeholders into five sequential layers that contain each other, like an onion: Operation (innermost layer), Contribution, Source, Market, and Community (outermost layer). The idea is that, the closer to the center, the more directly involved with the design product the stakeholder is. Hence, towards the edge are the stakeholders who are not actively involved in the design process, but who somehow affect or are affected by it.

The second artifact is the *Evaluation Frame (EF)*, and it is used to support the reasoning of problems and solutions associated with each stakeholder identified in the SID [2]. It is represented as a table, with a column for raising issues, a column for solutions or ideas, and five rows, one for each SID category. Therefore, this artifact allows the identification of requirements, as well as the anticipation of issues that may affect the design.

The third and last artifact is the *Semiotic Framework (SF)*. In the context of our work, it is used for identifying and organizing requirements, in six levels of knowledge [10]. The graphical representation of the SF shows these levels stacked on top of each other, in a progressive manner, similar to a ladder. From bottom to top, they are the following: Physical, Empirical, Syntactic, Semantic, Pragmatic and Social World. The bottom three levels are related to the structure of signs, how they are organized and transmitted. In turn, the upper three levels are related to how signs are used, in terms of meanings, intentions, and social impact they have. Hence, the SF, as we use it, allows for a view of the system requirements, going from their technical infrastructure (bottom), to their information system (top).

The three artifacts are shown in Figure 1. They are meant to be used by a group of people, preferably representatives from the stakeholders, in participatory practices. In the practices we have conducted [3], for the SID and the EF, the idea is to fill in the
layers collectively. To do so, participants are given post-its and can freely place them where they see fit, in a large printed version of the artifact's graphical representations, in workshops planned for this. During this process, participants are encouraged to discuss amongst themselves what to write on the post-its and where to place them. This way, they can all contribute to the decision-making process, and collectively reach an understanding of the problem at hand. The SF, in turn, uses the results from the other two artifacts to allow stakeholders to discuss the system requirements and, at the same time, gain an understanding of its different dimensions.

<figure><img src="image_2.png" /><figcaption>Fig. 1. To the left, the SID. In the middle, the EF. To the right, the SF.</figcaption></figure>

# 4 Preliminary Case Studies

The first case study is about taking a discussion artifact – the *Stakeholders Identification Diagram (SID)* – and redesigning it to become a tangible digital artifact. To do so, we adopted a work method, so that the prototype follows Universal Design principles. Then, in the second case study we take an ideation method – the BrainWriting – propose and explore a new dynamics with digital tools, so that it becomes accessible to all.

## 4.1 The SID Case Study

In this case study, to design the digital artifact, we divided it into three independent tiers: *Hardware*, *Aesthetics*, and *Software*. This separation provides us with the flexibility to make changes in a tier without affecting the others. In turn, this reduces costs of future alterations, and allows compatibility with multiple forms of input and output – an important feature for Universal Design.

The *Hardware* tier is where we work on the electronic components of the artifact, such as micro-controllers, buttons, wires and sensors. It involves the technology the user will not see, because it is covered by the *Aesthetics* tier. This, in turn, is where are the tangible materials that people interact with. They make the interface between the hardware, the person, and the software. In addition, it constitutes a major part of what the user sees, so, as the name suggests, it aims to be visually pleasant. Finally, the *Software* tier contains the virtual interface of the artifacts. We chose the web platform due to its multi-device capabilities, and to its easy integration with several micro-controllers currently available. This tier receives the signals from the Hardware tier and provides the appropriate feedback, either to the hardware or to the person.
Therefore, to make the SID a tangible and digital artifact, we started out by the Aesthetics tier. We drew its five onion-like layers on cardboard, and cut out each layer, making them five independent pieces. We trimmed the edges of each piece so that, when they were laid out inside each other, there would not be friction. This way, it is easier to make each layer interactable, and they can be differentiated by touch – an important feature for the visually impaired. To add on to that differentiation, we also made the inner layers progressively taller than the outer layers.

Next, we worked on the Hardware tier. Behind each cardboard piece, we placed a push-button switch, and also an extra one outside the SID layers, to act as a control button. They were all connected to an Arduino micro-controller. The push buttons we chose are simple but provide good accessibility, since they make a relatively loud click sound when pressed (an useful cue for the visually impaired), and they are neither too hard nor too soft to press, so even if who is pushing the button cannot hear the click (e.g. a deaf person), they can still feel the press. When a button is pushed, it sends a signal to the micro-controller, allowing us to program responses for each button.

Finally, the Software tier is composed by a web page with the graphical SID representation. The micro-controller communicates with the page so that, when a button is pushed, the page can provide the appropriate response. If one of the five SID layer buttons is pushed, the web page will visually highlight that layer on the graphical representation. It will also provide a text and sound feedback, listing all the stakeholders that were identified for that layer. Otherwise, if the separate control button is pushed, then the web page will ask (through sound and text) for a layer to be chosen, i.e., for one of the other five buttons to be pressed. When one of them is pushed, then the web page will prompt an input, either through voice or text, for a stakeholder to be added to that layer.

<figure><img src="image_3.png" /><figcaption>Fig. 2. To the left, the Aesthetics of the SID digital artifact. To the right, its Hardware.</figcaption></figure>

Figure 2 shows the Aesthetics and the Hardware tiers of our SID prototype. The separation between the tiers allowed us to design the digital artifact in modules, starting from the known part (its looks), and then thinking about the interactions with it based on how the paper artifact has been used. For the next case study, we need to start from the interaction itself, since the ideation methods we chose are not based on specific graphic representations.
## 4.2 BrainWriting Case Study

The main challenge in adapting the BrainWriting technique lies on its very description: it is a silent and written method for generating ideas. In particular, if we consider a visually impaired person, both of these aspects are issues. So the first step was to figure out how to substitute the written input for another form of non-visual input that would not violate the silence requirement. This led to reasoning that the silence is necessary so that people can work in parallel, without affecting each other's ideas. So, for a pilot test, the solution we came up with was to substitute writing for speaking. This, in turn, meant that, at least for a trial run, the creation of ideas would be in series, not in parallel. Hence, the dynamics we did was the following: (1) the problem statement is explained to all participants at the same time; (2) on the first round, one by one participants leave the room and record, in private, their initial idea in a 30-second audio file; (3) on the next round, each participant listens, in private, to the audio with someone else's original idea and records another 30-second audio, complementing that idea; (4) on the subsequent rounds, participants listen to the audio with someone else's original idea, as well as all the audios others recorded to complement that idea. The process stops when all participants have complemented everyone else's ideas.

Of course, the activity had a limitation that does not happen in the original BrainWriting: between recordings, participants had time to reflect while they waited their turn. However, this can be solved if, for instance, participants are not in the same room, and each one uses a device (e.g. smartphone), and records and listens to ideas through the web. Another limitation is low anonymity, since it is easy to recognize someone's voice – perhaps easier than identifying their handwriting. However, this can also be solved by using voice recognition to translate the original audio into text, and then using text-to-speech to read it with a synthesized voice.

At the end of the dynamics, which involved four participants, we had four different ideas, each with four audio files – one from its creator and three from the other participants. Then, the first problem at hand is how to consolidate this data, i.e., how to allow participants to have a general picture of what they accomplished and, together, find the best ideas. Using voice recognition to translate audio into text can also be a solution here as well, since text makes it easier to create a bigger picture than audio. This broader view could be achieved in several ways. One would be creating a tag cloud of words, a graphical representation where the size of the words is proportional to how much they were used in the text. In turn, this cloud of word can also be represented by sound, i.e., the higher its frequency of use, the louder a term is read. Another method would be creating a vocabulary with the most important terms that showed up in the audios. To come up with these terms, it would be necessary either a group discussion or the work of a curator, i.e., someone to look at the ideas and select the most relevant information. This curator could be either a person or a machine.

## 4.3 Extending to Other Artifacts

In the first case study, we were able to take a discussion tool, the SID, and use its graphical representation as a starting point for the design of the digital artifact. The
prototype we built, made with buttons and a micro-controller, is a good template for the other two OS artifacts, the EF and the SF, since they also already have graphical representations. For the design of their tangible digital artifacts, we will need to consider what could be a button, and what actions the buttons trigger. For instance, for the EF, there could be a button for each cell on the table, and for the SF, a button for each step on the ladder. They would each also need a confirmation button, to trigger the action of adding new information. Once we have a prototype for each OS artifact, the next step is to experiment with them in real problem discussions, to see if changes need to be made and how they can be improved.

From the second case study we learned that it is possible to adapt the round-robin dynamics in a way that preserves the individuality of contributions and, at the same time, fosters collaboration. The success in this adaptation encourages us to make similar reflections upon the OS artifacts, and how their use could be redesigned. We also learned that, to strive towards Universal Design, it is necessary for the data to be *transmitted* and *collected* through multiple channels, e.g., sound and text. In our experimentation with the BrainWriting, we only used audio because we were thinking about visual impairments, but if we were to consider, for instance, deafness, another form of input (e.g. text or sign language) would be required.

As for the BrainDraw, how to make “tangible drawings” is still an open question. Considering the round-robin dynamics, in parallel or in series, how could people use tangible materials to propose interface ideas? One solution would be to use LEGO pieces, since they provide an easy way to make well-defined shapes, and the pieces have a good fit with electronic components such as push buttons. Therefore, it would be possible to wire some pieces with electronics, and let participants use them along with normal pieces to build their interface ideas. In addition, we could pre-program some behaviors so that participants could test their ideas in real-time. Also, if a vocabulary is formed during a BrainWriting activity, it could be reused in the BrainDraw. For instance, with the BrainWriting, participants agree “clock” is essential for solving the problem at hand. For the BrainDraw, we provide pieces of interface that contemplate “clock” representations.

# 5 Conclusion

Including people in especial conditions, to the widest possible sense, in the process of designing a product was the challenge we took on. More specifically, in this paper we presented our initial efforts towards the redesign of some traditional techniques and artifacts from this context, to make them accessible to all people. We described the theoretical foundation for our work, the techniques and tools we intend to redesign, and then we presented two preliminary case studies. In the first case study we redesigned a tool for idea discussion – the SID – to become a digital artifact. We began from its graphical representation to create the aesthetics of our prototype, then we designed the interaction along with the hardware and the software. The method we applied to the SID serves as a template to design the digital artifacts of the EF and the SF as well.
In the second case study, we did a pilot experiment with an adaptation of the Brain-Writing technique. This entailed understanding the meanings of its core definitions, and reinterpreting them in ways that make them more accessible, while maintaining their essence. Despite limitations that we can overcome, the dynamics worked and gave us indications as to how to promote discussion and deliberation. In this sense, one of the lessons was that it is necessary to work the data so that it can be available in multiple formats (e.g. sound and text). This facilitates collaboration and, at the same time, is coherent with Design for All.

The efforts we presented in this paper contribute to the practice of Organizational Semiotics and of Participatory Design because they bring the possibility of including in the ideation and discussion processes people who were left out before. We proposed to rethink the techniques and tools used in these areas through the lens of Universal Design, a process for including as many people as possible in the design of a product. Hence, we showed the initial stages of such process, the prototypes and explorations that will open the path for further steps. Future work, then, involves studying what the digital artifacts have to offer in terms of meaning construction, and what are their limitations. We have to see if they maintain the original idea of their written counterparts, or if they have somehow altered the process, especially with a large group of people.

**Acknowledgements.** This work is financially supported by the São Paulo Research Foundation (FAPESP) through grants #2015/16528-0 and #2015/24300-9, by Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) through grant #01-P-04554/2013 and by National Council for Scientific and Technological Development (CNPq) through grants #160911/2015-0 and #306272/2017-2.

## References

1. Baranauskas, M.C.C., Bonacin, R.: Design – Indicating through Signs. Design Issues 24(3), 30–45 (2008), https://doi.org/10.1162/desi.2008.24.3.30

2. Baranauskas, M.C.C., Schimiguel, J., Simoni, C.A., Medeiros, C.B.: Guiding the Process of Requirements Elicitation with a Semiotic-based Approach – A Case Study. In: Proceedings of the 11th International Conference on Human-Computer Inter-action. pp. 100–111 (2005)

3. Buchdid, S.B., Pereira, R., Baranauskas, M.C.C.: Creating an IDTV Application from Inside a TV Company: A situated and Participatory Approach. In: Liu, K., Gulliver, S.R., Li, W., Yu, C. (eds.) Service Science and Knowledge Innovation. pp. 63–73. Springer Berlin Heidelberg, Berlin, Heidelberg (2014)

4. Ishii, H., Ullmer, B.: Tangible bits: Towards seamless interfaces between people, bits and atoms. In: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems. pp. 234–241. ACM (1997)

5. Kyng, M.: Designing for Cooperation: Cooperating in design. Commun. ACM34(12), 65–73 (Dec 1991), http://doi.acm.org/10.1145/125319.125323

6. Liu, K.: Semiotics in Information Systems Engineering. Cambridge University Press, New York, NY, USA (2000)

7. Liu, X.: Employing MEASUR Methods for Process Reengineering in China (6 2001)
8. Muller, M.J., Haslwanter, J.H., Dayton, T.: Chapter 11 - Participatory Practices in the Software Lifecycle. In: Helander, M.G., Landauer, T.K., Prabhu, P.V. (eds.) Handbook of Human-Computer Interaction (Second Edition), pp. 255–297. North-Holland, Amsterdam, second edition edn. (1997)

9. Schuler, D., Namioka, A.: Participatory Design: Principles and Practices. CRC Press (1993)

10. Stamper, R.K.: Information in Business and Administrative Systems. John Wiley & Sons, Inc., New York, NY, USA (1973)

11. Stamper, R.K.: Language and Computer in Organised Behaviour. In: Riet, R.P.V.d., Meersman, R.A. (eds.) Linguistic Instruments in Knowledge Engineering, pp. 143–163. Elsevier Science Inc., Amsterdam (1992)

12. Stephanidis, C., Salvendy, G., Akoumianakis, D., Arnold, A., Bevan, N., Dardailler, D., Emiliani, P.L., Iakovidis, I., Jenkins, P., Karshmer, A., Korn, P., Marcus, A., Murphy, H., Oppermann, C., Stary, C., Tamura, H., Tscheligi, M., Ueda, H., Weber, G., Ziegler, J.: Toward an Information Society for All: HCI Challenges and R&D Recommendations. International Journal of Human-Computer Interaction 11(1), 1–28 (1999), https://doi.org/10.1207/s15327590ijhc1101_1

13. Story, M.F., Mueller, J.L., Mace, R.L.: The Universal Design File: Designing for People of all Ages and Abilities. ERIC (1998)

14. VanGundy, A.B.: Brain writing for new product ideas: An Alternative to Brainstorming. Journal of Consumer Marketing 1(2), 67–74 (1984), https://doi.org/10.1108/eb008097

15. Weiser, M.: The computer for the 21st century. Scientific American 265(3), 94–104(1991)