# Interaction Design as a Multimodal Conversation

Arminda Lopes

► To cite this version:

Arminda Lopes. Interaction Design as a Multimodal Conversation. Second IFIP WG 13.6 Conference on Human Work Interaction Design: Usability in Social, Cultural and Organizational Contexts (HWID), Oct 2009, Pune, India. pp.235-251, 10.1007/978-3-642-11762-6_19. hal-01056239

HAL Id: hal-01056239

https://inria.hal.science/hal-01056239

Submitted on 18 Aug 2014

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Interaction Design as a Multimodal Conversation

Arminda Lopes,

Instituto Politécnico de Castelo Branco

aglopes@ipcb.pt

**Abstract.** This paper explores the concept of design as a multimodal conversation, in the context of observed case studies within interdisciplinary collaboration. Five case studies were video recorded and the verbal and non-verbal designers' behaviours as well as the designers' interactions with the artefacts were analysed. The analysis revealed the designers' experience based on dialogue which was considered a clear paradigm of interaction using words and or images.

**Keywords:** Design, Interdisciplinary collaboration, Behaviours, Artefacts, Dialogue

## 1 Introduction

The evidences presented in this paper are the result of the analysis of five case studies within two network groups of designers, as it will be mentioned.

The research question at hand was: How does dialogue enable us to get a richer view of what is happening in design within multidisciplinary design teams?

However, in this paper, it will be only considered the reflections about the designers' conversations and behaviours: what designers said and how they behaved during the design process, which contributed to partially answer the research question.

The data analysis permitted to understand that design is a facilitator of interactions between humans through objects that have some sort of ability to sense and respond to human input via communication. Design is also about behaviour, the behaviour of artefacts, with how artefacts work and the behaviour of designers interacting with others and with artefacts. Designers interact through verbal and non-verbal behaviours, which is reflected on the produced artefact/artwork, and that will contribute to facilitate the design process and to enhance design quality.

The kind of design practice observed in this study is focussed on what designers experience and the actions that create these experiences, the ideas, emotions, and memories they had. In conjunction with the presented approach a design thinking process was also identified for practical, creative resolution of problems or issues.

Design teams have conversations during the design process. Conversations are multimodal in the sense that communication occur with speech and also with visual media including non-verbal behaviours. During the conversations designers discuss their ideas, and it is apparent they come from different angles – differing levels of experience, differences in age and profession, differences in abilities to deal with technical or non technical tasks, among others. The use of dialogue as a tool
allows ideas exchange, finding connections and meaning, and interests in a real collaborative environment.

Design is actively constructed, with clear step by step progressions. Designers construct themselves in the production of dialogue by speaking and behaving in a certain way which is in accordance with themselves, with those around them and with what they are engaged in doing at the time. Due to both these factors, the way in which they cooperated to generate design was easily observed, helping to reveal a shared sense, and which also displayed the motivation and the enthusiasm they had or had not in the formal and informal environment that surrounded the design process. The dialogue conversations transversed all the process of design but the way the designers communicated was better understood by the way they used language.

Multimedia and multimodal communication is referred in this paper. Multimedia resources (i.e., text, audio, and video) and multimodal resources concerned with multiple modalities such as speech, hand gesture, gaze, facial expression, body posture, drawings, etc. Gesture and speech are integrated in feature structures that express syntactic, semantic and pragmatic features at the utterance and other levels. These comprehend the multimodal signs.

## 2 Interaction Design

Interaction design has been associated with computing and technology, but the focus of interaction design must also be on designing the way people interact with any artefact, be it an object, a system, an environment, or being a consequence of the use of digital technologies or not. The aim is to support either an interaction of a person with the artefact, or an interaction among people that is mediated by the artefact. Much of what is understood about the design of digital artefacts is also applicable to non-digital artefacts.

In recent years human computer interaction (HCI) and related fields, for example computer supported collaborative work, interaction design, and participatory design have produced an increase in interest from focusing on efficiency, functionality and usability, towards an increasing preoccupation in the aspects related to the user experience of technology and digital artefacts [1].

Interaction design as a discipline is a result of its interdisciplinary roots: in industrial design, information architecture, communication design, user-experience design, human factors, usability engineering, and human computer interaction, which all overlap.

Interaction design is inherent in all design, it is a facilitator of interactions between humans through objects that have some sort of ability to sense and respond to human input via communication, either one to one, one to many or many to many. It occurs in the design process, within objects; and with designers serving interaction between people. Interaction design is also about behaviour, the behaviour of objects and services, with how objects and services work and the behaviour of designers interacting with others.
## 3 Multimodal Design Conversations

Conversations are multimodal in various senses, for example, communication can occur not just with speech, but also with visual media including gesture and gaze. When people interact with others, face to face, they are constantly sending and receiving messages through signs, expressions, gestures, postures and vocal expressions. [2] Secondly, different media sets must be used for different communications.

We present here conversation features, verbal and non-verbal behaviours and visual elements conveyed through artefacts which were analysed within the observed case studies and which were considered conversation features.

Verbal and non-verbal behaviour go hand in hand, often simultaneously, and both either together or apart in face-to-face interactions. They have a significant impact on whether or not people achieve their objectives with other people. Verbal behaviours analysis is considered to be the process of breaking behaviour down into smaller elements – sentences or utterances.

The benefits of verbal behaviour analysis are that they give precise information for describing what was going on in the design and dialogue interaction processes: that they are a practicable means of monitoring, in this research, designer's behaviours.

The use of non-verbal behaviour has some advantages. First, it provides extra information, which aids understanding what people are in reality, what they are thinking, feeling or meaning. The other advantage permits us to have a more successful relationship with and understanding of people. However, non-verbal behaviours are easy to observe but difficult to interpret. The problem is deducing a correct meaning from what has been seen, and it cannot be generalised because of the diversity of people's cultures. Non-verbal behaviour can be described as a "relationship language" [3]. This language is the tool through which people, without stating feelings openly, communicate, for example, trust, boredom, submission, dislike and friendship. When decoding non-verbal behaviour, it is important to pay attention to the context, and to the pattern or cluster of verbal and non-verbal behaviours on display.

Artefacts are the designers' outcome. They reflect the designer's experience, skills, conversations, emotions and culture. Artefacts are expressions with different signs on different levels of design language.

Conversation influences the design, in several ways: through the language use, the meaning making, the expression of designers' attitudes and feelings, the process of exchange with purpose, the designers' experiences and ways of thinking, among others.

### 3.1 Dialogue as Conversation

Design as a communication process is a dialogue between all the participants in the design process and elements of design. All designs tell something, through text, image, symbols, or styles. It is a conversation between the designer and people.
Different authors [4], [5], [6] consider dialogue as a conversation: "conversation (...) is dialogue (...). A conversation has no overt goal in terms of the world outside the encounter; it serves simply to allow the participants to develop interpersonal ties, so that it is the relationship of the speakers (...) which becomes the goal of the talk."

Dialogue," I mean a dynamic generative kind of conversation in which there is room for all voices, in which each person is wholly present, and in which there is a two-way exchange and crisscrossing of ideas, thoughts, opinions and feelings" [5].

Dialogue is a conversation in which people think together in relationship; thinking together implies that you no longer take your own position as final [6].

Dialogue is also a multifarious process which encloses an amplitude of human experience including personal values and cultural myths, the nature of emotions, and ways of thinking. [7]

Design is conversation or dialogue in the sense that, for example, both are forms of inquiries; in design thinking there is no judgment; in dialogue suspension of judgments is recommended; design thinking is a creative process based around the building up of ideas; dialogue suggests the sharing of ideas amongst teams – collaborative design; design thinking promotes the effect of cultural and knowledge transfer in the design activity; exchange of knowledge is also an important goal within dialogue.

Our definition of dialogue, in a design context, is: a process of conversation between designers and other participants which is reflected on the produced artefact.

We believe that dialogue is the ideal form of human communication, where the interpretation allows for exchange and adjustment, and for the building and extending of a shared ground. Dialogue is very effective, paraphrasing [6] as a collaborative communication method. It is a process for gaining common understanding and common meaning among individuals in a group [8].

## 4 The Study

This research was conducted within the Leonardo Network group and the White Rose Network for Affective Communication in Consumer Product and Exhibition Design. From the former case studies were considered: Case Study one - Chindogu Challenge (Team1, Team2, Team3); Case Study two - Human Beans - Culture, Creativity and Interaction Design; Case Study three - TIDE - The Integrated Development Environment Art; Case Study four - Threshold. From the latter network one case study was analysed: Case Study five - Human Beans – Affective Communication.

There were about twenty institutions in the former and four in the latter; and about 55 artists and technologists that took part in the study. The methods used to collect data were centred on a qualitative study, a combination of research methods was used to collect the data, literature review, including documentation, records of individual and group's experiences and behaviours, case studies, interviews and observation.
Concerning conversations, data was gathered about the “actual” words of people and their conversations were reproduced to the best of our ability from the transcripts and participant observations. We attempted to preserve the context in which things were said and done.

The data analysis main concern was not just in how utterances were made cohesive, nor in how cohesion was achieved across turns. There was also interest in how interactivity was achieved: that was, what roles speakers took on, how they positioned other participants into particular roles, how turn taking and topic change occurred in contexts where one person was not in control, and the different kinds of feedback strategies that designers used.

To understand what designers said during the design process and in what ways they spoke some questions were considered: Were they interested in listening to the others' opinions or were they imposing their ideas, speaking all at once without allowing the others to speak? Was the conversation sequentially organised according to the design phases or were they mixing subjects along the process?

In pursuing these questions, Conversation Analysis (CA) methods were used:

* The turn taking mechanisms in conversation [9];

* The adjacency pair structure of conversation [10];

* How speakers initiate, shift and close topics, referred to as topic management [11];

* How conversations can keep going indefinitely and continue to make sense.

Verbal and non-verbal behaviours were presented as a complement to dialogue to understand what was said and in what conditions and also how designers were motivated throughout the design process.

## 4.1 Methodologies

Mixed research methodologies were undertaken: Grounded Theory Methodology (GTM), Ethnography, Actor-network Theory (ANT) and Discourse Analysis (DA). Some of their principles were deeply taken into consideration and others were not. The justification to not choose only one methodology was because of the multidisciplinary nature of the research and it was found that all of them could complement each other and give a richer strategy for the research.

The strategy was to employ more than one type of research method. Although a mixed methods research is generally concerned with the combination of quantitative and qualitative methods, in this research the paradigm mixing several qualitative methods was used because the strengths of one fulfil the weakness of other.

In GTM studies data gathering and analysis are tightly interwoven processes; data analysis guides future data collection. Data collection is not considered to be a specific phase that must be completed before analysis begins; after the first collection exercise it is a matter of carrying out the first analysis, finding indicators for particular concepts, expanding concepts into categories and, on the basis of these results, collecting further data. In this mode of procedure, data collection is never completely
excluded, since through the process of coding and memo writing new questions always arise which can only be dealt with if new data is collected or earlier data re-examined.

Ethnographic methods and techniques helped to guide the researcher through the swamp of personal observation and to accurately identify and classify the bewildering variety of events and actions that formed social situations. Ethnographic research has most of the following features which were followed in this research:

* People's behaviour was studied in informal contexts, rather than under conditions created by the researcher;

* Data was gathered from a variety of sources, observations and /or relatively informal conversations;

* The approach to data collection was unstructured, in the sense that it did not involve following through a detailed plan set up at the beginning, nor were the categories used for interpreting what people said and did entirely pre-given or fixed;

* The analysis of the data involved interpretation of the meaning and functions of human actions and mainly took the form of verbal descriptions and explanations.

ANT legitimated the interaction between humans (the designer) and non-humans (technology/artworks). The notion of network enhanced better understanding and aligned the interests of actors and their interaction with a multiplicity of different materials.

DA, using the Conversation Analysis method allowed exploration of the contributions of different designers dialoguing and their different types of working methods.

Data gathering methods included semi structured interviews, observations during meetings and workshops and document analysis. Data was gathered from the initial sample group in a cyclical process: observations; interviews; collection of sketches; more documents; more observations; more visual images such as photography and sketches. Video recordings were used to record observations – both visual aspects and verbal interaction.

The data analysis process was a complex task, especially because of the variety of information and diversity of methods used. Data was collected and dated; descriptions for each case was written; key themes and coding for them were identified; arguments/analysis with supporting evidence in the data/literature was built; transcriptions were made.

The participants were recorded with videotape and for example, hand-written notes were used. Video was used to allow the interaction analysis. The recorded activity was transcribed and involved recorded dialogue, description of non-vocal aspects of interaction, including gestures and body language; and description of interaction between people and artworks.
# 5 Discussion

This section presents a summary analysis of the conversations and behaviours observed within the observed case studies. Concerning non-verbal behaviours, it is not the aim of this study to judge people's behaviours according to the gestures that could occur as a demonstration of personality and also because all of the observed gestures could be misunderstood and not correspond to people's intentions. Some people create impressions of their personality through posture.

Another reason to not judge is based on the time and the environment during which the observations took place. The atmosphere was informal and the time schedule to develop each artefact was very short which was not, probably enough, to understand all the gesture occurrences or other behaviours. The interpretation made will ensure that the meaning of gestures are clarified and not misunderstood along the dialogue established. To avoid reaching untrue conclusions, no judgements will be made.

## 5.1 Conversation Organization

Within conversation analysis, there were some relevant questions to be answered in the description of conversation. The following answers are given in relation to designers because they were the actors on this study. However, these answers could be given by anyone in any conversation. The goal here was to apply conversation analysis rules to design teams' dialogue – a specific type of individual performing specific tasks - to show that when they are producing an artefact, at they behave in the same way as any individual behaves in doing any task.

* Why did the designers speak one at a time? – In conversation there are no pre-set rules, for who talks when or for how long wasn't defined. Designers seemed to respect themselves and they made use of turn taking to take the floor.

* How do designers know when to change turns? – Transitions from one turn to another with no overlap were regular. The current speaker selected the next speaker by addressing a question or speakers self-selected in starting to talk.

* How do designers know when to initiate new topics? – They know because, generally, the last speaker ended with a sentence like: "Okay it's done. Now (..."); or the next speaker introduces the new topic by himself, by using a question or even a declarative sentence.

* How do designers know it is appropriate to interrupt? - Frequently, one speaker talked at a time but it happened that, by the use of gaze or eye contact, and also through some gestures, a speaker interrupted another. In some cases they made overlaps.
* How can a designer complete another speaker's utterance? - Usually they interrupted the other, or waited for their turn to complete the first speaker's utterance.

* How do designers recognise when a speaker wants to close a conversation? – The length of conversation was not specified in advance. When a speaker wanted to close a conversation they said something like: “Well”, “Okay” or direct information “You need to leave”, “We’ve finished”.

Sometimes a speaker positioned another participant into a particular role when he wanted more information to be added and the other was the expert in a specific subject.

In both Human Beans workshops the problem was fully explained to the group. Each group worked separately to generate their own ideas and possible solutions. All of the written or drawn ideas were put down on paper. During the design process they decided on the idea to present and they conversed about how to reach the proposed goal. No one took a significant position or defined role; they worked together as a team. Each idea was presented and clarified to the other groups at a scheduled time. A kind of evaluation was performed by the workshop leaders and other groups. It was understood that designers, in general, enjoyed these tasks and engaged themselves in solving it, as can be interpreted from the case description presented before.

In the Chindogu case studies, they carried out, in some of the cases, a more or less structured approach to accomplish their tasks, but overall they answered the following questions: What do we have? What can we create from these objects? Can we connect these things together? Where does this solution fit into the challenge themes and Chindogu tenets? Why should we or not develop this idea? In all cases, after brainstorming and dialogue, they reached their goal, and a great enthusiasm was found, as the following expressions reveal:

S – “I quite like it!

N – I quite like it as well!

D – I like that!

S - It has got potential!” – Group I

Al – “Wow (...) Yeah (...) that would be really useful (...) to charge up our phones (...) we could communicate while we are in the wild (...)” – Group II

P – I quite like the idea of interacting and I like the idea of cats. (...) There is a lot of potential!” - Group III

In TIDE, although they didn't develop the artwork during the phase of presenting ideas, they also displayed being engaged and enthusiastic in settling ideas for the artwork as observed in the following transcripts:
C - "The great satisfaction, I think, may be we would share that (...) the great satisfaction is producing something that works and that can be an aesthetic work or can be an audience work (...)

S - (...) what do you want?

C - Acclaim, applause!

C - "The project it's faulty, it's incomplete but I think it fulfilled the criteria of the original brief. And in so much that there has been a general collaboration."

Threshold had a different analysis; the only information gathered from it was from interviews. However, reading peoples expressions as opinions or comments, it could be understood that they were happy with the results:

N - "It was successful to a certain extent in that we've got along very well."

C - "I think that is an interesting project and Nadia has collected a lot of data by recording how interaction took place and that will provide raw data for analysis about interaction design and how spaces can be augmented potentially by that kind of interaction technology."

J - "I think that there's lot of potential in this particular installation. I think that I learned a lot about architecture and design by working with these people."

N - "It seems like a great project to me."

Designers adapted their speech according to the person they were talking to and also according to the point behind the speech. The transcript's use of language functions, during conversations helped to understand verbal behaviours variations through the way designers influenced and interacted with the kind of language used and how they made use of associated functions which language fulfils in different situations. There were some dominant functions including especially the referential and emotive. It was found that when one function was accentuated, it tended to diminish the importance of all the others; the emotive function did not refer to emotions but with the conditions of their senders; the referential function was related to the "things" spoken of, [12]; metalingual function was used to establish mutual agreement, for example, through definitions or questions like "What do you mean by?" This function was manifested directly, when asking a question or indirectly, when only the answer was presented.

In considering if the designers used the main dialogue features, the designers presented their assumptions and opinions. The way they carried this out was by stating not by defending against somebody who had another opinion. They contributed their knowledge to the dialogue in progress: it was a process of sharing. Suspension involved attention, listening and looking and it also involved exposing a designer's reactions, impulses, feelings and opinions in such a way that they could be reflected back to the others in the group; Inquiry and reflection was made by asking open-ended questions as a form of exploration of assumptions and beliefs. Reflection provided the opportunity to review and connect with what had been said. Reflection also provided the opportunity to slow down and collect their thoughts. This was done with questions like: "What if..?" or "what does it mean to you?"; Listening - Good listening was both an active and passive skill, it took an effort to really hear and digest
what was being said. One of the goals of a dialogue is to learn by clarifying what people don't understand, and to open their minds to other approaches and perspectives. In so doing, they were uncovering of what was getting in the way of effective communication. The tools they used to accomplish this were: listening actively, asking clarifying questions to make sure they understood, and repeating back what they had heard to confirm that they were interpreting accurately.

## 5.2 Verbal Behaviours

Verbal behaviour has to do with the way people express themselves. Behaviour has nine categories that occurred as steps within the design and dialogue processes [13]. Others considered eight verbal behaviour categories: Seeking ideas; Suggesting/Proposing; Supporting; Seeking Clarification; Disagreeing; Clarifying/Explaining/Informing; Expressing Feelings; Relationships within Group. I considered only four categories: seeking ideas and clarification and added expressing feelings and relationships within groups [14].

During the design process whilst designers were working to produce the artefact and talking, they made use of these categories to dialogue which permitted to observe their structured behaviour.

In some way they had the preoccupation to share knowledge and they behaved in an ideal mood for the collaboration being productive and proficient. Designers had the preoccupation to share not only their knowledge but also their experiences to reach the goal of constructing an artefact and they created propitiate collaborative and communicative environment for that.

**Seeking Ideas** - In this category designers requested facts, or relevant information, or even they asked ideas. They asked more information using declarative or interrogative sentences, as verified, they used especially interrogative sentences. In both TIDE and Chindogu projects, the sentences were longer than in Human Beans case studies. The justification for this, respect to the characteristics of these case studies: people talked during short periods of time being concerned with the goal of producing, at the same time, drawings representing their ideas. Conversely, each task had a short time schedule.

**Suggesting/Proposing** - Suggestions/propositions were given in an explicit form: by using the personal pronoun in the plural making the sense of agreement between participants "we could create a random (...); "we can use (...); "It seems to me that this is viable and we can (...)

"S how about (...)?"; "why we don't do what S suggested?"; or also by using "let's". They were also given in an implicit form, which means that through the content it could be inferred that the speaker was proposing: "Now, we need to get the dreams sequence, we need to see if it works."; "We can use the mouse to interact in a different way". Or finally, they used sentences that content information appealing to a change or modification: "I think the problem is the frustration of not contact the outside world".
Supporting - Supporting was a category easily understood by the use of agreement expressions such as: "Yes, I agree, Right, Yeah, Uhm". Or by the use of adjectives: "remarkable, very, really well" or by the verb: to "like" and finally by using the possessive pronoun: "your".

Seeking Clarification - In this category, demands were expressed almost of the time by using open questions, for example: "How do you do that?"; "What do you think about (...)?" "Do you think (...)?" "Did you think (...)?". Another form of asking clarification was by the phatic expression "Isn't it".

Disagreeing - There were few sentences expressing disagreement: "I don't think."; I'm really reluctant to (...)"; "It's the opposite with me.". Or by the use of adjectives and exclamatory sentences "(...) rubbish ideas!"; "(...) ridiculous!."

Clarifying/Explaining/Informing – In this category it was found many examples and it was quite straightforward to read them and understand that they are exemplificative of each subcategory.

Expressing Feelings – Designers expressed their feelings by using emotional expressions and adjectives: "Uhm, much better."; "very good!"; "beautiful"; "I like that!" or by the content of the sentence: "I got the feeling of a cell (...)"; "We want to be at the beach, and we want to be relaxing."

Relationships within Group – The relationships within the group and the mood of the group was analysed in terms of the formality and informality – the way people treated themselves and expressed their feelings. In general, participants referred to each other using first names and they expressed freely their emotions by laughing and non verbal behaviours. It can be said that those meetings had an informal characteristic and some of the participants already met others in different occasions.

Concerning TIDE case, they also had some relationship before meeting for the project purpose: two elements were husband and wife and they already met with one of the others.

Expressions such as: "folks"; "pals" or sentences informing the group's engagement: "you fell part of a story" – the story was the meeting, the goals and the feelings about the artwork.

In Chindogu case study it was found the mood of the group through expressions like: "Looking into a who we are and what we learned of each other in this 45 minutes (...); or by the use of the personal pronoun "we" all the time they refer to ideas and tasks. The sense of belonging and cohesion was showed: "Can we?"; "We got (...); "were we?".

Each group was formed by different members and they come to play different roles: participants and leaders.

Factors such as knowledge, skills, ability, competence and experience could influence the exact position each member could acquire within the group structure. However, other factors such as personality traits or social factors, for example, being a good communicator or being socially skilled may also had impact.

It was sometimes perceived as dominance, the way someone, not reluctant to communicate openly in front of the others. And in those case studies this personal characteristic was identified by the period of time each member spoke, as it as presented before. Although, according to what was observed, dialogue occurred, in a
free way, between members, and they worked closely together without any individual
prominence.

The observations showed that there wasn't leadership and instead, the leader
behaved in all the cases as a facilitator.

There are some differences between patterns of behaviour in the observed
groups, such happened in TIDE group, which existed over a period of time, near two
years, however, it drawn together, solely, for the purpose of this particular project.
The same happened with the group of Threshold project, although, the characteristics
of the group were different, people met, talked but they split tasks per participants.

For the case studies of both Human Beans projects, members of the group
came together for the first time, and they devoted some energy to getting acquainted
with one another and with the group's task. Even though, they managed very well and
some of the members already knew others. They came to the workshop with some
expectations about it but they had some ideas concerning the meeting's agenda,
except that they didn't know the strategies to follow neither the themes to develop.

It was observed that the cohesiveness of each group was shown by the
informal, and sometimes friendly and relaxed way people behaved and also by the use
of the plural of the personnel pronoun “we” and the way each member supported one
another and the use of informal communication – people addressing each other by
first names.

Other remarks are: in general, not speaking with whole sentences; relaxed
rules of turn taking; interruptions; change in conversation and several people talking
at once; expression of feelings and emotions.

The success of each group was also been measured in terms of how much
work it got done. In all the case studies they reach the goal and they produced an
artwork or a drawing according to the case as it will be analysed in the next section.

From the explanation above it can be argued that design was informed as
dialogue along the design process stages. Designers thought directed towards some
end product the nature of which was communicated to other participants helping to
design it. They grouped together information and they reflected and produced
artefacts or artworks. They made design as a communicative tool and the
communicative quality of the artefacts was explained to others.

They used different behavioural categories in different parts of the design
process by generating a meaningful understanding by the tone, voice, look and feel
they expressed.

5.3 Non-Verbal Behaviours

When using non-verbal behaviours during dialogue, designers were
influenced by them and, whether intentional or not they expressed some emotional
state.

The problem was inferring an accurate meaning from what had been
observed. The possibility of a wrong interpretation could be reduced by staying
focused and not bound to a conclusion based on the observations of isolated pieces of
behaviour. Therefore it was decided to base the interpretation on a number of
different non-verbal behaviours that fitted together into a coherent pattern.
Another aspect to consider when analysing non-verbal behaviours had to do with generalising. It is one of those topics where there are more exceptions than agreements to every generalisation or rule. It is, therefore, misleading to assume that because doodling, for example, signifies boredom in some people it does for all people.

One can recognise somebody else's attitudes and emotions from their non-verbal behaviour [15]. We consider that this is not always possible. One must be careful when judging this non-verbal behaviour because each gesture and expression can not be completely understood even in a specific context. People can behave differently spontaneously or can disguise their true emotions by pretending, forcing a wrong interpretation. However, it is possible to exchange meaning with non-verbal codes as long as verbal and non-verbal components are evaluated together within the context, as it will be argued.

The following non-verbal behaviour categories were considered:

Body posture was understood as the way designers stood, sat or walked or was concerned with hands, arms and legs movements or positions.

Posture was mainly observed in sitting positions by being still, sitting back or sitting on the side of the chair with changes of positions on seats like perching or displaying a sterner posture by sitting up straight and rigidly. One of the teams sat around a table with just one person stood up to write on a board. They sat still and moved, frequently, standing up or walking around the room.

Designers used hands and arms to complement speech almost constantly. A variety of movements were made by hands and arms. Hands were either in a relaxed position, or in pockets, which demonstrated an informal talking form; and hands were used especially to draw in space or to emphasise speech. Hand movements in rapid motions and to draw visual images were used frequently, using their hands to explain visual images, making visual images with their hands. Also simple gestures of placing hands on the face or mouth often occurred. The same happened with arms. They were crossed or folded, sometimes showing a relaxed posture. Participants positioned their arms resting on the table frequently. Legs were crossed and uncrossed all the time.

Bodily behaviours contained information about all the movements made by body parts or observed positions, mainly using the head, face, neck, fingers and feet, such as resting the chin and elbows on their hands and hands sometimes on the table or at the waist. The body behaviours were comprised essentially of nodding movements for agreement, fiddling with pens and doodling, which could be considered as suggesting gestures of tension or boredom, but it was not the case, and touching the self. In the described contexts, it seems that people used those movements to occupy their hands while speaking and drawing or from habit or also as aids to concentration.

Facial expressions performed by lips, mouth, eyebrows etc or to express an emotion were made, and used to complement language, to substitute speech and also to indicate to others that the speaker had finished speaking.

Gestures, such as nodding and pointing were made as were movements such as fiddling, twiddling, playing, tapping and holding.

Gaze and eye contact were constant among participants sometimes, in an abusive way, to signify agreement and interest in what had been said. In general, they
looked at each one when speaking. Gaze and eye contact were used to convey
information about understanding, interest, reinforcement, or agreement. It seems, also,
that a number of personality dimensions were related to gaze; people used it in the
initiation of interaction or at the end of an encounter. There was a regular connection
between talking and looking.

Quality of speech presents information about a variety of speech related
elements, in different ways, such as tone, rate, silences and pauses. The differences
between silence and pauses, in this context, are that a pause of speech could be
speechless but with some sound production, such as nodding, tapping, expressions
like “Uhm”, and silence means no speech at all, no sound or gestures. The main
observations about quality of speech in the observed teams were laughter, the use of
louder voices, overlapping and sometimes, by speaking fairly quickly. However in
some situations a low tone voice and calm way of speaking was used. There were
pauses and silences frequently at the end of each turn taking within some teams
whereas within others, they were almost inexistent. The quality of speech was in some
cases, different from others, mainly through the changes in voice tone. It was
observed that in almost all the case studies people laughed frequently which
demonstrated the enjoyment and engagement during the interaction. In other cases
just smiles were prevalent.

It was also considered interesting to observe proximity in sitting - referring
to the place and closeness between people - proximity dealt with closeness which
suggest more intimacy, although different cultures have different rules, and especially
in HB - Affective Communication, groups were constituted by people from different
countries, and so, some people were more closed with others, for example British
persons, where cultural rules define a relatively distant position. Within some teams it
was observed that participants were seated, and a small amount of space was found
between them, in others they sat side by side and also in front of the others, and also
in some, a participant was in a more prominent position than the others.

In Human communication we cannot avoid using our face, hands and body
while we speak, and in face-to-face conversation people respond to the interlocutor's
words and gestures. Those verbal and non-verbal behaviours, were observed, during
the design process, and reflected in the produced artefacts. In some situations, what
designers said was written. Some of their gestures were drawn on the resulted
outcome or influenced the designers high and low participation on the design process.
Posture, for example, permitted to interpret the designer interest, involvement, among
others as well as the contextual environment. The following figures are examples of
the designers' verbal and non-verbal behaviour's transcriptions.
<figure><img src="image_2.png" /><figcaption>Fig. 1 Designers’ verbal and Fig.2 Designers’ non-verbal behaviours</figcaption></figure>

What verbal and non-verbal behaviours tell us about design as dialogue is how their uses are central to, and constitutive of the ways in which designers conduct their interactions. How each one talked about a topic during the design process reflected certain aspects of the speaker’s attitude towards that topic and that had implications on the ideas being presented and steps to follow to attain the teams’ goals.

Concerning non-verbal behaviours, it is considered that all the behaviours had more to do with the designers’ personality and the influence of the meeting’s contexts. The relation between gestures and some aspects of personality depended on several different procedures: some gestures reflected a prevailing emotional state, or a style of behaviour; designers could control and manipulate their behaviours, and they could even produce the opposite gesture to their true state; some designers gesture style was, partly, a result of his/her cultural and professional, background, age, and sex or even health or fatigue state. Some designers made unconscious movements which could be considered as being closely integrated with the content of the speech itself.

## 5.4 Artefacts

Artefacts reflect the dialogue conversations occurred during dialogue and the design processes. A detailed discussion of the analysis undertaken is out of the scope of this paper. However, some clues are presented to underline that dialogue conversations, really, took place during the design process.

The approach followed in the analysis was a social semiotic and a multimodal social semiotics. The former concerns pictures analysed as a meaningful texts, as social semiotics is interested in deconstructing a text to identify the elements that make up its structure.

The latter, the multimodal social semiotics is concerned with the way people use semiotic 'resources' both to produce communicative artefacts and events and to interpret them in the context of specific social situations and practices [16].
Figures or texts were analysed within this perspective and also following Olesen who consider that to describe a 'thing' we need to describe its qualities and the 'context of the things'. The qualities referred in this context are related to the object properties such as colour, shape, weight and size.

The analysis was an exploratory process involving a visual analysis of a dialogue through a series of drawings that were made in a dialogue context.

Far from being a subjective experience, we consider to be a profound dialogical achievement. During the visual analysis of the designers' interaction, the form of dialogue observed was: vocal and sketching in that they engaged in dialogue via both words and images.

The artefacts/artwork analysis was carried out in different deepening degrees based on the features presented in table 1.

<table><caption>Table 1 - Artefacts Analysis Features</caption><thead><tr><td>Representational Meaning</td><td>Compositional Meaning</td><td>Interactive Meaning</td></tr></thead><tbody><tr><td>Syntax (qualities of the artwork - lines, shapes, colours and textures) and Materials</td><td>Semantic (forms, purpose, and meaning)</td><td>Pragmatic (relationships)</td></tr></tbody></table>

In representational meaning, materials were included since the choice of materials is the vocabulary used to convey message, to allow dialogue. Donald Schon suggests a characterisation of designing as a conversation with materials [17].

Designers' attitudes, emotions and skills were reflected in the designed artefacts through the type of drawings (lines, shapes, colours and textures), the objects' forms and ways of sheet's presentation, the included texts, the use of more or less technology resources, and the reference to each object's interaction.

Two texts are presented as examples of the outcome produced by the designers involved in two of the case studies.

Figure 3 is the final presentation they did. The given story was "Dad waiting in a hospital for five hours with injured six year old boy" and the designers did the representation suggesting the slogan "A waiting room that you won't want to leave" which meant a room with communication facilities: internet connections, telephones, a blue tooth or a bleeper system which would enable those waiting to be able to wander around the site without worrying that they would miss a call for the next phase of the medical process.

Figure 4 presents the design process of the artefact different phases "Remote wild animal interaction device".
<figure><img src="image_3.png" /><figcaption>Fig.3 Dad waiting in a hospital with injured boy</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig.4 Remote Wild Animal Interaction Device</figcaption></figure>

According to the analysed artefacts/artworks, designers are, in general, preoccupied with social interaction solutions.

# 6 Conclusions

In this paper we focus on the conversations that designers had during the design processes. They were in a dialogic form [7], [6]. The focus was to understand how conversation was organised, how designers behaved and a reference to the designers’ outcome, the artefact, was given.

We also proposed a definition of dialogue within design which was identified along the observed case studies: a process of conversation between designers and other participants which is reflected on the produced artefact.

Our main point was that, in practical terms, the use of dialogue conversations in collaborative design practice permit, if designers real use it, that the relationships established along the design process and that the changed and sharing experiences might create an ambience more accurate. For that, all the design participants have a similar level of engagement and then, the ended result, the artefact, may have to be better created and so, better used or interpreted by the final user.

The researcher advice for other researchers to carry out similar endeavours is: in the design process it should be considered four inter-related strands – dialogue, behaviours, artefacts and contextual features (communication, creativity, collaboration and culture) to structure and analyse design as dialogue.

# Acknowledgements

I would like to thank, Professor Antony Bryant and Professor Janet Finlay from Leeds Metropolitan University. I am very grateful to the management team of the Leonardo Network in Culture, Creativity and Interaction Design and to White Rose Network for Affective Communication in Consumer Products and Exhibition
Design. To all those participants in the case studies of this research, that gave me
permission to observe and video record them I present my gratitude.

References

1. McCarthy, John and Wright, Peter, (2004), Technology as Experience, The MIT Press

2. Buchanan David and Huczynski, Andrzej (2004) Organizational Behaviour, An Introductory Text, Fifth Edition, Prentice Hall, England

3. Guirdham, M. (2002), Interactive behaviour at Work, (3rd Edition), Essex: Pearson Education

4. Cheepen, C., (1995), Discourse Considerations in Transcription and Analysis, in eds. G. Leech, G. Myers & J.Thomas, Spoken English on Computer: Transcription, Mark-up and Application, Longman Group, Ltd

5. Anderson, Rob, Baxter, Leslie, Cissna, Kenneth, (2004), Dialogue, Theorizing Differences in Communication Studies, Sage Publications, Thousand Oaks, London, New Delhi

6. Isaacs, William (1999), Dialogue, and the Art of Thinking Together, Published by Doubleday, New York

7. Bohm, David, and Garrett, Peter, (1991) "Dialogue: A Proposal", Unpublished paper

8. Nelson Harold G. and Stolterman, Erik (2003), The Design Way, Intentional Change in an Unpredictable World, Educational Technology Publications, Englewood Cliffs, New Jersey

9. Sacks, Schegloff and Jefferson, Sacks, H. Schegloff E.A. and Jefferson, G. (1974), "A Simplest Systematic for the Organisation of Turn-Taking for Conversation" Language 50: 696-735

10. Schegloff and Sacks, (1973/1974) "Opening up Closing" In Turner, R (ed) Ethnomethodology, Harmondsworth:Penguin

11. Sacks, H. (1992), Lectures on Conversation, Vol II, Cambridge, MA: Blackwell

12. Honey, Peter, (1988), Face to Face Skills, Gower Publishing Company Limited, England, USA

13. Jakobson, Roman (1960), Essais de Linguistique Générale, vol. 1, Paris, ed. de Minuit

14. Lopes, A. (2009), Design as Dialogue: Encouraging and Facilitating Interdisciplinary Collaboration, VDM Verlag Dr. Müller

15. Pease, Allan and Garner, Alan (1985), Talk Language, Simon and Schuster, London

16. Van Leeuwen, T. (2005), Introducing Social Semiotics, London: Routledge

17. Schon, D. (1995), The Reflective Practitioner: How professionals think in action, Arena, Alershot, UK