# Identifying the Interplay of Design Artifacts and Decisions in Practice: A Case Study

Judy Bowen, Anke Dittmar

► To cite this version:

Judy Bowen, Anke Dittmar. Identifying the Interplay of Design Artifacts and Decisions in Practice: A Case Study. 16th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2017, Bombay, India. pp.503-512, 10.1007/978-3-319-67744-6_31. hal-01676179

HAL Id: hal-01676179

https://inria.hal.science/hal-01676179

Submitted on 5 Jan 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Identifying the Interplay of Design Artifacts and Decisions in Practice: A Case Study

Judy Bowen<fnref n="1" /> and Anke Dittmar<fnref n="2" />

**Abstract.** Interaction design is a complex and challenging process. It encompasses skills and knowledge from design in general as well as from HCI and software design in particular. In order to find better ways to support interaction design and propose methods and tools to further the research in this area we must first better understand the nature of interaction design in practice. In this paper we present two small case studies which attempt to analyse design and decision-making through the lens of one particular theoretical framework. The framework seeks to focus design activities via its artifacts and the design spaces that exist in order to support reasoning about the process and the evolution of the artifacts. Our case studies show that we can use such a framework to consider real-world design projects, and also that there are further considerations that might usefully be included in such a framework.

**Keywords:** Design, design artifacts, case-study

## 1 Introduction

Interaction design benefits from multidisciplinary collaboration, bringing in knowledge and experiences from related disciplines such as design theory and practice, human-computer interaction, and software design and development. At the same time, the challenges and complexity of interaction design are heightened, for example, by these different design practices. Understanding and reasoning about interaction design is, therefore, also challenging, but it is important as we seek to ensure that the interactive systems being built are usable, correct and relevant in the current world of ubiquity and increasing use of technological solutions.

In other work, Bowen and Dittmar proposed a framework which introduces the concept of complex design spaces to describe multidisciplinary design work [4]. The framework is based on an understanding of design, and in particular interaction design, from the literature and incorporates ideas from design theory, traditional HCI practices such as user-centred design, and software engineering practices such as requirements engineering and refinement. Design is basically considered to be a process of constructing, using, discarding, and refining design artifacts or resources.

In this paper we start to investigate the relevance of the proposed theoretical framework as an instrument for empirical studies that helps to deepen our understanding of

<footnote n="1">¹ University of Waikato, New Zealand
jbowen@waikato.ac.nz</footnote>

<footnote n="2">² University of Rostock, Germany
anke.dittmar@uni-rostock.de</footnote>
real-world design activities. Two small case studies of design in practice have been conducted. We report on the results of these case-studies and discuss how they can be viewed within the proposed framework. We also report on aspects identified which suggest the framework may be extended to consider additional factors.

## 2 Background on Studying Interaction Design Practices

Interaction designers are faced with complex design problems, often characterized as ‘wicked problems’ [13]. Goodman et al. [7] point out that studying the complexity of interaction design processes is challenging and “needs a diverse set of research methods, each bringing complementary aspects and perspectives to an overall understanding”. The actual use of particular design methods is a frequently investigated subject in empirical studies. For example, Vredenburg et al. [14] used the survey method to get an overview about which user-centred design methods are applied by practitioners, or the interview study in [11] investigates how the persona method is integrated into existing design practices. Generally, interview-based approaches allow experienced designers to reflect upon particular aspects of their practice either individually or in groups [15]. Observational techniques have been applied to study design meetings of teams working on both artificial [1, 6] and real-world problems [12]. Olson et al.’s study with two companies is related to interactive software development in general but is interesting because of its application of design rationale concepts to analyse the designers’ discussion.

Goodman et al. [7] argue that HCI research has influenced most interaction design studies. While theoretical approaches such as activity theory [3, 10] or technology as experience [9] have shaped empirical studies of technology use “there has been little theorizing of interaction design practices within HCI” [7]. Our work centres on the concept of a design space as it is used by Buxton [5] and others. All stages of problem setting and solving can be represented within such spaces which represent an iterative generation of ideas and a gradual convergence or refinement towards solutions [5].

In the work of [4], a design space can be hierarchically decomposed into sub-spaces. Every such (sub-)space describes the ‘moves’ of a design (sub-)team. It has an entry and exit point and is populated by design artifacts. In this context, all external design representations such as prototypes, scenarios, behavioural specifications, or the final interactive system are considered to be design artifacts. Designers are provided via the entry point with some initial artifacts representing requirements, design constraints and resources. Their activities result in the creation, modification, use or discarding of design artifacts within their design (sub-)space and they provide via the exit point some of their products to other sub-teams.

The case studies presented in this paper are guided by the above framework. However, the analysis is focussed on tracking the creation, modification, discarding and provision of design artifacts at a macro level. It is less focussed on a detailed content analysis of design artifacts and how designers relate design artifacts at a micro level.
# 3 Case Studies

Two case studies were conducted. The first was with a commercial web design company who were undertaking the redesign of a web site for an optometrist, and the second was with a group of computer science and graphic design students undertaking a pre-defined (artificial) design project. The motivation for the studies was to investigate design in real-world situations with two goals:

(i) to identify the applicability of the framework described in [4],

(ii) to identify real-world design practices and see if additional considerations were needed in the framework (above).

The studies focussed on tracking the design process by direct observation of the teams involved and by identification and categorisation of the design artifacts used.

## 3.1 Case Study 1

The first case study was conducted with a locally based commercial web development company. They were undertaking a 6 month project to update the website of an optometrist. There were 6 team members from the company involved in the project: a project manager, a data analyser, a designer, two developers and a content manager. The project manager and clients were based in the same city, while all other team members worked from an office in a different city located 160km away.

The study was conducted as an observational study by a single researcher, supported by note-taking, audio recordings and access to all design artifacts. The main factors identified and recorded were when decisions were taken and when design artifacts were created, amended or accepted. Ethical consent was obtained to perform the study with permissions from the design company and client to access all materials required and report findings in an anonymised fashion. Design meetings and discussions were conducted by way of face-to-face meetings, online meetings and email discussion. Bespoke online tools are used by the company which enable all team members to collaborate. Analysis of the materials gathered during the process led to the categorisation of the key elements as follows:

- any concrete materials or reports produced were categorised as 'Design artifacts' and labelled DA_1 .. DA_n

- specific decisions that were made following discussions, reviews or choices were categorised as 'Decision Points' and labelled DP_1.. DP_n

- factors affecting decisions were categorised as 'Constraints' and labelled C_1 .. C_n

The design process began with the data analyst preparing background material from an investigation into the client's industry and similar companies. An online report template was used to capture all of the information from this (DA_1). The project manager reviewed DA_1 and an online meeting was conducted between the project manager and data analyst where this was reviewed, feedback was provided and two decisions were made (DP_1, DP_2). A further report was prepared during this meeting which detailed recommendations for the next phase of the project (DA_2). A meeting was held
between the project manager and the client where DA_2 was discussed and feedback
provided by the client. Based on this DA_2 was updated (DP_3) and a meeting of the
co-located design team was held where this was used as the basis for the creation of a
site map (DA_3). DA_3 was emailed to the client who returned it with some changes
(DP_4) and the site map was updated to reflect these (DA_4). The project manager held
an online meeting with the design team leader and provided DA_4 along with a brief for
the design (DA_5). Following discussion of these, specific decisions were taken in line
with the client's corporate design requirements (colour schemes, fonts etc) which acted
as constraints on the design (C_1). Following this meeting the design team met and
produced a series of sketches (DA_6) in a collaborative design process. At the end of
the meeting these were approved by the project manager (DP_5). One of the designers
then produced a wireframe (DA_7) which was sent to the project manager for approval
(DP_6) and then on to the client. The wireframe contained a number of alternatives
from which the client made selections (DP_7) and which were then incorporated into
the final wireframe and prototype (DA_8).

In addition to the explicit constraint described (C_1) the researcher also observed
a number of implicit constraints. The project manager had a defined timescale to work
to, which meant that no additional functions over and above the initial requirements
were ever offered to the client (C_2). This had a direct effect on some of the design
decisions taken (DP_3 and DP_5). The design company had access to several different
technologies that were used across a number of their client solutions. All new solutions
had to adhere to these existing technologies and no solutions or functions could be
offered which would require additional technology. This had an over-arching effect on
the whole project as it acted as an implicit requirement that could not be broken. The
final implicit constraint (C_3) was due to organisational culture which meant that the
developers and designers would always agree with the project manager irrespective
of the decisions made. Figure 1 summarises the interplay of design artifacts, decision
points and constraints identified in the first case-study.

## 3.2 Case Study 2

The second study consisted of an artificial design project created for the purpose of
the study. Four undergraduate students took part, and were given the task of creating a
mobile application to monitor and save battery life on a mobile phone. The participants
were all students with no commercial experience, two were computer science students
in the fourth year of their studies, while the other two were graphic design students in
the third year of their studies. They did not know each other prior to the study.

As for the first case study, this was conducted as an observational study by a single
researcher, supported by note-taking, audio recordings and access to all design artifacts.
Ethical consent was obtained to perform the study. The same categorisation of key
elements occurred as in study 1, but the time over which the study was conducted was
limited to four meetings, each of one hour in length. All four participants took part in
all four meetings.

In the first meeting the participants discussed the design problem, brainstormed their ideas and noted down what they decided were the most important factors (DA_1). They also searched online for ideas of existing applications and any online resources that
<figure><img src="image_2.png" /><figcaption>Fig. 1. Case Study 1</figcaption></figure>

they felt could be helpful. They extended their initial notes with useful features from existing apps (DA_2). Following a discussion around these materials three personas were developed along with associated scenarios (DA_3).

The second meeting started with a review of the personas and scenarios created in the previous meeting and a feature list was created (DA_4) from DA_2. This was subsequently prioritised (DP_1) into high and low importance. Between the second and third meetings the participants worked individually on sketches of initial design ideas (DA_5) and brought them along to the third meeting. Following discussions one of the sketches was selected as base sketch to work from (DP_2) and some of the features on the list DA_4 were removed as not being necessary (DP_3).

In the final meeting, the aim was to come up with a final design. The desired elements were reviewed and then a layout was created (DA_6). It was observed that one of the students was designated for drawing the design and the others gave suggestions and comments. Initially the participants wanted all the desired elements to show up on the homepage but this would have resulted in a cluttered look. At this point, they went back to reviewing existing related apps and websites and based on existing different designs, managed to create their final design sketch (DP_4, DA_7). As was seen in study 1, constraints existed which had an effect on the decision-making process. The most evident constraint was that of time (C_1), towards the end of each hourly meeting there was an obvious pressure to achieve something which led to ideas being accepted or discarded hurriedly in order to reach a resolution. This particularly affected DP_3 and was directly responsible for DP_4 which led to a final result more based on the review of existing solutions than all of the previous work undertaken. The second constraint was the skill-level of the participants (C_2) which meant that they looked at superficial aspects of the design only (no discussions of technical aspects) and having created the personas and scenarios, DA_3, they used these only in the creation of feature list DA_4,
<figure><img src="image_3.png" /><figcaption>Fig. 2. Case Study 2</figcaption></figure>

but otherwise they never made use of them again. Figure 2 shows the interplay of design artifacts, decision points and constraints of the second case study.

Although we have identified the students’ skill level as a constraint, based on their inability to incorporate the personas they developed into subsequent development activities, there are other possible causes that could lead to similar decision-making. A longer term (12 week) study of design teams conducted by Blomquist and Arvola [2] found a similar problem with incorporating personas into the design process and it therefore may not be just the skill-level of the student participants which led to this. We now consider such aspects further in the discussion of our results.

# 4 Discussion

To consider the two design processes in light of the framework we first create design space diagrams which match the definition given in [4]. Each design space is represented by a box with input shown at the left-hand side and outputs on the right. Within the design space are the local design artefacts and decisions. Figure 3 shows the design spaces of case study 1. The numbered artefacts correspond to those described earlier, e.g. DA_1 is the online report of background material. There is a defined ordering, represented by the number of the spaces from DS_1 to DS_4, which is based on the passing of design artifacts from one space to the next. Arrows between artefacts imply a direct relationship, so either an artefact has been 'refined' into something more concrete, or has been used as the basis for a further artefact. The groupings within design spaces, however, may incorporate multiple meetings or discussions. In the final design space, DS_4 there are several options (or alternatives) offered to the client via the entry point (denoted by DA_7alt.i, DA_7alt.ii...) and they make a selection which is then used to inform the final design, DA_8. In the original framework [4] a distinction is made between alternatives and variants, with alternatives being either/or choices between particular options and variants being different ways of enabling the same thing, which may even co-exist in the final implementation to provide a user choice. The identification of
the alternatives from the decision-making process in DS_4 is made possible by the use
of the framework here, however no variants were identified in either study.

In figure 1 we have clearly identified where decisions were made (by way of the
decision points) and also shown how the constraints identified affected particular parts
of the design and decision-making. In the original framework decision-making is rep-
resented as just another artifact in a design space so we can still see that there is a
relationship between a decision-making process and an design but it is not as explicit.
In fact the design spaces of the framework can be considered an abstraction of the in-
formation shown in figures 1 and 2. If we were to ‘zoom in’ on any of the individual
design spaces, or design artifacts then we might imagine we would see something more
akin to these figures.

<figure><img src="image_4.png" /><figcaption>Fig. 3. Case Study 1 Design Space</figcaption></figure>

Figure 4 shows the design sub-spaces of case study 2 which can be combined to create the overall design space. The initial problem brief is the input to the first design space, and the idea generations are included as well as the previously defined design artifacts. This reflects the fact that within the framework everything (designs, ideas, considerations etc.) is considered an artifact. There are 5 sub-spaces, DS_1, DS_2, DS_4 and DS_5 represent the four meetings and DS_3 represents the work done at home by each of the team members to produce design sketches which are combined into DA_5. DS_3 is described as a collection of sub-spaces which are closed, this reflects the fact that the sketching was done at home by the participants and therefore not observed.

The design spaces described are ordered 1..5 as there was a single team working
on the entire process. The inner design spaces of DS_3 are unordered, however as we
can consider them to have occurred in tandem. We do not explicitly consider temporal
<figure><img src="image_5.png" /><figcaption>**Fig. 4.** Case Study 2 Design Space</figcaption></figure>

properties or time beyond simple ordering. In general, therefore, we see a process that is fairly linear, but this is primarily an artifact of the constrained process that was set up for the student group (with defined meetings for all team members) rather than a reflection of a typical design process.

While the detail of the figures 1 and 2 suggest that the framework can not capture all of the detail and subtlety that occurs, the design spaces shown in figs 3 and 4 can be viewed as a suitable abstraction of these. As such we are able to consider each of the case-studies in light of the framework although with some elements absent from each. For example the framework does not directly consider constraints (although these may be ‘hidden’ with decision-making processes represented in the framework as QOC diagrams which explicitly consider relationships between questions, options and criteria [8]), and there are aspects within the framework, such as the use of variants, which were not seen in the case studies.

# 5 Conclusions

In this paper we have presented two small case studies designed to investigate design practices. We analysed the studies in light of a proposed framework for considering design as proposed in [4]. From the studies we were able to identify design entities that were not included in the original framework (namely implicit and explicit design constraints). We were also able to view the framework in practice and see how it enables us to identify where design decisions are made, and in light of constraints consider what might have led to them.

These small initial studies suggest that we can make use of such a framework to consider design practices, however there are further factors that need to be taken into account before we can propose this as a suitable mechanism for any design project. The case studies were necessarily small to fit with these initial investigations, the first step for any future work should be to apply the same process across a longer and larger
design project. As the complexity of multi-team interactions increases it will be useful
to see how well the concept of design spaces and the identification of artifacts supports
a fuller understanding of the history of the process once it is complete.

References

1. Baker, A., van der Hoek, A.: Ideas, subjects, and cycles as lenses for understanding the software design process. Design Studies 31(6), 590 – 613 (2010), special Issue Studying Professional Software Design

2. Blomquist, A., Arvola, M.: Personas in action: Ethnography in an interaction design team. In: Proceedings of the Second Nordic Conference on Human-computer Interaction. pp. 197–200. NordiCHI ’02, ACM, New York, NY, USA (2002)

3. Bødker, S.: A human activity approach to user interfaces. Hum.-Comput. Interact. 4(3), 171–195 (Sep 1989)

4. Bowen, J., Dittmar, A.: A semi-formal framework for describing interaction design spaces. In: Proceedings of the 8th ACM SIGCHI Symposium on Engineering Interactive Computing Systems, EICS 2016, Brussels, Belgium, June 21-24, 2016. pp. 229–238 (2016)

5. Buxton, B.: Sketching User Experiences : Getting the design right and the right design. M. Kaufmann (2007)

6. Christiaans, H., Almendra, R.A.: Accessing decision-making in software design. Design Studies 31(6), 641 – 662 (2010), special Issue Studying Professional Software Design

7. Goodman, E., Stolterman, E., Wakkary, R.: Understanding Interaction Design Practices. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 1061–1070. CHI ’11, ACM (2011)

8. MacLean, A., Young, R., Bellotti, V., Moran, T.: Questions, Options, and Criteria: Elements of design space analysis. Human Computer Interaction 6(3), 201–250 (1991)

9. McCarthy, J., Wright, P.: Technology as experience. interactions 11(5), 42–43 (Sep 2004)

10. Nardi, B.A. (ed.): Context and Consciousness: Activity Theory and Human-computer Inter-action. Massachusetts Institute of Technology, Cambridge, MA, USA (1995)

11. Nielsen, L., Storgaard Hansen, K.: Personas is applicable: A study on the use of personas in denmark. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 1665–1674. CHI ’14, ACM (2014)

12. Olson, G.M., Olson, J.S., Carter, M.R., Storrøsten, M.: Small Group Design Meetings: An Analysis of Collaboration. Hum.-Comp. Interaction 7(4), 347–374 (1992)

13. Stolterman, E.: The Nature of Design Practice and Implications for Interaction Design Research. International Journal of Design 2(1), 55–65 (2008)

14. Vredenburg, K., Mao, J.Y., Smith, P.W., Carey, T.: A survey of user-centered design practice. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 471–478. CHI ’02, ACM (2002)

15. Zhao, M.: Seek it or let it come: How designers achieve inspirations. In: CHI ’13 Extended Abstracts on Human Factors in Computing Systems. pp. 2779–2784. CHI EA ’13, ACM, New York, NY, USA (2013)