<figure><img src="image_1.png" /></figure>

HAL
open science

# Assessing Short-Term Human-Robot Interaction in Public Space

Jakub Zlotowski

<figure><img src="image_2.png" /></figure>

## To cite this version:

Jakub Zlotowski. Assessing Short-Term Human-Robot Interaction in Public Space. 13th International Conference on Human-Computer Interaction (INTERACT), Sep 2011, Lisbon, Portugal. pp.370-373, 10.1007/978-3-642-23768-3_35. hal-01596999

HAL Id: hal-01596999

https://inria.hal.science/hal-01596999

Submitted on 28 Sep 2017

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Assessing Short-Term Human-Robot Interaction in Public Space¹²

Jakub Złotowski

HCI and Usability Unit, ICT&S Center, University of Salzburg
Sigmund-Haffner-Gasse 18, 5020 Salzburg, Austria
{jakub.zlotowski, manfred.tscheligi, astrid.weiss}@sbg.ac.at

**Abstract.** This thesis discusses an evaluation approach for assessing social acceptance of short-term HRI in public space with special emphasis on robots seeking information from pedestrians.

**Keywords:** Human-Robot Interaction, Evaluation Methodology, Measures and Metrics, Social Acceptance.

## 1 Context and Motivation

In recent years the research on Human-Robot Interaction (HRI) started moving from highly-controlled laboratory experiments to field studies in natural human environments. However, despite increased interest in spontaneous and time limited (short-term) HRI, due to lack of common benchmarks and theoretical concepts being evaluated by researchers, it is not possible to compare the results of these studies [4]. Moreover, the context of public space is challenging for conducting reliable studies, as new methods are needed in order to address the problem of less controlled environment compared to laboratory studies [2, 9].

There is little doubt that successful integration of robots in human environments can accelerate the speed of further development of that technology. High usability of robots is required so that people will be eager to use them. Furthermore, there are also two other key aspects which must be addressed in order to accelerate that process: social acceptance and user experience. HRI, as a relatively young discipline, suffers from lack of reproducibility of studies and comparability of results since currently they are often platform specific. However, Bartneck [1] pointed out that the field reached the stage when it is necessary to conduct studies in a consistent manner in order for it to progress further.

The work presented in this paper was developed and is being evaluated as part of the EU funded FP7 project Interactive Urban Robot (IURO). The scenario explored in this project involves development of a robot that is capable of navigating in densely populated human environments using only information obtained from encountered

<footnote n="1">¹ Supervisor: Manfred Tscheligi</footnote>

<footnote n="2">² Advisor: Astrid Weiss</footnote>
humans. As an alternative to GPS enabled and Internet access provided robots, the IURO must find its way to a destination located in another part of a city by asking pedestrians for directions. A robotic system must be socially accepted so that it receives help from strangers. Therefore, it is necessary to create a platform independent set of benchmarks and methods suitable for evaluation of robots which are interacting with bystanders.

## 2 Background and Related Work

Human-Computer Interaction (HCI) and HRI have provided several evaluation methods to assess various aspects of human interaction with social robots. However, there has been relatively little work done which aims at providing a holistic evaluation approach for short-term HRI. Weiss [12] proposed an evaluation framework for human-robot collaboration that included four main factors: Usability, Social acceptance, User experience and Societal impact (USUS). Each of these factors was then operationalized by several indicators, which were linked with suitable research methods for addressing them.

In many aspects HRI in public space differs from interaction in other areas investigated in the field. Usually such interaction is only short-term and people spend limited time with a robot [10]. Therefore, their ability to adapt to a robot is reduced. Furthermore, since interaction in public space is spontaneous, there is no extensive preparation on a person's side, such as reading manual. As a result using natural human interaction channels is critical for robots operating in public space [10]. In addition, specific aspects of this context require adapting HRI evaluation frameworks and providing new methods for addressing them during studies in natural human environments. As Weiss [11] noted the USUS evaluation framework needs to be adjusted for different contexts of HRI.

Human beings are a reach source of information in rapidly changing environments, such as public space. Since modern robots do not have human capabilities yet, and even if they had, it will be often necessary for a robot to approach strangers in order to obtain information required for successful accomplishment of various tasks. This will be only possible if people accept robots as social actors.

There is strong evidence under the Computers are Social Actors paradigm that human interaction with technology is basically social, e.g. [3, 7]. Furthermore, compared with other interactive technologies humanoid robots encourage more anthropomorphic mental models [5]. Therefore, it seems appropriate to expect that people providing help to robots will base their decision on similar criteria as when other human requests help. Latané and Darley [6] proposed the Five Stage Model of Helping, which a bystander goes through before help is provided for a person in need. Each stage represents an important decision and if at any point the outcome is negative, it will result in no help being provided. The stages proposed by them are:

1. Noticing the situation

2. Labeling the situation correctly as emergency

3. Assuming responsibility for helping
4. Deciding how to help

5. Implementing the decision

Furthermore, they identified factors, which can influence the decision at various stages. The model begins before the interaction is initiated (pre-beginning sequence [8]) and ends when help is being provided. Therefore, if pedestrians' mental model will be similar for human strangers and robots alike, it would be possible for robots to impact people's decision for providing help.

## 3 Research Hypothesis and Methods

My assumption is that if the Five Stage Model of Helping is also applicable for HRI it should be possible to increase a robot's chances of receiving help from strangers by addressing critical benchmarks on each of the stages. Therefore, the main goal of my dissertation is to create an evaluation approach for assessing social acceptance of short-term HRI in public space with special emphasis on robots seeking information from pedestrians. It should constitute of two parts: theoretical and methodological.

The theoretical part of the evaluation approach is based on the described above Five Stage Model of Helping [6]. However, compared with a human being, usability of a robotic system can affect whether help will be provided completely and successfully. Hence, an additional sixth stage (Process of helping) has been added to this model. This extended framework called the MOdel of HElping Robots (MOHER) should allow evaluation on all stages from noticing the situation until a person finishes providing help to a robot. Robot benchmarks, which are important in order to increase probability of bystander passing to the next stage, are assigned to each stage of the MOHER. There are multiple indicators identified at each stage and the same benchmark can be important at various stages.

The USUS evaluation framework [11] is used as the starting point for identifying social acceptance indicators. Only benchmarks which are meaningful in the new context are used and some of them had to be redefined. Furthermore, based on broad literature review from Psychology, HCI and HRI new indicators were added. However, it is not possible to obtain full picture of short-term HRI without analyzing the relevance of two other factors: usability and user experience. Thus, they are also incorporated in the MOHER.

The methodological part provides means for the evaluation of the MOHER. An evaluation plan has been created and each stage of the model is explored in a dedicated case study. The methods used include qualitative and quantitative studies, such as: field trial, heuristic evaluation or focus troupe. In addition to existing HRI practices, various methods from HCI and Ubiquitous computing, such as Co-discovery or Bluetooth device scanning, will be utilized in order to address the theoretical concepts. The first video-based case study on approach trajectories with 30 participants has just been conducted and the results are promising for feasibility of the proposed evaluation approach. Moreover, SoAc and UX questionnaires developed by Weiss [11] will be adjusted and validated in the IURO project's scenario in order to learn more about HRI in public space.
## 4 Expected Contributions

Based on these 6 case studies, guidelines, which will indicate what methods are suitable for different stages and what benchmarks should be evaluated, will be created for fellow researchers. As a result, it is believed that the evaluation process of short-term HRI in public space will be simplified and identification of the stage at which a robot's performance has to be improved will be aided. The PhD research should also improve reproducibility of studies as the same benchmarks can be used for various studies and the results could be compared.

As an exploratory work, the conducted case studies will not be able to guarantee holistic evaluation of the concept. However, identified stages, benchmarks and validated methods for evaluating them, together with the guidelines for other evaluators can have positive impact on HRI design process.

## References

1. Bartneck, C., Kulić, D., Croft, E., Zoghbi, S.: Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. International Journal of Social Robotics 1, pp. 71–81 (2009)

2. Dix, A., Finlay, J., Abowd, G.: Human-computer interaction. Prentice hall (2004)

3. Fogg, B., Nass, C.: How users reciprocate to computers: an experiment that demonstrates behavior change. In: CHI '97 extended abstracts on Human factors in computing systems: looking to the future. CHI EA '97, pp. 331–332. ACM, New York, NY, USA (1997)

4. Fong, T., Nourbakhsh, I., Dautenhahn, K.: A survey of socially interactive robots. Robotics and Autonomous Systems 42(3-4), pp. 143–166 (2003)

5. Kiesler, S., Hinds, P.: Introduction to this special issue on human-robot interaction. Hum.-Comput. Interact. 19, pp. 1–8 (2004)

6. Latané, B., Darley, J.: The unresponsive bystander: why doesn't he help? Prentice Hall (1970)

7. Lee, E.J., Nass, C., Brave, S.: Can computer-generated speech have gender?: an experimental test of gender stereotype. In: CHI '00 extended abstracts on Human factors in computing systems. CHI EA '00, pp. 289–290. ACM, New York, NY, USA (2000)

8. Schegloff, E.: On the organization of sequences as a source of "coherence" in talk-in-interaction. Conversational organization and its development 38, pp. 51–77 (1990)

9. Tanaka, F., Movellan, J.R., Fortenberry, B., Aisaka, K.: Daily hri evaluation at a classroom environment: reports from dance interaction experiments. In: Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction. HRI '06, pp. 3–9. ACM, New York, NY, USA (2006)

10. Thrun, S., Schulte, J., Rosenberg, C.: Interaction with mobile robots in public places. IEEE Intelligent Systems, pp. 7–11 (2000)

11. Weiss, A.: Validation of an evaluation framework for human-robot interaction. The impact of usability, social acceptance, user experience, and societal impact on collaboration with humanoid robots. Ph.D. thesis, University of Salzburg (2010)

12. Weiss, A., Bernhaupt, R., Lankes, M., Tscheligi, M.: The USUS evaluation framework for human-robot interaction. In: AISB2009: proceedings of the symposium on new frontiers in human-robot interaction. Citeseer (2009)