Results Of The User Studies And Requirements On "Individual Reflection At Work"
Creators: name: Pammer-Schindler, Viktoria
name: Fessl, Angela
name: Wessel, Daniel
name: Krogstie, Birgit
Publisher: name: Zenodo
Year: 2011
Subjects: reflective learning in the workplace
designing for reflection
technology-enhanced learning
human-computer interaction
workplace learning
user studies

description: Within this deliverable, we (i) show evidence that supports the assumption that people at selected companies (called testbeds)  learn by reflection and by observing others, (ii) identify and categorise concrete situations in which people reflect, what they reflect upon, and what they learn from reflection, (iii) identify categories of triggers for reflection, (iv) collect existing work tools and data as well as reflective practices at selected companies  as connecting points into which future reflection Apps can hook into, and (v) discuss potential technological support for reflection.

<strong>Assumptions and Research Questions</strong>

In particular, we investigated the following assumptions and research question based on data collected at MIRROR testbeds:


	Assumption 1: People reflect upon their work
	Assumption 2: People learn by observing others at work
	Research Question 1a: When and how does individual reflection about work and learning practices occur at work?
	Research Question 1b: Are there any typical triggers for reflection?
	Research Question 2: Is individual reflection currently aided by tools?
	Research Question 3: Which kind of technological support could be used to support reflective learning at work?


Method

In order to verify the above assumptions and answer the research questions, we designed research instruments that would help us elicit answers from the target users in the MIRROR testbeds. Tools that we used were a reflection diary, a reflection questionnaire, a job description questionnaire and a focus group on “Technology Support for Learning by Reflection at Work”. Tool design here means that we formulated the precise questions, and prepared mock-ups and discussion inputs. These tools can of course also be used in the future to investigate other workplaces with regard to reflection.

<strong>Analysis and Results</strong>

We support Assumptions 1 and 2 with a descriptive analysis of the returned reflection questionnaires, i.e. we find that on average employees of NBN agree with the statement that they reflect upon their work to a degree of 3.7 (where 1 denotes no agreement and 5 strong<br>
agreement). We also support Assumptions 1 and 2 with reflection stories, i.e. real concrete stories about everyday work situations of user study participants in which reflection happens and direct quotes from user study participants that indicate that learning. An exemplary reflection story illustrates both the format of a reflection story, and reflection at a hospital:<br>
 

<em>The participants did a simulation of reanimation. This was video-recorded from three perspectives. “We have observed others and then ourselves. Also in order to compare how I felt with how it looks. I had thought during the reanimation that I was very slow, and then I realized that I was nearly too fast. […] The perception was very different, this has confirmed me in not needing to become even faster.” (Interview)</em>

<br>
In order to answer Research Question 1a systematically, we developed the following classification scheme. This scheme was developed based on the existing theory of reflection, ongoing theoretical work within the MIRROR project, as well as existing user study data.

<br>
Characteristics of a Reflection Situation


	At Work, Outside Work (Where, when)
	Individual, Supervised, Collaborative (With whom).


Content Characteristics


	Single Experience, Set of Experiences
	Activity, Interaction with Colleagues, Interaction with Patient or Client, Own Reaction (Topics of Reflection and Observation)
	Individual Experience, Vicarious Experience, Shared Experience. (Ownership of Experience)


Outcome Characteristics


	Individual Learning, Organisational Learning (Who learns).
	Individual, Team, Organisational Scope (Scope of Reflection)


We clustered evidence for learning by reflection that we found in user study data into the above classification scheme, and can thus provide a good impression of the diversity of reflection in these three real workplaces.

The result of our analysis of the given data according to Research Question 1b is the following categorisation of triggers for reflection:


	Different from Baseline
	Emotion
	Information Input
	Feedback from / Reaction of Stakeholders


Note that from a theoretical viewpoint, we assume that what triggers reflection is a “discrepancy between expectations and the current experience” (see Sect. 2.1.1). However, for practical purposes, we wanted to be able to differentiate between different kinds of triggers. We expect that an understanding of concrete triggers will constitute a significant contribution to the advancement of theory on reflective learning. Additionally, we expect insights into triggers to be a first step towards intelligent prompting for reflection, or “reflection recommenders” that recommend people to reflect.<br>
<br>
In order to answer Research Question 2, we looked for tools that already support reflection, as well as data or data sets, which are created during work, and tools, which are used during work, as well as reflective practices.<br>
We found hardly any direct technical support for reflection. The closest support we found was the lysis protocol at the involved hospital, where the protocol requires a reasoning both where the lysis (a treatment for stroke) was applied and where it was not applied. On the other hand, we found a rich amount of data and data sets, as well as technical support for work and mature reflective practices in place. All these can serve as starting point for the development of further reflection apps.

<br>
Research Question 3 was most deeply explored together with the participating hospital through the focus groups. Apart from this, we also excerpted various statements concerning wishes for technological support from other research instruments. We explored how (i) data that captures externally observable data, (ii) physiological data, (iii) mood data can support learning by reflection, and whether target users have a strategic approach to learning. Additionally, we identified barriers and constraints for technology support, and collected explicit App ideas.<br>
Overall we found that users clearly see video or audio trails of their work as a good potential basis for reflection, as well as their “usual” work data. Target users are more sceptical about physiological and mood data, but mostly in connection with the explanation that “we cannot do anything about it anyway”, i.e. users seem to see little possibility to act on knowledge about their physiological or emotional state. Except for physicians in specialist training, we did not find much resonance with users concerning a strategic approach to learning.

<br>
<strong>Implications and Outlook</strong>

<br>
A major constraint of target users is that they have little time to do something “in addition” during their working hours. We conclude from this that future reflection Apps must be integrated into existing work practices. Additionally, we will strive to use existing data, and to “insert reflection” unobtrusively into everyday work.<br>
Concerning the content of reflection, the user study results indicate that the development of domain-specific knowledge and self-organisation capabilities, as well as the communication with clients, patients or residents is more important to target users than the communication with colleagues or reflection on one"s own reaction. This result aligns well with the above discussed constraint on time. We therefore plan to develop reflection Apps that are testbed-specific in the immediate future. This focus will give us the possibility to well integrate Apps into existing work environments.
description: Even though the file says "Draft" on the title page, this is the final Deliverable that has been internally peer-reviewed within the EU FP7 MIRROR project, and submitted and accepted by the EU as project result.