# Work and Speech Interactions among Staff at an Elderly Care Facility

Tetsuro Chino, Kentaro Torii, Naoshi Uchihira, Yuji Hirabayashi

► To cite this version:

Tetsuro Chino, Kentaro Torii, Naoshi Uchihira, Yuji Hirabayashi. Work and Speech Interactions among Staff at an Elderly Care Facility. 3rd Human Work Interaction Design (HWID), Dec 2012, Copenhagen, Denmark. pp.35-45, 10.1007/978-3-642-41145-8_4. hal-01463376

HAL Id: hal-01463376

https://inria.hal.science/hal-01463376

Submitted on 9 Feb 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Work and Speech Interactions among Staff at an Elderly Care Facility

Tetsuro Chino*¹, Kentaro Torii*¹, Naoshi Uchihira*¹, Yuji Hirabayashi*²

*¹Corporate R&D Center, Toshiba Corporation, Kawasaki, Japan.

*²Institute of Technology, Shimizu Corporation, Tokyo, Japan.

tetsuro.chino@toshiba.co.jp

**Abstract.** We observed bathing assistance, night shift operations, and handover tasks at a private elderly care home for 8 days. We collected approximately 400 h of recorded speech, 42,000 transcribed utterances, data from an indoor location tracking system, and handwritten notes by human observers. We also analyzed speech interaction in the bathing assistance task. We found that (1) staff members are almost always speaking during tasks, (2) remote communication is rare, (3) about 75% of utterances are spoken to the residents, (4) the intended recipient of utterances is frequently switched, and (5) about 17% of utterances contain personal names. We also attempted clustering utterances into passages, and about 33% of passages contained only one person's name. These results should be applicable in semi-automatic long-term care record taking.

**Keywords:** cooperative work, speech interaction, care

## 1 Introduction

Japan's increasingly aging population is an important issue, one urgent aspect of which is caring for elderly persons with disabilities. Nowadays, care services are often provided by the devoted efforts of care staff at long-term care facilities. Care services are characterized by what we call “action-oriented intellectual services.” Care staff makes many decisions regarding medical care which require specialized knowledge. They also assist elderly persons with disabilities in many activities of daily living. Care service staff members have many types of professions and specialties. Also, care services must operate at all hours, throughout the year, and successfully doing so requires a high degree of information sharing among staff, for example, keeping records of care performed. [1] pointed out that medical staff spend much time on indirect care, including record keeping and information sharing. Conventional IT systems are fundamentally designed for desk work, however, and do not support the needs for hands-free and eyes-free operations suited to action-oriented intellectual services.

[2] and [3] proposed a “voice tweet system” to overcome these problems. In that system, “voice tweets” spoken by a staff member are tagged with the staff member’s
location and motion, spoken keywords, and associations with background knowledge, and based on these tags, the tweets are automatically delivered to an appropriate staff-er. This system provides semi-hands-free and eyes-free communication among medi-cal and care staff. Such a system requires knowing what kind of speech communica-tion supports cooperative work in medical and care domains. We therefore performed field studies at an elderly care home in Japan to analyze cooperative work and speech interaction among care staff.

We report on a series of field studies and the insights extracted from analysis of the collected data. The remainder of this paper is organized as follows. Section 2 describes previous works. Section 3 describes the field and the tasks examined. Section 4 describes the study methodology, and Section 5 presents an analysis of the collected data. Finally, Section 6 gives our conclusions and proposed areas for future study.

## 2 Related Works

[4] analyzed dynamically formed teams in a hospital emergency room, and identified three key factors in the design of team communication technology: (1) maintaining awareness within the team, (2) making informative interruptions, and (3) supporting role-based calling. However, this study was limited to a few directly connected rooms on a single floor. Further investigation of more distributed locations, such as in larger hospitals or care facilities with multiple stories, may therefore be necessary.

[5], [6], and [7] reported on the use of a commercial voice communication system named “Vocera” in actual hospitals. They pointed out (1) that selecting communication artifacts is necessary, (2) that person-locating features can reduce human movement, and (3) that remote voice communication has positive effects on the prioritization and scheduling of tasks. They also described risks and problems associated with voice communication devices.

[8] reported on an analysis of nurse-patient conversations in a Japanese hospital. They found that most utterances effective for risk sharing are spoken by nurses who are highly skilled at risk recognition and communication. In the “voice tweet system” described above, sharing this kind of informative utterances as “voice tweets” can have positive educational effects for the whole team.

## 3 Study Location

### 3.1 Care Facility

We performed field studies at a Japanese elderly care home over 8 days between September 2011 and February 2012. Figure 1 shows an overview of the study location. The building has four stories, with 40 private resident rooms, common living and dining rooms, 1 care station, 1 bath area, and other rooms and spaces. Corridors, elevators, and stairways connect the rooms.
<figure><img src="image_2.png" /><figcaption>Fig. 1. Overview of the elderly care facility</figcaption></figure>

## 3.2 Residents with Disabilities

About 30 residents were living in this building during our field study. For convenience, we classify residents into the following groups:

- **Residents with slight disability** are independently mobile with the aid of a device such as a cane or walker.

- **Residents with moderate disability** require a wheelchair and staff assistance to move around.

- **Residents with severe disability** are bedridden, and can only be moved by multiple staff using specially designed devices such as a stretcher.

## 3.3 Target Tasks

We observed and collected data on 3 tasks.

- The **bathing assistance task** involves bathing residents on the first floor of the building, and is performed by a special day shift team.

- The nighttime staff performs **night shift tasks**, which involve monitoring resident sleep, performing evening and morning care, checking vital signs on a schedule, responding to nurse and sensor calls, and attending to other needs as they arise.

- At scheduled shift changes, leaders of the incoming and outgoing shift personnel perform the **handover task**.
# 4 Methodology

## 4.1 Speech Data Collection

We collected over 142 h of “subject sounds” via microphones attached to staff members’ clothing. We furthermore collected another 142 h of “intercom sounds” via intercom lines, and more than 98 h of “environmental sounds” via microphones set up at various locations. We used IC recorders to record environmental sounds in 44.1 KHz 16-bit linear PCM stereo, and recorded subject sounds and intercom sounds in 44.1 KHz 16-bit linear PCM mono. Figure 2 shows the equipment used.

<figure><img src="image_3.png" /><figcaption>Fig. 2. Sound data collection equipment</figcaption></figure>

## 4.2 Extraction and Transcription of Utterances

About half of the audio data was selected and automatically split into sound fragments. We set the threshold parameter for length of silence between fragments to 500 ms, and human transcribers flagged fragments that were human utterances. Finally, the human utterances were transcribed with time stamps and speaker identifiers. This resulted in more than 42,000 transcribed utterances.

## 4.3 Other Collected Data

We collected more than 64,000 staff location samples using a Bluetooth-based indoor location tracking system that we developed. Human observers traced staff movements onto paper, and investigated their actions. The observers recorded the time, place, participants, media, and a brief memo on the content of observed conversations involving staffers. Observers also videotaped subjects to get precise movement data with timestamps.
# 5 Work and Speech Interaction Analysis

## 5.1 Work Analysis of the Bathing Assistance Task

We chose the **bathing assistance task** for analyzing work and speech interactions, because this was the most complicated collaborative activity among our target tasks. The following is an overview of the bathing assistance task.

1. An **"inner support" staff** in the dressing room determines the next resident to be bathed according to a schedule. Staff performs the bathing task according to the disability level of the resident as follows:

* For residents with slight disability, the inner-support staff calls the resident's room via intercom and asks the resident to come to the bath area.

* For residents with moderate disability, one or two "outer support" staff members go to the resident's room, check vital signs, and then ask the resident to come to the bath area by wheelchair with their assistance. If the resident agrees, then the staff members assist the resident from the bed to the wheel chair, and bring the resident to the bath area on the first floor.

* For residents with severe disability, two or three outer support staff members go to the resident's room with a stretcher designed for bathing bedridden residents, and check vital signs. If the resident agrees and conditions are met, then the staff members assist the resident from the bed to the stretcher, and bring the resident to the bath area on the first floor.

2. After arriving at the bath area, the resident undresses with inner or outer staff assistance, according to need and availability.

3. Assisting staff also check the resident's body and apply ointments or medicines if necessary.

4. The inner support staff helps the resident bathe in a specially designed barrier-free bathroom or a mechanized bathroom, as needed.

5. After bathing, mainly inner support staff helps the resident get dressed.

6. Outer support staff assists the resident from the first-floor dressing room to the second-floor common living and dining rooms.

7. In the living and dining room, other staff members provide support by drying the resident's hair, bringing food, or helping the resident rest in the living room.

8. Finally, staff helps the resident return to his or her room.

Since multiple residents bathe at the same time, some procedures are performed simultaneously. Many factors can force changes in procedure. For example, residents may not be in their rooms, or their mental or physical condition may preclude bathing.
It is worth noting that the tasks of getting dressed and undressed provide residents with valuable opportunities to train and maintain their physical and intellectual abilities, so the staffers provide minimal support and focus on safety. The time needed for getting dressed and undressed differs from resident to resident and can reach 30 min in the longest case, but efficiency is not the primary consideration in providing high-quality services at elderly care facilities. Flexible collaboration is therefore necessary to ensure these tasks are performed smoothly.

## 5.2 Utterances Spoken during Bathing Assistance

We choose one bathing assistance task involving all female residents for our speech interaction analysis. The task was performed over one morning, and represents about 10% of the speech data described in section 4.1.

According to records provided by the elderly care home, 6 staff members had bathing assistance duties that morning. Among them, we focused on two outer support staff members, who we refer to as A and B. We selected them because they moved throughout the study location and interacted with many residents in many different situations at multiple times and places. Table 1 shows a summary of utterances by A and B and other staff and residents.

<table><caption>**Table 1.** Summary of the extracted utterances</caption><caption>Table 1.<br>Number and length of extracted utterances of staffers A and B of the outer support staff and others (other persons, including other staff and residents).</caption><thead><tr><th rowspan="2">Speaker</th><th>Number</th><th>Percentage</th><th rowspan="2">Min.</th><th>Ave.</th><th>Max.</th><th>Stdev.</th></tr><tr><th colspan="2">Extracted utterances</th><th colspan="3">Length of utterances (s)</th></tr></thead><tbody><tr><td>A</td><td>2,293</td><td>22.7%</td><td>0.074</td><td><strong>1.683</strong></td><td>5.697</td><td>1.214</td></tr><tr><td>B</td><td>1,692</td><td>16.8%</td><td>0.121</td><td><strong>1.724</strong></td><td>5.059</td><td>1.125</td></tr><tr><td>Others</td><td>6,110</td><td>60.5%</td><td>0.055</td><td><strong>1.635</strong></td><td>18.053</td><td>1.441</td></tr><tr><td>Total</td><td><strong>10,095</strong></td><td>100.0%</td><td>-</td><td>-</td><td>-</td><td>-</td></tr></tbody></table>

From the data, 10,095 utterances were extracted. The outer support staff spoke relatively more than others, perhaps because of numerous stock phrases used with residents. The lengths of each utterance were similar for all speakers, perhaps due in part to the influence of our automatic splitting process.

Table 2 shows the spatial conditions of each conversation participant. Almost all utterances (99.4%) were face-to-face communications. The proportion of remote communication was only 1.0% for staffer A, and 0% for staffer B. Inner support staff calls slightly disabled residents in this task, so there is little opportunity for remote communication by outer-support staff. Introducing the voice tweet system at this location would have a significant effect, because currently outer support staff does not have access to remote communication media, which may force unnecessary movement.
<table><caption>**Table 2.** Summary of special conditions of each conversation participant</caption><caption>Table 2.<br>Spatial conditions of conversational participants in utterances of staffers A and B (outer support staff)</caption><thead><tr><th rowspan="2">Speaker</th><th colspan="2">Face-to-face</th><th colspan="2">Remote</th><th colspan="2">Total</th></tr><tr><th>Number</th><th>Percentage</th><th>Number</th><th>Percentage</th><th>Number</th><th>Percentage</th></tr></thead><tbody><tr><td>A</td><td>2,269</td><td>99.0%</td><td>24</td><td>1.0%</td><td>2,293</td><td>100.0%</td></tr><tr><td>B</td><td>1,692</td><td>100.0%</td><td>0</td><td>0.0%</td><td>1,692</td><td>100.0%</td></tr><tr><td>Total</td><td>3,961</td><td><strong>99.4%</strong></td><td>24</td><td>0.6%</td><td>3,985</td><td>100.0%</td></tr></tbody></table>

## 5.3 Frequency and Intended Recipients of Utterances

We analyzed utterance frequency and the intended recipients of each utterance to know how often and to whom the care staff talks during this task. Figure 3 shows a time plot of utterances spoken by outer support staffer A (the upper half of the graph) and staffer B (the lower half of the graph) during the 2.5 h of the bathing assistance task. Dots at the upper, middle, and lower positions denote utterances spoken to residents, other staff, and oneself (monologues), respectively. Both A and B are almost always speaking during the task, and the intended recipients quickly changes. In one instance, the collected data show an outer support staff's simultaneous conversation with a resident and other staff in the dressing room.

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Time plot and intended recipients of utterances</figcaption></figure>
Figure 4 shows the distribution of intended recipients for utterances by staffers A and B. For both, about 75% of utterances are spoken to residents, about 17% to other staff, and the remaining utterances are monologues. The frequency of utterances to residents may be evidence of the importance of direct face-to-face verbal communication between staff and long-term residents.

<figure><img src="image_5.png" /><figcaption>Fig. 4. Distribution of intended recipients of utterances of staffers A and B (outer support staff)</figcaption></figure>

## 5.4 Names in Utterances

Accurate long-term care records are important for maintaining and improving the quality of care services. Meaningful long-term care records require several key types of information, such as who performed the care task, what kind of care was performed, for whom the care was performed, when the care was performed, and where it was performed. Mobile device-based support systems like the voice tweet system can automatically record data on who (staff), when (time), and where (location). But data about for whom care is performed (resident) may be difficult to acquire, because devices such as physical tags attached to residents may be considered dehumanizing. We therefore analyzed the appearance of personal names in the utterances spoken by outer support staffers A and B. Personal names appeared 703 times, 378 times for staffer A and 325 times for staffer B. Table 3 summarizes the distribution of personal names in each utterance. About 16.9% of utterances have at least one name that could be a clue as to who is providing care or to whom care is being provided. Such information could be used by an advanced voice tweet system to semi-automate long-term care record maintenance in future care staff support systems.
<table><caption>**Table 3.** Distribution of personal names in each utterance</caption><caption>Table 3.<br>Distribution of number of personal names in utterances of A (outer support staff A) and B (outer support staff B).</caption><thead><tr><th rowspan="2">Speaker</th><th colspan="2">Number</th><th colspan="2">Percentage</th><th colspan="2">Number</th><th colspan="2">Percentage</th></tr><tr><th colspan="2">No (0)</th><th colspan="2">Only 1</th><th colspan="2">2 or more</th><th colspan="2">Total</th></tr></thead><tbody><tr><td>A</td><td>1,938</td><td>84.5%</td><td>335</td><td>14.6%</td><td>20</td><td>0.9%</td><td>2,293</td><td>100.0%</td></tr><tr><td>B</td><td>1,377</td><td>81.4%</td><td>305</td><td>18.0%</td><td>10</td><td>0.6%</td><td>1,692</td><td>100.0%</td></tr><tr><td>Total</td><td>3,315</td><td>83.2%</td><td>640</td><td><strong>16.1%</strong></td><td>30</td><td><strong>0.8%</strong></td><td>3,985</td><td>100.0%</td></tr></tbody></table>

## 5.5 Clustering Sequences of Utterances into Passages

To get more information about who provides care and for whom, from utterances for a semi-automatic long-term care record system, we attempted to cluster sequences of utterances into what we call “passages.” A threshold parameter of the time gap between sequential utterances allows merging series of utterances spoken by each staffer into passages. Figure 5 shows distributions of passages with specified numbers of names, for 13 different settings (from 0.1 to 60 s) of the threshold for the time gap between passages.

Given an appropriate parameter value (3 s in this case, at the vertical dotted arrow in Figure 5.), the percentage of passages with only one person-name (shown as the solid line in the Figure 5.) can be dramatically improved from 16.1% (the unmerged utterances case in Table 3) to 32.6%, and the average time length of the passage will be 16.7 s. This rate is sufficiently high and this length is sufficiently short to be utilized for precise semi-automatic long-term care records in future care staff support systems.

<figure><img src="image_6.png" /><figcaption>**Fig. 5.** Relation between the average length of passages (horizontal axis), and the distribution of passages with a specified number of personal names (vertical axis), for multiple thresholds</figcaption></figure>
## 5.6 Possible Extensions and Applications

Background knowledge on each person who works or lives in the facility (e.g. name, role, gender, shift/work schedule, profession, behavioral preference or tendency, etc.) can be utilized to disambiguate the referent of each referring expression (e.g., personal names or pronouns) appearing in each utterance or passage. Based on this information, together with automatically acquirable information (e.g., who, where, and when) from a mobile device-based platform, the future stuff support system can generate and suggest estimated candidates or outlines of long-term care records to the responsible staff member, and ask for confirmation. This mechanism may contribute to lower costs and reduced load of care staff for the indirect care, as well as to improved quality of long-term care records.

# 6 Conclusions and Future Work

We reported on a series of field studies at an elderly care facility. For eight days we observed three tasks: bathing assistance, night shift operations, and shift handover. We collected approximately 400 h of recorded speech, 42,000 transcriptions, data from a location tracking system, and handwritten notes by human observers. We also analyzed speech interaction in a bathing assistance task. We found that (1) staff members are almost always speaking during the task, (2) remote communication is rare, (3) about 75% of staff utterances are spoken to residents, (4) utterance targets are frequently switched, and (5) about 17% of utterances contain at least one personal name. We also attempted clustering utterances into passages. The rate of passages with only one personal name can be raised to 33%. These results could be utilized in semi-automatic long-term care record taking.

Future targets for study include (1) an integrated analysis of not only the collected speech data, but also the location, action, and observed descriptions of staff actions, and (2) comparison with data from other hospitals to clarify differences and common aspects, helping to create general models applicable to the broader healthcare domain.

## Acknowledgements

This project was joint research between the Toshiba Corporation, the Shimizu Corporation, and the Japan Advanced Institute of Science and Technology, and was supported by the Service Science, Solutions and Foundation Integrated Research Project Program of the Japan Science and Technology Agency under the Ministry of Education, Culture, Sports, Science and Technology, Japan.
References

1. Lemonidou C., C. Plati, H. Brokalaki, J. Mantas, V. Lanara. Allocation of nursing time. Scand J Caring Sci. 1996; 10(3):131-6, (1996).

2. Uchihira Naoshi., et. al., Innovation for Service Space Communication by Voice Tweets in Nursing and Caring: Concept and Approach in Japanese National Project, 20th Annual Frontiers in Service Conference, (2011).

3. Kentaro Torii, Naoshi Uchihira, et. al., Service Space Communication by Voice Tweets in Nursing, in proc. of AHFE2012, July, (2012).

4. Soyoung Lee, Charlotte Tang, et. al., Loosely Formed Patient Care Teams: Communication Challenges and Technology Design, in proc. of CSCW2012, pp.867-876, February, (2012).

5. Joshua E. Richardson, The Effects of Hand Free Communication Devices on Clinical Communication: Balancing Communication Access Needs with User Control, in proc. of AIMA2008, pp.621-625, (2008).

6. Charlotte Tang, Sheelagh Carpendale, A Mobile Voice Communication System in Medical Setting: Love it or Hate It?, in proc. of CHI2009, pp.2041-2050, April, (2009).

7. Charlotte Tang, Sheelagh Carpendale, et. al., InfoFlow Framework for Evaluating Information Flow and New Health Care Technologies, in Intl. Journal of Human-Computer Interaction, Vol.26, No.4, pp.477-505, (2010).

8. Misako Nambu, Etsuko T. Harada, et.al. Risk Sharing Communication in Medical Settings: Analysis of Nurses' Conversation, Cognitive Studies, Vol.13, No.1, pp.62-79, (in Japanese), March, (2006).

9. Ole Andreas Alsos, Awareness and Usability Issues in Clinical CSCW Systems, in proc. of CSCW2010, pp.501-502, (2010).