# From Text to Treatment: How Medical Discharge Letters Are Used as a Key Artifact for Managing Patient Care

Anastasiya Zakreuskaya, Daniel Buschek, Wendy E. Mackay, Ignacio Avellino, Graham Dove, Bjoern Eskofier

<figure><img src="image_1.png" /></figure>

## To cite this version:

Anastasiya Zakreuskaya, Daniel Buschek, Wendy E. Mackay, Ignacio Avellino, Graham Dove, et al. "From Text to Treatment: How Medical Discharge Letters Are Used as a Key Artifact for Managing Patient Care." MuC 2024 - Mensch und Computer, Sep 2024, Karlsruhe, Germany. pp.99-110, 10.1145/3670653.3670665. hal-04738259

HAL Id: hal-04738259

https://hal.science/hal-04738259v1

Submitted on 22 Oct 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
# From Text to Treatment: How Medical Discharge Letters Are Used as a Key Artifact for Managing Patient Care

## Anastasiya Zakreuskaya

Université Paris-Saclay,
CNRS, Inria Saclay
Machine Learning and Data Analytics
Lab, FAU Erlangen-Nürnberg
Paris, France

## Daniel Buschek

Mobile Intelligent User Interfaces
University of Bayreuth
Bayreuth, Germany

## Wendy E. Mackay

Université Paris-Saclay,
CNRS, Inria Saclay
Paris, France

## Ignacio Avellino

Sorbonne Université, CNRS, INSERM,
Institut des Systèmes Intelligents et
de Robotique, ISIR
Paris, France

## Graham Dove

NYU Tandon Dept. of Technology
Management and Innovation
New York University
New York, USA

## Bjoern Eskofier

Machine Learning and Data Analytics
Lab, FAU Erlangen-Nürnberg
Erlangen, Germany

## ABSTRACT

Hospital physicians must navigate through vast quantities of patient information represented in text-based reports. Although intended to improve patient care, their effectiveness hinges on each physician's ability to successfully handle and interpret fragmented information from diverse sources. The increasing automation of text interactions are a potential support but are still at the early phase of implementation in real-world scenarios. We observed 144 hours of clinical shifts in a German internal medicine hospital and collected structured field notes on physicians' current practices with text-based reports to enrich existing understanding of the requirements for including automation to clinical text. We identified medical discharge letters as most frequently consulted text document and a qualitative analysis of the field notes revealed that this document acts as a key artifact that serves different roles and purposes in the hospitalization of a patient. Based on our findings we discuss possible loss of these nuanced uses through automation and propose design implications for medical text reports.

## CCS CONCEPTS

* Human-centered computing → Empirical studies in HCI

## KEYWORDS

text-based reports, medical discharge letters, text automation, electronic health records

## ACM Reference Format:

Anastasiya Zakreuskaya, Daniel Buschek, Wendy E. Mackay, Ignacio Avellino, Graham Dove, and Bjoern Eskofier. 2024. From Text to Treatment: How Medical Discharge Letters Are Used as a Key Artifact for Managing

Patient Care. In *Proceedings of Mensch und Computer 2024 (MuC '24)*, September 01-04, 2024, *Karlsruhe, Germany*. ACM, New York, NY, USA, 12 pages. https://doi.org/10.1145/3670653.3670665

# 1 INTRODUCTION

As long as the centuries continue to unfold, the number of books will grow continually. [...] It will be almost as convenient to search for some bit of truth concealed in nature as it will be to find it hidden away in an immense multitude of bound volumes (-Denis Diderot, 1755)

Text-based reports offer a well-established medium for exchanging information in collaborative settings. In hospitals, the core medical record consists of conventional text blocks with clinical notes to summarize a patient's history and further procedures [7]. Healthcare professionals then coordinate patient care by sharing these records among each other and collaboratively augment this data from their clinical expertise. The level of completeness in these documents and thus the understanding of a patient's condition is a crucial factor for the success of follow-up consultations [49, 60]. With growing amounts of available data, physicians need to know how to navigate through given documents or systems and select specific subsets of information from a large fragmented quantity of patient data [36]. Over the years, this interpretation and documentation of data has become an integral part of a physician's work and often exceeds the time spent on direct patient care [2]. The lack of interoperability between clinical information systems hinders these efforts [56], as medical information about a patient is distributed over an imprecise number of healthcare facilities. In the German healthcare system, hospitals lack a nationwide information exchange structure and each hospital implements individual software, based on their size and resources [29]. Several initiatives like the digitization strategy by the German Federal Ministry of Health [22] or the proposed regulation for a European Health Data Space [57] create the foundation for new technologies that solve current problems in healthcare and improve quality of care. The sensitivity of patient information and privacy requirements emphasize the need for throughout research on integrating new innovations into standard care.

<figure><img src="image_3.png" /></figure>

This work is licensed under a Creative Commons Attribution International 4.0 License.
Given recent advances in AI-based systems, there is a trend to include automation in healthcare processes to support clinicians with growing information overload [19]. The use of Natural Language Processing (NLP) introduces new workflows, which improve clinical query systems with speech recognition adapted for medical language and generating ICU summaries in formats familiar to clinicians, thereby aiding to clinical decision support [40]. However, current clinical user interfaces often suffer from exceeding complexity, low usability and low integration into clinical workflows [24]. In response, there has been increasing efforts from the HCI community to investigate the socio-technical factors for successful integration of AI-based applications into clinical care [3]. This includes the need for further investigation of meaningful doctor-AI relationships in automated text technologies [21].

Clinical care is highly collaborative and requires a variety of interwoven non-electronic coordination artifacts and platforms [8, 12] to generate a successful overview of each patient. Previous studies show that physicians create individual workarounds and integrate information systems differently according to their intended use [18, 41]. Understanding the requirements for text automation systems thus requires to look at this interaction as a work practice within a hospital environment rather than a situated task [27].

In this work, we examine physician's current interaction with text-based documents to enrich existing understanding of requirements for new text automation systems. We are particularly interested in addressing the following research questions:

(1) Which **clinical documents** do physicians consult during their workflow,

(2) for which **purpose**,

(3) and which **challenges** do they encounter?

We first review related research and approaches to deal with existing information overload in the hospital. Then, we present the results of a field study of clinical shifts in a German internal medicine department and discuss implications for design.

# 2 RELATED WORK

We review research related to hospital information systems and the attempts to standardize clinical documentation. Then we review work on the increased information overload of physicians that come along with medical record keeping. Finally, we present work looking into how physicians are adapting to increased automation.

## 2.1 Tensions Between Standardization and Physician's Preference for Expressivity

Today's hospitals implement a wide variety of information systems, largely dedicated to clinical documentation, which describes "*a process in which healthcare providers record the observations, impressions, plans and other activities arising from episodes of patient care [that] generally occurs with each interaction between patients and the healthcare system*" [47]. This documentation serves multiple purposes, which include clinical tasks (e.g., identify diagnosis, decide on further treatment) as well as administrative tasks (e.g., coordinate examinations) and functions as regulatory justification for treatment decisions [30]. Text-based notes provide a flexible and adaptable mean to effectively document and reason about a patient's disease trajectory. Therefore, different documents like

medical discharge letters aim to give concise summaries and support information exchange between health providers. There has been growing interest to standardize clinical documents by creating structured and a predefined vocabulary [15, 39], to increase consistency and data quality. Looking at the big picture, these reports are a valuable resource for real-world insights into healthcare practices and can help improve quality of care [37]. Analyzing discharge letters can reveal important care specific breakdowns [33] or be used to make predictions about disease progression [48]. However, this standardization does not always align with the work practices and there is a recognized tension between physician's documentation preferences and the demand for standardization [23, 46].

Healthcare providers rely on expressivity to transport different layers of information about a patient's case in respect to their personal medical competencies [46]. A closer look at possible use cases has revealed that not every department can integrate standardized overviews in their care. Medical specialties that treat complicated and long-term patients with multiple comorbidities have very individual or even unpredictable information needs that vary for each patient and cannot simply be summarized based on one scheme [28]. Obtaining an overview of a patient is a multi-dimensional process in which a wide range of interwoven non-electronic coordination artefacts are consulted, and where the same information is represented in multiple spaces [8]. To identify these crucial underlying practices, we need to investigate current workflows with clinical notes and understand their purposes in respect to patient care.

## 2.2 Physician's Confrontation With Growing Information Overload and Missing Interoperability

Data-driven practices can improve quality of care, connecting caregivers and providing necessary information. Still, they bring new challenges, as they demand effort to already-busy practitioners, reducing productivity and efficiency [62, 65], and often their success is labor-driven, as considerable amounts of practitioner work is needed to guarantee the benefits of information systems [53]. This is known as *data work*, the socio-technical interventions to make data work [13]. Research in data work has studied the impacts of both feeding and consuming information. Feeding information systems involves skills such as assessing sources of disorganized information to create data structured as required by systems [43], sanitizing data [59], or, translating data from one system to another [10]. Consuming information, similarly, is also not always straightforward. In complex and comprehensive disease trajectories, the patient understanding is constructed through complex iterative sorting, filtering and ordering of information from different sources [28]. This is shown in work by Even Chorev [20], who presented a predictive information system for personalized care that actually hindered decision making as doctors struggled to interpret ambiguous data. Cabitza et al. [16] also showed how multiplicity in data work, when a phenomenon is recorded by multiple experts, leads to data variability, uncertainty, and ambiguity, and to experts adapting to cope with such variations when interpreting data. The fact that clinical text is consulted for different purposes depending on the clinical phase [52] adds to this complexity. These phases include (1) preparation (i.e., obtain an understanding of a
patient's case), (2) consultation (i.e., interaction with the patient) and (3) wrap-up (i.e., consolidating the documentation afterwards). Moreover, the use of these systems impacts healthcare practices, for example one study shows that the introduction of the Electronic Health Record (EHR), physicians' documentation time almost doubled due to inefficiencies in simultaneous paper and digital work [9]. More than half of this time is dedicated to clinical notes for synthesizing patient information [45], as notes serve as a primary source of information [52].

The resulting information overload has potentially become ob-
structive to patient care as physicians do not always succeed in ex-
tracting the clinically relevant information from available data [38].
Work with information systems in hospital settings is fragmented
and does not incorporate cognitive aids, leading to frequent task
switching and workarounds [26, 35, 64], showing a need to consider
the attributes of clinical practice to measure the success of new
technologies [32].

## 2.3 Physician's Acceptance of Automation Efforts

As a response to growing amount of data, new applications and research areas emerge aiming to relieve relieve physicians in their conventional data work and direct their efforts towards everyday care. These applications deploy different methods to facilitate documentation and interpretation of clinical text [31, 58, 63]. However, their success hinges on physician's willingness to adopt and integrate them in their workflows. Previous studies have shown that healthcare professionals trust data differently depending on its source as it allows for better skill and competence assessment [54]. The textual progress notes format and content vary depending on the author, the particular issue of a patient [36] and the document structures differ across institutions and specialties [52]. This variability introduces a complexity in automation due to a wide range of possible formats and a fragmentation in different documents. Clinicians therefore wish to establish constructive synergies with text automation technologies, that, like digital assistants, support them in their work with functions such as extracting clinical information, automated voice documentation, creating record summaries or making text reports searchable [21]. A deep understanding of these needs and existing clinical practices is needed for the design of efficient health record systems of the future.

3 METHOD

This analysis is part of a large observational study investigating physician's data retrieval activities during hospital shifts. In this analysis, we specifically focus on our observations on how physicians interact with text-based documents.

3.1 Participants and Research Site

This study was conducted in the internal medicine department of a 340-bed municipal teaching hospital in Germany, which can be considered a typical German hospital according to official statistics [29]. Such hospital usually provides acute medical care to patients with severe or chronic diseases in cases of deterioration and is equipped for short hospitalization times. We observed all wards that are included in a patient journey in internal medicine.

The majority of these patients are admitted through **emergency care** either because of unidentified symptoms or the deterioration of a disease. After hospitalization, patients are referred to one of the specialized internal medicine wards (e.g., cardiology and pneumology, gastroentrology, intensive care or single bed in other departments as outsourced area due to capacity limits) depending on their condition and diagnosis. The internal medicine team comprises two chief physicians, 12 senior physicians and a fluctuating number of 22 residents, who rotate through the different wards. This department accumulates data from various sources, such as imaging techniques, patient monitoring and score assessment to find treatment strategies for a broad range of diseases.

We obtained ethics committee approval for this study from one
of the author’s university. We disclose that we avoided collecting
any patient data or personal identifiers.

3.2 Hardware and Software

We needed a lightweight method for capturing both images and written notes without wireless network access, due to lacked WiFi connectivity and inconsistent mobile network coverage. Our goal was to capture data interactions systematically for available data sources and data types. We implemented a data collection tool, called **SNAPCUTS**, using the **SHORTCUTS** app on an Apple **IPhone 12 Pro**. We first designed a general template to capture an identified context and defined a structure to describe occurring data retrieval events. For each observed context (e.g., morning meeting, patient ward, etc.), the observer was able to create the template dynamically consisting of the following elements:

(1) Automatic timestamp.

(2) Field note to describe the current context and other relevant information, which is described by identifying new situations, activities, tasks or changes of location.

(3) Current task: *direct patient contact, indirect contact, other professional activities, personal activities* (from Weigl et al. [61])

(4) Location: *physicians' office, nursing room, examination room, corridor, patient room, meeting room, outside the assigned ward*

(5) Data-Interaction-Table: *Table of data source and data type combinations, in which the rows represent the different data sources and columns the data type.*

Recording a field observation consists of selecting the appropri-
ate elements for each category, and optionally adding open-ended
field notes in the table of data source and data type combinations.
An example of a filled out template can be found in Appendix A.

## 3.3 Procedure

The first author observed physicians from the internal medicine team during their shifts from June to August 2023 on different days of the week. At the beginning of the study, the team was informed about the study procedure and goals and a flyer with further information and contact information was distributed in the physicians' office. After attending the morning meeting, the researcher opportunistically asked either a resident, physician, or a senior physician for verbal consent to follow the shift throughout
<table><caption>Table 1: Overview of observations at different wards. Most observations were conducted at the cardiology and pneumology department (61 hours, 7 shifts). In total, the observer followed eleven different physicians in 17 shifts, where some residents were observed over multiple shifts.</caption><thead><tr><th>Ward</th><th>Hours (shifts)</th><th>Physicians</th><th>Collected templates</th><th>Used for data analysis</th></tr></thead><tbody><tr><td>Cardiology and pneumology</td><td>61 (7)</td><td>Resident 1 (f), Resident 2 (m), Resident 3 (f), Resident 4 (f), Resident 9 (m)</td><td>153</td><td>46</td></tr><tr><td>Gastroenterology</td><td>25 (3)</td><td>Senior 1 (m), Resident 5 (f), Resident 6</td><td>50</td><td>14</td></tr><tr><td>Outsourced Area</td><td>24 (3)</td><td>Resident 6 (f), Resident 8 (m)</td><td>32</td><td>11</td></tr><tr><td>Emergency Department</td><td>24 (3)</td><td>Resident 7 (m), Senior 2</td><td>122</td><td>20</td></tr><tr><td>Intensive care</td><td>8 (1)</td><td>Resident 7 (m)</td><td>35</td><td>3</td></tr><tr><td>Total</td><td>144 (17)</td><td>9 Residents, 2 Seniors</td><td>398</td><td>94</td></tr></tbody></table>

the day. At the beginning of each observation, the researcher explained the study aims to the physician and emphasized the focus on systems. In order to mitigate the concerns of being observed, the researcher reassured the physician that the quality of work will not be assessed and possible questions were clarified. Physicians were encouraged to ignore the researcher and perform their work as usual and tell when they do not want to be followed. For each observed context (e.g., morning meeting, patient ward, etc.) the researcher used SNAPCUTS to fill out a new template, that was saved as a note file in Apple Notes and recorded observed interactions in the Data-Interaction-Table. Each cell summarized the observed interactions for a specific data source and data type within this context. The content of the notes did not include any personal identifiers or patient data, physicians were recorded using acronyms to limit traceability and the note focused on observed interactions.

Short informal interviews with the observed physician were held between tasks or during lunch that reflected situations observed during the day to obtain further background knowledge. Whenever possible, the researcher shadowed a whole shift. At the end of each shift, the researcher captured personal reflections on the day's events in a journal.

## 3.4 Data Collection and Analysis

Our observations included 17 shifts for a total of 144 hours. We followed seven residents and two senior physicians. We spent a majority of the time at the cardiology and pneumology ward (61 hours) and almost equally divided time at the gastroenterology ward, the outsourced area and emergency department. Overall, we recorded 398 filled out templates, out of which 94 fit our inclusion criteria for further analysis as they included interactions with text-based documents. One filled-out template represented the observed interactions in one context, e.g., patient round, group meeting, etc.. Table 1 summarizes the hours we observed at each ward and number of filled out templates that we collected and then used for data analysis. We did not have access to the text-based documents generated by clinicians.

We analyzed the data through reflexive thematic analysis [14], combining an inductive (data-driven) and deductive (question-driven approach). The template for data retrieval was constructed bottom-up to identify combinations for data source and data type. After completion of all observations, the first author screened the filled-out templates, and extracted those that included interaction with

text-based documents with medical patient information. These templates were then discussed among co-authors and the final contexts (represented in a filled-out template) were analyzed in an inductive approach. For this, the first author screened each template and summarized the interaction with the text document, based on the available information regarding the context, also relying on field notes when needed. Afterwards, the first author reviewed the summaries performing open coding for the type of text document used, the purpose of the report in this situation, and, if available, associated challenges with its use. The codes were discussed with the co-authors and grouped into categories. These categories then were further grouped into themes, guided by our initial research questions in a deductive step. The first author then drafted a first version of the themes, which two of the authors contributed to refine.

# 4 RESULTS

The coding regarding the type of text document used, resulted in three distinct document types: (1) a final report from emergency care, (2) examination results and (3) medical discharge letters. Throughout the analysis we identified that physicians engage most frequently with the medical discharge letter, which is why we focus the presentation of our findings on this document.

Our observations reveal that medical discharge letters serve purposes beyond their anticipated role as summaries of medical care, they are also multifaceted artifacts used throughout a patient's hospitalization. In the following, we present our findings about their life cycle and composition through different documents and different purposes for which they are consulted. Furthermore we identified four challenges that physicians need to overcome to obtain necessary information for patient care.

## 4.1 Medical Discharge Letters Summarize the Patient Record and Give a Concise Patient Overview During Their Hospitalization

At the beginning of a patient's admission to the ward, the responsible physician usually creates a medical discharge letter, which is embedded in the patient's EHR of the hospital information system. Overall, this document follows a predefined structure: it starts with a concise summary of the patient's condition and main diagnoses, followed by a list of medication at admission. Then, it summarizes
<figure><img src="image_4.png" /><figcaption>**Figure 1:** The medical discharge letter is not merely a summary of a hospitalization at the end of a patient stay but rather embedded in the patient journey as an internal working document used for expansion of medical information and Internal communication. After discharge, this document is shared with other healthcare providers for further treatment.</figcaption></figure>

the results of conducted examinations during the hospitalization in separate blocks, which are usually a synthesis of different documents such as the final report of the emergency care unit or reports of examinations throughout the hospital stay. Finally, the report is condensed in a narrative summary and a list of medication at discharge. Any adjustments of medication are highlighted in this list. Although the length of this document varies depending on the complexity and duration of the hospital stay, it is usually over 3 pages long. Patients receive a printed copy at discharge. We observed that there were several nuances in how physicians used these documents based on their experience and personal preferences, which we further elaborate in the following sections.

## 4.2 The Life Cycle of a Medical Discharge Letter During a Patient's Hospitalization

We summarize in Figure 1 how a discharge letter evolves over a patient's hospitalization based on the different interactions that we have observed. We elaborate on the life cycle steps below.

4.2.1 *Synthesis.* At the beginning of a patient's hospitalization, the discharge letter is a document for synthesis of available information. Clinicians therefore first considered the **final emergency care**

report<fnref n="1" /> and if available previous discharge letters. These are only available in the system, if (a) they have been created in the same hospital or (b) manually transferred via fax from another clinician upon request. The first version of the discharge letter is an internal working document that duplicates previous information concisely in one place.

4.2.2 *Expansion and Internal Communication.* Upon creation, the internal document of the medical discharge letter functions as a succinct tracker of the patient's hospitalization progress, including medical as well as coordination efforts. This information is extracted or copied from documents of examination results<fnref n="2" />, which have been either written manually by another specialist or created automatically through a diagnostic software. Information which is not available yet, is implied through yellow highlighted placeholders, such as 'XXXX' or including designated blocks to examination results through a headline. We observed how different physicians used this structure, for example: *We conducted a [examination/procedure] that revealed XXXX.*

In other cases, physicians included the headline of a block and left it empty to indicate that this information needed to be obtained

<footnote n="1">¹German: *Abschlussbericht der zentralen Notaufnahme* (ZANA)</footnote>

<footnote n="2">²German: *Untersuchungsbefund*</footnote>
yet. As a physician explained in informal conversation, this struc-
ture allowed for implicit communication with colleagues to indicate
treatment plans and served as a mental note, that this procedure
had yet to be performed for further clarification. To verify if this
procedure had been scheduled already, physicians need to consult
further resources in the hospital information system. Overall, physi-
cians would mostly spend time on updating the letters with new
information in the afternoon after rounding all patients.

4.2.3 *Summary and External Communication.* The final discharge letter is the summary of the hospital stay and contains information intended for the physician responsible for further treatment, e.g., a general practitioner. In this step, the internal working document transforms to an external document that archives the relevant information of the hospital stay and is shared with the patient and other care providers. We observed, that this final summary depends on the style of the physician, and varies in style as well as level of detail of information. After finalizing and verifying this document, the letter is printed out with a table of the last laboratory results and physically handed out to the patient, who is responsible for sharing it with their care providers. Obtaining this physical discharge letter is the crucial perquisite for discharge; without this document the patient is not approved to leave the hospital. We observed nonetheless several incidents of inpatient patients who wished to leave. Remarkably, in two cases, patients chose to sign out against regulation because of their unwillingness to wait for the physical discharge document, resulting in considerable concern for the attending physicians.

## 4.3 Physicians Consult Discharge Letters for Different Purposes

The coding of purposes for which physicians consult medical dis-
charge letters resulted in eleven codes, that we have summarized
in four categories: (1) obtain an overview, (2) search for specific in-
formation, (3) Implicit communication, and, (4) create a final report.
Each category with its included codes is illustrated in Table 2 and
is further explained below.

4.3.1 *Obtain an Overview.* Medical discharge letters serve as an entry point to a patient's case. We observed different interactions in several situations to obtain an overview of either existing or newly assigned patients. Physicians consult them to familiarize themselves with a patient's medical history and read through the available information from top to bottom. At the beginning of a shift, residents at the ward sit down and read or skim through available information in the current discharge letter. After this familiarization, the residents round the patients with a mobile computer and a drawer with the respective paper records. Before visiting a patient, physicians read through the discharge letter to recollect the details of the case. This also includes looking through previous discharge letters (if available) and compare the disease progression.

4.3.2 *Search for Specific Information.* We found that medical dis-
charge letters are not only used to obtain an overview about a
patient case, but also serve as a source for specific information.
We mostly recorded these events during the afternoon as different
questions arose in the follow-ups and planning of further treatment.

This includes specific medical details like medication or previously-
known diagnoses. If this information was related to previous cases
or could not be retrieved from the medical chart right away, physi-
cians consulted available discharge letters and needed to retrieve
this information from the text.

4.3.3 *Implicit Communication.* We observed that physicians followed individual approaches to writing the discharge letter. Most of the physicians gave a textual short summary of the main conditions at the beginning. One physician however explained in a short informal interview that their senior physician taught them to furthermore summarize the main treatments that have been carried out for each of the diagnoses. As explained in Section 4.2.3, physicians included implicit information through flagging or arranging blocks in the letter for pending examinations. Furthermore, if text has been copied and pasted from examination reports, this was visually distinguishable through a different font, similar to the Courier font from a typewriter. These examination reports are either written by specialists or are automatic output from a diagnostic software and the differences in font style and text structure convey implicit information or hints about the text source.

4.3.4 *Create a Final Report.* The final version of the medical dis-
charge letter is the summary of the hospital stay and contains
information for the physician responsible for further treatment,
e.g., a general practitioner. Before a patient discharge, the respon-
sible physician verifies the discharge letter by following a very
similar procedure. That is, skim or read through the documents and
check the reported examinations. Afterwards, they read through
and finalize the narrative summary about the hospital stay and,
finally, they list the medication at discharge. Therefore, the physi-
cian compares the medication at admission from the top of the
medical discharge letter with the last recorded medication in the
patient’s EHR and provides a final list with highlighted changes.
We observed that each physician had an individual approach to
tagging these modifications and either wrote the new medications
in bold letters or added a tag such as (NEW) to the respective list
element. Most physicians used a voice recorder to write the final
letter. In one case, a physician relied on a 80-page document, that
they had created themselves, which included text snippets for nar-
rative descriptions of common conditions. Therefore, they used a
keyword search to find the text snippet that they then copied to
the discharge letter and modified to fit the respective case. After
finalizing and verifying the letter, it is printed out with a table of
the last laboratory results and physically handed out to the patient.

4.4 Challenges Associated With Interaction With Medical Discharge Letters

We observed several challenges related to the interaction and ease of
use with medical discharge letters. We summarize them as (1) avoid-
ance to use auto-generated text, (2) uncertainty about completeness
of information, and (3) duplication and missing interoperability.

4.4.1 Physicians Avoid Copying Auto Generated Text. We noted a reoccurring rejection of an auto-generated ECG text report. During our observations of four residents (Resident 2, Resident 9, Resident 1, Resident 5) interacting with the system in ten different situations, we could observe a similar pattern each time. The report is first
<table><caption>Table 2: The medical discharge letter for a patient's hospitalization was not just a document that was created at the end, but was consulted for different purposes, which we have grouped in four categories with respective situations.</caption><thead><tr><td>Category</td><td>Purpose for consulting the discharge letter</td></tr></thead><tbody><tr><td>Obtain an overview</td><td>Overview of all assigned patients at the beginning of a shift<br>Overview of a newly assigned patient<br>More detailed overview during patient round</td></tr><tr><td>Search for specific information</td><td>Specific value in previous or current discharge letter<br>Implicit information about further procedures<br>Updates on a patient's case</td></tr><tr><td>Implicit communication</td><td>Track progress of hospital stay<br>Coordinate further tasks<br>Compare with previous discharge letters</td></tr><tr><td>Create final report</td><td>Summarize procedures and results<br>Verify the final report</td></tr></tbody></table>

generated on request by a physician, and after its creation: (1) the physician skims through the text, (2) they write a new text that is much shorter than the automatic one, and, (3) they delete the previous auto-generated text. One collected field note illustrates a situation in which a physician detects that their colleague did not previously review the automated report and they discovered an inconsistency at the cardiology ward when reading through the medical discharge letter.

Based on the text structure and style, the physician notices that a colleague has copied the automatic report to the discharge letter and performs the analysis again to verify the result. This reveals errors in the recorded findings and the physician emphasises: "You just can't rely on them [medical notes]... This is so dangerous" — vignette depicting Resident 9 during post-processing of tasks (time: 3:22 PM).

### 4.4.2 Physicians Do Not Assume Available Text Reports Are Complete.
During work with data and text reports, there was a continuous uncertainty about the completeness of information. With a fragmentation of documents and their sources, "not all of the existing previous discharge letters are displayed in the system" (quote by senior physician during a patient round with Resident 4 at the cardiology ward, 11:57 AM). Therefore, physicians have to search different sources to build up the understanding of a patient case. When trying to obtain an overview, physicians read through available resources but need to keep in mind, that previous reports might exist, although missing from sight. This deficiency was a observed recurring with previous medical discharge letters from external care providers. This included not specifying certain information that were crucial for further treatment, as described in this situation:

Resident 8 reads a report on a patient with a pancreatic cancer. However, the report does not seem to provide further information about its localisation, which the resident expresses with the following quote: "The records we have on the case are just bad - but we won't get the records from the cardiologist until they are back from holiday" — vignette depicting Resident 8 trying to get updated on a patient (time: 02:30 PM)

Furthermore, lack of completeness was usually revealed when physicians would search for a very specific piece of information or report and could not find it. If this information could not be found, we observed that physicians consulted other sources such as colleagues, external physicians or patients and their relatives. There is no notification if new examination reports are available and therefore physicians either need to keep track of the information they need to obtain, or, they sometimes seemed to stumble upon it by sheer chance. Lastly, we observe that physicians did not always seem to know what they were looking for in the current or previous medical discharge letters given the variety of complex cases and individuality of physical parameters.

### 4.4.3 Physicians Must Navigate Through Duplicated Snippets to Find Information.
As patient data is represented in text reports that can take various shapes that extend multiple pages, information is replicated over different print and digital documents. The medical discharge letters were handled in a text editor that was not connected to the patient's EHR and thus needs manual transferal of information. This impedes straightforward data retrieval and physicians need to manually go through several documents to identify the required information. Oftentimes, this involves manually gathering these text documents as reflected in the following situation:

Resident 8 searches for their colleague to ask whether they know where to find the reports of a patient. They are displeased by the situation, as reflected in their exclamations: "I don't have any previous reports from the colleagues [...] It can't be true that I have to chase these documents [...] I don't have the time." — vignette depicting Resident 8 trying to obtain an overview about a patient (time: 08:41 AM)

The ward did not have access to tools that would facilitate this data retrieval. Instead, physicians needed to first identify the source of previous medical discharge letters (who is the principal general practitioner) and then try to obtain these reports, if they were missing in the system. Once they obtained the documents, they needed to scan various pages to find the necessary data. We did not observe physicians using search functions or other tools that would visually summarize the medical discharge letters in a broader picture.
The process to retrieve information from multilateral text-based documents rather resembled a situation of looking for a needle in a haystack, as shown in the following situation:

The attending resident and a senior physician read through the medical report to find the dosage of a specific medication. Therefore, they scan through multiple pages and seem to have difficulties to find what they are looking for. Finally, when they identify the dosage, the senior physician says: “*Oh this shows again, how important it is to look through the reports multiple times*” — attending Resident and a senior physician search for a specific piece of information.

Due to the overload and missing structure in the available data, we observed recurring situations in which physicians expressed a high perceived overload, for example, when reading through reports and claiming: “*What do I do first? That is actually the important question now?*” (Resident 3 at 12:53 PM), or “*You feel all day like you’re doing nothing*” (Resident 6 at 09:44).

# 5 DISCUSSION AND IMPLICATIONS FOR DESIGN

Our observations of 17 shifts at internal medicine of a municipal German hospital revealed heterogeneous uses of text-based reports and several challenges related to inaccurate, missing and fragmented data. We identified two key results, that are:

(1) Medical discharge letters are not just a document for the final summary of a hospital stay, but rather an artifact that physicians consult for different purposes to manage patient care.

(2) Physicians rejected a system that automatically created textual reports for ECG diagnoses and instead manually rewrote a concise version themselves.

With these results, we derive in the following section implications for the design of text-based systems and their automation potential in real-world clinical contexts.

## 5.1 Document Personalization: Implicit Messages and Individual Styles in Medical Discharge Letters

Several observations revealed an underlying personalization of the discharge letter that transported implicit messages to coordinate care. While the documents followed a general structure, the text paragraphs varied in style and text based on the author of the text, something that has also been observed in previous studies [7, 36]. Examining the use of these text-based reports from the perspective of different residents, we note that this personalization carries implicit messages besides the written facts, achieved through the flexibility and narrative nature of text. This includes another layer of information about procedures that are expected to be performed, by starting new (yet incomplete) sections or adding landmarks like (NEW) or XXXX to give further information about the context. Furthermore, we observed that the final medical discharge letter looked differently revealing who was the author, as style is more divergent than machine-generated text. As one resident explained, the structure of the summary in the medical discharge letter followed

a structure they have been taught by their supervisor. Moreover, reports with no personalization (machine-generated) were replaced by the physician's version of text, encapsulating the same message but formulated in their personal style.

In a collaborative setting, where hospital professionals know each other and are aware of other's expertise, identifying the author and even information about their supervisors based on personal style gives much background information that can be relevant for further actions. Internal medicine treats various complex cases, with multiple diagnoses that require individual treatment decisions associated with certain clinical uncertainty [55]. Such implicit messages can thus help to assess previous therapy decisions and coordinate further actions. Practices of coordination in hospitals are common, as identified in previous HCI research, notably the use of EHR systems to explicitly coordinate the inter-department transferring of patients [1], how the physical layout of hallways and rooms, along with spaces around information displays, facilitate coordination [50], or, how work is implicitly divided among surgeons when summarizing video recordings [5].

## 5.2 Implications for Moving Towards Automatic Text-Based Report Systems

With the recent and growing rise of new methods that automate data or text generation, it is important to discuss how the observed practices will be impacted in the near future. New technologies aim to support the documentation and administrative work by automating tasks like documenting patient encounters [58], the creation of medical discharge letters [42] or the generation of reports about examinations [25]. Our observations of medical discharge letters use in clinical care show that this document is used beyond summarizing the patient's hospitalization at the end of the stay. Rather it is built successively throughout the patient journey and physicians consult it for different purposes like obtaining a concise summary about a patient and include implicit messages about the hospitalization process.

In contrast, previous work has shown that automatic text-based reports can increase standardization to health records with the aim of homogenizing vocabulary or increasing data quality [15, 39] and many existing studies have focused on the technical feasibility and accuracy of such algorithms [44, 63]. As these solutions move towards clinical practice, with an overall positive attitude from practitioners towards these tools [17, 21, 31], our work provides more nuances on the collaborative work practices in a hospital. We suggest that medical text-documents, and specifically discharge letters should not only be considered as a report that consists of text blocks, but rather as a dynamic system that incorporates multiple interconnected processes. We argue that text automation should consider the use of implicit messages and reflection in discharge letters through, for example, offering different versions and prioritize to automate generation tasks rather than the content of the documents.

### 5.2.1 Distinguish Between Communication and Archiving Versions of Text Documents.
Despite their promised benefits, we believe automatic text generation systems potentially lessen the value of text-based reports if they remove space for personalization. Our observations support the need for a collaborative approach [21] between
physicians and text-generating systems that assist in text managing and incorporation of personalized comments. This aligns with notions from previous research that has identified that documentation interfaces should distinguish between repetitive and individual tasks [46]. The implication we present is that, by standardizing medical discharge letters, there is a potential risk to be avoided of not reflecting important information conveyed through the understanding of personalized text in the formal medical records, which then can only be transported through implicit communication. A possible example of avoiding this risk could involve a system that contains concise text snippets of examination results, presents them to the physician, and lets the physician generate, personalise and review the text. Furthermore, the observed information overload highlighted the need for elaborated text search functions that can help to make sense of the available data.

### 5.2.2 Automate Text Generation Tasks, Not Content.
We identified a recurring situation in which a deployed system that aimed to facilitate text generation by creating an automatic examination report, actually complicated this process. Four different residents deleted the proposed text from the system and replaced it with their individual version. One physician even claimed that the automatic report contained errors and blindly copying it would be "dangerous" to patient care. This is known as an irony of automation [6]: although automation is meant to alleviate problems, it can lead to their increase, discarding the original intended benefits. Moreover, this recurring situation reflects a common threat among HCI and UX designers that is known as the gap between "Work as Imagined (WAI) versus Work as Done (WAD)" [11] and observed in different clinical contexts [4, 18]. The non-use of the generated text shows the disparities between the envisioned idealized workflow and the actual, real-world execution of the task and a resulting workaround. Our second implication then is that, when automatic generated text contains errors that physicians detect, this will inevitably lead to their non-use. It is thus important to regularly assess the technology in real-world scenarios as the domain evolves, to understand when errors are being produced, to safely integrate automatic systems into clinical processes. Reasons beyond deficient implementation can also include fast changing environment. As new diseases, treatments, new guidelines or medications appear, systems may not be able to produce their reports correctly and potentially produce results that are not grounded in the factual information [34, 51]. Regular assessments should be carefully designed, as physicians are already dealing with information overload [38] and as our observations show, their capacity to oversee the systems is decreasing, which requires new interaction strategies.

## 5.3 Limitations and Future Work

Several limitations of this work need to be taken into account. We acknowledge that our population is limited to one municipal German hospital and thus do not claim generalizability of our findings to other institutions or cultures. However, we argue that text-based discharge letters and the prevalence of scattered documents and systems reflect the reality that we most likely encounter in health centers. Future work can complement and contrast out results looking at different institutions to create comparisons across different health system. Furthermore, we believe that the implications on

automation of text-based systems are interesting for other domains, but should be included with considerations. Also, the main focus of the hospital in this study is acute care and some deployed technologies might be outdated. In this study, we directed our focus towards internal medicine physicians, that usually need to make decisions about a variety of complex cases based on many heterogeneous data types from different sources. We acknowledge that healthcare is provided by various professionals, who are equally vital in the provision of care, like nurses, case managers, pharmacists, etc. Therefore, different specialists create and interact with a patients data and future studies should consider these perspectives.

During our observational study, many participants were residents who had just begun their medical training and we experiences challenges to convince senior physicians to participate in the shadowing. Three of the senior physicians we approached, referred us to a resident they were responsible for. While working with residents provides valuable insights into early stages of medical careers and the residents were much more involved in the daily data work, it may not fully capture the best practices for data retrieval during clinical care. The design of future medical systems would highly benefit from insights of senior physicians and find solutions how to integrate this expertise in the innovation.

# 6 CONCLUSION

This paper explores how internal medicine physicians navigate through text-based documents of patients with long-term medical conditions during clinical shifts. Therefore, we conducted an extensive observational study and identified how medical discharge letters were used beyond merely summarizing a patient's hospitalization. Our primary contribution is to bring into view the importance of understanding the different nuances and purposes for which these discharge letters are consulted throughout hospitalization to build up a collective understanding of a patient's case. We observed several challenges that were related to inaccurate, incomplete and fragmented data. Furthermore, we consistently observed how clinicians rejected a text-automation system designed to ease documentation burden, opting instead to rewrite the text manually. In respect to current developments of automatic text generation, this paper elaborates on the possible loss of implicit messages through personalization and suggest to distinguish between different layers of text-based documents (communication and archiving versions) and focus on automating tasks, not content.

## ACKNOWLEDGMENTS

We acknowledge that ChatGPT-4 has been used to restructure some sentences in the manuscript following current ACM policies. This research was supported by the Federal Ministry for Economic Affairs and Climate Action of Germany (BMWK) under Grant No. 68GX21004F (TEAM-X). This work was partially supported by European Research Council (ERC) grant n°695464 ONE: Unified Principles of Interaction. This research was supported in part by NSF awards #1928614 and #2129076. This project is partly funded by the Bavarian State Ministry of Science and the Arts and coordinated by the Bavarian Research Institute for Digital Transformation (bidt).
REFERENCES

[1] Joanna Abraham and Madhu C. Reddy. 2013. Re-coordinating activities: an investigation of articulation work in patient transfers. In *Proceedings of the 2013 conference on Computer supported cooperative work (CSCW '13)*. Association for Computing Machinery, San Antonio, Texas, USA, 67-78. https://doi.org/10.1145/2441776.2441787

[2] E. Ammenwerth and H.-P. Spötl. 2009. The time needed for clinical documentation versus direct patient care. A work-sampling analysis of physicians' activities. *Methods of Information in Medicine* 48, 1 (2009), 84-91.

[3] Tariq Osman Andersen, Francisco Nunes, Lauren Wilcox, Enrico Coiera, and Yvonne Rogers. 2023. Introduction to the Special Issue on Human-Centred AI in Healthcare: Challenges Appearing in the Wild. *ACM Trans. Comput.-Hum. Interact.* 30, 2 (June 2023), 25:1-25:12. https://doi.org/10.1145/3589961

[4] Ahmed Ashour, Darren M. Ashcroft, and Denham L. Phipps. 2021. Mind the gap: Examining work-as-imagined and work-as-done when dispensing medication in the community pharmacy setting. *Applied Ergonomics* 93 (May 2021), 103372. https://doi.org/10.1016/j.apergo.2021.103372

[5] Ignacio Avellino, Sheida Nozari, Geoffroy Canlorbe, and Yvonne Jansen. 2021. Surgical Video Summarization: Multifarious Uses, Summarization Process and Ad-Hoc Coordination. *Proc. ACM Hum.-Comput. Interact.* 5, CSCW1 (April 2021), 140:1-140:23. https://doi.org/10.1145/3449214

[6] Lisanne Bainbridge. 1983. Ironies of automation. *Automatica* 19, 6 (Nov. 1983), 775-779. https://doi.org/10.1016/0005-1098(83)90046-8

[7] Jörgen P. Bansler, Erling C. Havn, Kjeld Schmidt, Troels Mønsted, Helen Høgh Petersen, and Jesper Hastrup Svendsen. 2016. Cooperative Epistemic Work in Medical Practice: An Analysis of Physicians' Clinical Notes. *Computer Supported Cooperative Work (CSCW)* 25, 6 (Dec. 2016), 503-546. https://doi.org/10.1007/s10606-016-9261-x

[8] Jakob E. Bardram and Claus Bossen. 2005. A web of coordinative artifacts: collaborative work at a hospital ward. In *Proceedings of the 2005 ACM International Conference on Supporting Group Work (GROUP '05)*. Association for Computing Machinery, New York, NY, USA, 168-176. https://doi.org/10.1145/1099203.1099235

[9] Lisa Ann Baumann, Jannah Baker, and Adam G. Elshaug. 2018. The impact of electronic health record systems on clinical documentation times: A systematic review. *Health Policy (Amsterdam, Netherlands)* 122, 8 (Aug. 2018), 827-836. https://doi.org/10.1016/j.healthpol.2018.05.014

[10] Camilla Bjørnstad and Gunnar Ellingsen. 2019. Data work: A condition for integrations in health care. *Health Informatics Journal* 25, 3 (Sept. 2019), 526-535. https://doi.org/10.1177/1460458219833114

[11] Ann Blandford, Dominic Furniss, and Chris Vincent. 2014. Patient safety and interactive medical devices: Realigning work as imagined and work as done. *Clinical Risk* 20, 5 (Sept. 2014), 107-110. https://doi.org/10.1177/1356262214565500

[12] Claus Bossen and Lotte Groth Jensen. 2014. How physicians' achieve overview: a case-based study in a hospital ward. In *Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing (CSCW '14)*. Association for Computing Machinery, New York, NY, USA, 257-268. https://doi.org/10.1145/2531602.2531620

[13] Claus Bossen, Kathleen H Pine, Federico Cabitza, Gunnar Ellingsen, and Enrico Maria Piras. 2019. Data work in healthcare: An Introduction. *Health Informatics Journal* 25, 3 (Sept. 2019), 465-474. https://doi.org/10.1177/1460458219864730
Publisher: SAGE Publications Ltd.

[14] Virginia Braun and Victoria Clarke. 2019. Reflecting on reflexive thematic analysis. *Qualitative Research in Sport, Exercise and Health* 11, 4 (Aug. 2019), 589-597. https://doi.org/10.1080/2159676X.2019.1628806 Publisher: Routledge_eprint: https://doi.org/10.1080/2159676X.2019.1628806.

[15] S. H. Brown, S. Hardenbrook, L. Herrick, J. St Onge, K. Bailey, and P. L. Elkin. 2001. Usability evaluation of the progress note construction set. *Proceedings. AMIA Symposium*** (2001), 76-80.

[16] Federico Cabitza, Angela Locoro, Camilla Alderighi, Raffaele Rasoini, Domenico Compagnone, and Pedro Berjano. 2019. The elephant in the record: On the multiplicity of data recording work. *Health Informatics Journal* 25, 3 (Sept. 2019), 475-490. https://doi.org/10.1177/1460458218824705

[17] Carrie J Cai, Samantha Winter, David Steiner, Lauren Wilcox, and Michael Terry. 2019. "Hello AI": uncovering the onboarding needs of medical practitioners for human-Al collaborative decision-making. *Proceedings of the ACM on Human-computer Interaction* 3, CSCW (2019), 1-24.

[18] Rory Stuart Clark, Tom Owen, Matt Jones, Martin Porcheron, and Phillip Wardle. 2023. It Works Better When I Do That: Interaction and Communication In Radiology Departments. In *Proceedings of the 41st ACM International Conference on Design of Communication (SIGDOC '23)*. Association for Computing Machinery, New York, NY, USA, 55-62. https://doi.org/10.1145/3615335.3623011

[19] Thomas Davenport and Ravi Kalakota. 2019. The potential for artificial intelligence in healthcare. *Future Healthcare Journal* 6, 2 (June 2019), 94-98. https://doi.org/10.7861/futurehosp.6-2-94

[20] Nadav Even Chorev. 2019. Data ambiguity and clinical decision making: A qualitative case study of the use of predictive information technologies in a personalized cancer clinical trial. *Health Informatics Journal* 25, 3 (Sept. 2019), 500-510. https://doi.org/10.1177/1460458219827355

[21] David Fraile Navarro, A. Baki Kocaballi, Mark Dras, and Shlomo Berkovsky. 2023. Collaboration, not Confrontation: Understanding General Practitioners' Attitudes Towards Natural Language and Text Automation in Clinical Practice. *ACM Trans. Comput.-Hum. Interact.* 30, 2 (April 2023), 29:1-29:34. https://doi.org/10.1145/3569893

[22] Bundesministerium für Gesundheit. 2023. Gemeinsam Digital. Digitalisierungsstrategie für das Gesundheitswesen und die Pflege.

[23] Peter E. Gabriel, Aditi P. Singh, and Lawrence N. Shulman. 2023. Standardized Documentation Is Not the Solution to Reduce Physician Time in the Electronic Health Record-Reply. *JAMA oncology* 9, 8 (Aug. 2023), 1152. https://doi.org/10.1001/jamaoncol.2023.1526

[24] Xinning Gui, Yunan Chen, Xiaomiu Zhou, Tera L. Reynolds, Kai Zheng, and David A. Hanauer. 2020. Physician champions' perspectives and practices on electronic health records implementation: challenges and strategies. *JAMIA open* 3, 1 (April 2020), 53-61. https://doi.org/10.1093/jamiaopen/ooz051

[25] Philipp Harzig, Moritz Einfalt, and Rainer Lienhart. 2019. Automatic Disease Detection and Report Generation for Gastrointestinal Tract Examination. In *Proceedings of the 27th ACM International Conference on Multimedia (MM '19)*. Association for Computing Machinery, New York, NY, USA, 2573-2577. https://doi.org/10.1145/3343031.3356066

[26] Robert G. Hill, Lynn Marie Sears, and Scott W. Melanson. 2013. 4000 clicks: a productivity analysis of electronic medical records in a community hospital ED. *The American Journal of Emergency Medicine* 31, 11 (Nov. 2013), 1591-1594. https://doi.org/10.1016/j.ajem.2013.06.028

[27] Kasper Hornbæk and Morten Hertzum. 2011. The notion of overview in information visualization. *International Journal of Human-Computer Studies* 69, 7 (July 2011), 509-525. https://doi.org/10.1016/j.ijhcs.2011.02.007

[28] Lotte Groth Jensen and Claus Bossen. 2016. Factors affecting physicians' use of a dedicated overview interface in an electronic health record: The importance of standard information and standard documentation. *International Journal of Medical Informatics* 87 (March 2016), 44-53. https://doi.org/10.1016/j.ijmedinf.2015.12.009

[29] J. Klauber. 2023. Krankenhaus-Report 2023. https://www.springermedizin.de/krankenhaus-report-2023/25804918

[30] Thomson Kuhn, Peter Basch, Michael Barr, Thomas Yackel, and Medical Informatics Committee of the American College of Physicians. 2015. Clinical documentation in the 21st century: executive summary of a policy position paper from the American College of Physicians. *Annals of Internal Medicine* 162, 4 (Feb. 2015), 301-303. https://doi.org/10.7326/M14-2128

[31] Brenna Li, Noah Crampton, Thomas Yeates, Yu Xia, Xirong Tian, and Khai Truong. 2021. Automating Clinical Documentation with Digital Scribes: Understanding the Impact on Physicians. In *Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI '21)*. Association for Computing Machinery, New York, NY, USA, Article 658, 12 pages. https://doi.org/10.1145/3411764.3445172

[32] Lena Mamykina, David K. Vawdrey, Peter D. Stetson, Kai Zheng, and George Hripcsak. 2012. Clinical documentation: composition or synthesis? *Journal of the American Medical Informatics Association: JAMIA* 19, 6 (2012), 1025-1031. https://doi.org/10.1136/amjiajnлр-200200901

[33] Stephanie Medlock, Saeid Eslami, Marjan Askari, Erik Jan van Lieshout, Dave A. Dongelmans, and Ameen Abu-Hanna. 2011. Improved communication in post-ICU care by improving writing of ICU discharge letters: a longitudinal before-after study. *BMJ quality & safety* 20, 11 (Nov. 2011), 967-973. https://doi.org/10.1136/bmjqs-2011-000074

[34] Bertalan Meskó and Eric J. Topol. 2023. The imperative for regulatory oversight of large language models (or generative AI) in healthcare. *npr Digital Medicine* 6, 1 (July 2023), 1-6. https://doi.org/10.1038/s41746-023-00873-0 Publisher: Nature Publishing Group.

[35] Amanda J Moy, Mollie Hobensack, Kyle Marshall, David K Vawdrey, Eugene Y Kim, Kenrick D Cato, and Sarah C Rossetti. 2023. Understanding the perceived role of electronic health records and workflow fragmentation on clinician documentation burden in emergency departments. *Journal of the American Medical Informatics Association* 30, 5 (May 2023), 797-808. https://doi.org/10.1093/jamia/ocad038

[36] Troels Mønsted, Madhu Reddy, and Jørgen Bansler. 2011. *The Use of Narratives in Medical Work: A Field Study of Physician-Patient Consultations*. "", "". https://doi.org/10.1007/978-0-85729-913-0_5_Pages: 100.

[37] National Research Council (US) Committee on Engaging the Computer Science Research Community in Health Care Informatics. 2009. *Computational Technology for Effective Health Care: Immediate Steps and Strategic Directions*. National Academies Press (US), Washington (DC). http://www.ncbi.nlm.nih.gov/books/NBK20640/

[38] Sohn Nijor, Gavin Rallis, Nimit Lad, and Eric Gokcen. 2022. Patient Safety Issues From Information Overload in Electronic Medical Records. *Journal of Patient Safety* 18, 6 (Sept. 2022), e999-e1003. https://doi.org/10.1097/PTS.0000000000001002

[39] Rita Noumeir. 2006. Benefits of the DICOM structured report. *Journal of Digital Imaging* 19, 4 (Dec. 2006), 295-306. https://doi.org/10.1007/s10278-006-0631-7
[40] Lucila Ohno-Machado. 2011. Realizing the full potential of electronic health records: the role of natural language processing. *Journal of the American Medical Informatics Association* : JAMIA 18, 5 (2011), 539. https://doi.org/10.1136/amiajnl-2011-000501

[41] Sun Young Park and Yunan Chen. 2012. Adaptation as design: learning from an EMR deployment study. In *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12)*. Association for Computing Machinery, New York, NY, USA, 2097-2106. https://doi.org/10.1145/2207676.2208361

[42] Sajan B Patel and Kyle Lam. 2023. ChatGPT: the future of discharge summaries? *The Lancet Digital Health* 5, 3 (2023), e107-e108.

[43] Kathleen H. Pine. 2019. The qualitative dimension of healthcare data interoperability. *Health Informatics Journal* 25, 3 (Sept. 2019), 536-548. https://doi.org/10.1177/1460458219833095

[44] Rimma Pivovarov and Noémie Elhadad. 2015. Automated methods for the summarization of electronic health records. *Journal of the American Medical Informatics Association: JAMIA* 22, 5 (Sept. 2015), 938-947. https://doi.org/10.1093/jamia/ocv032

[45] Daniel Reichert, David Kaufman, Benjamin Bloxham, Herbert Chase, and Noémie Elhadad. 2010. Cognitive analysis of the summarization of longitudinal patient records. *AMIA ... Annual Symposium proceedings. AMIA Symposium* 2010 (Nov. 2010), 667-671.

[46] S. Trent Rosenbloom, Joshua C. Denny, Hua Xu, Nancy Lorenzi, William W. Stead, and Kevin B. Johnson. 2011. Data from clinical notes: a perspective on the tension between structure and flexible documentation. *Journal of the American Medical Informatics Association: JAMIA* 18, 2 (2011), 181-186. https://doi.org/10.1136/jamia.2010.007237

[47] S. Trent Rosenbloom, William W. Stead, Joshua C. Denny, Dario Giuse, Nancy M. Lorenzi, Steven H. Brown, and Kevin B. Johnson. 2010. Generating Clinical Notes for Electronic Health Record Systems. *Applied Clinical Informatics* 1, 3 (Jan. 2010), 232-243. https://doi.org/10.4338/ACI-2010-03-RA-0019

[48] A. Rumshisky, M. Ghassemi, T. Naumann, P. Szolovits, V. M. Castro, T. H. McCoy, and R. H. Perlis. 2016. Predicting early psychiatric readmission with natural language processing of narrative discharge summaries. *Translational Psychiatry* 6, 10 (Oct. 2016), e921. https://doi.org/10.1038/tp.2015.182

[49] Christine Maria Schwarz, Magdalena Hoffmann, Petra Schwarz, Lars-Peter Kamolz, Gernot Brunner, and Gerald Sendlhofer. 2019. A systematic literature review and narrative synthesis on the risks of medical discharge letters for patients' safety. *BMC health services research* 19, 1 (March 2019), 158. https://doi.org/10.1186/s12913-019-3989-1

[50] Peter G. Scupelli, Yan Xiao, Susan R. Fussell, Sara Kiesler, and Mark D. Gross. 2010. Supporting coordination in surgical suites: physical aspects of common information spaces. In *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10)*. Association for Computing Machinery, New York, NY, USA, 1777-1786. https://doi.org/10.1145/1753326.1753593

[51] P. G. Shekelle, E. Ortiz, S. Rhodes, S. C. Morton, M. P. Eccles, J. M. Grimshaw, and S. H. Woolf. 2001. Validity of the Agency for Healthcare Research and Quality clinical practice guidelines: how quickly do guidelines become outdated? *JAMA* 286, 12 (Sept. 2001), 1461-1467. https://doi.org/10.1001/jama.286.12.1461

[52] Nicole Sultanum, Michael Brudno, Daniel Wigdor, and Fanny Chevalier. 2018. More Text Please! Understanding and Supporting the Use of Visualization for Clinical Text Overview. In *Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18)*. Association for Computing Machinery, New York, NY, USA, 1-13. https://doi.org/10.1145/3173574.3173996

[53] Yuling Sun, Xiaojuan Ma, Silvia Lindtner, and Liang He. 2023. Data Work of Frontline Care Workers: Practices, Problems, and Opportunities in the Context of Data-Driven Long-Term Care. *Proc. ACM Hum.-Comput. Interact.* 7, CSCW1 (April 2023), 42:1-42:28. https://doi.org/10.1145/3579475

[54] Helena Tendedez, Maria-Angela Ferrario, Roisin McNaney, and Adrian Gradinar. 2022. Exploring Human-Data Interaction in Clinical Decision-making Using Scenarios: Co-design Study. *JMIR human factors* 9, 2 (May 2022), e32456. https://doi.org/10.2196/32456

[55] S. Timmermans and A. Angell. 2001. Evidence-based medicine, clinical uncertainty, and learning to doctor. *Journal of Health and Social Behavior* 42, 4 (Dec. 2001), 342-359.

[56] Amir Torab-Miandoab, Taha Samad-Soltani, Ahmadreza Jodati, and Peyman Rezaei-Hachesu. 2023. Interoperability of heterogeneous health information systems: a systematic literature review. *BMC medical informatics and decision making* 23, 1 (Jan. 2023), 18. https://doi.org/10.1186/s12911-023-02115-5

[57] European Union. 2022. Proposal for a REGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL on the European Health Data Space. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A52022PC0197

[58] Marieke M. van Buchem, Hileen Boosman, Martijn P. Bauer, Ilse M. J. Kant, Simone A. Cammel, and Ewout W. Steyerberg. 2021. The digital scribe in clinical practice: a scoping review and research agenda. *npj Digital Medicine* 4, 1 (March 2021), 1-8. https://doi.org/10.1038/s41746-021-00432-5 Publisher: Nature Publishing Group.

[59] Polyxeni Vassilakopoulou and Margunn Aanestad. 2019. Communal data work: Data sharing and re-use in clinical genetics. *Health Informatics Journal* 25, 3 (Sept. 2019), 511-525. https://doi.org/10.1177/1460458219833117

[60] Katharine Weetman, Rachel Spencer, Jeremy Dale, Emma Scott, and Stephanie Schnurr. 2021. What makes a "successful" or "unsuccessful" discharge letter? Hospital clinician and General Practitioner assessments of the quality of discharge letters. *BMC health services research* 21, 1 (April 2021), 349. https://doi.org/10.1186/s12913-021-06345-z

[61] Matthias Weigl, Andreas Müller, Andrea Zupanc, and Peter Angerer. 2009. Participant observation of time allocation, direct patient contact and simultaneous activities in hospital physicians. *BMC Health Services Research* 9, 1 (June 2009), 110. https://doi.org/10.1186/1472-6963-9-110

[62] Qingyu Xu, Yuling Sun, and Liang He. 2021. Design Considerations for Information Collaboration in Long-term Senior Care System. In *2021 IEEE 24th International Conference on Computer Supported Cooperative Work in Design (CSCWD).***, 1040-1045. https://doi.org/10.1109/CSCWD49262.2021.9437812

[63] Peng Zhang and Maged N. Kamel Boulos. 2023. Generative AI in Medicine and Healthcare: Promises, Opportunities and Challenges. *Future Internet* 15, 9 (Sept. 2023), 286. https://doi.org/10.3390/fi15090286 Number: 9 Publisher: Multidisciplinary Digital Publishing Institute.

[64] Kai Zheng, Raj M. Ratwani, and Julia Adler-Milstein. 2020. Studying workflow and workarounds in EHR-supported work to improve health system performance. *Annals of internal medicine* 172, 11 Suppl (June 2020), S116-S122. https://doi.org/10.7326/M19-0871

[65] Xiaomu Zhou, Mark S. Ackerman, and Kai Zheng. 2010. Doctors and psychosocial information: records and reuse in inpatient care. In *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10)*. Association for Computing Machinery, New York, NY, USA, 1767-1776. https://doi.org/10.1145/1753326.1753592
A IMPLEMENTING AN OBSERVATION TOOL WITH TEMPLATES

10.08.2023, 08:20

**Context:** Explanation of the context* (e.g., *Obtaining an overview of the patients*)
**Category:** Indirect patient contact
**Location:** Doctor's office

**Combination of Interaction:** Data source / Data type

<table><caption>Figure 2: A translated version from a template as it would show up after completing the macros for an observed context. Expression that are displayed in bold letters would be fixed input as generated by the macro. The note starts with a timestamp, followed by a note on the description about the observed template and an assignment of some parameters as described in Section 3.2. The displayed table shows exemplary field notes that would be assigned for a combination of data type and source.</caption><thead><tr><th></th><th>Status of a patient</th><th>Free text</th><th>Medication</th><th>Laboratory results</th><th>Next procedures</th><th>Timeseries of signals (e.g. ECG)</th><th>Information workflow</th><th>Medical image</th></tr></thead><tbody><tr><td>Hospital Information system</td><td></td><td></td><td></td><td>Opens the Lab results from the previous day and checks critical values</td><td></td><td></td><td></td><td></td></tr><tr><td>Patient curve</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Text-processing system</td><td></td><td>Reads through current admission letter</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Paper-based: personal Notes</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Paper-based: print-out document</td><td></td><td></td><td>Reads through a printed medication plan brought by the patient</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Other system</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Phone Call</td><td></td><td></td><td></td><td></td><td>Calls the senior physician to ask about the scheduled procedures for a patient</td><td></td><td>Asks a nurse who is responsible for a patient</td><td></td></tr><tr><td>In a conversation</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>From memory</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table>