SlowGuess/NAACL_2022_OCR huggingface dataset:
If the information was not recorded by the system, they would take their own supplementary notes.
Real-Time Augmenters
Although the output of the system was of sufficient quality, the speed of transcription was not fast enough to prompt follow-up questions.
In order to overcome this, they adopted a hybrid approach where they would take their own quick notes to use during the call, and replace these with the more detailed system output at the end of the call.
Post-Call Comparers
The clinician would take their own notes as normal and not observe the Note Generation system output at all while they are speaking to the patient.
Once the call had ended, they would compare the generated notes to their own manual notes and add any additional detail to their own notes from the AI model.
sultations), or copied selected statements into their own notes (one third of all consultations).
4 Note Generation system
The UX Exploration detailed so far gave us, among other insights, indications that: (i) a Note Generation tool could be of value to clinicians; (ii), it would need to generate notes in real time, in order for clinicians to evaluate its output as the consultation is progressing; and (iii), clinicians would still want to be able to take manual notes during the consultation.
Prior to this UX Exploration, we had developed a Note Generation system based on a publicly available BART sequence-to-sequence model (Lewis et al., 2020) pre-trained on the CNN summarisation dataset (Nallapati et al., 2016), which we further fine-tuned on a proprietary Babylon dataset of 10,000 transcripts and consultation notes. In Moramarco et al. (2022) we detail a non-production evaluation of the system, and in Papadopoulos Korfiatis et al. (2022) we release the mock transcript and note dataset used to evaluate it. Examples of model inputs and outputs from this mock dataset can be found in Appendix A.2.
The following modifications were made as a result of the three studies in the UX Exploration:
- The previous system could only generate notes in an end-to-end fashion from the complete consultation transcript. This is not compatible with the clinicians' preferred design, so we retrained the model to produce notes from incremental chunks of the transcript. This was achieved by mapping sentences of the notes and sentences of the transcript with
a similarity function and generating incremental training pairs for each consultation. This system will only append text to the previously generated note, preserving anything that the clinician might have previously validated.
- Automatically generated notes often contain inaccuracies or omissions (Moramarco et al., 2022). Using a Note Generation system in a live clinical setting might mislead the clinician into introducing errors in their consultation notes or influence the clinical decision making process. In order to mitigate those risks, we re-trained the Note Generation system to only generate the history and examination portions of the notes (Pearce et al., 2016), preventing it from producing any diagnosis or treatment advice by removing it from the training data.
5 Live Test
In order to test the usability of the updated Note Generation system in clinical practice, we incorporated the real-time system into the Clinical Portal and allowed a limited number of clinicians to use it in a supervised environment for a period of three weeks. Our goal from this live test was to answer the following questions:
- How well is the Note Generation system able to cope in a live setting?
- How many consultations does it take clinicians to start making use of the output from the Note Generation system?
- Are there any safety issues that need to be addressed?
- What is the current and potential value of the Note Generation system during consultations?
- Are there any usability issues that need to be addressed?
In order to maximise the learnings from the live test, the research approach needed to allow us to observe the Note Generation system being used live, ask follow-up questions and understand the clinicians' overall experience. The research approach for the live test had the following steps:
- Interviews with clinicians before and after the three week live test.
- Weekly observation of clinicians using the Note Generation system during real-world consultations.
- A post-consultation survey on the use and value of the system during that particular consultation.
- Ad-hoc questions about the clinician experience so far via private messages.
- Analytics on the Clinical Portal (such as UI event timestamps) to quantify Note Generation system usage.
As the Note Generation software is a clinician-facing tool and falls under Babylon normal service delivery, the patients were not participants of the live test. However, for all consultations observed, explicit consent to do so was obtained from the patients (consent form in Appendix A), and the observations were completely passive (no interaction) to minimise inconvenience and distress both to patient and clinician. As an extra mitigation for any generated note inaccuracy, we also clearly instructed clinicians to continue manually recording the key information of the consultation.
Five Babylon clinicians (GPs, three women, two men) were invited to participate in the three-week live test. The inclusion criteria were: (i) UK clinicians having worked at Babylon for at least 6 months, (ii) consulting at Babylon for at least 10 hours a week, (iii) willing to use the Note Generation system during their consultations for the three-week period, (iv) agreeing to have a small number of their consultations with patients observed by the researchers, (v) minimum bandwidth (download  $5\mathrm{Mb / s}$ , upload:  $3\mathrm{Mb / s}$ ; to support the observing researchers), (vi) be willing to participate in all our research activities related to the project (such as the interviews and post-consultation surveys). The participants' time in consultations was under their normal working hours at Babylon. The extra time
Participant	# Cons.	% copied	Overlap
A	101	97%	50%
B	81	23%	13%
C	69	42%	22%
D	18	50%	34%
E	38	13%	19%
Table 1: # Cons. shows the number of consultations carried out in the live test, % copied the amount of generated note copied into clinicians' own notes (calculated with UI copy events), and Overlap an indication of how much of the final note comes from the system. This is estimated with ROUGE-L (Lin, 2004) recall<sup>6</sup>, which measures the n-gram overlap between generated and final notes divided by the n-gram count in the final note.
for interviews and training was paid at the study standard rate.
Prior to the start of the live test, the clinicians were provided with 60 minutes one-on-one up-front training on how to use the tool. The training consisted of showing them videos of the Note Generation system during a mock consultation, warning them that the system is not able to capture nonverbal observations, and that the medico-legal responsibility for the notes still rested with them (the clinicians). They were then provided with access to the Note Generation system and enabled to use it during their real-world consultations. Through initial interviews, we gathered that three of our participants fit in the Sketchers note-taking style; one in Doodlers; and one in Touch Typers (see Figure 2). Throughout the experiment, we observed a total of 4 hours of live consultations for each clinician. Finally, we collected their summary feedback through 60-minute, one-on-one closure interviews.
5.2 Findings
Here are the key points uncovered by observing and interviewing the clinicians using the Note Generation system in live practice:
- In the initial observation sessions, all clinicians took manual notes, ignoring the output of the Note Generation system. However, by the end of the experiment, all clinicians were making use of the generated notes by copying them at least partly into their notes. Copying occurrence varied between clinicians (see Table 1).
- During difficult consultations (for example, consultations with multiple interconnected medical issues, over-explaining patients, or mental health cases), we observed all clinicians reverting to
manual note-taking instead of using the Note Generation output.
- Analysis of the notes showed that clinicians always made amendments to the Note Generation output before using it. These included: fixing any identified errors, reorganising the output into distinct paragraphs, changing references of days to dates ('on Saturday' -> '14/01/21'), and changing the use of pronouns to match their preferred style.
- In order to measure the impact of the Note Generation tool, we asked all clinicians the following questions both before and after their usage of the tool. The answers to these questions were collected through a rating scale (from very difficult '1' to very easy '10'); we report the average for all 5 clinicians.
- How easy is it to write up your notes within the ten minutes allocated for the consultation? (before: 4.8, after: 5.5)
- How easy is it to focus on the patient during consultations? (before: 6.2, after: 8.3)
- How easy is it to multitask during consultations with patients? (before: 5.8, after: 6.8)
- Finally, when asked in the post-experiment interviews, all clinicians reported that they would like to continue using the Note Generation tool in their clinical practice.
In this paper we presented a series of user research studies and described how they influenced the development of a Natural Language Processing system for automatically generating consultation notes.
Some of the main challenges in putting such a system into clinical practice are not due to the system's technology or limitations, but instead to the user experience design choices around it. User research and live testing is essential to ensure the system is actually useful for clinicians. Among the list of tools available, we found that a low-fidelity Wizard-of-Oz prototype was both incredibly valuable to gather hands-on feedback from clinicians and significantly easier to implement than a functional product prototype.
More specifically, the need to generate notes in real time is a requirement that was identified
through the UX Exploration and confirmed in both the Wizard-of-Oz and live test phases. To our knowledge, this has not been covered in similar studies and we believe it will be a key challenge for future systems. Similarly, the user interviews uncovered three clinical use cases that could prove challenging to a Note Generation system: patient dishonesty, multiple presenting complaints, and clinicians' non-verbal observations.
We observed an interest for the tool beyond the live test, even from clinicians who were initially skeptical of automatically generated notes. We also highlight the individual differences in personal note-taking styles, which mean that clinicians use it in different, but still valuable ways.
One limitation of our work so far is the relatively small sample of clinicians (no more than 7 in each round) and the short time frame (3 weeks) in which they used the tool in our live test. The results we present from three weeks of live testing might not be representative of long-term usage of the Note Generation tool. Another limitation is that the design, based on data and processes from the UK, may not transfer easily to other regions without substantial changes.
In conclusion, we find that user experience studies, currently not commonly carried out in NLP research, revealed essential requirements for the design of our Note Generation system. We believe that to design an interactive tool it is crucial to test with active users from the start and hope that this work may help further studies in the field of medical Note Generation.
We would like to thank all the clinicians who participated in the UX studies and the live test of the Note Generation system. We are grateful to Oriol Valldeperas for the beautiful diagram of the Wizard of Oz setup. Lastly, a warm 'thank you' to all the patients who have consented to their consultations being observed for the purpose of this study.
Brian G Arndt, John W Beasley, Michelle D Watkinson, Jonathan L Temte, Wen-Jan Tuan, Christine A Sinsky, and Valerie J Gilchrist. 2017. Tethered to the ehr: primary care physician workload assessment using ehr event log data and time-motion observations. The Annals of Family Medicine, 15(5):419-426.
$^{6}$ https://pypi.org/project/rouge-score/
Timothy W. Bickmore, Laura M. Pfeifer, and Brian W.
Jack. 2009. Taking the time to care: Empowering low health literacy hospital patients with virtual nurse agents. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '09, page 1265-1274, New York, NY, USA. Association for Computing Machinery.
Bharath Chintagunta, Namit Katariya, Xavier Amatriain, and Anitha Kannan. 2021. Medically aware gpt-3 as a data generator for medical dialogue summarization. In Proceedings of the Second Workshop on NLP for Medical Conversations, pages 66-76.
Martina A Clarke, Linsey M Steelege, Joi L Moore, Jeffery L Belden, Richelle J Koopman, and Min Soon Kim. 2013. Addressing human computer interaction issues of electronic health record in clinical encounters. In International Conference of Design, User Experience, and Usability, pages 381-390. Springer.
Nils Dahlback, Arne Jonsson, and Lars Ahrenberg. 1993. Wizard of oz studies—why and how. Knowledge-based systems, 6(4):258-266.
Seppo Enarvi, Marilisa Amoia, Miguel Del-Agua Teba, Brian Delaney, Frank Diehl, Stefan Hahn, Kristina Harris, Liam McGrath, Yue Pan, Joel Pinto, et al. 2020. Generating medical reports from patient-doctor conversations using sequence-to-sequence models. In Proceedings of the first workshop on NLP for medical conversations, pages 22-30.
Sakib Jalil, Trina Myers, Ian Atkinson, and Muriel Soden. 2019. Complementing a clinical trial with human-computer interaction: Patients' user experience with telehealth. *JMIR Hum Factors*, 6(2):e9481.
Anirudh Joshi, Namit Katariya, Xavier Amatriain, and Anitha Kannan. 2020. Dr. summarize: Global summarization of medical dialogue by exploiting local structures. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pages 3755-3763.
Amanda E. Lackey, Mariam Moshiri, Tarun Pandey, Chandana Lall, Neeraj Lalwani, and Puneet Bhargava. 2014. Productivity, part 1: Getting things done, using e-mail, scanners, reference managers, notetaking applications, and text expanders. Journal of the American College of Radiology, 11(5):481-489.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871-7880, Online. Association for Computational Linguistics.
Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81.
Sean MacAvaney, Sajad Sotudeh, Arman Cohan, Nazli Goharian, Ish Talati, and Ross W Filice. 2019. Ontology-aware clinical abstractive summarization. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1013-1016.
Herlind Megges, Silka Dawn Freiesleben, Christina Rösch, Nina Knoll, Lauri Wessel, and Oliver Peters. 2018. User experience and clinical effectiveness with two wearable global positioning system devices in home dementia care. *Alzheimer's & Dementia: Translational Research & Clinical Interventions*, 4:636-644.
Nir Menachemi and Robert G Brooks. 2006. Reviewing the benefits and costs of electronic health records and associated patient safety technologies. Journal of medical systems, 30(3):159-168.
Francesco Moramarco, Alex Papadopoulos Korfiatis, Aleksandar Savkov, and Ehud Reiter. 2021. A preliminary study on evaluating consultation notes with post-editing. In Proceedings of the Workshop on Human Evaluation of NLP Systems (HumEval), pages 62-68.
Francesco Moramarco, Alex Papadopoulos Korfiatis, Mark Perera, Damir Juric, Jack Flann, Ehud Reiter, Anya Belz, and Aleksandar Savkov. 2022. Human evaluation and correlation with automatic metrics in consultation note generation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics.
Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Caglar Gulçehre, and Bing Xiang. 2016. Abstractive text summarization using sequence-to-sequence rnns and beyond. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 280-290.
John J. Palmieri and Theodore A. Stern. 2009. Lies in the Doctor-Patient Relationship. Primary Care Companion to The Journal of Clinical Psychiatry, 11(4):163-168.
Alex Papadopoulos Korfiatis, Francesco Moramarco, Radmila Sarac, and Aleksandar Savkov. 2022. (in press): Primock57: A dataset of primary care mock consultations. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics.
Patricia F Pearce, Laurie Anne Ferguson, Gwen S George, and Cynthia A Langford. 2016. The essential soap note in an ehr age. *The Nurse Practitioner*, 41(2):29-36.
François Portet, Ehud Reiter, Albert Gatt, Jim Hunter, Somayajulu Sripada, Yvonne Freer, and Cindy Sykes. 2009. Automatic generation of textual summaries from neonatal intensive care data. Artificial Intelligence, 173(7-8):789-816.
Ehud Reiter and Anya Belz. 2009. An investigation into the validity of some metrics for automatically evaluating natural language generation systems. Computational Linguistics, 35(4):529-558.
Ehud Reiter, Roma Robertson, and Liesl M. Osman. 2003. Lessons from a failure: Generating tailored smoking cessation letters. Artificial Intelligence, 144(1):41-58.
Kimberly Stowers and Mustapha Mouloua. 2018. Human computer interaction trends in healthcare: an update. In Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care, volume 7, pages 88-91. SAGE Publications Sage CA: Los Angeles, CA.
Wen-wai Yim and Meliha Yetisgen-Yildiz. 2021. Towards automating medical scribing: Clinic visit dialogue2note sentence alignment and snippet summarization. In Proceedings of the Second Workshop on NLP for Medical Conversations, pages 10-20.
Longxiang Zhang, Renato Negrinho, Arindam Ghosh, Vasudevan Jagannathan, Hamid Reza Hassanzadeh, Thomas Schaaf, and Matthew R Gormley. 2021. Leveraging pretrained models for automatic summarization of doctor-patient conversations. In *Findings of the Association for Computational Linguistics: EMNLP* 2021, pages 3693-3712.
Yuhao Zhang, Daisy Yi Ding, Tianpei Qian, Christopher D Manning, and Curtis P Langlotz. 2018. Learning to summarize radiology findings. In Proceedings of the Ninth International Workshop on Health Text Mining and Information Analysis, pages 204-213.
A.1 Patient consent form
The following is the consent form we obtained from patients prior to a consultation observation.
"At Babylon, we sometimes have training doctors and support staff observe our clinics for learning how we use our internal systems. I have [X] participants observing today who have completed all relevant security checks and received specific training to enable them to safely observe consultations. So before we continue, we always check consent with patients - are you happy to proceed?"
A.2 Example mock transcript and corresponding note
Transcript	Note
Clinician	So, um, tell me what's been going on. You've been saying there's a problem with your hearing. Is that right?	History:
Hx of difficulty hearing left ear for 6 weeks with tinnitus and slight nausea/ dizziness.
One previous similar episode in the past- resolved spontaneously.
No discharge/fever/itchiness/pain
Doesn't use cotton wool buds
No Pmhx of note
Ex: Looks well, not in pain.
Imp: need to exclude impacted wax in ear canal first
Pln: for face to face GP
appointment in 5 days to examine ear
If any problems in interim to ring us back
Pt happy with and understands plan
Patient	Yeah, so I just feel I can't really hear as well as I used to, like my hearing is kind of deteriorating in some way.
Clinician	Right, OK. How long has this been going on for?
Patient	Uh about six weeks.
Clinician	Six weeks, OK. Um, and before that have you had any hearing problem at all?
Patient	Um I had something maybe, about a year ago, but it only lasted a couple of days, it wasn't anything as long as this.
Clinician	Right, OK, OK. And, um, in this six week period, have you had anything else happen? Have you had any other ear symptoms at all?
Patient	Um, I occasionally get like a ringing in my left ear, uh just on the one side and um there's actually been a few times when I felt kind of a bit sick or a bit dizzy as well.
Table 2: Snipet of a mock consultation transcript and the corresponding note, written by the consulting clinician. Text in violet remarks the diagnosis and treatment advice, which we exclude in the Live Test.
(a) Design 1: Live transcript of the consultation (red box) and generated note displayed at the end (green box).
EXAMINATION
PC SOBAR, c section 2-3 weeks ago HPC;
SOBAR while watching tv, acute onset. Cough 1/7 previously, unproductive.
No fever, no chills, non positional.
No chest pain, no pain on inspiration
C section 2-3 weeks ago. No complications, baby well. No symptoms.
No of blood thinners.
PMHx nill
Dhx. allergies to ibuprofen
Oex HR via telephone roughly 120bpm. tachy, regular
FHx,
Mother hx of clots on legs.
36D
and tinnitus is something that your medicine doctors. We don't really understand. It fully we haven't in people can't give you a specific cause for why some pieces get tempted and some patients do it can be triggered by a wide range of different things. And in fact, I'm glad that you mentioned that what I might do in the time between now and when you come for the appointment I'm going
37 D To put paste the link for a leaflet about tinnitus in the notes for this appointment. I really appreciate it. If you could start reading about it. Okay, and often that awareness of what it is and what the treatment options are. There are limited options that helps us in terms of conversation about what we do next if you're it was clear. Okay, so I'm going to paste a link to that and you can read about that. And what we'll do is we'll see you in about five this time. You won't be with me will be another GP, but they'll have my notes and they'll know exactly.
38 D See what what my question is, which is just a check your ears. Is that okay? 67
39 P Yes, that's correct.
40 D All right, great. So bear with me five minutes and then you come into the support team and give the number in the notes of this appointment bring them and then they'll stop you in okay.
41 P Okay, really?
42 D And if there's any problem any issues between now and when the clinic appointment is just book another appointment. You can speak to other also video, okay?
43 P Okay. 81
44 D All right, then take a nice to see this money. All right. 79
45 P Thank you.
(b) Design 2: No feedback during the consultation and generated note displayed at the end (green box).
PC SOBAR, c section 2-3 weeks ago
HPC;
SOBAR while watching tv, acute onset. Cough 1/7 previously,
unproductive.
C section 2-3 weeks ago. No complications, baby well. No
symptoms.
PMHxnil
FHx
(c) Design 3: Live note generated during the consultation (red box).
SOBAR while watching tv, acute onset. Cough 1/7 previously.
PMHx pill
Figure 5: The three designs presented to clinicians in Discovery Phase 2. A red box indicates a textual pane updated during the consultation. A green box indicates a pane updated at the end of the consultation.