Original: Exploring Alternative Methods of Visualizing Patient Data
Revised: Exploring Alternative Methods of Visualizing Patient Data

Original: A BSTRACT
Revised: A BSTRACT

Original: Patient data visualization can help healthcare providers gain an overview of their patient’s condition and assist in decision-making about the next steps on management and communication.
Revised: Patient data visualization can help healthcare providers gain an overview of their patient’s condition and assist in decision-making about the next steps on management and communication.

Original: We explore the acceptance and opinion of five different visualizations that can be used to summarize patient data, including a Text
Revised: We explore the acceptance and opinion of five different visualizations that can be used to summarize patient data, including a Text

Original: Summary, text and frequency-based Word Cloud, a Bar Graph, a time-based Line Graph and a newly developed Text Graph that combines text and time-based distribution.
Revised: Summary, text and frequency-based Word Cloud, a Bar Graph, a time-based Line Graph and a newly developed Text Graph that combines text and time-based distribution.

Original: Results from a user study with 15 professional healthcare providers, 16 first- or secondyear medical students, and 17 third or greater year medical students show that most visualizations are useful in extracting patient information and are received positively by the users.
Revised: Results from a user study with 15 professional healthcare providers, 16 first- or secondyear medical students, and 17 third or greater year medical students show that most visualizations are useful in extracting patient information and are received positively by the users.

Original: In addition,
Revised: In addition,

Original: Text Summary and Text Graph are rated to be the most useful visualizations in extracting patient health information.
Revised: Text Summary and Text Graph are rated to be the most useful visualizations in extracting patient health information.

Original: 1 I NTRODUCTION
Revised: 1 I NTRODUCTION

Original: Health care providers (HCPs) are health professionals whom a person sees when they are in need of medical care or advice.
Revised: Health care providers (HCPs) are health professionals whom a person sees when they are in need of medical care or advice.

Original: This may include physicians, medical specialists, nurses, etc., [1].
Revised: This may include physicians, medical specialists, nurses, etc., [1].

Original: Part of the continuity of this care/advice is documenting the interactions with, and measurements of, the people they see, which becomes the patient’s medical record.
Revised: Part of the continuity of this care/advice is documenting the interactions with, and measurements of, the people they see, which becomes the patient’s medical record.

Original: HCPs use the information found in a patient’s medical record to support their decisions on patient care and management [2].
Revised: HCPs use the information found in a patient’s medical record to support their decisions on patient care and management [2].

Original: Patient medical records consist of clinical notes and patient data including demographics, laboratory results, radiographic images, problem lists, medication lists, etc., that have been gathered via official requisitions and use approved, validated measurement techniques [3], [4].
Revised: Patient medical records consist of clinical notes and patient data including demographics, laboratory results, radiographic images, problem lists, medication lists, etc., that have been gathered via official requisitions and use approved, validated measurement techniques [3], [4].

Original: Patient-generated data is a recent trend in medical record data collection, where patients (or their caregivers) record or gather the patient’s own health data.
Revised: Patient-generated data is a recent trend in medical record data collection, where patients (or their caregivers) record or gather the patient’s own health data.

Original: The information collected may include health symptoms, lifestyle choices, biometric data, etc.
Revised: The information collected may include health symptoms, lifestyle choices, biometric data, etc.

Original: Specialized technology that can either be provided by an HCP, or a publicly available technology such as a FitBit™, is often used for collecting patient -generated data.
Revised: Specialized technology that can either be provided by an HCP, or a publicly available technology such as a FitBit™, is often used for collecting patient -generated data.

Original: However, these data are less formal and possibly less trustworthy because of periodic inaccuracies [6].
Revised: However, these data are less formal and possibly less trustworthy because of periodic inaccuracies [6].

Original: The quantity of patient data collected can be overwhelming to process by HCPs as there can be many individual data items in different styles and formats from a large variety of sources [7] – [9]
Revised: The quantity of patient data collected can be overwhelming to process by HCPs as there can be many individual data items in different styles and formats from a large variety of sources [7] – [9]

Original: The information overload increases even further for patients with chronic illnesses as patient data accumulates over time [8], [10].
Revised: The information overload increases even further for patients with chronic illnesses as patient data accumulates over time [8], [10].

Original: In addition, sorting through and interpreting patient records can be time consuming and intensive.
Revised: In addition, sorting through and interpreting patient records can be time consuming and intensive.

Original: HCPs are under constant time pressure due to the amount and complexity of patient cases they have under their care.
Revised: HCPs are under constant time pressure due to the amount and complexity of patient cases they have under their care.

Original: Gathering information, interpreting it, and deciding the next step for their patient must be done as quickly as possible [11], [12].
Revised: Gathering information, interpreting it, and deciding the next step for their patient must be done as quickly as possible [11], [12].

Original: There is a need to have patient data presented in a concise and summarized manner allowing healthcare providers to efficiently access relevant data and it to manage their patient care [9].
Revised: There is a need to have patient data presented in a concise and summarized manner allowing healthcare providers to efficiently access relevant data and it to manage their patient care [9].

Original: MyHealthMyRecord (MHMR) is designed to allow patients to self-produce brief audio-video recordings of their experiences inbetween visits to their healthcare provider [13].
Revised: MyHealthMyRecord (MHMR) is designed to allow patients to self-produce brief audio-video recordings of their experiences inbetween visits to their healthcare provider [13].

Original: Currently the system lacks a method for visualizing data contained in the recordings.
Revised: Currently the system lacks a method for visualizing data contained in the recordings.

Original: This paper presents an evaluation of five methods for summarizing the patient-generated data from MHMR.
Revised: This paper presents an evaluation of five methods for summarizing the patient-generated data from MHMR.

Original: The five methods are a Text-Summary, text-based Word Cloud, frequency and time-based graphs, and a newly developed Text Graph that combines text and time-based distribution.
Revised: The five methods are a Text-Summary, text-based Word Cloud, frequency and time-based graphs, and a newly developed Text Graph that combines text and time-based distribution.

Original: The research questions we aim to answer are: 1) How do healthcare providers interpret visual summaries of patientgenerated data presented in the different MHMR visualization formats?
Revised: The research questions we aim to answer are: 1) How do healthcare providers interpret visual summaries of patientgenerated data presented in the different MHMR visualization formats?

Original: and 2) What are the preferences and acceptability of these visualizations for managing patient care?
Revised: and 2) What are the preferences and acceptability of these visualizations for managing patient care?

Original: In this paper, we will present the design and implementation of five MHMR visualizations that include a Text Summary, Word Cloud, Bar
Revised: In this paper, we will present the design and implementation of five MHMR visualizations that include a Text Summary, Word Cloud, Bar

Original: Graph, Line Graph, Text Graph, and Text Summary.
Revised: Graph, Line Graph, Text Graph, and Text Summary.

Original: We will then present and discuss our findings from a qualitative evaluation with 15 professional HCPs and 33 medical students.
Revised: We will then present and discuss our findings from a qualitative evaluation with 15 professional HCPs and 33 medical students.

Original: Because these individuals vary in their level of medical training, we want to investigate whether there is a difference between how they perceive the five visualizations and how useful they find them in extracting patient health information.
Revised: Because these individuals vary in their level of medical training, we want to investigate whether there is a difference between how they perceive the five visualizations and how useful they find them in extracting patient health information.

Original: The results and discussion cover the users’ rating of different visualizations based on usefulness and their opinions of the visualizations.
Revised: The results and discussion cover the users’ rating of different visualizations based on usefulness and their opinions of the visualizations.

Original: 2 B ACKGROUND
Revised: 2 B ACKGROUND

Original: Data visualization is the use of graphics to illustrate information [14] that can then be used to support decisions [2].
Revised: Data visualization is the use of graphics to illustrate information [14] that can then be used to support decisions [2].

Original: Visualization enables one to efficiently find trends and outliers within a dataset and understand underlying patterns.
Revised: Visualization enables one to efficiently find trends and outliers within a dataset and understand underlying patterns.

Original: It also allows the detection of trends and patterns, which can then be presented and communicated to others so they can understand and make sense of the data as well [15].
Revised: It also allows the detection of trends and patterns, which can then be presented and communicated to others so they can understand and make sense of the data as well [15].

Original: 2. Methods of visualizing data
Revised: 2. Methods of visualizing data

Original: One common method of visualizing data is using a graph that displays a relationship between two or more variables within a dataset [16].
Revised: One common method of visualizing data is using a graph that displays a relationship between two or more variables within a dataset [16].

Original: One type of graph that is often used to depict continuous data is a line graph [16].
Revised: One type of graph that is often used to depict continuous data is a line graph [16].

Original: A line graph connects data points displayed on a two-dimensional scale [16].
Revised: A line graph connects data points displayed on a two-dimensional scale [16].

Original: An advantage of a line graph is that it can highlight trend [17], and multiple continuous datasets can be plotted on the same graph for comparison.
Revised: An advantage of a line graph is that it can highlight trend [17], and multiple continuous datasets can be plotted on the same graph for comparison.

Original: But, when reading line graphs, individuals spend less time viewing the trends and more time relating different graphical features such as axis and graph titles or data point labels to one another to make sense of the graph [18].
Revised: But, when reading line graphs, individuals spend less time viewing the trends and more time relating different graphical features such as axis and graph titles or data point labels to one another to make sense of the graph [18].

Original: Another example of a common graph is a bar graph, created with the use of vertical or horizontal columns.
Revised: Another example of a common graph is a bar graph, created with the use of vertical or horizontal columns.

Original: A bar graph compares a single variable (often the dependent variable) against several variables, and each column represents one group [16].
Revised: A bar graph compares a single variable (often the dependent variable) against several variables, and each column represents one group [16].

Original: Word clouds are another method of data visualization.
Revised: Word clouds are another method of data visualization.

Original: They summarize a body of text by illustrating the words that occur most frequently [19].
Revised: They summarize a body of text by illustrating the words that occur most frequently [19].

Original: A high-frequency word will be shown in the word cloud in large font, and any words mentioned less frequently in comparison will be displayed in a smaller font or not included at all [20].
Revised: A high-frequency word will be shown in the word cloud in large font, and any words mentioned less frequently in comparison will be displayed in a smaller font or not included at all [20].

Original: Text features and word placements are often used to create a word cloud [21].
Revised: Text features and word placements are often used to create a word cloud [21].

Original: Text features describe the font colours, fontweight, and font size.
Revised: Text features describe the font colours, fontweight, and font size.

Original: Word placement then describes the layout of the word cloud ranging from sorted (e.g., alphabetically), to semantically clustered (e.g., placing all nouns together), and to spatially laid out (i.e., unordered placement of words)
Revised: Word placement then describes the layout of the word cloud ranging from sorted (e.g., alphabetically), to semantically clustered (e.g., placing all nouns together), and to spatially laid out (i.e., unordered placement of words)

Original: Word clouds are useful for four main activities: searching, browsing, impression forming and recognizing or matching [21].
Revised: Word clouds are useful for four main activities: searching, browsing, impression forming and recognizing or matching [21].

Original: When an individual uses the word cloud to search, they look for cues such as font size or colour to get a sense of the organization and frequencies of words as proxies for underlying concepts.
Revised: When an individual uses the word cloud to search, they look for cues such as font size or colour to get a sense of the organization and frequencies of words as proxies for underlying concepts.

Original: When users browse a word cloud, they get an overview of the text properties, forming impressions of which concepts are important and inferring information about the text data underlying the world cloud, and in some cases identifying themes that emerge within the dataset [21].
Revised: When users browse a word cloud, they get an overview of the text properties, forming impressions of which concepts are important and inferring information about the text data underlying the world cloud, and in some cases identifying themes that emerge within the dataset [21].

Original: 2. Healthcare data visualization
Revised: 2. Healthcare data visualization

Original: Some examples of early work in healthcare data visualization include a one-page detailed graphical summary proposed by
Revised: Some examples of early work in healthcare data visualization include a one-page detailed graphical summary proposed by

Original: Powsner and Tufte [22], and Lifelines by Plaisant et al. , [23].
Revised: Powsner and Tufte [22], and Lifelines by Plaisant et al. , [23].

Original: The graphical summary could reveal patient condition status to physicians by plotting numerical patient data as a variation of a scatter plot, then adding doctor’s notes and relevant medical images on the same page [22].
Revised: The graphical summary could reveal patient condition status to physicians by plotting numerical patient data as a variation of a scatter plot, then adding doctor’s notes and relevant medical images on the same page [22].

Original: Lifelines display a patient's history as a timeline where patient visitation dates are on the horizontal axis and information such as patient concerns, diagnosis, medications etc. are presented on the vertical axis as dots or horizontal lines depending on their duration [23].
Revised: Lifelines display a patient's history as a timeline where patient visitation dates are on the horizontal axis and information such as patient concerns, diagnosis, medications etc. are presented on the vertical axis as dots or horizontal lines depending on their duration [23].

Original: Both of these studies only have one type of visualization for users.
Revised: Both of these studies only have one type of visualization for users.

Original: The graphical summary by
Revised: The graphical summary by

Original: Powsner and Tufte displays the patient data as a variation of a scatter plot and Lifelines has a single mode of display using a timeline.
Revised: Powsner and Tufte displays the patient data as a variation of a scatter plot and Lifelines has a single mode of display using a timeline.

Original: The MHMR system provides five different visualizations for the users to be able to choose what is best for them in viewing and understanding patient data.
Revised: The MHMR system provides five different visualizations for the users to be able to choose what is best for them in viewing and understanding patient data.

Original: Sultanum et al.
Revised: Sultanum et al.

Original: , [24] developed the Doccurate system to present patient records on a timeline similar to Lifelines and also included a text panel showing associated clinical notes for each patient session.
Revised: , [24] developed the Doccurate system to present patient records on a timeline similar to Lifelines and also included a text panel showing associated clinical notes for each patient session.

Original: The text panel was added because the clinical text was found to be the physician’s primary source of information as they used it to obtain a general understanding of the patient condition, to recall information, and to assist in answering patient-related questions [12].
Revised: The text panel was added because the clinical text was found to be the physician’s primary source of information as they used it to obtain a general understanding of the patient condition, to recall information, and to assist in answering patient-related questions [12].

Original: Additionally, since text-based patient records and annotations have been a dominant part of physicians’ practice , text could be useful as a “familiar place to return to” [12, p. 10].
Revised: Additionally, since text-based patient records and annotations have been a dominant part of physicians’ practice , text could be useful as a “familiar place to return to” [12, p. 10].

Original: A filtering system in Doccurate allowed users to select a particular medical condition term.
Revised: A filtering system in Doccurate allowed users to select a particular medical condition term.

Original: As more mentions of these filtering terms occurred in the written notes, the terms appeared in larger fonts on the timeline.
Revised: As more mentions of these filtering terms occurred in the written notes, the terms appeared in larger fonts on the timeline.

Original: A user study with one physician and five residents was conducted to evaluate the system.
Revised: A user study with one physician and five residents was conducted to evaluate the system.

Original: The participants were asked to compare the patient's information gathered with the system using a set of filter terms they generated themselves, with a set of predefined terms.
Revised: The participants were asked to compare the patient's information gathered with the system using a set of filter terms they generated themselves, with a set of predefined terms.

Original: They attempted to gather information using both sets of terms for two patients.
Revised: They attempted to gather information using both sets of terms for two patients.

Original: The main findings from this study were that the participants were satisfied with the system but generally had a low level of trust in the automation used in Doccurate, because it made classification errors [24].
Revised: The main findings from this study were that the participants were satisfied with the system but generally had a low level of trust in the automation used in Doccurate, because it made classification errors [24].

Original: Similar to graphical summaries [25] and Lifelines [23],
Revised: Similar to graphical summaries [25] and Lifelines [23],

Original: Doccurate only had one method of visualization (a timeline), limiting the user’s choice in how they could display the patient data.
Revised: Doccurate only had one method of visualization (a timeline), limiting the user’s choice in how they could display the patient data.

Original: The use of different font-sizes to highlight words in Doccurate is similar to MHMR’s Word Cloud and Text Graph features bu t in
Revised: The use of different font-sizes to highlight words in Doccurate is similar to MHMR’s Word Cloud and Text Graph features bu t in

Original: Doccurate different font-sizes are intended to draw attention to certain visitation dates, and the Word Cloud and Text Graph text are intended to focus on the symptoms themselves.
Revised: Doccurate different font-sizes are intended to draw attention to certain visitation dates, and the Word Cloud and Text Graph text are intended to focus on the symptoms themselves.

Original: The Harvest system [26] displayed patient data as a longitudinal timeline, problem cloud, and doctor’s notes.
Revised: The Harvest system [26] displayed patient data as a longitudinal timeline, problem cloud, and doctor’s notes.

Original: The problem clouds (word clouds) demonstrated concepts extracted from the notes, based on the frequency of mention.
Revised: The problem clouds (word clouds) demonstrated concepts extracted from the notes, based on the frequency of mention.

Original: This system was based on the work of Reichert et al. , [8] who asked physicians to create patient record summaries.
Revised: This system was based on the work of Reichert et al. , [8] who asked physicians to create patient record summaries.

Original: The aim was to determine which section of the patient record the physicians spent most of their time in creating summaries.
Revised: The aim was to determine which section of the patient record the physicians spent most of their time in creating summaries.

Original: Similar to the findings of Sultanum et al .
Revised: Similar to the findings of Sultanum et al .

Original: , [12], the notes section was used the most by the participants, perhaps indicating that reading text summaries is the easiest way for physicians to achieve an overview.
Revised: , [12], the notes section was used the most by the participants, perhaps indicating that reading text summaries is the easiest way for physicians to achieve an overview.

Original: Since users preferred text, the researchers recommended the use of problem clouds and provision of a functionality to view notes.
Revised: Since users preferred text, the researchers recommended the use of problem clouds and provision of a functionality to view notes.

Original: The Word Cloud visualization in MHMR displays the most frequently mentioned words in the patient videos.
Revised: The Word Cloud visualization in MHMR displays the most frequently mentioned words in the patient videos.

Original: In contrast to
Revised: In contrast to

Original: Harvest, it displays day-to-day variation of particular symptoms.
Revised: Harvest, it displays day-to-day variation of particular symptoms.

Original: The Line Graph and Text Graph in MHMR illustrate the distribution of a particular symptom over time.
Revised: The Line Graph and Text Graph in MHMR illustrate the distribution of a particular symptom over time.

Original: The MHMR system is a mobile application that patients can use to audio-video record their experiences in-between HCP visits.
Revised: The MHMR system is a mobile application that patients can use to audio-video record their experiences in-between HCP visits.

Original: A case study with one patient was conducted to evaluate the use of
Revised: A case study with one patient was conducted to evaluate the use of

Original: MHMR, and to explore the topics, and issues that arose during the patient’s journey [13].
Revised: MHMR, and to explore the topics, and issues that arose during the patient’s journey [13].

Original: The patient, who was diagnosed with a chronic disease, used a tablet version of MHMR for three months and documented the frustrations or barriers that were faced.
Revised: The patient, who was diagnosed with a chronic disease, used a tablet version of MHMR for three months and documented the frustrations or barriers that were faced.

Original: Results from this study showed that the patient was willing and able to create videos about their experience and that there were readily identifiable themes related to health, pain, and accessibility issues.
Revised: Results from this study showed that the patient was willing and able to create videos about their experience and that there were readily identifiable themes related to health, pain, and accessibility issues.

Original: While the task of making video entries may be doable and worthwhile for patients, the information contained in the videos could also be useful to their HCP in understanding events, activities and issues that arise between visits and that may affect the patient’s ability to manage their health conditions.
Revised: While the task of making video entries may be doable and worthwhile for patients, the information contained in the videos could also be useful to their HCP in understanding events, activities and issues that arise between visits and that may affect the patient’s ability to manage their health conditions.

Original: However, patients may generate a large number of video materials.
Revised: However, patients may generate a large number of video materials.

Original: Asking HCPs to watch, analyze and understand a large set of videos in the time that is usually allocated to individual patients for an HCP visit is unrealistic and may interfere with the quality of the interaction (e.g., through factors such as reduced eye contact).
Revised: Asking HCPs to watch, analyze and understand a large set of videos in the time that is usually allocated to individual patients for an HCP visit is unrealistic and may interfere with the quality of the interaction (e.g., through factors such as reduced eye contact).

Original: Thus it was important to organize the large quantity of data in MHMR so that it could be useful to HCPs in managing their patient’s care by incorporating the concerns, activities and progress identified in the videos into consultations and decision-making [13].
Revised: Thus it was important to organize the large quantity of data in MHMR so that it could be useful to HCPs in managing their patient’s care by incorporating the concerns, activities and progress identified in the videos into consultations and decision-making [13].

Original: 3 M ETHOD
Revised: 3 M ETHOD

Original: A user study was designed to evaluate the usability and visualization preferences of MHMR from three different groups of
Revised: A user study was designed to evaluate the usability and visualization preferences of MHMR from three different groups of

Original: HCPs (first- or second-year medical students, third or greater year medical students, and professional healthcare providers).
Revised: HCPs (first- or second-year medical students, third or greater year medical students, and professional healthcare providers).

Original: The study was conducted online with a desktop computer using Zoom conferencing services [27].
Revised: The study was conducted online with a desktop computer using Zoom conferencing services [27].

Original: A prototype web application for the visualization aspects of MHMR was developed and deployed to
Revised: A prototype web application for the visualization aspects of MHMR was developed and deployed to

Original: GitHub Pages [28] for the online study.
Revised: GitHub Pages [28] for the online study.

Original: In addition to the five visualizations, the prototype also had a set of six samples of 30second personal health videos.
Revised: In addition to the five visualizations, the prototype also had a set of six samples of 30second personal health videos.

Original: These videos represented examples of a patient’s perspective on pain over a period of time.
Revised: These videos represented examples of a patient’s perspective on pain over a period of time.

Original: This study was approved by Ryerson University and the University of
Revised: This study was approved by Ryerson University and the University of

Original: T oronto’s ethics board.
Revised: T oronto’s ethics board.

Original: 3. MHMR Visualizations
Revised: 3. MHMR Visualizations

Original: Iterative design of the MHMR visualizations was carried out, with three versions being generated over a two-month period based on feedback from the MHMR team.
Revised: Iterative design of the MHMR visualizations was carried out, with three versions being generated over a two-month period based on feedback from the MHMR team.

Original: The MHMR team consisted of individuals with experience in computer science, human computer interaction, and 5-10 years of medical experience.
Revised: The MHMR team consisted of individuals with experience in computer science, human computer interaction, and 5-10 years of medical experience.

Original: Although not part of the system initially, filters/sorting options were added to the system to provide customization, based on feedback from the team.
Revised: Although not part of the system initially, filters/sorting options were added to the system to provide customization, based on feedback from the team.

Original: Over the second and third iterations, the correct type (e.g., physical health symptoms, all words, top ten words mentioned, etc.) and wording (e.g., health symptoms vs. physical health symptoms) for the filters were selected.
Revised: Over the second and third iterations, the correct type (e.g., physical health symptoms, all words, top ten words mentioned, etc.) and wording (e.g., health symptoms vs. physical health symptoms) for the filters were selected.

Original: Currently, there are five visualizations that are used in the data visualization study for MHMR:
Revised: Currently, there are five visualizations that are used in the data visualization study for MHMR:

Original: Word Cloud, Bar Graph, Line graph, Text Graph, and Text Summary.
Revised: Word Cloud, Bar Graph, Line graph, Text Graph, and Text Summary.

Original: These are made available in a mobile app that simulates the visualization functionality that could be used in the prototype MHMR application.
Revised: These are made available in a mobile app that simulates the visualization functionality that could be used in the prototype MHMR application.

Original: To generate the visualizations, audio from the patient videos was first transcribed into text using IBM Watson’s speech to text feature [29].
Revised: To generate the visualizations, audio from the patient videos was first transcribed into text using IBM Watson’s speech to text feature [29].

Original: Common “stop” words such as “the, and, but, how, a, etc.” are eliminated from the transcript using a natural language toolkit inside a python script.
Revised: Common “stop” words such as “the, and, but, how, a, etc.” are eliminated from the transcript using a natural language toolkit inside a python script.

Original: The remaining words and their frequency of occurrence in the videos are then used for creating the five visualizations.
Revised: The remaining words and their frequency of occurrence in the videos are then used for creating the five visualizations.

Original: In this section, the examples show the occurrences of words from a sample patient video set.
Revised: In this section, the examples show the occurrences of words from a sample patient video set.

Original: 3.1.1 Word Cloud
Revised: 3.1.1 Word Cloud

Original: The Word Cloud example used in this MHMR data visualization is generated by adding all words extracted from the transcript to an online tool, WordItOut [30] (see Figure 1).
Revised: The Word Cloud example used in this MHMR data visualization is generated by adding all words extracted from the transcript to an online tool, WordItOut [30] (see Figure 1).

Original: This tool allows for the selection of font colour and style that match the other visualizations.
Revised: This tool allows for the selection of font colour and style that match the other visualizations.

Original: Within the MHMR data visualization tool, the user is able to sort and organize the Word Cloud by selecting a minimum number of word occurrences
Revised: Within the MHMR data visualization tool, the user is able to sort and organize the Word Cloud by selecting a minimum number of word occurrences

Original: e.g., 5 or more instances, 10 or more instances and more than 30 instances.
Revised: e.g., 5 or more instances, 10 or more instances and more than 30 instances.

Original: The user is also able to filter between “All words” and “Physical health symptoms”.
Revised: The user is also able to filter between “All words” and “Physical health symptoms”.

Original: The “All words” option displays all words that are present in the transcript providing a birds-eye-view of all of the experiences reported by the patient during the recording period.
Revised: The “All words” option displays all words that are present in the transcript providing a birds-eye-view of all of the experiences reported by the patient during the recording period.

Original: When the word cloud is limited to only “Physical health symptoms”, HCP ca n focus on healthrelated issues.
Revised: When the word cloud is limited to only “Physical health symptoms”, HCP ca n focus on healthrelated issues.

Original: W ords such as “pain” and “swollen” in the “Physical health symptoms” option are extracted from the “All - words” list.
Revised: W ords such as “pain” and “swollen” in the “Physical health symptoms” option are extracted from the “All - words” list.

Original: 3.1.2 Bar Graph
Revised: 3.1.2 Bar Graph

Original: The Bar Graph example illustrates the ten most frequently mentioned words in the videos and displays them with five sorting and filtering options (see Figure 2).
Revised: The Bar Graph example illustrates the ten most frequently mentioned words in the videos and displays them with five sorting and filtering options (see Figure 2).

Original: The x-axis represents the words, and the y-axis represents the frequency of the word in the videos.
Revised: The x-axis represents the words, and the y-axis represents the frequency of the word in the videos.

Original: The filters allow users to choose between the top ten words mentioned, physical health symptoms, or the top ten words with additional adjectives or nouns.
Revised: The filters allow users to choose between the top ten words mentioned, physical health symptoms, or the top ten words with additional adjectives or nouns.

Original: For example, the word “swollen” from the “Top ten words mentioned” graph becomes “Ankle swollen” in the “More description” graph.
Revised: For example, the word “swollen” from the “Top ten words mentioned” graph becomes “Ankle swollen” in the “More description” graph.

Original: The user is able to sort the order of the word frequencies in the graph alphabetically, highest-to-lowest-mention, group by positive or negative sentiments, and group-by-ranges.
Revised: The user is able to sort the order of the word frequencies in the graph alphabetically, highest-to-lowest-mention, group by positive or negative sentiments, and group-by-ranges.

Original: Grouping by positive and negative organizes the Bar Graph based on the sentiment of the videos.
Revised: Grouping by positive and negative organizes the Bar Graph based on the sentiment of the videos.

Original: Colour is used to indicate whether the word has a positive or negative sentiment (orange is used for negative and green for positive).
Revised: Colour is used to indicate whether the word has a positive or negative sentiment (orange is used for negative and green for positive).

Original: For example, one of the words in the Bar Graph is “walking”; once a user applies the positive/negative sentiment filter, the bar becomes green indicating that this word is associated with positive sentiments in the videos.
Revised: For example, one of the words in the Bar Graph is “walking”; once a user applies the positive/negative sentiment filter, the bar becomes green indicating that this word is associated with positive sentiments in the videos.

Original: The group-by-ranges filter groups words together based on frequency size.
Revised: The group-by-ranges filter groups words together based on frequency size.

Original: For example, all words between 10-29 mentions are grouped together, 30-49 in a second grouping and finally more than 50 in a third.
Revised: For example, all words between 10-29 mentions are grouped together, 30-49 in a second grouping and finally more than 50 in a third.

Original: A dotted rectangle surrounding the “Pain” bar represents a button that takes the user to the associated Text Graph of a certain word.
Revised: A dotted rectangle surrounding the “Pain” bar represents a button that takes the user to the associated Text Graph of a certain word.

Original: Since there is only one Text Graph in the prototype and it represents pain, this
Revised: Since there is only one Text Graph in the prototype and it represents pain, this

Original: Bar Graph only has one button.
Revised: Bar Graph only has one button.

Original: 3.1.3 Line Graph
Revised: 3.1.3 Line Graph

Original: The Line Graph example (see Figure 3) represents the data as occurrences of words over specific time intervals.
Revised: The Line Graph example (see Figure 3) represents the data as occurrences of words over specific time intervals.

Original: The example in
Revised: The example in

Original: Figure 3 shows the number of negative mentions of pain per day over a two-month recording period.
Revised: Figure 3 shows the number of negative mentions of pain per day over a two-month recording period.

Original: The x-axis represents the time interval in days over the two-month period, and the y-axis represents the number of times the word pain is mentioned with negative sentiment in the videos for a particular day.
Revised: The x-axis represents the time interval in days over the two-month period, and the y-axis represents the number of times the word pain is mentioned with negative sentiment in the videos for a particular day.

Original: The clear or black-filled circles plotted at the 0 points are either when there was no video created that day or no videos where there were negative pain words respectively (the legend below the graph indicates the meaning of the clear and black-filled circle).
Revised: The clear or black-filled circles plotted at the 0 points are either when there was no video created that day or no videos where there were negative pain words respectively (the legend below the graph indicates the meaning of the clear and black-filled circle).

Original: The two points surrounded by the dotted rectangle indicate a button that when clicked will take users to the videos created for that particular day.
Revised: The two points surrounded by the dotted rectangle indicate a button that when clicked will take users to the videos created for that particular day.

Original: 3.1.4 Text Graph
Revised: 3.1.4 Text Graph

Original: The Text Graph (see Figure 4) adds text markers to maxima and minima points on the time-based Line Graph.
Revised: The Text Graph (see Figure 4) adds text markers to maxima and minima points on the time-based Line Graph.

Original: In the example shown in Figure 4, the maximal and minimal of the graph are labelled either “More pain” or “Less pain” indicating whether the video(s) for that day mentioned pain in a positive or negative manner.
Revised: In the example shown in Figure 4, the maximal and minimal of the graph are labelled either “More pain” or “Less pain” indicating whether the video(s) for that day mentioned pain in a positive or negative manner.

Original: The font-sizes of the text vary with the frequency of mentions as in a
Revised: The font-sizes of the text vary with the frequency of mentions as in a

Original: Word Cloud.
Revised: Word Cloud.

Original: The graph also has filters for users to choose to show positive only points, negative only points, or both.
Revised: The graph also has filters for users to choose to show positive only points, negative only points, or both.

Original: In addition, users are able to toggle between a coloured and black/white representation.
Revised: In addition, users are able to toggle between a coloured and black/white representation.

Original: We wanted to determine whether colour could help people interpret a text graph that was populated with a large variety of information displayed (e.g., lines, labels/text, points, and button indicators).
Revised: We wanted to determine whether colour could help people interpret a text graph that was populated with a large variety of information displayed (e.g., lines, labels/text, points, and button indicators).

Original: When there is a day where no video is created, or there is no mention of pain, the graph shows clear or black filled-in circles, respectively.
Revised: When there is a day where no video is created, or there is no mention of pain, the graph shows clear or black filled-in circles, respectively.

Original: Points on the graph shown in Figure 4 with a dotted rectangle are buttons that when selected lead users to the videos created for that day, thus offering users a drill-down option.
Revised: Points on the graph shown in Figure 4 with a dotted rectangle are buttons that when selected lead users to the videos created for that day, thus offering users a drill-down option.

Original: 3.1.5 Text Summary
Revised: 3.1.5 Text Summary

Original: The Text Summary example (see Figure 5) is a general summary of the videos presented as text-based bullet points created from high-frequency words.
Revised: The Text Summary example (see Figure 5) is a general summary of the videos presented as text-based bullet points created from high-frequency words.

Original: It is created using the data from the transcripts.
Revised: It is created using the data from the transcripts.

Original: The purpose of the summary is to briefly describe the main patient experiences over the duration of the entire video set.
Revised: The purpose of the summary is to briefly describe the main patient experiences over the duration of the entire video set.

Original: For example, “aches” and “pain” were mentioned often in conjunction with the word “morning”.
Revised: For example, “aches” and “pain” were mentioned often in conjunction with the word “morning”.

Original: The Text Summary then shows “The patient complained of pain and aches multiple times...
Revised: The Text Summary then shows “The patient complained of pain and aches multiple times...

Original: On most occasions, complaints of pain and aches were mentioned with “morning””.
Revised: On most occasions, complaints of pain and aches were mentioned with “morning””.

Original: The user is able to toggle between a general summary and a quantitative summary.
Revised: The user is able to toggle between a general summary and a quantitative summary.

Original: The quantitative summary displays the number of videos that mention a certain word.
Revised: The quantitative summary displays the number of videos that mention a certain word.

Original: For example, “59/72 videos mentioned “pain” wi
Revised: For example, “59/72 videos mentioned “pain” wi

Original: th a total account of 90 mentions”.
Revised: th a total account of 90 mentions”.

Original: 3. Participants
Revised: 3. Participants

Original: A total of 48 healthcare providers and individuals in medical school (20 males, 27 females, 1 did not answer) were recruited for the user study.
Revised: A total of 48 healthcare providers and individuals in medical school (20 males, 27 females, 1 did not answer) were recruited for the user study.

Original: The three groups were: students currently in their first or second year of medical school (16 in total), students in their third or greater year of medical school (17 in total), and finally healthcare providers with two or more years of work experience in the healthcare industry (15 in total).
Revised: The three groups were: students currently in their first or second year of medical school (16 in total), students in their third or greater year of medical school (17 in total), and finally healthcare providers with two or more years of work experience in the healthcare industry (15 in total).

Original: The healthcare providers were from a range of disciplines including registered nurses, graduate nursing students, medical residents, a general practitioner, a nutritionist, and a behaviour therapist.
Revised: The healthcare providers were from a range of disciplines including registered nurses, graduate nursing students, medical residents, a general practitioner, a nutritionist, and a behaviour therapist.

Original: These three groups were chosen because they vary in their experience of medical training.
Revised: These three groups were chosen because they vary in their experience of medical training.

Original: Previous studies [8], [12] have shown that professional HCPs like to use clinical notes to support their decisions, but we wanted to see if the amount of training can play a role in how HCP like their patient data presented.
Revised: Previous studies [8], [12] have shown that professional HCPs like to use clinical notes to support their decisions, but we wanted to see if the amount of training can play a role in how HCP like their patient data presented.

Original: The first group were freshly starting medical school and lacked exposure to traditional patient data summarizations such as clinical notes.
Revised: The first group were freshly starting medical school and lacked exposure to traditional patient data summarizations such as clinical notes.

Original: The second group of students had slightly more practical medical training, perhaps including clinical rounds, so they had more experience in handling patient data than the first group.
Revised: The second group of students had slightly more practical medical training, perhaps including clinical rounds, so they had more experience in handling patient data than the first group.

Original: Finally, the third group were individuals working in the field and had the most exposure and experience in handling patient data.
Revised: Finally, the third group were individuals working in the field and had the most exposure and experience in handling patient data.

Original: Age, gender and years of experience were collected to ensure that there was a representative sample of the target populations (see Figure 7).
Revised: Age, gender and years of experience were collected to ensure that there was a representative sample of the target populations (see Figure 7).

Original: Participants varied in age between 18 and 41 years with the majority of participants (31 of 48) aged between 18 and 22 years.
Revised: Participants varied in age between 18 and 41 years with the majority of participants (31 of 48) aged between 18 and 22 years.

Original: All participants were given a small token of appreciation for their participation in the study.
Revised: All participants were given a small token of appreciation for their participation in the study.

Original: 3. Study design
Revised: 3. Study design

Original: Each study lasted around 90 minutes and began with a pre-study questionnaire that gathered demographic data as well as comments on the healthcare provider’s/medical student’s current routine of practice.
Revised: Each study lasted around 90 minutes and began with a pre-study questionnaire that gathered demographic data as well as comments on the healthcare provider’s/medical student’s current routine of practice.

Original: This was followed by a short training period where the user was introduced to the visualization system as well as a patient vignette and scenario.
Revised: This was followed by a short training period where the user was introduced to the visualization system as well as a patient vignette and scenario.

Original: The vi gnette explained the patient’s condition and their experiences to the user, whereas the scenario set the tone for the user study explaining what exactly the user will be required to do.
Revised: The vi gnette explained the patient’s condition and their experiences to the user, whereas the scenario set the tone for the user study explaining what exactly the user will be required to do.

Original: Participants were then invited to complete ten tasks while thinking aloud followed by a short semi-structured interview that gathered their opinion of the visualizations.
Revised: Participants were then invited to complete ten tasks while thinking aloud followed by a short semi-structured interview that gathered their opinion of the visualizations.

Original: Three versions of the ten user tasks were created, and the participant was randomly assigned a version.
Revised: Three versions of the ten user tasks were created, and the participant was randomly assigned a version.

Original: Each version had the same user tasks but in a different order to eliminate sequence bias.
Revised: Each version had the same user tasks but in a different order to eliminate sequence bias.

Original: The tasks were designed so that the user had to use the application and the visualizations to answer the questions, but users could use the application as they preferred, even if it meant that they only used one or two of the visualizations throughout the study.
Revised: The tasks were designed so that the user had to use the application and the visualizations to answer the questions, but users could use the application as they preferred, even if it meant that they only used one or two of the visualizations throughout the study.

Original: However, for each task one or two visualizations were more appropriate for answering the question.
Revised: However, for each task one or two visualizations were more appropriate for answering the question.

Original: For example, one task was “How did the patient’s pain vary over the course of 2 months?”.
Revised: For example, one task was “How did the patient’s pain vary over the course of 2 months?”.

Original: Since the task asks a time-based question, the Line Graph or Text Graph would be more appropriate to use.
Revised: Since the task asks a time-based question, the Line Graph or Text Graph would be more appropriate to use.

Original: The users were free to skip any question they did not feel comfortable or had trouble answering.
Revised: The users were free to skip any question they did not feel comfortable or had trouble answering.

Original: Notes were also generated on what visualizations the participants used the most for the study tasks, and which ones they were struggling to understand.
Revised: Notes were also generated on what visualizations the participants used the most for the study tasks, and which ones they were struggling to understand.

Original: The study ended with a post-study questionnaire that allowed participants to rate the system usability, reflect on their experiences, discuss what they liked/disliked about the system, and make recommendations.
Revised: The study ended with a post-study questionnaire that allowed participants to rate the system usability, reflect on their experiences, discuss what they liked/disliked about the system, and make recommendations.

Original: In this paper, the results of the post-study questionnaire and observational notes are reported.
Revised: In this paper, the results of the post-study questionnaire and observational notes are reported.

Original: 3. Data collection
Revised: 3. Data collection

Original: The post-study questionnaires consisted of ten System Usability
Revised: The post-study questionnaires consisted of ten System Usability

Original: Scale [31] questions, two questions that allowed participants to choose which visualization(s) they liked the most and which they liked the least, and one 4-point rating questions on the perceived usefulness of each visualization.
Revised: Scale [31] questions, two questions that allowed participants to choose which visualization(s) they liked the most and which they liked the least, and one 4-point rating questions on the perceived usefulness of each visualization.

Original: Usefulness was rated with four possible responses: Very useful, Useful, Somewhat Useful, Not useful at all.
Revised: Usefulness was rated with four possible responses: Very useful, Useful, Somewhat Useful, Not useful at all.

Original: There were also five open-ended questions that allowed participants to use freeform text to write about their opinion and interest in working with the system and its visualizations.
Revised: There were also five open-ended questions that allowed participants to use freeform text to write about their opinion and interest in working with the system and its visualizations.

Original: 3. Data analysis
Revised: 3. Data analysis

Original: The questionnaire responses were analyzed using nonparametric statistical methods.
Revised: The questionnaire responses were analyzed using non-parametric statistical methods.

Original: A Kruskal-Wallis non-parametric analysis of variance was used to determine whether there were significant differences between the three participant groups for usefulness and ratings of each visualization.
Revised: A Kruskal-Wallis non-parametric analysis of variance was used to determine whether there were significant differences between the three participant groups for usefulness and ratings of each visualization.

Original: Then, Friedman OneWay Repeated Measure Analysis of Variance by Ranks was used to test whether there was a significant difference in ratings of all visualizations.
Revised: Then, Friedman One-Way Repeated Measure Analysis of Variance by Ranks was used to test whether there was a significant difference in ratings of all visualizations.

Revised: A

Original: A Wilcoxon Signed Rank Test was then used to check where these differences occur.
Revised: Wilcoxon Signed Rank Test was then used to check where these differences occur.

Original: Finally, the strength of association between the ratings of different visualization was assessed using Kendall’s tau.
Revised: Finally, the strength of association between the ratings of different visualization was assessed using Kendall’s tau.

Original: 4 R ESULTS
Revised: 4 R ESULTS

Original: The mean SUS score was 74.90 (SD=13.80).
Revised: The mean SUS score was 74.90 (SD=13.80).

Original: According to Bangor et al.
Revised: According to Bangor et al.

Original: , [32] a score above 68 demonstrates average usability obtained across a range of studies.
Revised: , [32] a score above 68 demonstrates average usability obtained across a range of studies.

Original: In addition, 60.4% of the participants were very likely (rating of 5 on a 1-5 scale) to recommend this system to a friend or colleague.
Revised: In addition, 60.4% of the participants were very likely (rating of 5 on a 1-5 scale) to recommend this system to a friend or colleague.

Original: 4. Statistical analysis
Revised: 4. Statistical analysis

Original: The Shapiro-Wilk test was used to check for statistical normality of the usefulness ratings.
Revised: The Shapiro-Wilk test was used to check for statistical normality of the usefulness ratings.

Original: The test results indicated that the ratings distribution departed significantly from normality ( p <0.05).
Revised: The test results indicated that the ratings distribution departed significantly from normality ( p <0.05).

Original: A
Revised: A

Original: Pearson chi-square test of independence was used to determine the significance of usefulness ratings for each visualization.
Revised: Pearson chi-square test of independence was used to determine the significance of usefulness ratings for each visualization.

Original: The test results were statistically significant ( p <0.05).
Revised: The test results were statistically significant ( p <0.05).

Original: A Kruskal-Wallis test showed no significant differences between groups and their rating of each visualization ( p >0.017; after Bonferroni adjustment).
Revised: A Kruskal-Wallis test showed no significant differences between groups and their rating of each visualization ( p >0.017; after Bonferroni adjustment).

Original: However, there was a significant difference in the overall rating of each visualization (Friedman Test, p<0.017) and also some pairs of visualizations (see Table 1).
Revised: However, there was a significant difference in the overall rating of each visualization (Friedman Test, p<0.017) and also some pairs of visualizations (see Table 1).

Original: In addition, a contingency table analysis was performed between each visualization, but the results were not statistically significant at the .017 level.
Revised: In addition, a contingency table analysis was performed between each visualization, but the results were not statistically significant at the .017 level.

Original: 4. Frequency responses
Revised: 4. Frequency responses

Original: Figures 7 and 8 show which visualizations were preferred or disliked for the different participant groups.
Revised: Figures 7 and 8 show which visualizations were preferred or disliked for the different participant groups.

Original: Overall, the most preferred visualizations were Text Summary and Text Graph.
Revised: Overall, the most preferred visualizations were Text Summary and Text Graph.

Original: The most disliked graph was the Line Graph.
Revised: The most disliked graph was the Line Graph.

Original: Figure 9 illustrates the frequency of usefulness ratings for each visualization.
Revised: Figure 9 illustrates the frequency of usefulness ratings for each visualization.

Original: Text summary and Text Graph were also rated the most useful visualizations (rating of very useful or 4/4), and the Word Cloud and Bar Graph are rated as useful (rating of 3/4).
Revised: Text summary and Text Graph were also rated the most useful visualizations (rating of very useful or 4/4), and the Word Cloud and Bar Graph are rated as useful (rating of 3/4).

Original: The Line Graph was rated as somewhat useful (rating of 2/4) by 18 participants.
Revised: The Line Graph was rated as somewhat useful (rating of 2/4) by 18 participants.

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 4. Written responses
Revised: 4. Written responses

Original: All 48 participants commented on their experience and opinion of the visualizations, and the application.
Revised: All 48 participants commented on their experience and opinion of the visualizations, and the application.

Original: Most participants had positive reviews of MHMR and its visualization techniques.
Revised: Most participants had positive reviews of MHMR and its visualization techniques.

Original: Participants mentioned that the Text Graph or Text Summary were the most useful visualization to work with, e.g., “Text Graph, Text
Revised: Participants mentioned that the Text Graph or Text Summary were the m ost useful visualization to work with, e.g., “Text Graph, Text

Original: Summary [are the most useful] because they give a better and a quick picture [answering] my questions” (P18).
Revised: Summary [are the most useful] because they give a better and a quick picture [answering] my questions” (P18).

Original: Some part icipants mentioned that what they liked least about the system was the Line
Revised: Some participants mentioned that what they liked least about the system was the Line

Original: Graph e.g., P6 wrote “Line graphs, I think it is a lot more time consuming and comparatively less helpful than the other techniques.”
Revised: Graph e.g., P6 wrote “Line graphs, I think it is a lot more time consuming and comparatively less helpful than the other techniques.”

Original: P46 also mentioned that “I think some of the visualizations were redundant (the text graph was a better version of the line graph)”.
Revised: P46 also mentioned that “I think some of the visualizations were redundant (the text graph was a better version of the l ine graph)”.

Original: With respect to design and layout, some participants had negative comments on the x-axis labels on the Line or Text Graph because they found them “unclear” (P28) or did not understand the difference between the open and closed circle (P17).
Revised: With respect to design and layout, some participants had negative comments on the x-axis labels on the Line or Text Graph because they found them “unclear” (P28) or did not understand the difference between the open and closed circle (P17).

Original: Participants also stated that they would be willing to use this system in their practice especially when monitoring a patient’s condition over time, or prescribing medication.
Revised: Participants also stated that they would be willing to use this system in their practice especially when monitoring a patient’s condition over time, or prescribing medication.

Original: For example, P wrote, “I do weekly check -ins with my clients, it would help me to see their progress as well as help me to pinpoint where changes in their nutritional and exercise plans need to be made.”
Revised: For example, P wrote, “I do weekly check -ins with my clients, it would help me to see their progress as well as help me to pinpoint where changes in their nutritional and exercise plans need to be made.”

Original: P31 stated, “As a nurse, you can under stand at what time of the day the [patient] experiences more pain, and you can advocate for the [patient] to get pain meds prescribed at certain times of the day.”
Revised: P31 stated, “As a nurse, you can understand at what time of the day the [patient] experiences more pain, and you can advocate for the [patient] to get pain meds prescribed at certain times of the day.”

Original: But there were concerns on how compliant patients would be with using MHMR, and how they would be encouraged to record their symptoms as often as possible (P45).
Revised: But there were concerns on how compliant patients would be with using MHMR, and how they would be encouraged to record their symptoms as often as possible (P45).

Original: 5 D ISCUSSION
Revised: 5 D ISCUSSION

Original: This study evaluated opinions about, and acceptance of, the data visualizations presented in the MHMR application.
Revised: This study evaluated opinions about, and acceptance of, the data visualizations presented in the MHMR application.

Original: In the ratings of preference, usefulness (Figures 7-9) and written responses, the Text
Revised: In the ratings of preference, usefulness (Figures 7-9) and written responses, the Text

Original: Summary and Text Graph visualizations tended to be more preferred.
Revised: Summary and Text Graph visualizations tended to be more preferred.

Original: There was a significant difference in rating of Text Summary compared to the Word Cloud, Bar Graph, and Line Graph.
Revised: There was a significant difference in rating of Text Summary compared to the Word Cloud, Bar Graph, and Line Graph.

Original: In their research, Sultanum et al. , [12] concluded that text is a familiar method of generating and consuming information about patients for physicians (or HCP).
Revised: In their research, Sultanum et al. , [12] concluded that text is a familiar method of generating and consuming information about patients for physicians (or HCP).

Original: Thus, it is not surprising that the Text
Revised: Thus, it is not surprising that the Text

Original: Summary was preferred, since it resembled the type of information provided in clinical notes.
Revised: Summary was preferred, since it resembled the type of information provided in clinical notes.

Original: Given that word clouds are used for searching, browsing, impression forming and recognizing or matching [21] it was anticipated that participants would find them useful in finding patterns for matching specific health conditions.
Revised: Given that word clouds are used for searching, browsing, impression forming and recognizing or matching [21] it was anticipated that participants would find them useful in finding patterns for matching specific health conditions.

Original: For some participants this was the case, and they found the Word Cloud to be useful for this purpose, either alone or along with Text Graph or
Revised: For some participants this was the case, and they found the Word Cloud to be useful for this purpose, either alone or along with Text Graph or

Original: Text Summary (e.g., P4, P14, P24, and P34).
Revised: Text Summary (e.g., P4, P14, P24, and P34).

Original: However, for others there was too much disorganized information contained in the
Revised: However, for others there was too much disorganized information contained in the

Original: Word C loud, e.g., one participant (P12) stated: “[ Word Cloud] appeared scattered and packed” , and thus they did not find it as useful as the Text Summary and Text Graph in gaining information about the patient ’s condition.
Revised: Word C loud, e.g., one participant (P12) stated: “[ Word Cloud] appeared scattered and packed” , and thus they did not find it as useful as the Text Summary and Text Graph in gaining information about the patient ’s condition.

Original: The Bar Graph was the only visualization that had a good usefulness rating (Figure 9) and no negative comments associated with it.
Revised: The Bar Graph was the only visualization that had a good usefulness rating (Figure 9) and no negative comments associated with it.

Original: This could be due to familiarity with its style and the information it conveyed.
Revised: This could be due to familiarity with its style and the information it conveyed.

Original: For example, P37 liked the Bar Graph because it displayed “the most words said and how many times the patient actually said them.”
Revised: For example, P37 liked the Bar Graph because it displayed “the most words said and how many times the patient actually said them.”

Original: Plus, the Bar Graph has a filter that allows participants to sort from highest to lowest frequency words.
Revised: Plus, the Bar Graph has a filter that allows participants to sort from highest to lowest frequency words.

Original: The usage of this feature was repeatedly observed in the user studies and as an example, the Bar Graph was liked by P11 because “it gives you [the] highest symptoms experienced vs. lowest”.
Revised: The usage of this feature was repeatedly observed in the user studies and as an example, the Bar Graph was liked by P11 because “it gives you [the] highest symptoms experienced vs. lowest”.

Original: The Line Graph was collectively the least favourite visualization among the participants.
Revised: The Line Graph was collectively the least favourite visualization among the participants.

Original: This could be because it requires attention to understand the trend and to investigate each point, or because participants had less experience with interpreting line graphs.
Revised: This could be because it requires attention to understand the trend and to investigate each point, or because participants had less experience with interpreting line graphs.

Original: For example, P16 said “I think there should be more detail to the line graph because I couldn't understand that graph much”, and P30 said “I think the line graph was a bit difficult to read” .
Revised: For example, P16 said “I think there should be more detail to the line graph because I couldn't understand that graph much”, and P30 said “I think the line graph was a bit difficult to read” .

Original: The Text Graph provided more or less the same information as the Line Graph but combined text labels with the graphical, timebased representation of the data.
Revised: The Text Graph provided more or less the same information as the Line Graph but combined text labels with the graphical, timebased representation of the data.

Original: The Text Graph was rated more highly than the Line Graph and there was a significant difference between their ratings.
Revised: The Text Graph was rated more highly than the Line Graph and there was a significant difference between their ratings.

Original: 11 participants that rated Text Graph as a 4/ (Very useful) rated the Line Graph as a 2/4 (Somewhat useful).
Revised: 11 participants that rated Text Graph as a 4/ (Very useful) rated the Line Graph as a 2/4 (Somewhat useful).

Original: Carpenter and Shah [18] found that, when viewing line graphs, individuals spent more time relating the different graphical features axis and data point labels to make sense of the data and less time viewing the pattern or trends.
Revised: Carpenter and Shah [18] found that, when viewing line graphs, individuals spent more time relating the different graphical features axis and data point labels to make sense of the data and less time viewing the pattern or trends.

Original: The Text Graph may have helped participants make sense of the data because it highlights the important information for them, and so they can focus more on understanding the overall pattern.
Revised: The Text Graph may have helped participants make sense of the data because it highlights the important information for them, and so they can focus more on understanding the overall pattern.

Original: In addition, The Text Graph allowed users to drill down to extract more detailed information by viewing specific videos related to those data points.
Revised: In addition, The Text Graph allowed users to drill down to extract more detailed information by viewing specific videos related to those data points.

Original: This may have provided the additional detail as suggested by P16 or it may have added a sufficient amount of text to take advantage of the familiarity of text favoured in the Text Summary.
Revised: This may have provided the additional detail as suggested by P16 or it may have added a sufficient amount of text to take advantage of the familiarity of text favoured in the Text Summary.

Original: For example, P said “in my opinion, the most useful technique is text -graph [because] it shows day to day variation of patient symptoms...
Revised: For example, P said “in my opinion, the most useful technique is text -graph [because] it shows day to day variation of patient symptoms...

Original: it will help me get a better understanding of my patient [to] evaluate necessary management”.
Revised: it will help me get a better understanding of my patient [to] evaluate necessary management”.

Original: Researchers have found that healthcare providers prefer the notes section in a patient record [8], [12].
Revised: Researchers have found that healthcare providers prefer the notes section in a patient record [8], [12].

Original: The Text Summary in MHMR was similar to clinical notes so it was expected that most participants would show a preference for the Text Summary and would find it useful.
Revised: The Text Summary in MHMR was similar to clinical notes so it was expected that most participants would show a preference for the Text Summary and would find it useful.

Original: Nonetheless, participants also saw benefits of other visualizations, particularly the Text Graph, and formed mainly positive opinions of them.
Revised: Nonetheless, participants also saw benefits of other visualizations, particularly the Text Graph, and formed mainly positive opinions of them.

Original: The Text Graph was newly developed for this research and was new to all participants.
Revised: The Text Graph was newly developed for this research and was new to all participants.

Original: The
Revised: The

Original: Text Graph was designed to exploit the preference for notes and the benefits of visually representing patterns over time as a line graph.
Revised: Text Graph was designed to exploit the preference for notes and the benefits of visually representing patterns over time as a line graph.

Original: The Text Graph and Text Summary had very similar ratings as well.
Revised: The Text Graph and Text Summary had very similar ratings as well.

Original: 16 participants rated both visualizations as 4/4 (Very useful) andrated both graphs as 3/4 (Useful) indicating that both visualizations were useful in extracting information about patients’ status and conditions.
Revised: 16 participants rated both visualizations as 4/4 (Very useful) andrated both graphs as 3/4 (Useful) indicating that both visualizations were useful in extracting information about patients’ status and conditions.

Original: The Word Cloud was also text-based but there was a significant difference between its rating compared to Text Graph and Text Summary.
Revised: The Word Cloud was also text-based but there was a significant difference between its rating compared to Text Graph and Text Summary.

Original: I t was mainly rated to be “useful” (3/4) rather than “very useful” (4/4) like the Text Summary and Text Graph.
Revised: I t was mainly rated to be “useful” (3/4) rather than “very useful” (4/4) like the Text Summary and Text Graph.

Original: This could be because the Word Cloud was something new for them and they were more comfortable with the Text Summary and Text
Revised: This could be because the Word Cloud was something new for them and they were more comfortable with the Text Summary and Text

Original: Graph but were open to trying the Word Cloud as well.
Revised: Graph but were open to trying the Word Cloud as well.

Original: In terms of design, the Text Graph, Text Summary, Word Cloud, and Bar Graph seem to be acceptable ways of visualizing qualitative patient-generated data for MHMR.
Revised: In terms of design, the Text Graph, Text Summary, Word Cloud, and Bar Graph seem to be acceptable ways of visualizing qualitative patient-generated data for MHMR.

Original: The Line Graph is not as useful and can be given less importance in the MHMR implementation since it does not clearly convey patient information.
Revised: The Line Graph is not as useful and can be given less importance in the MHMR implementation since it does not clearly convey patient information.

Original: In addition, the use of colours on visualizations allows users to distinguish between different aspects and it is beneficial to use them to communicate the results of different filters to the end user.
Revised: In addition, the use of colours on visualizations allows users to distinguish between different aspects and it is beneficial to use them to communicate the results of different filters to the end user.

Original: 5. Limitations
Revised: 5. Limitations

Original: This study evaluated the acceptance and opinion of data visualizations presented in the MHMR application.
Revised: This study evaluated the acceptance and opinion of data visualizations presented in the MHMR application.

Original: The statistical analysis of post-study questionnaires showed no significant differences between the groups (HCP, first or second-year medical students, third-year or greater medical students).
Revised: The statistical analysis of post-study questionnaires showed no significant differences between the groups (HCP, first or second-year medical students, third-year or greater medical students).

Original: One of the reasons for this could be that there is not enough data to work with because although data of 48 participants were analyzed, there were only around 16 participants in each group.
Revised: One of the reasons for this could be that there is not enough data to work with because although data of 48 participants were analyzed, there were only around 16 participants in each group.

Original: 5.1.1 Demographics
Revised: 5.1.1 Demographics

Original: There were no differences in the usefulness ratings between groups, and we suggest that a larger sample may elicit differences.
Revised: There were no differences in the usefulness ratings between groups, and we suggest that a larger sample may elicit differences.

Original: The HCP were mostly nurses who may have different experiences than doctors or other types of HCP.
Revised: The HCP were mostly nurses who may have different experiences than doctors or other types of HCP.

Original: Future studies should incorporate a more diverse set of participants varying in roles.
Revised: Future studies should incorporate a more diverse set of participants varying in roles.

Original: The students recruited were mainly from the same geographic location and thus diverse geographic samples, and the impact of different training regimes between different jurisdictions should also be studied.
Revised: The students recruited were mainly from the same geographic location and thus diverse geographic samples, and the impact of different training regimes between different jurisdictions should also be studied.

Original: 5.1.2 Online study
Revised: 5.1.2 Online study

Original: One technical limitation was that this was an online study.
Revised: One technical limitation was that this was an online study.

Original: Technical difficulties such as Internet issues with several participants slowed the process of viewing and interpreting visualizations causing frustration and impatience by participants.
Revised: Technical difficulties such as Internet issues with several participants slowed the process of viewing and interpreting visualizations causing frustration and impatience by participants.

Original: This may have impacted their views and they may have been distracted by the technical issues.
Revised: This may have impacted their views and they may have been distracted by the technical issues.

Original: Another limitation of the online study was that the MHMR was intended as a mobile application, but this study used a responsive web application that mimics the user interface of a mobile application.
Revised: Another limitation of the online study was that the MHMR was intended as a mobile application, but this study used a responsive web application that mimics the user interface of a mobile application.

Original: Users saw a simulation of a mobile screen on a desktop (height of the interface was 980px and the width was set to 50% of the displaying screen) instead of a display on an actual mobile device.
Revised: Users saw a simulation of a mobile screen on a desktop (height of the interface was 980px and the width was set to 50% of the displaying screen) instead of a display on an actual mobile device.

Original: Because users were using screens with different aspect ratios, the user interface could appear wider than intended or disproportional depending on the size of the screen, and sometimes the participants were not able to see the entire application at once and had to scroll up or down.
Revised: Because users were using screens with different aspect ratios, the user interface could appear wider than intended or disproportional depending on the size of the screen, and sometimes the participants were not able to see the entire application at once and had to scroll up or down.

Original: This may have had an impact on the participants’ view of the application and the visualizations as some information may have been hidden or not clearly visible on their screen.
Revised: This may have had an impact on the participants’ view of the application and the visualizations as some information may have been hidden or not clearly visible on their screen.

Original: In addition, the MHMR system was designed for a touch screen but the participants used a mouse in the study as they were working on their desktops.
Revised: In addition, the MHMR system was designed for a touch screen but the participants used a mouse in the study as they were working on their desktops.

Original: This may have affected the ease of use, for example some elements on the graphs may have been easier to touch rather than click using a mouse.
Revised: This may have affected the ease of use, for example some elements on the graphs may have been easier to touch rather than click using a mouse.

Original: 5.1.3 Visualizations
Revised: 5.1.3 Visualizations

Original: Another limitation of the study was the number of visualizations presented.
Revised: Another limitation of the study was the number of visualizations presented.

Original: Our study presented five visualizations, however, there are a number of other ways to present data.
Revised: Our study presented five visualizations, however, there are a number of other ways to present data.

Original: Other common graphs include scatter plots, pie charts, histograms, etc.
Revised: Other common graphs include scatter plots, pie charts, histograms, etc.

Original: In addition, there are a number of ways to add or remove details from graphs to create variation.
Revised: In addition, there are a number of ways to add or remove details from graphs to create variation.

Original: The Text Graph added text labels on top of a time-based line graph for more information but even simply removing data labels from the Bar Graph can potentially create a difference in understanding of the graph.
Revised: The Text Graph added text labels on top of a time-based line graph for more information but even simply removing data labels from the Bar Graph can potentially create a difference in understanding of the graph.

Original: Future studies should incorporate different types of visualization techniques and assess how different graphical features play a role in the understanding of the data.
Revised: Future studies should incorporate different types of visualization techniques and assess how different graphical features play a role in the understanding of the data.

Original: 6 C ONCLUSION
Revised: 6 C ONCLUSION

Original: The design and implementation of five visualizations depicting patient-generated data were presented in this paper.
Revised: The design and implementation of five visualizations depicting patient-generated data were presented in this paper.

Original: Participant data from three different groups representing a spectrum of healthcare providers in terms of their education and experience was evaluated for comparison and correlation.
Revised: Participant data from three different groups representing a spectrum of healthcare providers in terms of their education and experience was evaluated for comparison and correlation.

Original: Quantitatively, there was no difference between the groups and their preference and opinion of the visualizations, but there were overall differences in ratings and preferences towards the different visualizations.
Revised: Quantitatively, there was no difference between the groups and their preference and opinion of the visualizations, but there were overall differences in ratings and preferences towards the different visualizations.

Original: The results showed positive attitudes towards some of the visualizations in addition to the Text Summary, particularly the Text Graph.
Revised: The results showed positive attitudes towards some of the visualizations in addition to the Text Summary, particularly the Text Graph.

Original: The
Revised: The

Original: Text Summary was similar to the notes section in a patient record so, as anticipated it was the most preferred and was rated to be the most useful by the users.
Revised: Text Summary was similar to the notes section in a patient record so, as anticipated it was the most preferred and was rated to be the most useful by the users.

Original: However, the Text Graph, despite being something the users have not seen before, was as useful as the Text
Revised: However, the Text Graph, despite being something the users have not seen before, was as useful as the Text

Original: Summary.
Revised: Summary.

Original: Many participants were also interested in using this application in their future clinical practice.
Revised: Many participants were also interested in using this application in their future clinical practice.

Original: Future work needs to incorporate a larger sample size and a diverse group of participants.
Revised: Future work needs to incorporate a larger sample size and a diverse group of participants.

Original: The visualizations also need to be automated and tested for their accuracy in depicting the correct words spoken by the patient and associating the correct sentiment to those words.
Revised: The visualizations also need to be automated and tested for their accuracy in depicting the correct words spoken by the patient and associating the correct sentiment to those words.

Original: A CKNOWLEDGMENT
Revised: A CKNOWLEDGMENT

Original: Funding for this project was generously provided NSERC, project #355320-07.
Revised: Funding for this project was generously provided NSERC, project #355320-07.

Original: The authors also thank the participants for their time and generosity as well as the MHMR team for their time and effort on this project.
Revised: The authors also thank the participants for their time and generosity as well as the MHMR team for their time and effort on this project.