JMIR MEDICAL INFORMATICS

Stanhope et al

Original Paper

The Impact of Collaborative Documentation on PersonCentered Care: Textual Analysis of Clinical Notes
Victoria Stanhope1, MSW, PhD; Nari Yoo1, MA; Elizabeth Matthews2, MSW, PhD; Daniel Baslock3, MSW, PhD;
Yuanyuan Hu4, MSW, PhD
1Silver School of Social Work, New York University, New York, NY, United States
Graduate School of Service, Fordham University, New York, NY, United States
3
School of Social Work, Virginia Commonwealth University, Richmond, VA, United States
4
School of Social Work, University of Minnesota, St Paul, MN, United States
2

Corresponding Author:
Victoria Stanhope, MSW, PhD
Silver School of Social WorkNew York University
1 Washington Square N
New York, NY, 10003
United States
Phone: 1 3016931203
Email: ves1@nyu.edu

Abstract
Background: Collaborative documentation (CD) is a behavioral health practice involving shared writing of clinic visit
notes by providers and consumers. Despite widespread dissemination of CD, research on its effectiveness or impact on
person-centered care (PCC) has been limited. Principles of PCC planning, a recovery-based approach to service planning that
operationalizes PCC, can inform the measurement of person-centeredness within clinical documentation.
Objective: This study aims to use the clinical informatics approach of natural language processing (NLP) to examine theimpact of CD on person-centeredness in clinic visit notes. Using a dictionary-based approach, this study conducts a textual
analysis of clinic notes from a community mental health center before and after staff were trained in CD.
Methods: This study used visit notes (n=1981) from 10 providers in a community mental health center 6 months before andafter training in CD. LIWC-22 was used to assess all notes using the Linguistic Inquiry and Word Count (LIWC) dictionary,
which categorizes over 5000 linguistic and psychological words. Twelve LIWC categories were selected and mapped onto
PCC planning principles through the consensus of 3 domain experts. The LIWC-22 contextualizer was used to extract sentence
fragments from notes corresponding to LIWC categories. Then, fixed-effects modeling was used to identify differences innotes before and after CD training while accounting for nesting within the provider.
Results: Sentence fragments identified by the contextualizing process illustrated how visit notes demonstrated PCC. The
fixed effects analysis found a significant positive shift toward person-centeredness; this was observed in 6 of the selected
LIWC categories post CD. Specifically, there was a notable increase in words associated with achievement (β=.774, P<.001),
power (β=.831, P<.001), money (β=.204, P<.001), physical health (β=.427, P=.03), while leisure words decreased (β=−.166,
P=.002).
Conclusions: By using a dictionary-based approach, the study identified how CD might influence the integration of PCC
principles within clinical notes. Although the results were mixed, the findings highlight the potential effectiveness of CD inenhancing person-centeredness in clinic notes. By leveraging NLP techniques, this research illuminated the value of narrative
clinical notes in assessing the quality of care in behavioral health contexts. These findings underscore the promise of NLP for
quality assurance in health care settings and emphasize the need for refining algorithms to more accurately measure PCC.
JMIR Med Inform 2024;12:e52678; doi: 10.2196/52678
Keywords: person-centered care; collaborative documentation; natural language processing; concurrent documentation;clinical documentations; visit notes; community; health center; mental health center; textual analysis; clinical informatics;
https://medinform.jmir.org/2024/1/e52678

JMIR Med Inform 2024 | vol. 12 | e52678 | p. 1
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Stanhope et al

behavioral health; mental health; linguistic; linguistic inquiry; dictionary-based; sentence fragment; psychology; psychological;
clinical information; decision-making; mental health services; clinical notes; NLP

Introduction
Collaborative documentation (CD) is a specified behavioral health practice where clinicians complete visit notes
jointly with consumers during the session [1]. Throughdeliberate clinical strategies, such as sharing the computer
screen, reading visit notes aloud, and actively seeking
consumer’s input into the content of the session note [2],CD is a person-centered strategy that aims to engage andempower individuals and facilitate a mutual agreement on
treatment progress, service goals, and session activities. Bothas a means to promote person-centered care (PCC) andmake health information more accessible and transparent to
consumers, the practice of CD is being widely disseminatedthrough formal and informal training for community mental
health providers [3].
PCC, also referred to as patient-centered care, is aparadigm shift in health care that is defined by the Instituteof Medicine as care that is responsive to individual client
preferences, needs, and values [4]. A key part of behavioral health reform, PCC moves away from disease-centeredtreatment to a more holistic approach that engages individuals as active, empowered partners in their care. Person-centered care planning (PCCP) is a recovery-oriented practicethat integrates principles of PCC into the service planning
process. PCCP orients service planning and documentation
to the unique personal life goals of the consumer [5], andprovides a framework for operationalizing PCC in practicethrough six core principles, which are as follows: (1) PCC isbased on the person’s own unique life goals and aspirations;(2) PCC is oriented toward promoting recovery rather than
only minimizing illness and symptoms; (3) PCC articulates
the person‘s own role and the role of both paid practitioners
and natural supports in assisting the person to achieve his or
her own goals; (4) PCC focuses and builds on the person’s
capacities, strengths, and interests; (5) PCC emphasizes theuse of natural community settings rather than segregatedprogram settings; and (6) PCC anticipates and allow for
uncertainty, setbacks, and disagreements as inevitable steps
on the path to greater self-determination [6].
While CD has emerged as a recognized person-centeredpractice strategy by fully engaging the consumer in decisions
about their care, there remains a very limited evidence basedemonstrating its clinical effectiveness, including its impact
on PCC. Existing research demonstrates that CD is alignedwith consumer preferences; a recent scoping review foundthat the ability to read their visit notes improved consumers’
experience in mental health care, including their ability toremember their plans of care, understand their treatment,
and trust decisions made with providers [7]. In community
mental health, the use of CD specifically has been found tostrengthen the therapeutic alliance [8], and improve service
engagement, both in terms of visit attendance and medication
adherence [1].
https://medinform.jmir.org/2024/1/e52678

Despite this preliminary support for CD, more work isneeded to examine its impact on quality of care. Adding
urgency to this knowledge gap, regulatory changes to the
21st Century Cures Act now mandate organizations to makeelectronic health and mental health information, including
many types of visit notes, accessible online to service users[9]. As a consequence, best practices for using visit notes tosupport the provision of high-quality PCC are needed.
Clinical informatics, which provides highly efficient ways
to mine data within the electronic health record (EHR), is apromising methodological approach to examining the impactof CD on clinical quality. Although behavioral health has
lagged behind medical settings in the adoption of EHRs,
now the majority of behavioral health settings document
visit notes via the EHR [10]. The shift from paper records
to EHRs provides an unprecedented opportunity for clinicalinformatics to inform quality improvement in behavioral
health care. Visit notes include nuanced information aboutcare processes, session content, provider perspective, and theconsumer experience that are not captured in other morestructured fields of the EHR, offering valuable insight into
clinical quality that has not yet been systematically targetedin mental health services research [11]. Researchers haveapplied manual content analysis to visit notes to evaluate
dimensions of PCC [12,13], but these efforts are inevitably
limited in scope and dependent on the interpretive lens of theresearcher [14].
One clinical informatics strategy that can parse large
volumes of unstructured narrative information into quantitative data is natural language processing (NLP) [15]. Whilesome text mining approaches use words as the unit ofanalysis, NLP is able to capture the complexity of unstructured narrative using underlying metadata, which examines
how words relate to each other in a sentence and thesemantic context of a sentence [16]. The method involves
syntactic processing, information extraction, and capturing
meaning and relationships across concepts. NLP has been
predominantly used for detecting pathology and predicting
behavior [17] including measuring the following: alcohol
misuse in trauma patients [18], suicidal behavior [19], adverse
childhood experiences among VA patients [20], smoking
status [21], and sentiment at discharge [22]. Recent studies
have used this method to measure quality and safety innursing care [23], identify integrated care elements within
primary care [24], and detect changes in clinical documentation after opening notes to service users [25], however,there have been fewer studies that have used NLP to analyze
indicators of mental health care quality, including PCC.
By providing a framework to systematically categorize andcompare the contents of clinical notes, NLP is well poised toprovide novel insight into how CD affects clinical quality and
PCC.
The dictionary-based approach in NLP is a method thatuses a predefined lexicon to identify and extract certain types
of words or phrases within a given text. This approach is
JMIR Med Inform 2024 | vol. 12 | e52678 | p. 2
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
often used in tasks such as part-of-speech tagging, namedentity recognition, and sentiment analysis. In the dictionarybased approach, the dictionary consists of a list of words
or phrases along with their associated tags or labels. For
example, in part-of-speech tagging, the dictionary might
contain a list of common nouns, verbs, and adjectives, eachwith a corresponding tag that indicates the word’s part ofspeech. One of the benefits of the dictionary-based approachis its simplicity and ease of implementation, as it targets onlya predefined lexicon, or set of words, and does not require theuse of complex algorithms or machine learning models. Oneof the most dictionary-based approaches is LIWC (Linguistic
Inquiry and Word Count), and LIWC analysis has been foundto be particularly useful for identifying and analyzing theemotional state of individuals in mental health–related text
data [26]. Despite its potential, such as its application in
oncology settings to examine the changes in clinical notes
after patient access [27], the dictionary-based approach has
not yet been applied to clinical notes in behavioral health caresettings for its change after CD.
This study examines the effect of CD on the person-centeredness of documentation within a community mental healthsetting using a quasi-experimental pretest-posttest design.
The study adapted a well-established dictionary to conduct
sentiment analysis of provider clinic visit notes before andafter providers were trained in CD.

Methods
Ethical Considerations
This study (IRB-FY2022-5838) was granted an exemption by
the New York University Institutional Review Board.

Data Source
The study setting was a community mental health center, which provided a range of services to people withsevere mental illnesses including outpatient therapy, assertive
community treatment, community support programs, andpsychiatric rehabilitation. The data set, or corpus, is visit
notes completed by providers trained in CD. Providers weretrained in CD by MTM Services [28], a leader in CD training.
Their training consisted of tailored in-person workshops,
technical assistance, and a “train the trainer” series designed
for practice sustainability. The clinic training consisted of
8 hours of web-based training and customized consultation
support for implementation.

Stanhope et ala service user. Visit notes document the following: (1)sessions focused on developing or revising the service plan,a narrative clinical document completed every 6 months
with the service user detailing goals, strengths and barriers, short-term objectives, supports, professional/ billable
services, and natural support and self-directed actions, and(2) a visit focused on progress made towards completing thesteps in the service plan. Inclusion criteria for providers werethat they were: full-time employees of the clinic; provided
services to adults with severe mental illnesses; trained in CD;
and had been employed in their position a year prior and theyear after being trained.
This study sampled all visit notes completed by participating providers 6 months prior to CD training and all visit
notes completed during the sixth month following training(with a month lag for implementation time) between July 1,
2015, and March 10, 2020. Based on anticipated documentation rates of a clinic note being generated by each visit, we
recruited 10 providers, generating a total of 1981 visit notes.
On average, 198.1 notes were included per provider with astandard deviation of 37.8. Sampled notes were deidentified
but linked to providers through unique identifiers. In addition
to the session narrative, each note was comprised of thefollowing sections: (1) Provider ID, (2) Time and Date ofService, (3) Therapy Modality, and (4) Length of Stay.

Analytic Strategy
We used LIWC-22 to compute the scores for PCC-related
sentiment and linguistic categories [29] in each sampled note.
The LIWC method is a text analysis tool that uses linguistic algorithms to identify and categorize words in a text
according to their psychological properties. The method isbased on the LIWC dictionary, which contains over 5000
words and word stems organized into linguistic and psychological categories. In this study, 3 domain experts used a
consensus approach to select LIWC categories that mapped
onto the 6 principles of PCC [6] with several of the categories
mapping onto more than one domain. This iterative process
involved 8 rounds of independent coding by 3 raters using
Excel, followed by discussions to resolve discrepancies andachieve consensus on the final coding of all data points.
According to the Cohen κ measure, raters had an averageinterrater agreement of 88% across the LIWC categories [30].
Out of a possible 107 LIWC categories, we selected 12
categories. The selected LIWC domains and their associated
PCC principles are summarized in Tables 1 and 2 [6,30].

The visit notes were clinical narrative documentscompleted when the provider had an in-person contact with
Table 1. Mapping PCCPa principles onto LIWCb categories.

PCCP principles
PCCP is based on the person’s own unique life goals and aspirations.
PCCP is oriented toward promoting recovery rather than only minimizing
illness and symptoms.

https://medinform.jmir.org/2024/1/e52678

LIWC categories
Lifestyle (leisure, home, work, money, religion), social referents(family, friend)
Health (physical, wellness)

JMIR Med Inform 2024 | vol. 12 | e52678 | p. 3
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Stanhope et al

PCCP principles

LIWC categories

PCCP articulates the person’s own role and the role of both paidpractitioners and natural supports in assisting the person to achieve his or
her own goals.

Social referents (family, friend), physical (physical, wellness)

PCCP emphasizes the use of natural community settings rather thansegregated program settings.
PCCP anticipates and allows for uncertainty, setbacks, and disagreements
as inevitable steps on the path to greater self-determination.

Lifestyle (leisure, home, work, money, religion)

PCCP focuses and builds on the person’s capacities, strengths, andinterests.

Drives (achievement, affiliation, power), lifestyle (leisure, home,
work, money, religion)

Drives (affiliation, achievement, power)

aPCCP: person-centered care planning.
bLIWC: Linguistic Inquiry and Word Count.

Table 2. Sample fragments from clinic notes generated by contextualizing LIWCa subcategories.
LIWC subcategories
DrivesAchievement

Clinic note excerpt

Affiliation

Power

LifestyleHome
LeisureMoney
ReligionWork

PhysicalPhysicalWellness

Social referents
FamilyFriendWas able to identify her strengths, abilities, and self-identified progress in therapy reportsthat she has been engaging more socially with friends and that she has been trying to express
herself
Unexpected death of her pet was huge stressor that triggers increases in intensity of depression improvement with community resources and social networking due to becoming more
integrated with his new community and within his daughter’s school district
The client reports with a positive outlook that she feels more in control and is excited toreceive praise for using her therapy learning
Had no outbursts or over reactions recently and feels proud of her assertive but in control
mannerHighlighting that he doesn’t like the apartment “being so quiet” when his son is gone…
The client has increased productivity at home with baking and wrapping presents.
She is also making self-care more of a priority, “I scheduled a cruise and it’s just my sisterand I going”
Progress is that he has begun basketball
She admits that she has no savings of her own but she knows that she will get alimony
He has figured out a plan to pay for housing
Topics of no control include people’s religious actions and beliefs, elements within his own
church and community, as well as the political culture
She reported that she has been supported by her church and increased her faith significantly
He continues to apply for jobs and is now working with workforce development.
Not working currently, sent about a couple of job applications, continues with college course
work
Strengthen his tongue and swallowing skills, will occur to help reduce his concern regarding
health issues
Barriers to maintaining treatment plans goals, because her varying blood sugars have caused
severe mood swings
Emily is making progress in her goals to increase positive self-worth and applying healthy
coping skills
Trouble with motivation at times and needing to clear his mind, discussed option of yoga andmindfulness
Has been spending time with her family and working to express her feelings and needs whenappropriate
Progress is that she has figured out how to resolve some of her parenting issues
States that she feels she can “cut loose and have fun” with her friends
Regards to her recent trip to [place] which she really enjoyed she made new friends

aLIWC: Linguistic Inquiry and Word Count.

https://medinform.jmir.org/2024/1/e52678

JMIR Med Inform 2024 | vol. 12 | e52678 | p. 4
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

AnalysisUsing the LIWC categories described above, we comparednotes pre and post CD training to examine differences in
PCC. The complete visit note was used as the unit of analysis.
For data preprocessing, we cleaned the data and converted theinformation into a structured format that made it amenable
to identifying patterns in the data. The LIWC-22 dictionary
is case-insensitive and allows for matching 2-word phrases.
The LIWC-22 software removes extra whitespace charactersby default. While irrelevant words are not explicitly definedin the dictionary, we removed section headings (eg, Location)from the clinical notes before processing the text to eliminate
some irrelevant words. Negated phrases (eg, “not happy”)
are not treated differently from nonnegated phrases in thestandard scoring. To address this limitation, we included an
additional analysis in Multimedia Appendix 1 that controls
for the negations score.
To validate the team’s selection of LIWC categories, wefirst used the Contextualizer function of LIWC-22 to generate
sentence fragments containing words related to each LIWC
domain included in the analysis. We then analyzed changes
in the clinical note before and after CD training, using thecomplete visit note as the unit of analysis. To calculatechanges in the content characteristics of clinical notes before
and after CD training, we calculated frequency scores ofeach LIWC category for every clinical note since this study
focuses on examining the presence of words from the LIWC
dictionary.
Instead of using LIWC scores based on percentages, weused frequency scores. This decision was made based on
previous research indicating that CD enhances the lengthof clinical notes in terms of word and character count [31].
Additionally, we used frequencies to assess the presence of
PCC-related language in clinical notes. Our primary interest
was to capture whether clinicians used PCC-related words intheir documentation, even if these words did not constitutea large proportion of the total text. By focusing on word
frequencies, we aimed to mitigate the potential impact of note
length inflation due to CD practices, such as copy-and-pastingor using templates [32,33]. The LIWC frequency scores arecalculated by the following steps. First, we calculated LIWC
scores, which are determined by the percentage of words ina text that belong to specific linguistic categories. Then, tofind the frequency of each category within a clinical note,the respective LIWC percentage is multiplied by the total
word count of the note. For example, an LIWC value of 1.02
for achievement indicates that the note contains 1.02 words
related to achievement per the LIWC dictionary.
We used a fixed effects model that included the provider
as a categorical variable. The changes in the notes before
and after CD training were calculated while accounting fornesting within the therapist using individual fixed-effects
models. This approach allowed us to examine whether thechanges in PCC language use before and after the CD training
varied across the 10 providers in our sample. To further
investigate these differences, we calculated the intraclasscorrelation coefficient (ICC) and conducted paired sample t
https://medinform.jmir.org/2024/1/e52678

Stanhope et al
tests for each provider (Multimedia Appendix 2). The LIWC
version 22 was used for dictionary-based sentiment analysis
and STATA (version 17.0; Statacorp) was used for statisticalanalyses.

Results
Contextualizing LIWC Categories
The sentences generated by the Contextualizer function
illustrated how the LIWC categories mapped onto the PCCP
principles (see Table 2). The drives subcategories (achievement, affiliation, and power) reflected a strength-based
approach by describing the positive changes made by theclient; greater self-determination by feeling more in control;and interests by capturing how a client feels connected
to community, people, and pets. The lifestyle subcategories (home, leisure, money, religion, and work) captured
the unique details of the person’s life that are needed toindividualize treatment, including their beliefs, values, andpreferences which inform their personal life goals. Examples
included going on a cruise as part of self-care and seeking
employment. Lifestyle categories also illustrated people’sinterests such as playing sports and describing their life in
the community such as attending church. Physical categories (physical and wellness) demonstrated a more holistic
approach to the client by paying attention to how physicalhealth affects mental health and also to a focus on recovery
by including activities that promote wellness such as yoga
and health coping skills. The social referents category (family
and friend) demonstrated the role family and friends play as
natural supports such as going on vacation with your sister or
having fun with your friends.

Changes in LIWC Categories
Overall, there was a significant positive change in 4 ofthe selected LIWC categories indicating person-centeredness
after the providers had been trained in CD. As shown in
Table 3, the fixed effects regression analysis found thefollowing among the 12 selected characteristics: an increased
use in 4 categories, decreased use in 4 categories, and nochange in use in 4 categories, while controlling for lengthof sessions at the therapist level. Within the drives category, we observed a significant increase in words associated with achievement (β=.774, P<.001, ICC=0.146) andpower (β=.831, P<.001, ICC=0.072), with the ICC values
indicating that 14.6% and 7.2% of the variance in theseword categories, respectively, could be attributed to differences between therapists. In the lifestyle category, there wasan increase in the use of words related to home (β=.047,
P=.35, ICC=0.060) and work (β=.047, P=.12, ICC=0.285),
but these changes are not statistically significant. The ICC
values suggest that 6.0 and 28.5% of the variance inthese word categories, respectively, could be attributed todifferences between therapists. On the other hand, leisureand religion-associated words showed a significant decrease(β=−.166, P=.002, ICC=0.033; β=−.105, P<.001, ICC=0.031,
respectively), while words associated with money displayed
substantial increases (β=.204, P<.001, ICC=0.035). The ICC
JMIR Med Inform 2024 | vol. 12 | e52678 | p. 5
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Stanhope et alvalues for these categories indicate that 3.3%, 3.1%, and3.5% of the variance, respectively, could be attributed todifferences between therapists. In the health category, there
was a notable increase in the use of physical health–related words (β=.427, P=.03, ICC=0.159), with 15.9% of
the variance attributable to differences between therapists.
In contrast, wellness-related words decreased significantly(β=−.427, P<.001, ICC=0.211), with 21.1% of the varianceattributable to differences between therapists. In the socialreferents category, the use of family-related words did notshow any significant change (β=−.016, P=.89, ICC=0.085),
with 8.5% of the variance attributable to differences between
therapists. However, the frequency of friend-related words
decreased significantly (β=−.084, P=.005, ICC=0.028), with
only 2.8% of the variance attributable to differences between
therapists.

Table 3. Person-centeredness before and after collaborative documentation (CD). Coefficients were reported. Standard errors are in parentheses. β
denotes coefficients of fixed-effects models. Fixed-effects estimates were based on models from the STATA module “xtreg” commands, clustered bytherapist and with controls for length of session (minutes).
Category

Sample words

Frequency mean

DrivesAchievement

Before CD

After CD

Fixed effects
ICCa

β (SE)

P valuework, better, best, working

4.04

5.09

0.146

.774 (0.129)

<.001

Power

wn, order, allow, power

1.75

2.66

0.072

.831 (0.094)<.001

Affiliation

Lifestylewe, our, us, help

4.28

4.24

0.212

.086 (0.138)

.53

Home

home, house, room, bed

0.66

0.77

0.060

.047 (0.051)

.35

Money

business, pay, price, market

0.20

0.41

0.035

.204 (0.035)<.001

LeisureReligionWork

HealthPhysicalWellness

Social referents
FamilyFriendgame, fun, play, party

god, hell, christmas, church

work, school, working, class

0.78

0.21

6.20

0.62

0.10

6.68

0.033

0.031

0.285

−.166 (0.053)−.105 (0.029).234 (0.152)

.002

<.001
.12

medic, food, patients, eye

5.43

5.90

0.159

.427 (0.193).03

parent, mother, father, baby
friend, boyfriend, girlfriend,
dude

2.24
0.26

2.09
0.18

0.085
0.028

−.016 (0.116)−.084 (0.03)

.89
.005

healthy, gym, supported, diet

0.93

0.59

0.211

−.416 (0.057)<.001

aICC: intraclass correlation coefficient.

The contextualizing analysis provided insight into the
documentation content reflecting the selected LIWC
categories and demonstrated how person-centered principles
can be integrated into clinical documentation. Using LIWC
categories illustrated how providers described their clients inways that gave a sense of their lives beyond their mental
health. These details included what they care about and howthat related to their personal goals (getting a job, financial
situation, going on a cruise, or attending church), their life
beyond the clinic in the community (their home life, family
and friends, and their community). The sentences also showed
when clinicians used a strengths-based approach, the natureand content of their clinic notes changed in ways that moved
beyond symptoms [34].

pronounced positive increase was within the drives category,with words associated with power and achievement increasing. In terms of PCC, this indicates providers made morereference to self-determination, including how the client
has made progress, and their strengths. Lifestyle categories,
which include words related to hobbies and other social
activities were more mixed, showing that providers were notconsistent in increasing their focus on personal life goals
or taking a holistic view of the client. In health, there
was a significant increase in references to physical healthbut a decrease in references to wellness. This may reflect
the increasing efforts to integrate health into their clinicalinterventions [35], but does not indicate a more recovery-oriented focus. Finally, in terms of social referents, there was nochange in family references, which may be due to the fact thatfamily inclusion is a common best practice of PCC [36,37],
and a decrease in references to friends, often considered asource of natural supports within PCC.

The quantitative results indicating whether there was anincrease in person-centeredness of clinical documentation as
indicated by relevant LIWC categories were mixed, witha significant increase in half the subcategories. The mostExisting work has suggested that CD can improve
important indicators of PCC, including service engagement
and the quality of the working alliance [1,8], but research has
yet not illuminated how this practice impacts care processes,

Discussion
Principal Findings

https://medinform.jmir.org/2024/1/e52678

JMIR Med Inform 2024 | vol. 12 | e52678 | p. 6
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
including how person-centered principles are integrated into
clinical interventions. Through analysis of session notes,
this study found an increase in strengths-based approaches
to clinical documentation following CD training, which
may also reflect a shift towards interventions that emphasize self-determination. In addition to expanding the limited
evidence base around the impact of CD on clinical quality,
this study uniquely describes the mechanisms through which
CD supports alliance building and engagement in care.
To meet the much-documented challenges of measuring
PCC [38,39], a core component of health care reform, thisstudy sought to harness the richness of clinical narrative datausing a machine learning algorithm. While some studies haveused NLP to study psychotherapy sessions [25], this is oneof the first to mine narrative psychosocial documentation
in behavioral health settings. Overall, this methodologicalapproach has considerable potential to sample large quantities of documentation to measure different aspects of carequality, including PCC. NLP can be used both by researchers
to examine how quality of care predicts clinical outcomes andby clinics to promote and document quality improvement.
Despite this potential, there are considerable challenges
in calibrating algorithms so that they can accurately capture
more nuanced aspects of care, such as person-centeredness.
This study chose to use a well-validated dictionary designed
to capture psychological concepts within narrative data.
Although the study team was able to map existing LIWC
categories onto established principles of PCC, the algorithm
was not explicitly designed to measure these constructs,
which may have contributed to the lack of positive findings within certain subcategories. Furthermore, the LIWC
dictionary’s focus on single words limits the algorithm’s
ability to capture more nuanced meanings that occur whenwords are evaluated within the larger context of surroundingphrases or sentences. This suggests the need to develop an
algorithm focused specifically on PCC.
This study showed the potential for using NLP techniques to measure the quality of care within behavioral
health settings. As more care standards demand that clinics
demonstrate PCC [40] within mental health, there is apressing need for feasible methods to capture this quality
dimension. Being able to use at the aggregate level note datathat reflects more nuanced and individualized aspects of carewould help clinics document and report PCC.
In addition, our study highlights the value of clinical notes
for research in behavioral health settings. Clinical notes havebeen harnessed for research purposes but have mainly been
confined to hospital settings due to the scarcity of publicly
available data [41]. The nature of psychosocial documentation differs from other clinical notes, which require different analytics and models and there is a need for publicly
available data sets for analyzing psychotherapy notes in the

Stanhope et al
United States. Furthermore, mental health notes often containidentifiable and sensitive information different from otherclinical notes for physical illnesses, so an in-depth discussion
on ethics, privacy, and deidentification and the developmentof techniques such as word embedding models to improve theprivacy of clinical notes [42,43] for mental health notes isrequired.

Limitations
The study was limited to the scope of the LIWC categories
rather than an algorithm developed specifically to capture
PCC and therefore, was not able to measure the concept in its
entirety. Furthermore, the analysis limited itself to categories
with a positive valence rather than also measuring the inverseof PCC. Although well validated, the LIWC can still fail tocapture the meaning of words and mis-categorize them but
as it is used more, the algorithm will continue to be trained
and improved. Overall, while documentation is an important
indicator of PCC, it does not directly capture the interpersonal interactions between the provider and the clinician whichshape a client’s experience of PCC.
While our fixed effects model controlled for Provider ID
and Date of Service, we were unable to account for potential
differences in therapy modality or length of stay. In our study
setting, the therapy modality primarily consisted of individual
psychotherapy sessions excluding 2 notes (1 group therapy
and 1 family therapy) resulting in insufficient variation tocontrol for this variable. Additionally, our current data set did
not include sufficient information on length of stay to include
this variable in the model. Future research could benefit fromexamining the influence of therapy modality and longitudinal
factors on PCC language use in clinical documentation.
It is important to acknowledge that while our study
demonstrates an increase in person-centered content withinclinic visit notes following the implementation of the CD, we
did not directly assess whether the increased PCC in the clinic
visit notes was associated with improved PCC practices.
Future research should investigate the relationship between
the presence of PCC in clinical notes and its impact on PCC
practices and outcomes.

Conclusion
This study is an important first step in using NLP tomeasure the quality of care through narrative clinical notes
in behavioral health settings. We were able to identify
key PCC principles within the notes using a dictionarybased approach and examine whether CD changes the way
providers document with respect to PCC. This demonstrates
the potential for NLP to be used by both researchers andclinics as a quality improvement tool and the importance offurther developing algorithms that can capture the nuances of
PCC.

Acknowledgments
This work was funded by the Constance and Martin Silver Center on Data Science and Social Equity at New York University.

https://medinform.jmir.org/2024/1/e52678

JMIR Med Inform 2024 | vol. 12 | e52678 | p. 7
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Stanhope et al

Conflicts of Interest
None declared.
Multimedia Appendix 1
Person-centeredness before and after collaborative documentation: control for negation words.
[DOCX File (Microsoft Word File), 21 KB-Multimedia Appendix 1]
Multimedia Appendix 2
Paired sample t test.
[DOCX File (Microsoft Word File), 33 KB-Multimedia Appendix 2]
References
1.
Stanhope V, Ingoglia C, Schmelter B, Marcus SC. Impact of person-centered planning and collaborative documentation
on treatment adherence. Psychiatr Serv. Jan 2013;64(1):76-79. [doi: 10.1176/appi.ps.201100489] [Medline: 23280459]
2.
Matthews EB. Integrating the electronic health record into behavioral health encounters: strategies, barriers, andimplications for practice. Adm Policy Ment Health. Jul 2017;44(4):512-523. [doi: 10.1007/s10488-015-0676-3]
3.
DiCarlo R, Garcia YE. Chapter 4 - Electronic record keeping and psychotherapy alliance: the role of concurrent
collaborative documentation. In: Tettegah SY, Garcia YE, editors. Emotions, Technology, and Health. Academic Press;
2016:63-82.
4.
Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New
Health System for the 21st Century. National Academies Press; 2001. ISBN: 9780309072809
5.
Tondora J, Miller R, Slade M, Davidson L. Partnering for Recovery in Mental Health: A Practical Guide to PersonCentered Planning. John Wiley & Sons; 2014. ISBN: 9781118388556
6.
Tondora J, Pocklington S, Osher D, Davidson L. Implementation of person-centered care and planning: from policy topractice to evaluation. Substance Abuse and Mental Health Services Administration; 2005. URL: https://www.hsri.org/
files/uploads/publications/ImplementationOfPersonCenteredCareandPlanning.pdf [Accessed 2024-09-13]
7.
Schwarz J, Bärkås A, Blease C, et al. Sharing clinical notes and electronic health records with people affected by mental
health conditions: scoping review. JMIR Ment Health. Dec 14, 2021;8(12):e34170. [doi: 10.2196/34170] [Medline:
34904956]
8.
Matthews EB. Computer use in mental health treatment: understanding collaborative documentation and its effect on thetherapeutic alliance. Psychotherapy (Chic). Jun 2020;57(2):119-128. [doi: 10.1037/pst0000254] [Medline: 31599638]
9.
O’Neill S, Blease C, Delbanco T. Open notes become law: a challenge for mental health practice. Psychiatr Serv Am
Psychiatric Assoc. Jul 1, 2021;72(7):750-751. [doi: 10.1176/appi.ps.202000782] [Medline: 33971748]
10. Agency for Planning and Evaluation Certified Community Behavioral Health Clinics Demonstration Program: Report toCongress, 2019. Washington, DC: Department of Health and Human Services Sep 2019; URL: https://aspe.hhs.gov/
reports/certified-community-behavioral-health-clinics-demonstration-program-report-congress-2019 [Accessed2022-10-13]
11. Hyun S, Johnson SB, Bakken S. Exploring the ability of natural language processing to extract data from nursing
narratives. Comput Inform Nurs. Jul 2009;27(4):215-223. [doi: 10.1097/NCN.0b013e3181a91b58] [Medline: 19574746]
12. Haselden M, Dixon LB, Overley A, et al. Giving back to families: evidence and predictors of persons with serious
mental illness contributing help and support to families. Community Ment Health J. May 2018;54(4):383-394. [doi: 10.
1007/s10597-017-0172-1] [Medline: 29022227]
13. Butler JM, Gibson B, Patterson OV, et al. Clinician documentation of patient centered care in the electronic healthrecord. BMC Med Inform Decis Mak. Mar 12, 2022;22(1):65. [doi: 10.1186/s12911-022-01794-w] [Medline: 35279157]
14. Abbe A, Grouin C, Zweigenbaum P, Falissard B. Text mining applications in psychiatry: a systematic literature review.
Int J Methods Psychiatr Res. Jun 2016;25(2):86-100. [doi: 10.1002/mpr.1481]
15. Edgcomb JB, Zima B. Machine learning, natural language processing, and the electronic health record: innovations inmental health services research. Psychiatr Serv. Apr 2019;70(4):346-349. [doi: 10.1176/appi.ps.201800401] [Medline:
30784377]
16. Dreisbach C, Koleck TA, Bourne PE, Bakken S. A systematic review of natural language processing and text mining ofsymptoms from electronic patient-authored text data. Int J Med Inform. May 2019;125:37-46. [doi: 10.1016/j.ijmedinf.
2019.02.008] [Medline: 30914179]
17. Mascio A, Kraljevic Z, Bean D, et al. Comparative analysis of text classification approaches in electronic health records.
Presented at: Proceedings of the 19th SIGBioMed Workshop on Biomedical Language Processing. 86-94; 2020.[doi: 10.
18653/v1/2020.bionlp-1.9]https://medinform.jmir.org/2024/1/e52678

JMIR Med Inform 2024 | vol. 12 | e52678 | p. 8
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.

Stanhope et al

Afshar M, Phillips A, Karnik N, et al. Natural language processing and machine learning to identify alcohol misuse fromthe electronic health record in trauma patients: development and internal validation. J Am Med Inform Assoc. Mar 1,
2019;26(3):254-261. [doi: 10.1093/jamia/ocy166] [Medline: 30602031]
Carson NJ, Mullin B, Sanchez MJ, et al. Identification of suicidal behavior among psychiatrically hospitalizedadolescents using natural language processing and machine learning of electronic health records. PLoS ONE. Feb
2019;14(2):e0211116. [doi: 10.1371/journal.pone.0211116] [Medline: 30779800]
Hammond KW, Ben‐Ari AY, Laundry RJ, Boyko EJ, Samore MH. The feasibility of using large‐scale text mining todetect adverse childhood experiences in a VA‐treated population. J Trauma Stress. Dec 2015;28(6):505-514. [doi: 10.
1002/jts.22058] [Medline: 26579624]
Rajendran S, Topaloglu U. Extracting smoking status from electronic health records using NLP and deep learning.
AMIA Jt Summits Transl Sci Proc. May 30, 2020;2020:507-516. [Medline: 32477672]
McCoy TH, Castro VM, Cagan A, Roberson AM, Kohane IS, Perlis RH. Sentiment measured in hospital discharge notes
is associated with readmission and mortality risk: an electronic health record study. PLoS ONE. Aug 24,
2015;10(8):e0136341. [doi: 10.1371/journal.pone.0136341] [Medline: 26302085]
Zerden L de S, Lombardi BM, Richman EL, Fraher EP, Shoenbill KA. Harnessing the electronic health record toadvance integrated care. Fam Syst Health. Mar 2021;39(1):77-88. [doi: 10.1037/fsh0000584] [Medline: 34014732]
Rahimian M, Warner JL, Jain SK, Davis RB, Zerillo JA, Joyce RM. Significant and distinctiven-grams in oncology
notes: a text-mining method to analyze the effect of opennotes on clinical documentation. JCO Clin Cancer Inform. Dec
2019;3(3):1-9. [doi: 10.1200/CCI.19.00012] [Medline: 31184919]
Atkins DC, Steyvers M, Imel ZE, Smyth P. Scaling up the evaluation of psychotherapy: evaluating motivational
interviewing fidelity via statistical text classification. Impl Sci. Dec 2014;9(1). [doi: 10.1186/1748-5908-9-49] [Medline:
24758152]
Blease C, Torous J, Hägglund M. Does patient access to clinical notes change documentation? Front Public Health. Nov
27, 2020;8:577896. [Medline: 33330320]
Alpert JM, Morris BB, Thomson MD, Matin K, Sabo RT, Brown RF. Patient access to clinical notes in oncology: amixed method analysis of oncologists’ attitudes and linguistic characteristics towards notes. Patient Educ Couns. Oct
2019;102(10):1917-1924. [doi: 10.1016/j.pec.2019.05.008] [Medline: 31109771]
MTM Services. 2023. URL: https://www.mtmservices.org/ [Accessed 2023-08-31]
Boyd RL, Ashokkumar A, Seraj S, Pennebaker JW. The development and psychometric properties of LIWC-22. Feb
2022. [doi: 10.13140/RG.2.2.23890.43205]
Klein D. Implementing a general framework for assessing interrater agreement in stata. Stata J. Dec 2018;18(4):871-901.
[doi: 10.1177/1536867X1801800408]
Yoo N, Matthews E, Baslock D, Stanhope V. Impact of collaborative documentation on completeness and length ofclinical notes in behavioral health settings. Psychiatr Serv. Aug 2, 2023;75(2):186-190. [doi: 10.1176/appi.ps.20230118][Medline: 37528697]
Thornton JD, Schold JD, Venkateshaiah L, Lander B. Prevalence of copied information by attendings and residents incritical care progress notes. Crit Care Med. Feb 2013;41(2):382-388. [doi: 10.1097/CCM.0b013e3182711a1c] [Medline:
23263617]
Wang MD, Khanna R, Najafi N. Characterizing the source of text in electronic health record progress notes. JAMA
Intern Med. Aug 1, 2017;177(8):1212. [doi: 10.1001/jamainternmed.2017.1548]
Braun MJ, Dunn W, Tomchek SD. A pilot study on professional documentation: do we write from a strengths
perspective? Am J Speech Lang Pathol. Aug 15, 2017;26(3):972-981. [doi: 10.1044/2017_AJSLP-16-0117] [Medline:
28637055]
Scharf DM, Eberhart NK, Schmidt N, et al. Integrating primary care into community behavioral health settings:
programs and early implementation experiences. Psychiatr Serv. Jul 2013;64:660-665. [doi: 10.1176/appi.ps.201200269]
Frampton SB, Giuliano M. Patient-centered care: the North Star to guide us during uncertainty into a better day. Int J
Qual Health Care. Aug 9, 2023;35(3). [doi: 10.1093/intqhc/mzad061] [Medline: 37556113]
Boise L, White D. The family’s role in person-centered care: practice considerations. J Psychosoc Nurs Ment HealthServ. May 2004;42(5):12-20. [doi: 10.3928/02793695-20040501-04]
Stanhope V, Baslock D, Tondora J, Jessell L, Ross AM, Marcus SC. Developing a tool to measure person-centered carein service planning. Front Psychiatry. Aug 2, 2021;12:681597. [doi: 10.3389/fpsyt.2021.681597] [Medline: 34408678]
Burgers JS, van der Weijden T, Bischoff E. Challenges of research on person-centered care in general practice: a scoping
review. Front Med. Jun 24, 2021;8:1-9. [doi: 10.3389/fmed.2021.669491]

https://medinform.jmir.org/2024/1/e52678

JMIR Med Inform 2024 | vol. 12 | e52678 | p. 9
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
40.
41.
42.
43.

Stanhope et al

Person-centered service planning guidelines for Medicaid managed care organizations, local departments of social
services, and health homes. New York State Department of Health. 2022. URL: https://www.health.ny.gov/health_care/managed_care/plans/pcsp_guidelines.htm [Accessed 2023-08-30]
Zhou N, Wu Q, Wu Z, Marino S, Dinov ID. DataSifterText: partially synthetic text generation for sensitive clinicalnotes. J Med Syst. Nov 16, 2022;46(12):96. [doi: 10.1007/s10916-022-01880-6] [Medline: 36380246]
Abdalla M, Abdalla M, Rudzicz F, Hirst G. Using word embeddings to improve the privacy of clinical notes. J Am Med
Inform Assoc. Jun 1, 2020;27(6):901-907. [doi: 10.1093/jamia/ocaa038] [Medline: 32388549]
Abdalla M, Abdalla M, Hirst G, Rudzicz F. Exploring the privacy-preserving properties of word embeddings:
algorithmic validation study. J Med Internet Res. Jul 15, 2020;22(7):e18055. [doi: 10.2196/18055] [Medline: 32673230]

Abbreviations
CD: collaborative documentation
EHR: electronic health record
ICC: intraclass correlation coefficient
LIWC: Linguistic Inquiry and Word Count
NLP: natural language processing
PCC: person-centered care
PCCP: person-centered care planning

Edited by Jennifer Hefner; peer-reviewed by Jaffar Abbas, Kyoungmi Woo, Lisa de Saxe Zerden; submitted 12.09.2023;
final revised version received 07.06.2024; accepted 26.06.2024; published 20.09.2024
Please cite as:
Stanhope V, Yoo N, Matthews E, Baslock D, Hu Y
The Impact of Collaborative Documentation on Person-Centered Care: Textual Analysis of Clinical Notes
JMIR Med Inform 2024;12:e52678
URL: https://medinform.jmir.org/2024/1/e52678
doi: 10.2196/52678© Victoria Stanhope, Nari Yoo, Elizabeth Matthews, Daniel Baslock, Yuanyuan Hu. Originally published in JMIR Medical
Informatics (https://medinform.jmir.org), 20.09.2024. This is an open-access article distributed under the terms of the Creative
Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, andreproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The
complete bibliographic information, a link to the original publication on https://medinform.jmir.org/, as well as this copyrightand license information must be included.

https://medinform.jmir.org/2024/1/e52678

JMIR Med Inform 2024 | vol. 12 | e52678 | p. 10(page number not for citation purposes)