C are C orpus+: Expanding and Augmenting Caregiver Strategy Data to Support Pediatric Rehabilitation (2024)

Abstract: Caregiver strategy classification in pediatric rehabilitation contexts is strongly motivated by real-world clinical constraints but highly under-resourced and seldom studied in natural language processing settings. We introduce a large dataset of 4,037 caregiver strategies in this setting, a five-fold increase over the nearest contemporary dataset. These strategies are manually categorized into clinically established constructs with high agreement ( \kappa =0.68-0.89). We also propose two techniques to further address identified data constraints. First, we manually supplement target task data with publicly relevant data from online child health forums. Next, we propose a novel data augmentation technique to generate synthetic caregiver strategies with high downstream task utility. Extensive experiments showcase the quality of our dataset. They also establish evidence that both the publicly available data and the synthetic strategies result in large performance gains, with relative F _1 increases of 22.6% and 50.9%, respectively.

## introduction globally, over 50 million children aged 0-5 years experience disability @xcite . these young children and their families benefit from timely access to quality pediatric rehabilitation services in diverse contexts, ranging from hospital to home and community @xcite . when rehabilitation providers create conditions for families to share their expertise about their child's attendance and involvement in valued activities, perceived supports and strategies, and priorities for change, they can engage families in shared decision-making to design and monitor a meaningful service plan @xcite . providers benefit from gathering this information from families to direct equitable care (pinto et al., 2022; jarvis and fink, 2021; magnusson and khetani, * work completed at university of illinois chicago. 2022). however, parent-generated data is often collected and documented as free text narratives, necessitating efforts to extract and standardize content for clinical and research applications @xcite @xcite . development of web-based tools (e.g., the participation and environment measure @xcite , also known as pem) can diversify the capture and use of structured and narrative information from families to drive pediatric rehabilitation service design and improvement. recent work established benchmarks for the detection and classification of caregiver strategies collected using two available versions of a pem tool @xcite . however, model performance was constrained by a limited availability of caregiver strategy data involving children, across a subset of relevant age ranges and rehabilitation care contexts. a larger, more diverse data source is needed to strengthen applicability across the broader pediatric rehabilitation care continuum. we respond to that need in this work, making three primary contributions. first, we establish inclusion and exclusion criteria for data sources fitting one or more classes of caregiver strategies. we define primary class characteristics, identify relevant and irrelevant external sources, and construct or select prototypical samples. next, we identify and prepare a subset of strategies data from (a) three datasets from prior related research (@xmath0, and (b) publicly available data instances that also match established guidelines. data are preprocessed and stored in a format compatible with existing caregiver strategies data @xcite , and labeled according to strategy class with high reliability (κ=0.68-0.89). finally, we propose a novel data augmentation technique to generate and filter caregiver strategies. we perform quality checks and performance comparisons to assess augmentation feasibility within this task domain. ultimately, we find that our manually and synthetically augmented data improves strategy classification performance by a wide margin, establishing a new performance ceiling for this task (f 1 =0.80). ## discussion our findings broadly support the premise that diverse data can be leveraged for specialized pediatric rehabilitation contexts, justifying our manual data curation and our investigation of synthetic data expansion within this domain. it also replicates findings from @xcite across a broader age range and rehabilitation care continuum; given the heterogeneity of pediatric rehabilitation, these findings encourage the application of automated strategy classification for this use case. performance continues to improve with the addition of non-strategy data harvested from publicly available online forums pertaining to child health, suggesting an accessible avenue for increasing performance. useful applications of this may include initiating caregiver education when detecting non-strategy responses during pem completion @xcite , enabling skilled engagement for families on waitlists to access rehabilitation services @xcite . importantly, we also find that prompt-based synthetic data expansion improves model performance. as llms are nascent in rehabilitation @xcite , this establishes evidence that llms are capable of adequately replicating user-generated content in this domain, to an extent that it improves model performance when used as a training source. it also raises intriguing questions for future work, such as whether llms could be used to consolidate strategies of the same type. this could address known barriers to family-centered goal-setting practice @xcite . it could also decrease caregiver burden when searching through a large bank of user-generated strategies, such as that in the pem-plus goal setting application, sustaining longer-term feasibility of user-driven strategy exchange. despite promising technical merit and potential clinical utility, there are ethical considerations to synthetic data expansion, such as its potential to compromise family voice @xcite . this presents a ripe opportunity for further study. ## conclusion in this work, we introduced an expanded dataset (@xmath1,062, an approximately five-fold increase over prior work) of caregiver strategies in diverse pediatric rehabilitation contexts. the strategies were identified from prior pediatric rehabilitation studies and manually assigned to clinically-established constructs by trained researchers spanning a rigorous five-month annotation process, resulting in strong agreement (κ=0.68-0.89). we also identified non-strategies from publicly available online child health forums to supplement the data and address previously-identified class balance issues @xcite . additionally, we proposed a new technique for synthetic data augmentation in this domain, guided by three diverse prompts leveraging task-relevant contextual information while filtering for synthetic strategies with high anticipated task utility. we demonstrated the value of the additional manually curated strategies, publicly available task-relevant non-strategies, and our novel data augmentation approach for the downstream task of caregiver strategy classification. our results establish evidence that both publicly available non-strategies (f 1 =0.65, a 22.6% relative increase over the use of cc+ alone) and prompting-based synthetic strategies (f 1 =0.80, a 50.9% relative increase) can support impressive performance gains in this highly specialized and under-resourced domain.