SMARTM iner: Extracting and Evaluating SMART Goals from Low-Resource Health Coaching Notes (2025)

Abstract: We present SMARTMiner, a framework for extracting and evaluating specific, measurable, attainable, relevant, time-bound (SMART) goals from unstructured health coaching (HC) notes. Developed in response to challenges observed during a clinical trial, the SMARTMiner achieves two tasks: (i) extracting behavior change goal spans and (ii) categorizing their SMARTness. We also introduce SMARTSpan, the first publicly available dataset of 173 HC notes annotated with 266 goals and SMART attributes. SMARTMiner incorporates an extractive goal retriever with a component-wise SMARTness classifier. Experiment results show that extractive models significantly outperformed their generative counterparts in low-resource settings, and that two-stage fine-tuning substantially boosted performance. The SMARTness classifier achieved up to 0.91 SMART F1 score, while the full SMARTMiner maintained high end-to-end accuracy. This work bridges healthcare, behavioral science, and natural language processing to support health coaches and clients with structured goal tracking - paving way for automated weekly goal reviews between human-led HC sessions. Both the code and the dataset are available at: https://github.com/IvaBojic/SMARTMiner.

## introduction health coaching (hc) is a person-centered intervention designed to facilitate sustainable change in healthy behavior and support self-management of chronic diseases. recent systematic reviews demonstrated that hc can significantly enhance physical activity, reduce pain, and improve psychological outcomes such as self-efficacy and quality of life among individuals with chronic conditions @xcite . a cornerstone of hc is the co-creation of actionable short-term goals that drive long-term behavior change, which in turn forms the foundation of its effectiveness as a behavioral intervention @xcite . overall, by providing focus and measurable targets, the research suggests that specific and actionable goals tend to improve health behavior outcomes more than unclear or generic behavioral goals @xcite . specific, measurable, attainable, relevant, and time-bound (smart) goals (figure 1 ) often yield better short-term results and can help sustain behavior change @xcite , as seen in improved exercise levels, weight loss, and self-management behaviors in various studies @xcite . however, setting and evaluating smart goals are laborious and complex @xcite . some goals may not be fully smart as they could lack one or more smartness components. this may stem from patient's varying levels of readiness to engage in health-related behaviors @xcite . additionally, patients often struggle to recall goals set during hc sessions @xcite , underscoring the need for improved goal documentation and consistent support between the hc sessions. this is especially important given that hc sessions are typically scheduled biweekly or monthly-intervals that exceed the intended time frame for most goals. as a result, patients often face a multi-week gap in feedback and reinforcement between sessions. in response, we present a smartminer framework that automatically extracts multiple (smart) goals set during hc sessions from unstructured session notes. since these goals are embedded within free-text narratives and cannot be audited at scale, we address two key challenges: ## the smartspan dataset smartspan is a new dataset comprising 173 annotated hc notes collected from a randomized controlled trial (rct), designed to evaluate multispan extraction in low-resource, domain-specific scenarios. each hc note summarizes real-world goal-setting conversations between clients and health coaches, offering a unique testbed for behavior change modeling. unlike structured msrc datasets-where answers in the form of multiple discontinuous spans are extracted in response to specific questions-smartspan presents challenges rooted in the complexity of extracting goals that are often diffuse or repeated across different sections of an hc note (details in appendix a). ## methods we propose a smartminer framework for multispan behavioral smart goal extraction from unstructured hc notes, as shown in figure 2 . the goal extraction module formulates goal identification as a span-based question answering task, enabling the extraction of multiple goal mentions from free-text hc notes. the smartness classi-fication module subsequently evaluates these extracted goals along three key dimensions to determine their alignment with the smart criteria. ## results and discussion ## conclusions and future work this paper presents the smartminer framework to support health coaches in identifying and categorizing client goals-particularly those that may require reformulation-while making all extracted goals accessible to clients through a mobile app interface. the framework was evaluated in both synthetic (i.e., multispanqa) and real-world (i.e., smartspan) datasets to assess its generalizability and practical utility. our experiments reveal that extractive spanbased models, such as spanqualifier, consistently outperform fine-tuned generative models in identifying multiple goals per hc note. this performance gap was particularly pronounced in smartspan-a real-world, low-resource dataset where precision and span fidelity are crucial. these results emphasize the importance of architecture selection and data-aware fine-tuning strategies, especially in settings with limited annotated data. our findings are aligned with prior work showing that extractive models (e.g., bert-based) of-ten outperform generative models in zero-shot and few-shot setups. for example, huang et al. @xcite demonstrated that their debertabased extractive model outperformed several gptbased baselines, including gpt-3.5-turbo-0301, on multispanqa. similarly, zhang et al. @xcite found that a simple bert-base extractor surpassed few-shot prompting with chatgpt-3.5. looking ahead, we aim to extend smartminer into goalguardian-a fully autonomous system designed to conduct weekly goal review sessions in between monthly hc appointments. this system would initiate structured check-ins, evaluate client progress, reinforce accountability, and generate timely summaries for health coaches. ultimately, it seeks to sustain client engagement and foster behavior change even in the absence of direct human involvement.