## 1 Introduction  
Real-world data (RWD), defined as clinical data collected in the course of routine medical care, and real-world evidence (RWE), the insights gathered via analysis and interpretation of those data [1], have become major components of the clinical research landscape. RWD rely on several possible sources, such as administrative claims, registries or electronic health records (EHRs). The adoption of new technologies in the clinic, the digitization of clinical information management, and analytical advances in particular have unlocked the potential for the use of EHRs as a major RWD source for clinical research [2, 3, 4, 5, 6]. However, EHRs have known limitations as research data sources. Information collected during routine patient care within EHRs varies between practices and may be driven by different EHR software or clinical workflows. Critical data points related to oncology care may not be comprehensively captured in structured fields and rather captured in an unstructured format [7]. In addition, the dynamics of routine care affect documentation practices. For example, documentation for purposes of patient care may be less complete because data elements pertinent for supporting decision making or billing may be prioritized; thus, normal values may be less likely to be captured compared to abnormal. As a result, incomplete or missing data and documentation variability may be suboptimal in the use of EHR-derived data for research [8; 9; 10; 11]. One approach to optimize the completeness of EHR-derived data is to combine, via manual abstraction, unstructured information (e.g., from physician notes and scanned documents) with structured data captured in predefined EHR data fields [2; 12; 13]. This approach can unlock information found only in unstructured formats and also leverage potential overlaps between unstructured and structured sources. For instance, treating physicians may describe general health status using predefined EHR fields and document it in their notes. Consulting both these sources may unveil nuances in clinical information and help resolve discrepancies during quality control processes. However, manual abstraction requires curation, detailed procedures and policies, and careful quality controls; therefore, overall scalability may be limited by its resource intensiveness [2; 13; 14]. Natural language processing (NLP) has emerged as a valuable approach to automate information extraction from unstructured clinical data sources, becoming a fertile ground for the development of tools in the healthcare setting [15; 16; 17; 18; 19; 20; 21]. NLP applications to support clinical decision making range from interpretation of diagnostic tests [22; 23] to the stratification of patients based on unstructured EHR data [24; 25; 26; 27]. Within the field of oncology, NLP can also be deployed to improve the quality and completeness of important clinical details, including stage, recurrence, and labs [28; 29; 30; 31; 32; 33]. Eastern Cooperative Oncology Group performance status (ECOG PS) score is a critical variable in oncology retrospective research. ECOG PS is a qualitative numeric score indicative of the general health status of a patient with cancer. The score ranges from 0 (no limitations) to 5 (decased) and has been consistently shown to have strong prognostic value [34]. ECOG PS score is commonly evaluated by clinicians, particularly to inform management decisions, such as whether to initiate a new treatment. ECOG PS is also a standard clinical trial eligibility criterion, a key stratification factor, and an important variable to include in multivariable analyses in oncology studies [35; 36; 37; 38; 39; 40]. However, depending on EHR work-flows and physician preference, ECOG PS may not always be recorded as a structured data point, or even documented at all [34; 41; 42; 43]. In our study, we developed and evaluated an NLP algorithm to extract ECOG PS scores from unstructured EHR sources at the time of new treatment initiation, when ECOG PS most impacts clinical decision making.