## 6 Conclusions  
In conclusion, we developed an NLP algorithm that extracts ECOG PS scores from unstructured documents across an EHR network of diverse real-world clinical settings. The algorithm improves the completeness of a critical variable at the time of treatment initiation and performs well across 21 distinct cancer types. Furthermore, additional validation, including the comparison of rwOS of patients with extracted vs. structured ECOG PS scores within a cohort of patients with aNSCLC, demonstrated clinically expected results. The use of high-performing algorithms such as this can help to overcome key challenges in RWD, such as missingness, as well as make one of the key advantages of RWD more attainable: the capability to aggregate longitudinal routine clinical care information from large patient cohorts for high-quality clinical research, ultimately benefiting providers, regulatory stakeholders, and, most importantly, patients.  
**Supplementary Materials:** The following supporting information can be downloaded at: [[https://www.mdpi.com/article/10.3390/app13106209/s1](https://www.mdpi.com/article/10.3390/app13106209/s1)]([https://www.mdpi.com/article/10.3390/app13106209/s1](https://www.mdpi.com/article/10.3390/app13106209/s1)), Figure S1: rwOS in patients present in the study databases across all eligible diseases, stratified according to their ECOG PS score; Table S1: Detailed cohort eligibility criteria; Table S2: Impact of the application of the NLP algorithm on the ECOG PS completeness in EHR-derived databases for 21 diseases; Table S3: rwOS (months) in patients with aNSCLC stratified according to their ECOG PS score for the subcohort with ECOG PS scores available as structured data, and for the subcohort with ECOG PS scores extracted via algorithm; Table S4: HR for patients with structured ECOG (reference group) in patients with aNSCLC. Table S5. Median real world OS (months) for patients present in the testing set across all eligible diseases.  
**Author Contributions:** Concept and design: A.B.C., C.J., A.R., K.H., M.R., S.N., G.A., B.H. and R.M. Data collection, analysis and interpretation: A.B.C., C.J., A.R., K.H., M.R., S.N. and B.H. Manuscript writing and review: All. All authors have read and agreed to the published version of the manuscript.  
**Funding:** This study was sponsored by Flatiron Health Inc., which is an independent member of the Roche group.  
**Institutional Review Board Statement:** Institutional Review Board approval of the study protocol was obtained prior to study conduct, and included a waiver of informed consent.  
**Informed Consent Statement:** Patient consent was waived due to: the study does not involve greater than minimal risk; leverages observational research, which relies on data which was previously collected--as such it is not practicable to conduct the research without the waiver or alteration; and waiving or altering the informed consent will not adversely affect the subjects' rights and welfare.  
**Data Availability Statement:** Requests for data sharing by license/permission for the purpose of replicating results in this manuscript can be submitted to [EMAIL_ADDRESS].  
**Acknowledgments:** Editorial support was provided by [PERSON], [PERSON] and [PERSON] of Flatiron Health.  
**Conflicts of Interest:** A.B.C., C.J., A.R., K.H., M.R., S.N., G.A. and R.M. report employment in Flatiron Health Inc., which is an independent subsidiary of the Roche Group, and stock ownership in Roche. R.M. and A.B.C. report equity ownership in Flatiron Health. B.H. reports consulting fees for Flatiron Health, the National Kidney Foundation, and Value Analytics Labs.