Inaccuracies in Evaluation of the Veterans Affairs Colorectal Cancer Oncology Watch Program
Authors: Siddharth P. Sura, Matthew Moseley, Mohammed Wehbi, Yvette Williams
Journal: Journal of Clinical Oncology (2013)

In their recent evaluation of the Veterans Affairs (VA) Colorectal Cancer (CRC) Oncology Watch (OncWatch) program, Bian et al 1 concluded that the intervention did not improve CRC screening rates and, furthermore, had negative "unintended consequences" among nonelderly veterans in eight medical centers in the Veterans Integrated Service Network 7 (VISN7).We disagree with these assertions for several reasons.First,thepurposeofOncWatchinthestudybyBianetalismisstated. OncWatch is an advanced clinical reminder with branching logic, which was developed in VISN7 in 2006-2007 and introduced in 2008 to complement the pre-existing basic clinical reminder system.The stated purpose ofOncWatchwastoimprovefollow-upforabnormalCRCscreening,not to improve CRC screening rates.Through OncWatch, for example, gastroenterologists at the Atlanta VA Medical Center enter a reminder in the Computerized Patient Record System (CPRS) for a surveillance colonoscopy in 3 years for a 50-year-old veteran with three small tubular adenomas on screening colonoscopy.Such reminders, present for surveillance and diagnostic colonoscopies, are then visible to all providers when opening a veteran's chart in CPRS.Second,thecontrolcohortusedbyBianetalisinvalid.Specifically,to assess the intervention for improving CRC screening, the authors compare medical centers in VISN7 (the intervention group) with VA medical centers outside of VISN7 (the control group).This method is incorrect because a basic clinical reminder that prompts primary care nurses and physicians to give veterans fecal occult blood test cards if they have not had CRC screening within an appropriate interval has been in place in VISN7 and almost all VA medical centers throughout the country for more than 10 years.This basic clinical reminder tool, which seeks to improve CRC screening, is included in OncWatch, but its functionality is not unique to OncWatch and hospitals within VISN7, and thus is present in the Bian et al intervention and control cohorts.Third, the data used by the authors to quantify CRC screening rates in VISN7 hospitals are not comprehensive.When adding health factors-which are provider-or clinical staff-entered patient-specific information available in the Corporate Data Warehouse-to the Bian et al 1 data collection methods, our estimates of CRC screening rates in VISN7 are much higher than in the Bian et al study (Fig 1).From 2006 to 2010, screening rates increased from 52.7% to 64.7%, compared with 37.6% to 33.2% by the Bian et al estimates.Two sets of health factors may explain the discrepancy.First, health factors regarding CRC screening performed outside of the VA and VA contract facilities are not included in the Bian et al analysis and thus underestimate the number screened in any given year.This is important, as it has previously been shown that a substantial portion of veterans receive some of their care outside of the VA system. 2 Second,althoughpatientsbilledinhospiceareexcludedbyBianetal, clinical health factors identifying veteran health status, such as enrollment in hospice care, life expectancy less than 6 months, and enrollment in palliative care, may provide a more comprehensive assessment of individuals for whom CRC screening would not have been appropriate.Of note, the rise in overall CRC screening rates is even more impressive in the context of a 27.6% increase in the total number of veterans eligible for CRC screening from 2006 to 2010.In conclusion, we have found important inaccuracies in the evaluation of OncWatch by Bian et al.The misstated purpose of Onc-Watch, in addition to the use of an invalid control cohort and an incomplete data set, preclude accurate conclusions about OncWatch.