"Tips From Former Smokers" Can Benefit From Considering All Available Data: Reply to McAfee et al.

We agree1 that “Tips from Former Smokers” (Tips) has been incredibly important and has likely helped many smokers kick the habit. We also agree that campaign evaluations should be based on all available data. For these same reasons, however, our study2 should not be discounted simply because the findings suggest Internet searches for cessation and the health risks featured in Tips declined between the first and second years of the campaign. Moreover, careful scrutiny of the most relevant peer-reviewed works, including those cited by the commenters, yields findings similar to our original analysis of search queries (Figure 1).2–6 Figure 1 Peer reviewed evidence regarding Tips, 2012-2013 The commenters’ Lancet study found self-reported quit attempts significantly increased from 31% to 34% during the first year of Tips,3 along with increases in 1-800-Quit-Now calls and Smokefree.gov visits.4 This news was rightly celebrated and consistent with these findings; our results demonstrated a concurrent increase in cessation-related Internet searches (19%, 95% CI=14%, 26%).2 But, looking beyond the campaign’s first year, we demonstrated another important finding that traditional metrics also demonstrate: Tips’ appeal may be waning. First, self-reported quit attempts were not statistically significantly higher among those reporting exposure to the campaign during Tips 2013 after adjusting for covariates in multiple logistic regressions,5 mirroring what we found for cessation searches in 2013. Second, even though 1-800-Quit-Now calls increased during Tips 2013, this increase was significantly (p0.001) lower (by a factor of nearly half) than the increase during Tips 2012.4,6 Our results reflected similar patterns, with significantly fewer searches during Tips 2013 than Tips 2012, including for health risks in both repeated advertisements and new advertisements. There was one caveat. Smokefree.gov visits increased substantially more during Tips 2013 than Tips 2012 and this merits recognition, but also investigation of the cause so it can be exploited.6 That our search-based analytics were largely confirmed by traditional metrics underscores their probable validity for understanding cessation help seeking and interest in health risks. Admittedly, the commenters rightly acknowledge our method had limitations, as all methods do. For example, survey responses change as a function of the survey itself in addition to the intervention being studied.7 We have pioneered many new methods for search query surveillance,8 including a state-of-the-art revision to Google Flu Trends (the basis of the commenters’ critique of search data).9 This work and numerous validation studies elsewhere (including those for tobacco-related behaviors)10,11 suggests the commenters’ concerns do not nullify the use of search query surveillance for understanding Tips. It is widely accepted that media campaigns have the greatest appeal when they offer new content with outcomes spiking early and declining as content is repeated.12 Based on all available data, Tips appears to be following a similar pattern. Our findings do not diminish how Tips has helped smokers or our support for mass media campaigns. We remain enthusiastic about what could be next for Tips and how search query surveillance, along with careful analysis of all available data, can inform future evaluations.