Machine Learning and Natural Language Processing for Geolocation-Centric Monitoring and Characterization of Opioid-Related Social Media Chatter
Authors: Abeed Sarker, Graciela Gonzalez-Hernandez, Yucheng Ruan, Jeanmarie Perrone
Journal: JAMA Network Open (2019)

<h3>Importance</h3> Automatic curation of consumer-generated, opioid-related social media big data may enable real-time monitoring of the opioid epidemic in the United States. <h3>Objective</h3> To develop and validate an automatic text-processing pipeline for geospatial and temporal analysis of opioid-mentioning social media chatter. <h3>Design, Setting, and Participants</h3> This cross-sectional, population-based study was conducted from December 1, 2017, to August 31, 2019, and used more than 3 years of publicly available social media posts on Twitter, dated from January 1, 2012, to October 31, 2015, that were geolocated in Pennsylvania. Opioid-mentioning tweets were extracted using prescription and illicit opioid names, including street names and misspellings. Social media posts (tweets) (n = 9006) were manually categorized into 4 classes, and training and evaluation of several machine learning algorithms were performed. Temporal and geospatial patterns were analyzed with the best-performing classifier on unlabeled data. <h3>Main Outcomes and Measures</h3> Pearson and Spearman correlations of county- and substate-level abuse-indicating tweet rates with opioid overdose death rates from the Centers for Disease Control and Prevention WONDER database and with 4 metrics from the National Survey on Drug Use and Health for 3 years were calculated. Classifier performances were measured through microaveraged F1 scores (harmonic mean of precision and recall) or accuracies and 95% CIs. <h3>Results</h3> A total of 9006 social media posts were annotated, of which 1748 (19.4%) were related to abuse, 2001 (22.2%) were related to information, 4830 (53.6%) were unrelated, and 427 (4.7%) were not in the English language. Yearly rates of abuse-indicating social media post showed statistically significant correlation with county-level opioid-related overdose death rates (n = 75) for 3 years (Pearson<i>r</i> = 0.451,<i>P</i> &lt; .001; Spearman<i>r</i> = 0.331,<i>P</i> = .004). Abuse-indicating tweet rates showed consistent correlations with 4 NSDUH metrics (n = 13) associated with nonmedical prescription opioid use (Pearson<i>r</i> = 0.683,<i>P</i> = .01; Spearman<i>r</i> = 0.346,<i>P</i> = .25), illicit drug use (Pearson<i>r</i> = 0.850,<i>P</i> &lt; .001; Spearman<i>r</i> = 0.341,<i>P</i> = .25), illicit drug dependence (Pearson<i>r</i> = 0.937,<i>P</i> &lt; .001; Spearman<i>r</i> = 0.495,<i>P</i> = .09), and illicit drug dependence or abuse (Pearson<i>r</i> = 0.935,<i>P</i> &lt; .001; Spearman<i>r</i> = 0.401,<i>P</i> = .17) over the same 3-year period, although the tests lacked power to demonstrate statistical significance. A classification approach involving an ensemble of classifiers produced the best performance in accuracy or microaveraged F1 score (0.726; 95% CI, 0.708-0.743). <h3>Conclusions and Relevance</h3> The correlations obtained in this study suggest that a social media–based approach reliant on supervised machine learning may be suitable for geolocation-centric monitoring of the US opioid epidemic in near real time.