Detailed Instructions: In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
Problem:Sentence classification combines natural language processing (NLP) with machine learning to identify trends in sentence structure, BIBREF14 , BIBREF15 . Each tweet is converted to a numeric word vector in order to identify distinguishing features by training an NLP classifier on a validated set of relevant tweets. The classifier acts as a tool to sift through ads, news, and comments not related to patients. Our scheme combines a logistic regression classifier, BIBREF16 , with a Convolutional Neural Network (CNN), BIBREF17 , BIBREF18 , to identify self-reported diagnostic tweets.

It is important to be wary of automated accounts (e.g. bots, spam) whose large output of tweets pollute relevant organic content, BIBREF19 , and can distort sentiment analyses, BIBREF20 . Prior to applying sentence classification, we removed tweets containing hyperlinks to remove automated content (some organic content is necessarily lost with this strict constraint). Twitter allows users to spread content from other users via 'retweets'. We also removed these posts prior to classification to isolate tweets authored by patients. We also accounted for non-relevant astrological content by removing all tweets containing any of the following horoscope indicators: 'astrology','zodiac','astronomy','horoscope','aquarius','pisces','aries','taurus','leo','virgo','libra', and 'scorpio'. We preprocessed tweets by lowercasing and removing punctuation. We also only analyzed tweets for which Twitter had identified 'en' for the language English.
Question: What machine learning and NLP methods were used to sift tweets relevant to breast cancer experiences?
Solution:
ML logistic regression classifier combined with a Convolutional Neural Network (CNN) to identify self-reported diagnostic tweets.
NLP methods: tweet conversion to numeric word vector, removing tweets containing hyperlinks, removing "retweets", removing all tweets containing horoscope indicators, lowercasing and removing punctuation.