# Text Mining of Stocktwits Data for Predicting Stock Prices
## 4 Materials and Methods
### Data Analysis  
#### 4.2.1 Labelling Techniques  
For labelling the messages in the StockTwits dataset three labelling techniques were used--Binary Classification (positive and negative), Percentage Change--2 labels (positive and negative) and Percentage Change--3 labels (positive, negative, and neutral). For each of these three techniques in total, the labelling was done using two ways--using same day open price--close price and using same day close price--previous day close price. Figure 1 shows hierarchy of different labelling techniques used. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Symbol** & **Message** & **Datetime** & **User** & **Message\_Id** \\ \hline TSLA & \$TSLA trash & \begin{tabular}{c} 2020-07- \\ 16T23:08:47Z \\ \end{tabular} & \begin{tabular}{c} 3796654 \\ \end{tabular} & \begin{tabular}{c} 2.28E+08 \\ \end{tabular} \\ \hline TSLA & \begin{tabular}{c} \$TSLA https://www. \\ tesmanian.com/blogs/ \\ tesmanian-blog/tesla- \\ entering-greek-market \\ \end{tabular} & \begin{tabular}{c} 2020-07- \\ 16T23:06:01Z \\ \end{tabular} & \begin{tabular}{c} 335497 \\ \end{tabular} & \begin{tabular}{c} 2.28E+08 \\ \end{tabular} \\ \hline TSLA & \begin{tabular}{c} \$TSLA what\&\#39;s \\ happening here? \\ Considerin selling out \\ tomorrow!: Convince me \\ otherwise \\ \end{tabular} & \begin{tabular}{c} 2020-07- \\ 16T23:04:50Z \\ \end{tabular} & \begin{tabular}{c} 3572445 \\ \end{tabular} & \begin{tabular}{c} 2.28E+08 \\ \end{tabular} \\ \hline TSLA & \begin{tabular}{c} \$BB https: \\ //publishing.ninja/V4 \\ page/10630/414/270/1 \\ \end{tabular} & \begin{tabular}{c} 2020-07- \\ 16T23:03:55Z \\ \end{tabular} & \
\begin{tabular}{c} 1711636 \\ \end{tabular} &
\begin{tabular}{c} 228471610 \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table} 
Table 1: StockTwits Data file. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Date** & **Open** & **High** & **Low** & **Close** & **Adj Close** & **Volume** \\ \hline
10/07/2020 & 279.2 & 309.784 & 275.202 & 308.93 & 308.93 & 1.17E+08 \\ \hline
13/07/2020 & 331.8 & 358.998 & 294.222 & 299.412 & 299.412 & 1.95E+08 \\ \hline
14/07/2020 & 311.2 & 318 & 286.2 & 303.36 & 303.36 & 1.17E+08 \\ \hline
15/07/2020 & 308.6 & 310 & 291.4 & 309.202 & 309.202 & 81839000 \\ \hline
16/07/2020 & 295.432 & 306.342 & 293.2 & 300.128 & 300.128 & 71504000 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Yahoo Finance data. Since the Yahoo Finance data had missing data for weekends and public holidays, it was filled by comparing the previous date and next date or by using pandas dataframe functionality of date_range(). The values for the missing dates were filled using the logic cited in [3]. \[X_{mean}=\frac{X_{previous}+X_{next}}{2} \tag{1}\]  
where \(X_{mean}\) is the missing value, \(X_{previous}\) is the value of the field for the previously available date, and \(X_{next}\) is the value of the field for the next available date. The label for each date in Yahoo Finance data is calculated using the following equation -  
For binary classification technique,  
For same-day approach:  
\[\textit{Label}\left\{ \begin{array}{l}1\;(\textit{positive}),\;if\;\textit{ close price}>\textit{open price}\\ 0\;(\textit{negative}),\;if\;\textit{close price}<\textit{open price}\end{array} \right. \tag{2}\]  
For the previous day approach:  
\[\textit{Label}\left\{ \begin{array}{l}1\;(\textit{positive}),\;if\;\textit{ current day close price}>\textit{previous day close price}\\ 0\;(\textit{negative}),\;if\;\textit{current day close price}<\textit{previous day close price}\end{array} \right. \tag{3}\]  
For percentage change technique,  
For same-day approach:  
\[\textit{Percentage change}=\;\langle\frac{Close_{sametday}-Open_{sametday}}{ Open_{sametday}}\rangle\;*100 \tag{4}\]  
For the previous day approach:  
\[\textit{Percentage change}=\;\langle\frac{Close_{currentday}-Open_{previousday}}{ Open_{previousday}}\rangle\;*100 \tag{5}\]  
The label is assigned according to the following criteria:  
\[\textit{Label}\left\{