# Text Mining of Stocktwits Data for Predicting Stock Prices
## 5 Results  
We performed various experimentations on the StockTwits FAANG dataset. With more than 6.4 million data samples, the baseline models, traditional machine learning models, and proposed models were trained for all set of possibilities to extract results. All the experimentations were done for three labelling techniques. All the experimentations were done on FAANG data with 1 and 2 years of the subset. Result for different labelling methods trained on various baselines models are as follows: Table 3 shows all the results for Binary Classification with 1 and 2 years of data. Naive Bayes performs best for one-year data, whereas, for two-year data, BERT performs best with a macro average F1 score of 0.59. Boldface in each of the below table indicates the highest macro F1 score and accuracy among each of the labelling method for different year configuration. Table 4 shows all the results for Percentage change with two labels (Positive and Negative) with 1 and 2 years of data. BERT performs best for one-year data as well as for two-year data with a macro average F1 score of 0.59 and 0.60, respectively. Table 5 depicts the results for Percentage change with three labels (Positive, Neutral, and Negative) with 1 and 2 years of data. The results generated are the lowest compared to other labelling techniques. Naive Bayes performs better than any other model with a macro average F1 score of 0.41 on both years' dataset. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multicolumn{4}{c}{**Percentage Change with Positive and Negative Labels**} \\ \hline \hline \multirow{3}{*}{**Model**} & \multicolumn{2}{c}{**1 Year**} & \multicolumn{2}{c}{**2 Year**} \\ \cline{2-5}  & \multicolumn{1}{c}{**F1 Score-Macro**} & \multicolumn{1}{c}{**Accuracy**} & \multicolumn{1}{c}{**F1 Score-Macro**} & \multicolumn{1}{c}{**Accuracy**} \\ \hline
**Naive Bayes** & 0.58 & 62\% & 0.59 & 61\% \\ \hline
**Gradient Boosting** & 0.37 & 58\% & 0.37 & 55\% \\ \hline
**Logistic regression** & 0.49 & 60\% & 0.53 & 58\% \\ \hline
**Random Forest** & 0.55 & 56\% & 0.55 & 55\% \\ \hline
**BERT** & **0.59** & **64\%** & **0.60** & **62\%** \\ \hline
**FinBERT** & 0.54 & 54\% & 0.53 & 53\% \\ \hline
**FinALBERT** & 0.51 & 60\% & 0.35 & 55\% \\ \hline
**CNN with Word2 Vec** & 0.42 & 59\% & 0.46 & 56\% \\ \hline
**CNN with BERT embeddings** & 0.38 & 59\% & 0.35 & 55\% \\ \hline
**Word2 Vec with BiLSTM** & 0.52 & 60\% & 0.56 & 59\% \\
**BiLSTM and Attention** & 0.54 & 61\% & 0.54 & 59\% \\ \hline
**FinALBERT with BiLSTM and Attention** & 0.50 & 60\% & 0.55 & 59\% \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Results for Percentage change labels (Positive and Negative). \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multicolumn{4}{c}{**Binary Classification**} \\ \hline \hline \multirow{3}{*}{**Model**} & \multicolumn{2}{c}{**1 Year**} & \multicolumn{2}{c}{**2 Year**} \\ \cline{2-5}  & \multicolumn{1}{c}{**F1 Score-Macro**} & \multicolumn{1}{c}{**Accuracy**} & \multicolumn{1}{c}{**F1 Score-Macro**} & \multicolumn{1}{c}{**Accuracy**} \\ \hline
**Naive Bayes** & **0.58** & **59\%** & 0.58 & 59\% \\ \hline
**Gradient Boosting** & 0.37 & 55\% & 0.36 & 54\% \\ \hline
**Logistic regression** & 0.55 & 57\% & 0.53 & 57\% \\ \hline
**Random Forest** & 0.54 & 55\% & 0.54 & 54\% \\ \hline
**BERT** & 0.54 & 58\% & **0.59** & **61\%** \\ \hline
**FinBERT** & 0.54 & 54\% & 0.54 & 54\% \\ \hline
**FinALBERT** & 0.35 & 55\% & 0.32 & 46\% \\ \hline
**CNN with Word2 Vec** & 0.40 & 47\% & 0.41 & 54\% \\ \hline
**CNN with BERT embeddings** & 0.35 & 55\% & 0.35 & 54\% \\ \hline
**Word2 Vec with BiLSTM and Attention** & 0.56 & 57\% & 0.52 & 57\% \\ \hline
**Fasttext with BiLSTM and Attention** & 0.54 & 58\% & 0.54 & 57\% \\ \hline
**FinALBERT with BiLSTM and Attention** & 0.53 & 57\% & 0.55 & 56\% \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Results for Binary classified labels. Proposed model FinALBERT performs moderately with all three labelling techniques with the macro F1 score ranging between 0.29-0.51.