# Mining Textual and Imagery Instagram Data during the COVID-19 Pandemic
## 3 Results
Finally, the most important step of text preparation that was carried out was vectorization, a process that converts words (or tokens) to numerical feature representations. Popular vectorization approaches can utilize different models; Bag-of-Words (BoW) term frequency model, L1-normalized term frequency model, and L2-normalized TF-IDF (Term Frequency--Inverse Document Frequency) model. More recently, models that employ word embeddings are considered; Word2 Vec/Doc2 Vec (Google), GloVe (Stanford University) and fastText (Facebook). In our case, the corpus of 675 posts was TF-IDF vectorized using scikit-learn feature extractor. An excerpt from the vectorized, sparse dataset is shown in Figure 10. The vocabulary (attributes) consisted of 3024 words: \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **\#pfizer** & **\#astrazeneca** & **\#jnj** & **Totals** \\ \hline ACK & 155 & 120 & 7 & 282 \\ \hline ADV & 81 & 67 & 9 & 157 \\ \hline SEK & 6 & 4 & 1 & 11 \\ \hline EXP & 97 & 100 & 28 & 225 \\ \hline Totals & 339 & 291 & 45 & 675 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Annotation results. The final vectorized and annotated dataset was split, using 10-fold cross validation, into two subsets for training and testing respectively. These subsets were used to train nine weta classifiers with their default configuration: one linear one, Logistic Regression (LOG), seven non-linear ones, Naive Bayes (NB), C4.5 Decision Tree (DT), k-Nearest Neighbour (KNN), Support Vector Machine (SVM), Multi-Layer Perceptron (MLP), Bayes Net (BN), Logistic Model Tree (LMT), and one ensemble classifier, Random Forest (RF). The objective here was to evaluate the classifiers' performance and choose the best suited algorithm for building a predictive model. The baseline performance was set by using the Zero-Rule (ZR) classifier which simply predicts the dataset class mode. In our case this was 'ACK' with 282 instances over a total of 675, thus an accuracy of 41.8%. Results are reported on Table 5. Reducing the vocabulary by keeping only the, e.g., 1000 top words, did not improve the accuracy. Similarly, considering bigrams as well as unigrams led to worse results. However, it would be expected to improve with more data, i.e., more posts. Further individual parameter tuning for each algorithm might have also yielded better results. Different metrics are also reported (weighted averages) such as: Mathews Correlation Coefficient (MCC), Receiver Operating Characteristic (ROC) area, F-measure (harmonic precision-recall mean), and Cohen's Kappa metric, with the last two suggested as more appropriate for imbalanced classification tasks. Confusion matrices (Figure 11) are also useful for showing the numbers of true and false predictions for each class. \begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline  & **LOG** & **NB** & **DT** & **KNN** & **SVM** & **MLP** & **RF** & **BN** & **LMT** \\ \hline Accuracy (\%) & 60.4 & 52.9 & 62.1 & 44.0 & 68.6 & 51.7 & 68.0 & 61.5 & 61.5 \\ \hline MCC & 0.409 & 0.394 & 0.422 & 0.269 & 0.523 & 0.349 & 0.523 & 0.412 & 0.414 \\ \hline ROC area & 0.777 & 0.763 & 0.704 & 0.618 & 0.790 & 0.702 & 0.854 & 0.766 & 0.772 \\ \hline F- measure & 0.609 & 0.572 & 0.614 & 0.373 & 0.677 & 0.549 & 0.652 & 0.576 & 0.609 \\ \hline Kappa & 0.407 & 0.344 & 0.419 & 0.163 & 0.516 & 0.310 & 0.498 & 0.394 & 0.412 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Classifiers’ evaluation metrics. Figure 10: An excerpt from the vectorized dataset. From these initial results we can infer that the SVM and RF classifiers seem more promising and require further investigation. In fact, a simple voting (averaging probability rule) scheme derived from these two classifiers gave a joint accuracy of 69.3%, with improvements in all other metrics as well. Other popular ensemble techniques are bootstrap aggregation (bagging, as in RF), boosting, and stacked generalization. Boosting the SVM or the RF classifier alone, did not improve the accuracy, with either 10 or 100 models.