# Buzz Tweet Classification Based on Text and Image Features of Tweets Using Multi-Task Learning
## 6 Discussion  
In the prediction of the buzz class, based on the correct prediction rate with only one of the features (text or image), it can be inferred that the text features are important. The model with the highest correct prediction rate for image features alone was VGG16 at 0.75. With both text and image features used as the input, the correct prediction rate of VGG16 for the image features was 0.84, which was higher than that of the other features.  
To see if there were any similarities in the misclassification trends, the agreement rate of misclassified tweets was also examined when using either image or text features as the only input, as well as combining text and image features. As a result, the tweets with low predicted correctness using only image features exhibited a large number of misclassified tweets, but the agreement rate of misclassified tweets was approximately 0.5. This indicates that approximately half of the misclassified tweets were difficult to classify using only image features.  
Although the prediction accuracy of VGG16 was higher than that of other image features, even when only image features were used, the same tweets were more likely to be misclassified when both text and image features were used, suggesting that VGG16 is a feature that is more effective when combined with BERT.  
Based on the results of Figures 5 and 6, it is observed that the rate at which non-buzz tweets are recognized as buzz tweets is slightly higher for the misclassified tweets. In addition, although tweets with likes close to the threshold are expected to be misclassified, this is not the case. In the case of misclassified non-buzz tweets, the misclassification is attributed to some popular words often found in buzz tweets.  
With DenseNet, there were several cases in which the cosine similarity of the image features was high, such as screenshots of the same game or text in the image. With DenseNet, there were also a few images which had a high cosine similarity in the use of a white background. With VGG16, the similarity in the same category was high, and cases of no images with high similarity were attributed to the rarity of the image in the tweet. The images in Figure 5 do not seem to be similar to each other, and based on the cosine similarity results, VGG16 can discriminate the similarity of images more accurately than DenseNet and ResNet50. The correct response rate for a single image feature model also suggests that VGG16 contributes to the prediction of buzz.  
The similarity between the tweets with higher BERT features may have been due to the length of the sentences and the fact that both tweets had a word in common (\"made\" in this example). In this example, the non-buzz tweet has a \"coffee image\" attached to the text \"made coffee\". In contrast, the buzz tweet has a text containing the word \"star\" with an image of several people posing for a picture (a group photo) attached to it. This indicates that there is no direct relationship between text and images in buzz tweets, whereas a direct relationship is observed in non-buzz tweets. Thus, the difference in the relevance of text and image is considered to be the boundary between buzz and non-buzz tweets. For this reason, we believe that a model that can extract the relationship between text and images would be effective, rather than using only one or a combination of both.