# Polarity and Subjectivity Detection with Multitask Learning and BERT Embedding
## 2 Related Work  
This section introduces related work in multitask learning, sentiment analysis, and subjectivity detection. Sentiment analysis, in particular, has seen growth in multitasking usage [7; 8]. Ref. [9] uses MTL for adversarial text classification, mitigating the shared and private latent feature spaces from interfering with each other. Ref. [10] proposes an MTL approach that allows joint training of the primary and auxiliary tasks, improving the performance of rumour verification. Subjectivity detection is an NLP task that consists of differentiating objective data (\"factual\" or \"neutral\" content) from subjective data (opinions). [PERSON] et al. (2018) [11] propose a multitask deep neural framework for document-level sentiment analysis that learns to predict the overall sentiment expressed in the given input document as the primary task and simultaneously learning to predict human gaze behavior and auxiliary linguistic tasks such as part-of-speech and syntactic properties of words in the document as the secondary task. Extracting subjective text segments poses a tremendous challenge that only a few works have attempted. Ref. [6] applied a graph-min-cut based technique to separate the subjective portion of the text from the irrelevant objective portions. Ref. [12] introduced a novel architecture for filtering out neutral content in a time- and resource-effective manner. Glove [13] is a word vector technique that leverages both global and local statistics of a corpus in order to come up with a principled loss function that uses both these. We finetuned Glove with the dataset and created an embedding of **L X 300** dimension where L is the maximum length of the input text. The authors in [12] proposed a Bayesian network-based extreme learning machine (BNELM) model. BNELM augments the standard recurrent neural network structure to generate a predictor that can take advantage of the beneficial properties of extreme learning machines and Bayesian networks. Emotion recognition is a task very close to sentiment classification. [PERSON] et al. (2019) [14] used BERT to detect emotions and applied it to a dialogue system. The authors in [15] propose an evaluation of MTL on semantic sequence prediction on data-dependent conditions. They derive attribute of datasets that make them favorable for MTL by comparing performance with information-theoretical metrics of the label frequency distribution. Generic MTL [3; 16] has a rich history in machine learning. It has widespread applications in other fields, such as genomics [17], NLP [18; 19; 20; 21], and computer vision [22; 23; 24]. The MTL paradigm provides an effective platform for achieving generalization. The inspiration is that if the tasks are related, the model can learn jointly, taking into account the shared information, which is expected to increase its generalization ability. Various tasks can exploit the inter-relatedness to improve individual performance through a shared representation. Overall, it provides three principal benefits over the single-task learning paradigm:  
1. It helps in achieving generalization for multiple tasks;
2. Each task improves its performance in association with the other participating tasks;
3. Offers reduced complexity because a single system can handle multiple problems or tasks simultaneously. The performance improvement from MTL [3] is due to the extra information in the training signals of related tasks:* **Implicit data augmentation:** Learning only one task carries the risk of overfitting that task while learning jointly enables the model to obtain a better representation by averaging noise patterns. MTL effectively increases the sample size we are using to train our model by sharing the learnt features. * **Attention focusing:** If the data are insufficient and high-dimensional, it can be challenging for a model to distinguish between relevant and irrelevant features. * **Eavesdropping:** We can allow the model to eavesdrop through MTL; i.e., tasks challenging to learn for one model are learnt by the other model.