# Transferring Sentiment Cross-Lingually within and across Same-Family Languages
## 6 Methodology
### Model Details  
Transformer-based neural networks are the current gold standard for classification tasks [19; 113; 114]. Taking cues from previous work [115; 116], each of the fine-grained labels (1 (worst) to 5 (best)) and their corresponding coarse-grained labels (positive, neutral, and negative) were treated as two distinct tasks. The model was trained to perform both tasks simultaneously. Not all datasets in our study employ the same annotation scheme. This prompted us to conduct an annotation projection from fine-grained labels (such as 5-star or 11-star ratings) to coarse-grained labels (three-class, i.e., positive, negative, and neutral). Our model is based on the multi-task transfer learning setting [117] for training a sentiment classifier with multiple datasets. The model is a hierarchical network that performs end-to-end training and stacks two classifiers on top of one another. The encoder is shared by all classifier layers. We framed cross-lingual sentiment classification as a problem of multitask learning. We aimed to jointly learn a set of neural network parameters for classifiers in the source and target languages. This was accomplished by jointly optimizing a loss function that took coarse and fine-grained labels and resources from both languages into account. A transformer-based model fits a parameterized model to maximize the conditional probability of a target label \(y\) given a source sentence \(x\), i.e., \(z=\operatorname*{argmax}_{x}p(y|x)\) given a training instance \(x\), \(y\). By combining training data from various sources and languages, learning is extended to multiple languages. The objective function we optimized is the sum of the conditional probabilities of different datasetsfrom different languages based on the representations obtained using a shared pre-trained language model.  
\[L_{multi}(\theta) =\underbrace{\sum_{D_{i}}\log P_{\theta}(y^{5}|x^{L1},\theta,\omega)} _{1}+\] \[\underbrace{\sum_{D_{i}}\log P_{\theta}(y^{3}|x^{L1},\theta,\phi)} _{2}+\underbrace{\sum_{D_{i}}\log P_{\theta}(y^{5}|x^{L2},\theta,\omega)}_{3}+ \tag{1}\] \[\underbrace{\sum_{D_{i}}\log P_{\theta}(y^{3}|x^{L2},\theta,\phi)} _{4}\]  
In objective Equation (1), four distinct loss terms share a common parameter, \(\theta\). In addition, language-independent classifiers share the parameters \(\omega\) and \(\phi\), which are label-specific. The first and third terms optimize the source language loss for coarse and fine-grained labels, respectively. Similarly, the second and fourth terms enhance performance in the target language. At two points, parameterization is performed. First, we modified a PLM for sentiment classification in source and target languages jointly. Second, there are two distinct parameters for labels. The global loss function is capable of both cross-lingual and hierarchical classification.  
Consider the training examples \(x\) and \(y^{5}\), where \(y^{5}\) is a five-class label (1-5). The labeled five-class dataset is also realizable as a three-class dataset. One and two are mapped to the negative category, three to the neutral category, and four to the positive category, in that order. For a particular training instance, \(\langle x,\langle y^{3},y^{5}\rangle\rangle\), where \(y^{3}\) is a three-class label that can be positive, negative, or neutral based on the five-class label, i.e., 1, 2, 3, 4, 5, and the objective is to jointly maximize the conditional probability such that instances that belong to the negative class also receive a lower rating and vice versa. This is performed to optimize the model uniformly for various languages and labeled datasets. Two feed-forward neural networks with a softmax output layer define the architecture. Consequently, we have two classifiers trained on the same text but with distinct labels. The first label is coarse, while the second is fine. This is known as \"pseudo-multitask learning\" because two tasks are simultaneously trained on a shared representation from a single training instance.