# Multitask Learning-Based Affective Prediction for Videos of Films and TV Scenes
## 3 Method
### Loss Function
#### 3.4.2 Joint Loss Function  
We adopt a multitask modeling approach, with joint primary and secondary tasks for fine-grained affective prediction. In the main task, the loss function is based on the extraction of affective dimensions from the FA, and tasks of the same category are learned at the same speed. Hence, the tasks of each category have the same degree of importance and receive the same attention, balancing the loss and avoiding seesaw phenomena [50; 51; 52]. MAE is used for the affective regression task and categorical cross-entropy loss for the classification task. The joint loss function to be minimized is  
\[L=W_{pos}\sum_{1}^{n_{p}}loss_{pos}+W_{sce}\sum_{1}^{n_{s}}loss_{sce}+W_{neg} \sum_{1}^{n_{a}}loss_{neg}+\partial loss_{pp}, \tag{1}\]  
where \(Loss_{pos}\), \(Loss_{sce}\), \(Loss_{neg}\), and \(Loss_{pp}\) are the respective losses incurred in the positive emotions, scene emotions, negative emotions, and classification tasks.  
When training a multitask network, it is difficult to balance the training of these tasks, and the change rate of loss can reflect the difficulty of learning a task, so the average value of the loss change rate of different emotion tasks is considered. Considering the average value of the loss change rate of the same dimension of emotion after a certain number of  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Component** & **Factor 1** & **Factor 2** & **Factor 3** \\ \hline Warm & **0.913** & \(-\)0.298 & 0.100 \\ Hopeful & **0.798** & \(-\)0.352 & 0.275 \\ Happy & **0.872** & \(-\)0.335 & 0.005 \\ Romantic & **0.65** & \(-\)0.241 & 0.48 \\ Relaxed & **0.839** & \(-\)0.449 & 0.126 \\ Fresh & **0.711** & \(-\)0.357 & 0.359 \\ Cozy & **0.908** & \(-\)0.299 & 0.015 \\ Sunny & **0.872** & \(-\)0.326 & 0.151 \\ Magnificent & \(-\)0.054 & \(-\)0.239 & **0.849** \\ Dreamy & 0.432 & \(-\)0.235 & **0.791** \\ Lonely & \(-\)0.397 & **0.795** & \(-\)0.085 \\ Sentimental & \(-\)0.269 & **0.897** & \(-\)0.235 \\ Disappointed & \(-\)0.259 & **0.904** & \(-\)0.232 \\ Depressed & \(-\)0.349 & **0.893** & \(-\)0.209 \\ Oppressive & \(-\)0.511 & **0.739** & \(-\)0.21 \\ Anxious & \(-\)0.427 & **0.827** & \(-\)0.247 \\ \hline Indicator & & & \\ Eigenvalue & 10.863 & 1.826 & 1.208 \\ Variance (\%) & 67.897 & 11.414 & 7.551 \\ Cumulative (\%) & 67.863 & 79.311 & 86.862 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Results of factor analysis of 16 affective adjectives.  
rounds of training, the weight of corresponding tasks can be changed, which connects different categories of emotions and reduces the dominance of a certain category of emotion task, which could result in an inability to converge. The ratio of the mean values of loss change rates for different dimensions after a certain number of training rounds is  
\[Loss_{pos}:Loss_{sec}:Loss_{neg}=K_{pos}:K_{sec}:K_{neg}, \tag{2}\]  
The respective weights of positive emotion, scene emotion, and negative emotion for a proposed scheme \(a\) are defined as follows:  
\[W_{pos}=aK_{pos}, \tag{3}\]  
\[W_{sec}=aK_{sec}, \tag{4}\]  
\[W_{neg}=aK_{neg}, \tag{5}\]  
where the multiplicative factors \(K_{pos}\), \(K_{sec}\), and \(K_{neg}\) regulate the magnitudes of the weights.