Venue: NAACL 2021

# Framing Unpacked: A Semi-Supervised Interpretable Multi-View Model

of Media Frames

Shima Khanehazar\({}^{\dagger}\) Trevor Cohn\({}^{\dagger}\) Gosia Mikolajczak\({}^{\star}\) Andrew Turpin\({}^{\dagger}\) Lea Frermann\({}^{\dagger}\)

\({}^{\dagger}\) School of Computing and Information Systems

\({}^{\star}\)School of Social and Political Sciences

The University of Melbourne

skhanehzar@student.unimelb.edu.au {tcohn,mmikolajcza,aturpin,lfrermann}@unimelb.edu.au

###### Abstract

Understanding how news media frame political issues is important due to its impact on public attitudes, yet hard to automate. Computational approaches have largely focused on classifying the frame of a full news article while framing signals are often subtle and local. Furthermore, automatic news analysis is a sensitive domain, and existing classifiers lack transparency in their predictions. This paper addresses both issues with a novel semi-supervised model, which jointly learns to embed local information about the events and related actors in a news article through an auto-encoding framework, and to leverage this signal for document-level frame classification. Our experiments show that: our model outperforms previous models of frame prediction; we can further improve performance with unlabeled training data leveraging the semi-supervised nature of our model; and the learnt event and actor embeddings intuitively corroborate the document-level predictions, providing a nuanced and interpretable article frame representation.

+
Footnote †: 2019 IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (2018); 2018 IEEE/RSJ Web Conference (WACM); 2017

frame prediction from its own perspective. We incorporate the latent multi-view representation into a transformer-based document-level frame classification model to form a semi-supervised model, in which the latent representations are jointly learnt with the classifier.

We demonstrate empirically that our semi-supervised model outperforms current state-of-the-art models in frame prediction. More importantly, through detailed qualitative analysis, we show how our latent features mapped to events and related actors allow for a nuanced analysis and add interpretability to the model predictions2. In summary, our contributions are:

Footnote 2: Source code of our model is available at [https://github.com/shinyemimalef/FRISS](https://github.com/shinyemimalef/FRISS)

* Based on the concepts of story- and emphasis framing, we develop a novel semi-supervised framework which incorporates local information about core events and actors in news articles into a frame classification model.
* We empirically show that our model, which incorporates the latent multi-view semantic role representations, outperforms existing frame classification models, with only labeled articles. By harnessing large sets of unlabeled informain data, our model can further improve its performance and achieves new state-of-the-art performance on the frame prediction task.
* Through qualitative analysis, we demonstrate that the latent, multi-view representations aid interpretability of the predicted frames.

## 2 Background and Related Work

A widely accepted definition of frames describes them as a selection of aspects of perceived reality, which are made salient in a communicating context to promote a particular problem definition, causal interpretation, moral evaluation and the treatment recommendation for the described issue Entman (1993). While detecting media frames has attracted much attention and spawned a variety of methods, it poses several challenges for automatic prediction due to its vagueness and complexity.

Two common approaches in the study of frames focus either on the detailed issue-specific elements of a frame or, somewhat less nuanced, on generic framing themes prevalent across issues. Within the first approach, Matthes and Kohring (2008) developed a manual coding scheme, relying on Entman's definition Entman (1993). While the scheme assumes that each frame is composed of common elements, categories within those elements are often specific to the particular issue being discussed (e.g., "same sex marriage" or "gun control"), making comparison across different issues, and detecting them automatically difficult. Similarly, earlier studies focusing specifically on unsupervised models to extract frames, usually employed topic modeling Boydstun et al. (2013); Nguyen (2015); Tsur et al. (2015) to find the issue-specific frames, limiting across-issue comparisons.

Studies employing generic frames address this shortcoming by proposing common categories applicable to different issues. For example, Boydstun et al. (2013) proposed a list of 15 broad frame categories commonly used when discussing different policy issues, and in different communication contexts. The Media Frames Corpus (MFC; Card et al. (2015)) includes about 12,000 news articles from 13 U.S. newspapers covering five different policy issues, annotated with the dominant frame from Boydstun et al. (2013). Table 5 in the Appendix lists all 15 frame types present in the MFC. The MFC has been previously used for training and testing frame classification models. Card et al. (2016) provide an unsupervised model that clusters articles with similar collections of "personas" (i.e., characterisations of entities) and demonstrate that these personas can help predict the coarse-grained frames annotated in the MFC. While conceptually related to our approach, their work adopts the Bayesian modelling paradigm, and does not leverage the power of deep learning. Ji and Smith (2017) proposed a supervised neural approach incorporating discourse structure. The current best result for predicting the dominant frame of each article in the MFC comes from Khanehzar et al. (2019), who investigated the effectiveness of a variety of pre-trained language models (XLNet, Bert and Roberta).

Recent methods have been expanded to multilingual frame detection. Field et al. (2018) used the MFC to investigate framing in Russian news. They introduced embedding-based methods for projecting frames of one language into another (i.e., English to Russian). Akyurek et al. (2020) studied multilingual transfer learning to detect multiple frames in target languages with few or no annotations. Recently, Huguet Cabot et al. (2020) investigated joint models incorporating metaphor, 

emotion and political rhetoric within multi-task learning to predict framing of policy issues.

Our modelling approach is inspired by recent advances in learning interpretable latent representations of the participants and relationships in fiction stories. Iyyer et al. (2016) present Relationship Modelling Networks (RMNs), which induce latent descriptors of types of _relationships_ between characters in fiction stories, in an unsupervised way. RMNs combine dictionary learning with deep autoencoders, and are trained to effectively encode text passages as linear combinations over latent descriptors, each of which corresponds to a distinct relationship (not unlike topics in a topic model). Frermann and Szarvas (2017) extend the idea to a multi-view setup, jointly learning multiple dictionaries, which capture _properties_ of individual characters in addition to relationships. We adopt this methodology for modeling news articles through three latent views: capturing their events (predicates), and participants (Argo, Arg1). We combine the unsupervised autoencoder with a frame classifier into an interpretable, semi-supervised framework for article-level frame prediction.

## 3 Semi-supervised Interpretable Frame Classification

In this section, we present our Frame classifier, which is Interpretable and Semi-supervised (Friss). The full model is visualized in Figure 1. Given a corpus of news articles, some of which have a label indicating their primary frame \(y\) (Figure 1(a)), Friss learns to predict \(\hat{y}\) for each document by combining a supervised classification module (Figure 1(c)) and an unsupervised3 autocncoding module (Figure 1(b)), which are jointly trained. The unsupervised module (i) can be trained with additional unlabeled training data, which improves performance (Section 5.2); and (ii) learns interpretable latent representations which improve the interpretability of the model (Section 5.3).

Footnote 3: The autoencoder is unsupervised wrt. frame-level information, but it relies on an off-the-shelf semantic role labeler.

Intuitively, Friss predicts frames based on aggregated sentence representation (supervised module; Section 3.2) as well as aggregated fine-grained latent representations capturing actors and events in the article (unsupervised module; 3.1). The un

Figure 1: Our Friss model. **(a)** Given an input document with a frame label \(y\) , we perform semantic role labeling (SRL) to obtain the predicate (blue), Arg0 (purple) and Arg1 (green) for input sentences \(s_{1}\ldots s_{n}\). **(b)** The unsupervised module takes as input semantic role embeddings (\(v_{a_{0}},v_{p},v_{a_{1}}\)) and a sentence embedding (\(v_{s_{i}}\)), and learns latent, role-specific embedding matrices (**F**) in an auto-encoding framework. The latent representations are incorporated into the overall frame classification module. **(c)** We predict document-level frames based on transformer-based document embeddings (\(\hat{y}_{s}\)) and the view-specific latent representations (\(\hat{y}_{u}\)), using cross-entropy loss with the true frame label \(y\).

 

supervised module combines an auto-encoding objective with a multi-view dictionary learning framework (Iyyer et al., 2016; Frermann and Szarvas, 2017). We treat predicates, their Arg0 and Arg1 as three separate views, and learn to map each view to an individual latent space representative of their relation to the overall framing objective. Below, we will sometimes refer to views collectively as \(z\in\{p,\mathrm{a}_{0},\mathrm{a}_{1}\}\). We finally aggregate the view-level representations and sentence representations to predict a document-level frame. The following sections describe Friss in technical detail.

### Unsupervised Module

#### 3.1.1 Input

Each input document is sentence-segmented and automatically annotated by an off-the shelf transformer-based semantic role labeling model (Shi and Lin, 2019; Pradhan et al., 2013) to indicate spans over the three semantic roles: predicates, Arg0s and Arg1s.

We compute a contextualized vector representation for each semantic role span (\(s_{\text{p}}\), \(s_{\mathrm{a}_{0}}\), \(s_{\mathrm{a}_{1}}\)). We describe the process for obtaining predicate input representations \(\mathbf{v}_{p}\) here for illustration. Contextualized representations for views \(\mathrm{a}_{0}\) (\(\mathbf{v}_{\mathrm{a}_{0}}\)) and \(\mathrm{a}_{1}\) (\(\mathbf{v}_{\mathrm{a}_{1}}\)) are obtained analogously. First, we pass each sentence through a sentence encoder, and obtain the predicate embedding by averaging all contextualized token representations \(\mathbf{v}_{w}\) (of dimension \(D_{w}\)) in its span \(s_{p}\) of length \(|s_{p}|\):

\[\mathbf{v}_{\text{p}}=\frac{1}{|s_{p}|}\sum_{w\in s_{p}}\mathbf{v}_{w}. \tag{1}\]

We concatenate \(\mathbf{v}_{\text{p}}\) with an overall sentence representation \(\mathbf{v}_{s}\), which is computed by averaging all contextualized token embeddings of the sentence \(s\) of length \(|s|\),4

Footnote 4: We also experimented with representing the sentence as the \([CLS]\) token embedding, but found it to perform worse empirically.

\[\mathbf{v}_{s} =\frac{1}{|s|}\sum_{w\in s}\mathbf{v}_{w} \tag{2}\] \[\tilde{\mathbf{v}}_{\text{p}} =[\mathbf{v}_{\text{p}};\mathbf{v}_{s}], \tag{3}\]

where [;] denotes vector concatenation. If a sentence has more than one predicate, a separate representation is computed for each of them.

#### 3.1.2 Multi-view Frame Representations

We combine ideas from auto-encoding (AE) and dictionary learning, as previously used to capture the content of fictitious stories (Iyyer et al., 2016), and its multi-view extension (Frermann and Szarvas, 2017). We posit a latent space as three view-specific dictionaries (Figure 1 (b)) capturing events (predicates; \(\mathbf{F}_{p}\)), their first (Arg0; \(\mathbf{F}_{\mathrm{a}_{0}}\)) and second (Arg1; \(\mathbf{F}_{\mathrm{a}_{1}}\)) arguments, respectively. Given a view-specific input as described above, the autoencoder maps it to a low-dimensional distribution over "dictionary terms" (henceforth descriptors), which are learnt during training. The descriptors are vector-valued latent variables that live in word embedding space, and are hence interpretable through their nearest neighbors ( Table 3 shows examples of descriptors inferred by our model). By jointly learning the descriptors with the supervised classification objective, each descriptor will capture coherent information corresponding to a frame label in our supervised data set. We hence set the number of descriptors for each dictionary to \(K=15\), the number of frames in our data set. For each view \(z\in\{p,\mathrm{a}_{0},\mathrm{a}_{1}\}\), we define a dictionary \(\mathbf{F}_{z}\) of dimensions \(K\times D_{w}\).

More technically, our model follows two steps. First, we _encode_ the input \(\tilde{\mathbf{v}}_{z}\) of a known view \(z\) by passing it through a feed forward layer \(\mathbf{W}_{h}\) of dimensions \(2D_{w}\times D_{h}\), shared across all the views, followed by a ReLU non-lineararity, and then another feed forward layer \(\mathbf{W}_{z}\) of dimensions \(D_{h}\times K\), specific to each view \(z\). This results in a \(K\)-dimensional vector over the view-specific descriptors,

\[\mathbf{l}_{z}=\mathbf{W}_{z}\operatorname{ReLU}(\mathbf{W}_{h}\tilde{\mathbf{v}}_{z}), \tag{4}\]

Second, we reconstruct the original view embedding \(\mathbf{v}_{z}\) as a linear combination of descriptors. While previous work used \(\mathbf{l}_{z}\) directly as weight vector, we hypothesize that on our fine-grained semantic role level, only one or a few descriptors will be relevant to any specific span. We enforce this intuition using Gumbel-Softmax differentiable sampling with temperature annealing (Jang et al., 2017). This allows us to gradually constrain the number of relevant descriptors used for reconstruction. We first normalize \(\mathbf{l}_{z}\),

\[\mathbf{d}_{z}=\operatorname{Softmax}(\mathbf{l}_{z}), \tag{5}\]

and then draw \(\mathbf{g}\) from the Gumbel distribution, and add it to our normalized logits \(\mathbf{d}_{\mathbf{z}}\) scaled by temperature \(\tau\) , which is gradually annealed over the 

training phase:

\[\mathbf{g} \sim\mathrm{Gumbel}(0,1) \tag{6}\] \[\mathbf{g}_{z} =\frac{\exp(\log(\mathbf{d}_{z})+\tfrac{\mathbf{q}}{\tau})}{\sum_{f}\exp( \log(\mathbf{d}_{z})+\tfrac{\mathbf{q}}{\tau})}.\]

We finally reconstruct the view-specific span embedding as

\[\hat{\mathbf{v}}_{z}=\mathbf{F}^{T}_{z}\mathbf{g}_{z}. \tag{7}\]

#### 3.1.3 Unsupervised Objective

Contrastive LossWe use the contrastive max-margin objective function following previous works in dictionary learning Iyyer et al. (2016); Frermann and Szarvas (2017); Han et al. (2019). We randomly sample a set of negative samples (\(N_{-}\)) with the same view as the current input from the mini-batch. The unregularized objective \(J^{u}_{z}\) (Eq. 8) is a hinge loss that minimizes the L2 norm5 between the reconstructed embedding \(\hat{\mathbf{v}}_{z}\) and the true input's view-specific embedding \(\mathbf{v}_{z}\), while simultaneously maximizing the L2 norm between \(\hat{\mathbf{v}}_{z}\) and negative samples \(\mathbf{v}^{n}_{z}\):

Footnote 5: We empirically found that L2 norm outperforms the dot product, and cosine similarity.

\[J^{u}_{z}(\theta)=\frac{1}{|N_{-}|}\sum_{\mathbf{v}^{n}_{z}\in N_{-}} \max(0,1+l_{2}(\hat{\mathbf{v}}_{z},\mathbf{v}_{z})\\ -l_{2}(\hat{\mathbf{v}}_{z},\mathbf{v}^{n}_{z})), \tag{8}\]

where \(\theta\) represents the model parameters, \(|N_{-}|\) is the number of negative samples, and the margin value is set to 1.

Focal Triplet LossPreliminary studies (Section 5) suggested that some descriptors (aka frames) are more similar to each other than others. We incorporate this intuition through a novel mechanism to move the descriptors that are least involved in the reconstruction proportionally further away from the most involved descriptor.

Concretely, we select \(t\) descriptors in \(\mathbf{F}_{z}\) with smallest weights in \(\mathbf{g}_{z}\) as additional negative samples. We denote the indices of the selected \(t\) smallest components in \(\mathbf{g}_{z}\)

\[I=[i_{1},i_{2},\dots,i_{t}]. \tag{9}\]

We use \(\mathbf{F}^{t}_{z}\) to denote the matrix (\(t\times D_{w}\)) with only those \(t\) descriptors. We re-normalize the weights of the selected \(t\) descriptors, and denote the renormalized weights vector as \(\mathbf{g}^{t}_{z}=[g^{i_{1}}_{z},g^{i_{2}}_{z},\dots,g^{i_{t}}_{z}]\). For each element in \(\mathbf{g}^{t}_{z}\), we compute an individual margin based on its magnitude. Intuitively, the smaller the weight is, the larger its required margin from a given total margin budget \(|M|\),

\[m^{it}_{z}=|M|*(1-g^{i_{z}}_{z})^{2}. \tag{10}\]

We compute the standard margin-based hinge loss over the additional negative samples with sample-specific margins:

\[J^{t}_{z}(\theta)=\frac{1}{|T|}\sum_{i_{t}\in I}\max(0, m^{it}_{z}+l_{2}(\hat{\mathbf{v}}_{z},\mathbf{v}_{z})\\ -l_{2}(\hat{\mathbf{v}}_{z},\mathbf{v}^{i_{t}}_{z})). \tag{11}\]

We sum the focal triplet objective \(J^{t}_{z}\) with \(J^{u}_{z}\), and then sum over all specific spans \(s\in S_{z}\), while adding an additional orthogonality encouraging regularization term.

\[J_{z}(\theta)=\sum_{s\in S_{z}}(J^{u}_{z}+J^{t}_{z})+\lambda||\mathbf{F_{z}F^{ T}_{z}-I}||^{2}_{\mathbf{F}}, \tag{12}\]

where \(\lambda\) is a hyper-parameter that can be tuned. We finally aggregate the loss from all the views:

\[J(\theta)=\sum_{z\in\{\text{p},\text{a}o,\text{a}_{1}\}}J_{z}(\theta). \tag{13}\]

### Supervised Document-level Frame Classification

We incorporate the semantic role level predictions as described above into a document-level frame classifier consisting of two parts, which are jointly learnt with the unsupervised model described above: (i) a classifier based on aggregated span-level representations computed as described in Sec 3.1 (Fig. 1 (c; left); Sec. 3.2.1) and (ii) a classifier based on an aggregated sentence representations (Fig. 1 (c; right); Sec. 3.2.2).

#### 3.2.1 Span-based Classifier

The unsupervised module makes predictions on the semantic role span level, however, our goal is to predict document-level frame labels. We aggregate span-level representations \(\mathbf{d}_{z}\) (Eq. 5) by averaging across spans and then views:6

Footnote 6: We empirically found these representations to outperform the sparser \(\mathbf{g}_{z}\).

\[\mathbf{w}_{u} =\frac{1}{Z}\sum_{z\in\{\text{p},\text{a}o,\text{a}_{1}\}}\frac{1} {|S_{z}|}\sum_{s\in S_{z}}\mathbf{d}^{s}_{z}\] (14) \[\hat{y}_{u} =\operatorname{Softmax}(\mathbf{w}_{u}),\] 

where \(Z\) is the number of the views, and \(S_{z}\) are the set of view-specific spans in the current document. We finally pass the logits through a softmax layer to predict a distribution over frames.

#### 3.2.2 Sentence-based Classifier

We separately predict a document-level frame based on the aggregate sentence level representations computed in Eq. (2). We first pass each sentence embedding through a feed forward layer \(W_{r}\) of dimensions \(D_{w}\times D_{w}\), followed by a ReLU non-linearity, and another feed forward layer \(W_{t}\) to map the resulting representation to \(K\) dimensions. Then average across sentences of the current document \(S_{d}\) and pass the result through a softmax layer,

\[\begin{split}&\mathbf{w}_{s}=\mathrm{ReLU}(\mathbf{W}_{r}\mathbf{v}_{s})\\ &\hat{y}_{s}=\mathrm{Softmax}\left(\frac{1}{|S_{d}|}\sum_{s\in S_ {d}}\mathbf{W}_{t}\mathbf{w}_{s}\right).\end{split} \tag{15}\]

### Full Loss

We jointly train the supervised and unsupervised model components. The supervised loss \(X(\theta)\) consists of two parts, one for the sentence-based classification and one for the aggregated span-based classification:

\[X(\theta)=X(\hat{y}_{u},y)+X(\hat{y}_{s},y). \tag{16}\]

The full loss balances the supervised and unsupervised components with a hyper-parameter \(\alpha\):

\[L(\theta)=\alpha\times X(\theta)+(1-\alpha)\times J(\theta). \tag{17}\]

## 4 Experimental Settings

DatasetWe follow prior work on automatic prediction of a single, primary frame of a news article as annotated in the Media Frames Corpus (MFC; Card et al. (2015)). The MFC contains a large number of news articles on five contentious policy issues (immigration, smoking, gun control, death penalty, and same-sex marriage), manually annotated with document- and span-level frames labels from a set of 15 general frames (listed in Table 5 in the Appendix). Articles were selected from 13 major U.S. newspapers, published between 1980 and 2012. Following previous work, we focus on the immigration portion of MFC, which comprises 5,933 annotated articles, as well as an additional 41,286 unlabeled articles. The resulting dataset contains all 15 frames. Table 5 (Appendix) lists the corresponding frame distribution. We partition the labeled dataset into 10 folds, preserving the overall frame distribution for each fold.

Pre-processing and Semantic Role labelingWe apply state-of-art BERT-based SRL model Shi and Lin (2019) to obtain SRL spans for each sentence. The off-the-shelf model from AllenNLP is trained on OntoNotes5.0 (close to 50% news text). While a domain-adapted model may lead to a small performance gain, the off-the-shelf model enhances generalizability and reproducibility. Qualitative examples of detected SRL spans are shown in Table 4, which confirm that SRL predictions are overall accurate.

We extract semantic role spans for predicates, their associated first (Arg0) and second (Arg1) arguments for each sentence in a document. For the unsupervised component, we disregard sentences with no predicate, and sentences missing both Arg0 and Arg1.

Sentence EncoderIn all our experiments, we use RoBERTa Liu et al. (2019) as our sentence encoder, as previous work Khanehkar et al. (2019) has shown that it outperforms BERT Devlin et al. (2019) and XLNet Yang et al. (2019). We pass each sentence through RoBERTa and retrieve the token-level embeddings. To obtain the sentence embedding, we average the RoBERTa embeddings of all words (Eq. 2). To obtain SRL span embeddings, we average the token embeddings of all words in a predicted span (Eq. 1). Following Gururangan et al. (2020), we pre-train RoBERTa with immigration articles using the masked language model (MLM) objective. Only the labeled data is used for pre-training for fair comparison between Friss and previous models.

Parameter SettingsWe set the maximum sequence length to RoBERTa 64 tokens, the maximum number of sentences per document to 32, and the maximum number of predicates per sentence to 10.7 We set the number of dictionary terms \(K=15\), i.e., the number of frame classes in the MFC corpus. Each dictionary term is of dimension \(D_{w}=768\), equal to the RoBERTa token embedding dimension. We also fix the dimensions of hidden vector \(\mathbf{w}_{s}\) (Eqn. 15) and \(D_{h}\) to this value. We set the number of descriptors in Focal Triplet Loss 

\(t=8\) and the margin pool \(|M|=t\). We set the balancing hyper-parameter between the supervised and unsupervised loss \(\alpha=0.5\) , and \(\lambda=10^{-3}\). The dropout rate is set to \(0.3\).

We perform stochastic gradient descent with mini-batches of 8 documents. We use the Adam optimizer Kingma and Ba (2015) with the default parameters, except for the learning rate, which we set to \(2\times 10^{-5}\) (for the RoBERTa parameters) and \(5\times 10^{-4}\) (for all other parameters). We use a linear scheduler for learning rate decay. The weight decay is applied to all parameters except for bias and batch normalization. We update the Gumbel softmax temperature with the schedule: \(\tau=\max(0.5,\exp(-5\times 10^{-4}\times\mathrm{iteration}),\) updating the temperature every 50 iterations. For all our experiments, we run a maximum of 10 epochs, evaluate every \(50\) iterations, and apply early-stopping if the accuracy does not improve for 20 consecutive evaluations.

## 5 Evaluation

In this section, we evaluate the performance of Friss on primary frame prediction for issue-specific news articles against prior work (Sec 5.1), demonstrate the benefit of adding additional unlabeled data to our semi-supervised model (Sec 5.2), and present a qualitative analysis of our model output corroborating its interpretability (Sec 5.3).

Preliminary StudiesPreliminary analyses revealed that human annotators disagree on frame labels non-uniformly, suggesting that some pairs of frames are perceived to be more similar than others. This observation motivated the Focal Triplet Loss and Gumbel regularization components of our model. In particular, the following groups of frame labels are confused most frequently {"Policy Prescription and Evaluation", "Public Sentiment", "Political"}, {"Fairness", "Legality"}, {"Crime and Punishment", "Security and Defense"}, and {"Morality", "Quality of Life", "Cultural Identity"}. This observation is also corroborated through the empirical gain through the focal triplet loss (Table 2).

### Experiment 1: Frame Prediction

For the supervised model, we report accuracy, as has been done in previous work, as well as Macro-F1, which is oblivious to class sizes, shedding light on performance across all frames. Table 1 compares Friss against related work. Card et al. (2016) incorporate latent personas learnt with a Bayesian model; Field et al. (2018) derive frame-specific lexicons based on pointwise-mutual information; Ji and Smith (2017) incorporate a supervised discourse classifier, and Khanehzar et al. (2019) train frame classifiers on top of RoBERTa-based document embeddings. RoBERTa-S corresponds to the sentence-embedding based component of Friss (Fig 1(b); left) without and with (+MLM) unsupervised pre-training. Overall, we can see that all our model variants outperform previous work in terms of both accuracy and macro-F1. Experiments were run 5 times with 10-fold cross-validation. The results in Table 1 are statistically significant (p<0.05; paired sample t-test).

To better understand the impact of our various

\begin{table}
\begin{tabular}{l c c} \hline \hline
* \\ Field et al. (2018) & 57.3 &
- \\ Ji and Smith (2017) & 58.4 &
- \\ Khanehazar et al. (2019) & 65.8 &
- \\ \hline RoBERTa-S & 66.4 (0.008) & 58.1 (0.013) \\ RoBERTa-S+MLM & 67.1 (0.008) & 58.8 (0.013) \\ \hline FRISS (labeled only) & **68.8** (0.009) & **60.1** (0.011) \\ FRISS (labeled+unlabeled) & **69.7** (0.011) & **60.5** (0.015) \\ \hline \hline \end{tabular}
\end{table}
Table 1: Primary Frame prediction on the MFC immigration data set with mean (standard deviation) over 10-fold cross-validation. We compare our full model Friss against recent work, and the supervised RoBERTa component with and without pre-training. The results are statistically significant (p ¡ 0.05; paired sample t-test). FRISS (labeled only) vs RoBERTa-S+MLM: p=0.009; FRISS (labeled + all unlabeled) vs FRISS (labeled only): p=0.012; FRISS (labeled + all unlabeled) vs RoBERTa-S+MLM: p=0.003.

\begin{table}
\begin{tabular}{l c c} \hline \hline
* focal & 68.73 & 59.98 \\
- Gumbel & 68.51 & 59.70 \\
- focal, Gumbel & 68.47 & 59.72 \\ \hline p only & 68.35 & 59.50 \\ Arg0 only & 68.47 & 59.68 \\ Arg1 only & 68.53 & 59.71 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Ablation results for Friss on primary frame prediction. Top removing Focal Triplet Loss (focal; 3.1.3), and/or Gumbel distribution (Gumbel; 3.1.2). Bottom Friss trained with only a single view (predicate, Arg0 and Arg1) on primary frame prediction.
 

model components, we performed an ablation study on Focal Triplet Loss, the Gumbel regularization, and the impact of individual views. Table 2 shows that both the focal loss and the Gumbel regularization contribute to model performance. Training Friss with any single view individually leads to a performance drop, which is most drastic if the two arguments are omitted, suggesting that the model relies on both predicate and argument information, with arguments playing a slightly more important role.

### Experiment 2: Benefit of Unlabeled Data

Our semi-supervised model can leverage news articles without a frame label, in addition to a labeled training set. We investigated the impact of training Friss with different amounts of additional news articles, taken from the unlabeled immigration portion of the MFC. Figure 2 shows the impact of additional unlabeled data on accuracy and F1: Models with access to more unlabelled data tend to result in higher accuracy and Macro F1 scores. Given the abundance of online news articles, this motivates future work on minimally supervised frame prediction, minimizing the reliance on manual labels and maximizing generalizability to new issues, news outlets or languages.

### Experiment 3: Qualitative Evaluation

In this experiment, we explore the added interpretability contributed by the local latent frame representations. Table 4 contains two MFC documents, highlighted with the most highly associated frame for each identified span for \({\rm p}\), \({\rm a_{0}}\) or \({\rm a_{1}}\). We can observe that the frame associations (a) are intuitively meaningful; and (b) provide a detailed account of the predicted primary frame. For both documents the gold primary frame is 'Political', the bottom document is classified correctly, whereas the top document is mis-classified as 'Capacity & Resources'. The detailed span-level predictions help to explain the model prediction, and in fact add support for the the mis-prediction, suggesting that predicting a single primary document frame may be inappropriate. In the bottom document "a letter" serves as both \({\rm a_{1}}\) of "Republicans sent a letter", where it is predicted as 'Political', and as \({\rm a_{0}}\) of the clause "a letter [...] describing the legal challenges", where it is classified as 'Legality', another example of the nuance of our model predictions, which can support further in-depth study of issue-specific framing.

The potential of our model for fine-grained frame analysis is illustrated in Table 4, which shows how each particular SRL span contributes differently towards various frame categories. It adds a finer-grained framing picture, and estimate of the trustworthiness of model predictions. It allows to assess the main actors wrt. a particular frame (within and across articles), as well as the secondary frames in each article. Also, using SRL makes our model independent of human annotation, and more generalizable. Going beyond "highlighting indicative phrases", our model can distinguish their roles (e.g., the "ICE" as an actor vs. participant in a particular frame).

Table 3 shows the semantic role spans, which are most closely related to Capacity & Resources (blue), Political (red), Legality (purple) and Public Sentiment (green) descriptors in the latent space. We can observe that all associated spans are intuitively relevant to the {frame, view}. Furthermore, Arg0 spans tend to correspond to active participants (agents) in the policy process (including politicians and government bodies), whereas Arg1 spans illustrate the affected participants (patients such as foreign workers, applicants), pro

Figure 2: Friss frame prediction performance with different portions of the 41K unlabeled documents.

 

cesses (reforms, cases, movements), or concepts under debate (political asylum). In future work, we aim to leverage these representations in scalable, in-depth analyses of issue-specific media framing. A full table illustrating the learnt descriptors for all 15 frames in the MFC and all three views is included in Table 6 in Appendix.

## 6 Conclusion

We presented Friss, an interpretable model of media frame prediction, incorporating notions of emphasis framing (selective highlighting of issue aspects) and story framing (drawing on the events and actors described in an article). Our semi-supervised model predicts article-level frame of news articles, leveraging local predicate and argument level embeddings. We demonstrated its three-fold advantage: first, our model empirically outperforms existing models for frame classification; second, it can effectively leverage additional unlabeled data further improving performance; and, finally, its latent representations add transparency to classifier predictions and provide a nuanced article representation. The analyses provided by our model can support downstream applications such as automatic, yet transparent, highlighting of reporting patterns across countries or news outlets; or frame-guided summarization which can support both frame-balanced or frame-specific news summaries. In future work, we plan to extend our work to more diverse news outlets and policy issues, and explore richer latent models of article content, including graph representations over all involved events and actors.

## Acknowledgments

We thank the anonymous reviewers for their helpful feedback and suggestions. This article was written with the support from the graduate research scholarship from the Melbourne School of Engineering, University of Melbourne provided to the first author. The original news articles used in this work were obtained from Lexis Nexis under the institutional licence held by the University of Melbourne. This research was undertaken using the LIEF HPC-GPGPU Facility hosted at the University of Melbourne, established with the assistance of LIF Grant LE170100200.

\begin{table}
\begin{tabular}{|l||c 



## References

* Akyurek et al. (2020) Afra Feyza Akyurek, Lei Guo, Randa Elanwar, Prakash Ishwar, Margrit Betke, and Derry Tanti Wijaya. 2020. Multi-label and multilingual news framing analysis. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 8614-8624, Online. Association for Computational Linguistics.
* Boydstun et al. (2013) Amber E Boydstun, Justin H Gross, Philip Resnik, and Noah A Smith. 2013. Identifying media frames and frame dynamics within and across policy issues.
* Card et al. (2015) Dallas Card, Amber E. Boydstun, Justin H. Gross, Philip Resnik, and Noah A. Smith. 2015. The media frames corpus: Annotations of frames across issues. In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 2: Short Papers_, pages 438-444. The Association for Computer Linguistics.
* Card et al. (2016) Dallas Card, Justin H. Gross, Amber E. Boydstun, and Noah A. Smith. 2016. Analyzing framing through the casts of characters in the news. In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016_, pages 1410-1420. The Association for Computational Linguistics.
* Chong and Druckman (2007) Dennis Chong and James N Druckman. 2007. Framing theory. _Annu. Rev. Polit. Sci._, 10:103-126.
* Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In _NAACL-HLT (1)_.
* Entman (1993) Robert M Entman. 1993. Framing: Toward clarification of a fractured paradigm. _Journal of communication_, 43(4):51-58.
* November 4, 2018_, pages 3570-3580.
* Frermann and Szarvas (2017) Lea Frermann and Gyorgy Szarvas. 2017. Inducing semantic micro-clusters from deep multi-view representations of novels. In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017_, pages 1873-1883. Association for Computational Linguistics.
* Gururangan et al. (2020) Suchin Gururangan, Ana Marasovic, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don't stop pretraining: Adapt language models to domains and tasks. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 8342-8360, Online. Association for Computational Linguistics.
* Hallahan (1999) Kirk Hallahan. 1999. Seven models of framing: Implications for public relations. _Journal of public relations research_, 11(3):205-242.
* Han et al. (2019) Xiaochuang Han, Gunsol Choi, and Chenhao Tan. 2019. No permanent Friends or enemies: Tracking relationships between nations from news. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 1660-1676, Minneapolis, Minnesota. Association for Computational Linguistics.
* Cabot et al. (2020) Pere-Luus Huguet Cabot, Verna Dankers, David Abadi, Agneta Fischer, and Ekaterina Shutova. 2020. The Pragmatics behind Politics: Modelling Metaphor, Framing and Emotion in Political Discourse. In _Findings of the Association for Computational Linguistics: EMNLP 2020_, pages 4479-4488, Online. Association for Computational Linguistics.
* Iyyer et al. (2016) Mohit Iyyer, Anupam Guha, Snigdha Chaturvedi, Jordan Boyd-Graber, and Hal Daume III. 2016. Feuding families and former Friends: Unsupervised learning for dynamic fictional relationships. In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 1534-1544, San Diego, California. Association for Computational Linguistics.
* Jang et al. (2017) Eric Jang, Shixiang Gu, and Ben Poole. 2017. Categorical reparameterization with gumbel-softmax. In _5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings_.
* August 4, Volume 1: Long Papers_, pages 996-1005.
* Khanehzar et al. (2019) Shima Khanehzar, Andrew Turpin, and Gosia Mikolajczak. 2019. Modeling political framing across policy issues and contexts. In _Proceedings of the The 17th Annual Workshop of the Australasian Language Technology Association_, pages 61-66, Sydney, Australia. Australasian Language Technology Association.
* Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_

Siyi Liu, Lei Guo, Kate K. Mays, Margrit Betke, and Derry Tanti Wijaya. 2019a. Detecting frames in news headlines and its application to analyzing news framing trends surrounding U.S. gun violence. In _Proceedings of the 23rd Conference on Computational Natural Language Learning, CoNLL 2019, Hong Kong, China, November 3-4, 2019_, pages 504-514. Association for Computational Linguistics.
* Liu et al. (2019b) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b. Roberta: A robustly optimized BERT pretraining approach. _CoRR_, abs/1907.11692.
* Matthes and Kohring (2008) Jorg Matthes and Matthias Kohring. 2008. The content analysis of media frames: Toward improving reliability and validity. _Journal of communication_, 58(2):258-279.
* Nguyen (2015) Viet An Nguyen. 2015. _Guided Probabilistic Topic Models for Agenda-Setting and Framing_. Ph.D. thesis.
* Pradhan et al. (2013) Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Hwee Tou Ng, Anders Bjorkelund, Olga Uruyupina, Yuchen Zhang, and Zhi Zhong. 2013. Towards robust linguistic analysis using ontnotes. In _Proceedings of the Seventeenth Conference on Computational Natural Language Learning, CoNLL 2013, Sofia, Bulgaria, August 8-9, 2013_, pages 143-152.
* Shi and Lin (2019) Peng Shi and Jimmy Lin. 2019. Simple BERT models for relation extraction and semantic role labeling. _CoRR_, abs/1904.05255.
* Tsur et al. (2015) Oren Tsur, Dan Calacci, and David Lazer. 2015. A frame of mind: Using statistical models for detection of framing and agenda setting campaigns. In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers_, pages 1629-1638.
* Yang et al. (2019) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. In _Advances in neural information processing systems_, pages 5753-5763.

## Appendix A Frame Descriptors Extractions

We denote the set of spans that belong to a specific {frame, view} combination as \(C^{f}_{z}\), for all possible combinations of frame categories and views. We obtain the high confidence set \(\hat{C}^{f}_{z}\) for each specific {frame, view} combination by only preserving the spans whose \(g^{f}_{z}>0.8\). To obtain a more general sets of spans, we remove the stopwords and lemmatize the spans. To normalize the sets of high-confidence spans, we calculate the inverse-document frequency for each span \(C^{f}_{z}\) and sort the spans accordingly by the inverse-document frequency to obtain the representative descriptors. Table 6 illustrating the learnt descriptors for all 15 frames in the MFC and all three views 



\begin{table}
\begin{tabular}{l l c} \hline \hline
**Frame** & **Frame description** & **\% IMM** \\ \hline
**Economic: Capacity and Resources**: & costs, benefits, or other financial implications availability of physical, human or financial resources, and capacity of current systems Morality: religious or ethical implications religious or ethical implications & 7.0\% 3.5\% \\
**Morality**: & religious or ethical implications religious or ethical implications balance or distribution of rights, responsibilities, and resources rights, freedoms, and authority of individuals, corporations, and government discussion of specific policies aimed at addressing problems effectiveness and implications of laws and their enforcement threats to welfare of the individual, community, or nation health care, sanitation, public safety threats and opportunities for the individual’s wealth, happiness, and well-being traditions, customs, or values of a social group in relation to a policy issue attitudes and opinions of the general public, including polling and demographics considerations related to politics and politicians, including lobbying, elections, and attempts to sway voters international reputation or foreign policy of the U.S. any coherent group of frames not covered by the above categories & 2.2\% 0.2\% \\ \hline \end{tabular}
\end{table}
Table 5: Framing dimensions from (Boydstun et al., 2013). The final column (% IMM) denotes the frame prevalence in the Immigration portion of the MFC used in the experiments reported in this paper.



* [145] M. C. K. Chan, J. S. Lee, and D. R. Smith, _A review of the design of a new paradigm for multi-scale