An Ordinal Latent Variable Model of Conflict Intensity (2023)

Abstract: Measuring the intensity of events is crucial for monitoring and tracking armed conflict. Advances in automated event extraction have yielded massive data sets of “who did what to whom” micro-records that enable data-driven approaches to monitoring conflict. The Goldstein scale is a widely-used expert-based measure that scores events on a conflictual–cooperative scale. It is based only on the action category (“what”) and disregards the subject (“who”) and object (“to whom”) of an event, as well as contextual information, like associated casualty count, that should contribute to the perception of an event’s “intensity”. This paper takes a latent variable-based approach to measuring conflict intensity. We introduce a probabilistic generative model that assumes each observed event is associated with a latent intensity class. A novel aspect of this model is that it imposes an ordering on the classes, such that higher-valued classes denote higher levels of intensity. The ordinal nature of the latent variable is induced from naturally ordered aspects of the data (e.g., casualty counts) where higher values naturally indicate higher intensity. We evaluate the proposed model both intrinsically and extrinsically, showing that it obtains comparatively good held-out predictive performance.

## introduction on a scale from -10 for conflictual to +10 for cooperative, which of the following events should be considered more "intense": "soldiers injured two civilians" or "rebels detained fifty soldiers"? measuring the intensity of events is crucial for monitoring and tracking armed conflict. advances in the automated collection and coding of events have produced massive and systematized data sets of micro-records that enable data-driven one of the most frequently used measures is the goldstein scale @xcite . major event datasets like idea @xcite , keds @xcite , gdelt @xcite , icews @xcite , phoenix @xcite and navco @xcite all rely on it. the goldstein scale assigns intensity scores between -10.0 and +10 on a conflictual-cooperative scale to the action categories defined by the conflict and mediation event observations (cameo) event coding scheme @xcite . cameo specifies 204 low-level event types which are summarized into 20 high-level action categories. the goldstein scale ranks "use unconventional mass violence" and "fight" as the most conflictual of the 20 highlevel action categories (-10.0) and "provide aid" (+7.0) as the most cooperative; see tab. 1 . despite its usage, the goldstein scale has many well-known shortcomings @xcite . in particular, it applies only to action categories, and does not account for any contextual information of a given event, like which actors are involved, or how many fatalities resulted, among other bits of context that should contribute to the perception of an event's "intensity". this paper takes a latent-variable based approach to measuring conflict intensity. we introduce a probabilistic generative model that assumes each observed event n is associated with a latent intensity class z n . a novel aspect of this model is that it imposes an ordering on the classes, such that higher values of z n denote higher levels of intensity. the ordinal nature of z n is induced from naturally ordered aspects of the data (e.g., casualty counts) where higher values naturally indicate higher intensity. the model effectively learns to interpolate the ordered (i.e., cardinal or ordinal) elements of the data while inferring correlation structure with the non-ordered (e.g., categorical) elements of the data (e.g., actor types). we start with a discussion of the goldstein scale and introduce a political event dataset annotated with goldstein values in §2 and §3. then, we propose our model with an ordinal latent variable in §4. we evaluate the performance of the model intrinsically ( §5) and extrinsically ( §6) and find that it improves over measures based on the original goldstein scale or heuristics based on the raw data. ## limitations of the goldstein scale the goldstein scale is a widely-used measure of the conflictual versus cooperative nature of interactions between countries @xcite . the scale was created by a panel of international relations experts who ranked descriptions of interactions. it was initially created to score action categories in the weis event ontology @xcite and was later adapted to cameo @xcite . the goldstein scale applies only to the action category of an event (e.g., "fight" or "trade"). thus, two different "fight" events, which might involve two different pairs of actors, occur at different times, or differ dramatically with respect to the number of associated fatalities, will still be assigned the same goldstein value.the goldstein scale is thus a poor measure of a conflict's perceived "intensity", as it ignores much of the information that contributes to that perception. recent work in conflict studies, for instance, operationalizes "intensity" primarily using casualty counts @xcite , which the goldstein scale ignores entirely. in tab. 1, we show the empirical distribution of assigned goldstein values alongside the empirical distribution of casualty counts in a dataset of conflict events. the goldstein scale is very coarsegrained; while it ostensibly ranges between -10.0 and +10.0, only a small number of discrete values ever occur, with many action categories assigned the same value. for the purpose of measuring conflict intensity, a finer-grained and more contextual scale is desirable. ## conflict event data this paper considers the publicly available nonviolent and violent campaigns and outcomes (navco) data collection @xcite , specifically, the latest release navco 3.0 from november 2019 which comprises @xmath0,089 events between december 1990 and december 2012. an exemplary event description is "on 19 may 2012, soldiers injured two civilians in afghanistan". each part of this description has been parsed by human coders into standardized, structural features. we color-code the features that correspond to the semantic roles subject, predicate, quantifier, object, which are the focus of our modeling approach. each data point n thus consists of a four-element tuple {s n , p n , q n , o n }. we note that events are further coded for their location (in this case, afghanistan) and time (19 may 2012), among other bits of contextual information. let us discuss each feature in more detail: subject s n . navco contains columns termed "actor3", "actor6" and "actor9" which code for the subject (or agent) of a given action. the actor types are defined by the cameo actor codebook. we first merge the higher-level categories "actor3" and "actor6", resulting in 33 different actor types, and @xcite . goldstein values and casualty counts have intrinsic intensity orderings. goldstein values are reversed and transformed, so that 1.0 represents the most conflictual. we model the subject s n as categorical, the predicate p n as beta, the quantifier q n as zero-inflated geometric and the object o n as categorical. predicate p n (goldstein values). navco codes 1 each event description into one of the 20 cameo action categories in the column "verb10", which is by extension associated with a goldstein value p n . throughout, we refer to the goldstein value p n as an action's "predicate", since there is a one-to-one mapping between action categories and goldstein values. we scale goldstein values p n to a [0, 1] range and invert them (i.e., p n ← 1p n ) so that higher values close to 1 represent more conflictual action categories. quantifier q n (casualty counts). each event description is annotated with human-verified fatality and wounded counts. we add the two and 1 navco features a 21 st action category which we exclude since it is not specified by the cameo taxonomy and thus has no goldstein value. ## ordinal latent variable model we operationalize conflict intensity as a latent variable that expresses the association between the observed variables subject (s n ), predicate (p n ), quantifier (q n ) and object (o n ). each data point is a tuple {s n , p n , q n , o n } representing an event. our bayesian latent variable model is depicted in fig. 2 . we assume the following generative story. for each event n, we assume that its event intensity class z @xmath1, . . . , c} is a categorical random variable with concentration parameter α (z) ∈ r c + . conditioned on z n , we assume each of the observed sites per event tuple s n , p n , q n and o n are then drawn as ## intrinsic evaluation: imputation to evaluate the fit of our model, we conduct a series of predictive experiments. we randomly split the @xmath2,089 events into a 70% training and 30% held-out set. we fit the model on the training set x and approximate the posterior distribution of continuous parameters θ based on @xmath3,000 nuts samples after discarding the first 200 burn-in samples. for evaluation on the held-out set y , we report scaled pointwise predictive density (sppd), mean squared error and weighted f1 score as detailed in app. a. ## extrinsic evaluation: time series in the last section, we intrinsically evaluated the proposed model by having it impute held-out data. in this section, we consider extrinsic evaluations of the proposed model, which seek to evaluate it on downstream predictive tasks for which it was not explicitly designed. in this section, we rely on features of events coded by navco which our model does not access during training. in particular, navco codes for the location ℓ n and the timestamp τ n of each event. we discretize time so that each event is associated with a specific month m n . augmented with these extra characteristics, as well as with the mean intensity inferred by our model, each event n has the following attributes: {s n , p n , q n , o n , ℓ n , m n , zn }. in all of the tasks in this section, we split events according to their location ℓ @xmath4, q n , zn }. in particular, we ask two questions: 1) whether zn adds predictive information for forecasting p n and q n , and 2) whether the model-based measure of intensity zn correlates with google trends more than just p n and q n . ## related work there are a number of relevant papers that broadly seek to measure "latent concepts" @xcite pertaining to international relations, particularly using event data. terechshenko (2020) uses a bayesian item-response theory model to learn ordinal conflict intensity levels from observed event types. o'connor et al. (2013) present an unsupervised, probabilistic topic model to learn an ontology of event data and use the goldstein scale to evaluate it. schein et al. (2015, 2016, 2019) decompose four-way tensors (senders, receivers, actions, time steps) to infer latent classes of cameo-coded events. stoehr et al. (2023b) build on those models by further imposing an ordering on their latent space, which captures conflictcooperation intensity. another line of work models friend-enemy relationship trajectories using neural network-based (han et al., 2019) or hidden markov model-based (chaturvedi et al., 2017) approaches. there also exists work on signed network representations of relationships that are extracted from text @xcite or wikipedia conflict articles @xcite . ## future work assuming civilian casualties are "more intense" than military casualties, we could impose an additional ordering on object types o n . we could also condition observed sites on each other-e.g., casualty counts q n are conditionally dependent on o n or even p n under the model. we plan to further incorporate multiple latent variables to model multi-dimensional intensity concepts. future models could also condition on location and include a temporal component to account for how surprisal may affect the perceived intensity of an event. we also note that the latent variable model presented in this work could also be extended to a more general framework for learning interpretable, ordinal scales from a set of mixed-type data that include cardinal or ordinal observations. we plan to explore generalizations of the proposed model and applications beyond of international relations.