A New Task and Dataset on Detecting Attacks on Human Rights Defenders (2023)

Abstract: The ability to conduct retrospective analyses of attacks on human rights defenders over time and by location is important for humanitarian organizations to better understand historical or ongoing human rights violations and thus better manage the global impact of such events. We hypothesize that NLP can support such efforts by quickly processing large collections of news articles to detect and summarize the characteristics of attacks on human rights defenders. To that end, we propose a new dataset for detecting Attacks on Human Rights Defenders (HRDsAttack) consisting of crowdsourced annotations on 500 online news articles. The annotations include fine-grained information about the type and location of the attacks, as well as information about the victim(s). We demonstrate the usefulness of the dataset by using it to train and evaluate baseline models on several sub-tasks to predict the annotated characteristics.

## introduction it is essential for human rights organizations to track, analyze and summarize attacks on human rights defenders over time and across locations for better personnel protection and situational analysis. to do so, multiple event attributes denoting different aspects of the attacking event need to be extracted from textual sources. however, this would be a time-consuming process if done manually. figure 1 gives an example of the kinds of information that such organizations need to extract. in order to train and evaluate an nlp model to extract this information automatically, a relevant dataset is necessary. the ideal dataset requires accurate annotations for both the breadth (the number of extracted event attributes) and depth (the levels of granularity for each event attribute) of the events. however, all existing event extraction (ee) datasets (e.g. ace05 @xcite , ere @xcite , ace05-e @xcite algerian journalist sentenced to six months' jail for 'defamation' 2021-03-29 00:00:00 a court in algeria's capital monday sentenced journalist abdelhakim setouane to six months in jail for defamation of a former speaker of the lower house of parliament, his lawyer said. abdellah heboul told afp that the journalist, who has been detained since october 20, would be released on april 20, having spent six months behind bars. 2019), ace05-e+ @xcite ) do not contain annotations at a sufficiently fine-grained level. although some existing ontologies and datasets do include annotations related to attacking events, e.g. the attack event type in the ace05 dataset along with the associated agent attribute, they are incomplete with respect to many of the details of interest to human rights organizations and do not contain annotations relevant to victim characteristics or the time/location of the attacking event. as a result, existing open-source ee models trained on these datasets @xcite @xcite are unable to predict the complete set of relevant information. to mitigate the gap in existing resources, we present hrdsattack, a new dataset containing crowdsourced annotations on 500 online news articles (including article title, article body text, and publication time). each news article is annotated with 13 different event attributes to capture critical information about attacks on human rights defenders, including the type and location of the attacks, as well as information about the victim(s) and the perpetrator. with hrdsattack, we hope to support more research opportunities for including nlp in applications related to human rights, as well as for broader ai for social good (ai4sg) efforts. to summarize, our contributions are threefold: the hrdsattack dataset along with the code for model training and evaluation is available at https://github.com/dataminr-ai/ hrdsattack . ## dataset in this section, we describe the construction of the hrdsattack dataset, which contains 500 annotated news articles, including article title, article body text, and publication time. we select news articles as the data source rather than other data sources (such as social media posts) since online news articles generally have higher accessibility, better trustworthiness of the source, and longer content category event attribute labels label definitions perpetrator perpetrator mention yes there is one or more explicit mention of the perpetrator in the news article. no there is no explicit mention of the perpetrator in the news article. perpetrator type state security forces anyone employed by or representing a state institution. other state actors other actors that are a part of the state or other non-military authorities of a state. other non-state actors other actors /private actors that are not a part of the state and act without the state's permission, support, or acquiescence. other actors with permissions armed actors that are not a part of the state but act with the state's permission, support or acquiescence. other actors without permissions other actors that are not a part of the state. regional organizations person or group working for a regional or international organization. insufficient information there is insufficient information available to determine one of the categories described above. none not applicable, when perpetrator mention is no. ## our model with the construction of hrdsattack, we now turn to developing a model for the task. we noted earlier that existing state-of-the-art ee models are not suitable as baselines, as they rely on extensive human annotations based on token-level annotations, hence cannot easily be re-trained and evaluated on this dataset. for instance, amr-ie @xcite and graphie @xcite are trained on the ace05 dataset and ere dataset. some recent research casts the ee task as qa tasks or seq2seq tasks, such as rcee_er @xcite and text2event @xcite . in this section, we propose a new model for extracting fine-grained details regarding attacks on human rights defenders. ## conclusion in this paper, we present a new dataset that supports extracting detailed information about attacks on human rights defenders under a new task setting. compared with existing event extraction resources, we focus on the human rights domain and expand to more event attributes for capturing event details more comprehensively. our new dataset (hrdsat-tack) contains 500 human-annotated news articles with 13 different event attributes regarding the victim(s), the type of perpetrator and violation(s), as well as the time and location of the attacks. we demonstrate the usefulness of the dataset by developing a sequence-to-sequence-based question answering model tailored for this task. while it achieves decent performance on some event attributes, there are many where there is much room for improvement. we view this model as a strong baseline for future work. we believe models trained with hrdsattack could be generalized to detect attacking events in other domains or targeting a different population. and we hope that this work encourages additional research on the development of new ai4sg nlp resources in the future.