ymoslem/acl-abstracts huggingface dataset:
Developing a tool for fair and reproducible use of paid crowdsourcing in the digital humanities (2022)

This system demonstration paper describes ongoing work on a tool for fair and reproducible use of paid crowdsourcing in the digital humanities. Paid crowdsourcing is widely used in natural language processing and computer vision, but has been rarely applied in the digital humanities due to ethical concerns. We discuss concerns associated with paid crowdsourcing and describe how we seek to mitigate them in designing the tool and crowdsourcing pipelines. We demonstrate how the tool may be used to create annotations for diagrams, a complex mode of expression whose description requires human input.
Archive TimeLine Summarization (ATLS): Conceptual Framework for Timeline Generation over Historical Document Collections (2022)

Archive collections are nowadays mostly available through search engines interfaces, which allow a user to retrieve documents by issuing queries. The study of these collections may be, however, impaired by some aspects of search engines, such as the overwhelming number of documents returned or the lack of contextual knowledge provided. New methods that could work independently or in combination with search engines are then required to access these collections. In this position paper, we propose to extend TimeLine Summarization (TLS) methods on archive collections to assist in their studies. We provide an overview of existing TLS methods and we describe a conceptual framework for an Archive TimeLine Summarization (ATLS) system, which aims to generate informative, readable and interpretable timelines.
Prabhupadavani: A Code-mixed Speech Translation Data for 25 Languages (2022)

Nowadays, the interest in code-mixing has become ubiquitous in Natural Language Processing (NLP); however, not much attention has been given to address this phenomenon for Speech Translation (ST) task. This can be solely attributed to the lack of code-mixed ST task labelled data. Thus, we introduce Prabhupadavani, which is a multilingual code-mixed ST dataset for 25 languages. It is multi-domain, covers ten language families, containing 94 hours of speech by 130+ speakers, manually aligned with corresponding text in the target language. The Prabhupadavani is about Vedic culture and heritage from Indic literature, where code-switching in the case of quotation from literature is important in the context of humanities teaching. To the best of our knowledge, Prabhupadvani is the first multi-lingual code-mixed ST dataset available in the ST literature. This data also can be used for a code-mixed machine translation task. All the dataset can be accessed at: \url{https://github.com/frozentoad9/CMST}.
Using Language Models to Improve Rule-based Linguistic Annotation of Modern Historical Japanese Corpora (2022)

Annotation of unlabeled textual corpora with linguistic metadata is a fundamental technology in many scholarly workflows in the digital humanities (DH). Pretrained natural language processing pipelines offer tokenization, tagging, and dependency parsing of raw text simultaneously using an annotation scheme like Universal Dependencies (UD). However, the accuracy of these UD tools remains unknown for historical texts and current methods lack mechanisms that enable helpful evaluations by domain experts. To address both points for the case of Modern Historical Japanese text, this paper proposes the use of unsupervised domain adaptation methods to develop a domain-adapted language model (LM) that can flag instances of inaccurate UD output from a pretrained LM and the use of these instances to form rules that, when applied, improves pretrained annotation accuracy. To test the efficacy of the proposed approach, the paper evaluates the domain-adapted LM against three baselines that are not adapted to the historical domain. The experiments conducted demonstrate that the domain-adapted LM improves UD annotation in the Modern Historical Japanese domain and that rules produced using this LM are best indicative of characteristics of the domain in terms of out-of-vocabulary rate and candidate normalized form discovery for {\textquotedblleft}difficult{\textquotedblright} bigram terms.
Every picture tells a story: Image-grounded controllable stylistic story generation (2022)

Generating a short story out of an image is arduous. Unlike image captioning, story generation from an image poses multiple challenges: preserving the story coherence, appropriately assessing the quality of the story, steering the generated story into a certain style, and addressing the scarcity of image-story pair reference datasets limiting supervision during training. In this work, we introduce Plug-and-Play Story Teller (PPST) and improve image-to-story generation by: 1) alleviating the data scarcity problem by incorporating large pre-trained models, namely CLIP and GPT-2, to facilitate a fluent image-to-text generation with minimal supervision, and 2) enabling a more style-relevant generation by incorporating stylistic adapters to control the story generation. We conduct image-to-story generation experiments with non-styled, romance-styled, and action-styled PPST approaches and compare our generated stories with those of previous work over three aspects, i.e., story coherence, image-story relevance, and style fitness, using both automatic and human evaluation. The results show that PPST improves story coherence and has better image-story relevance, but has yet to be adequately stylistic.
To the Most Gracious Highness, from Your Humble Servant: Analysing Swedish 18th Century Petitions Using Text Classification (2022)

Petitions are a rich historical source, yet they have been relatively little used in historical research. In this paper, we aim to analyse Swedish texts from around the 18th century, and petitions in particular, using automatic means of text classification. We also test how text pre-processing and different feature representations affect the result, and we examine feature importance for our main class of interest - petitions. Our experiments show that the statistical algorithms NB, RF, SVM, and kNN are indeed very able to classify different genres of historical text. Further, we find that normalisation has a positive impact on classification, and that content words are particularly informative for the traditional models. A fine-tuned BERT model, fed with normalised data, outperforms all other classification experiments with a macro average F1 score at 98.8. However, using less computationally expensive methods, including feature representation with word2vec, fastText embeddings or even TF-IDF values, with a SVM classifier also show good results for both unnormalise and normalised data. In the feature importance analysis, where we obtain the features most decisive for the classification models, we find highly relevant characteristics of the petitions, namely words expressing signs of someone inferior addressing someone superior.
Automatized Detection and Annotation for Calls to Action in Latin-American Social Media Postings (2022)

Voter mobilization via social media has shown to be an effective tool. While previous research has primarily looked at how calls-to-action (CTAs) were used in Twitter messages from non-profit organizations and protest mobilization, we are interested in identifying the linguistic cues used in CTAs found on Facebook and Twitter for an automatic identification of CTAs. The work is part of an on-going collaboration with researchers from political science, who are investigating CTAs in the period leading up to recent elections in three different Latin American countries. We developed a new NLP pipeline for Spanish to facilitate their work. Our pipeline annotates social media posts with a range of linguistic information and then conducts targeted searches for linguistic cues that allow for an automatic annotation and identification of relevant CTAs. By using carefully crafted and linguistically informed heuristics, our system so far achieves an F1-score of 0.72.
The Distribution of Deontic Modals in Jane Austen`s Mature Novels (2022)

Deontic modals are auxiliary verbs which express some kind of necessity, obligation, or moral recommendation. This paper investigates the collocation and distribution within Jane Austen`s six mature novels of the following deontic modals: must, should, ought, and need. We also examine the co-occurrences of these modals with name mentions of the heroines in the six novels, categorizing each occurrence with a category of obligation if applicable. The paper offers a brief explanation of the categories of obligation chosen for this investigation. In order to examine the types of obligations associated with each heroine, we then investigate the distribution of these categories in relation to mentions of each heroine. The patterns observed show a general concurrence with the thematic characterizations of Austen`s heroines which are found in literary analysis.
Man vs. Machine: Extracting Character Networks from Human and Machine Translations (2022)

Most of the work on Character Networks to date is limited to monolingual texts. Conversely, in this paper we apply and analyze Character Networks on both source texts (English novels) and their Finnish translations (both human- and machine-translated). We assume that this analysis could provide some insights on changes in translations that could modify the character networks, as well as the narrative. The results show that the character networks of translations differ from originals in case of long novels, and the differences may also vary depending on the novel and translator`s strategy.
The COVID That Wasn`t: Counterfactual Journalism Using GPT (2022)

In this paper, we explore the use of large language models to assess human interpretations of real world events. To do so, we use a language model trained prior to 2020 to artificially generate news articles concerning COVID-19 given the headlines of actual articles written during the pandemic. We then compare stylistic qualities of our artificially generated corpus with a news corpus, in this case 5,082 articles produced by CBC News between January 23 and May 5, 2020. We find our artificially generated articles exhibits a considerably more negative attitude towards COVID and a significantly lower reliance on geopolitical framing. Our methods and results hold importance for researchers seeking to simulate large scale cultural processes via recent breakthroughs in text generation.
War and Pieces: Comparing Perspectives About World War I and II Across Wikipedia Language Communities (2022)

Wikipedia is widely used to train models for various tasks including semantic association, text generation, and translation. These tasks typically involve aligning and using text from multiple language editions, with the assumption that all versions of the article present the same content. But this assumption may not hold. We introduce a methodology for approximating the extent to which narratives of conflict may diverge in this scenario, focusing on articles about World War I and II battles written by Wikipedia`s communities of editors across four language editions. For simplicity, our unit of analysis representing each language communities' perspectives is based on national entities and their subject-object-relation context, identified using named entity recognition and open-domain information extraction. Using a vector representation of these tuples, we evaluate how similarly different language editions portray how and how often these entities are mentioned in articles. Our results indicate that (1) language editions tend to reference associated countries more and (2) how much one language edition`s depiction overlaps with all others varies.
Computational Detection of Narrativity: A Comparison Using Textual Features and Reader Response (2022)

The task of computational textual narrative detection focuses on detecting the presence of narrative parts, or the degree of narrativity in texts. In this work, we focus on detecting the local degree of narrativity in texts, using short text passages. We performed a human annotation experiment on 325 English texts ranging across 20 genres to capture readers' perception by means of three cognitive aspects: suspense, curiosity, and surprise. We then employed a linear regression model to predict narrativity scores for 17,372 texts. When comparing our average annotation scores to similar annotation experiments with different cognitive aspects, we found that Pearson`s r ranges from .63 to .75. When looking at the calculated narrative probabilities, Pearson`s r is .91. We found that it is possible to use suspense, curiosity and surprise to detect narrativity. However, there are still differences between methods. This does not imply that there are inherently correct methods, but rather suggests that the underlying definition of narrativity is a determining factor for the results of the computational models employed.
Towards a contextualised spatial-diachronic history of literature: mapping emotional representations of the city and the country in Polish fiction from 1864 to 1939 (2022)

In this article, we discuss the conditions surrounding the building of historical and literary corpora. We describe the assumptions and method of making the original corpus of the Polish novel (1864-1939). Then, we present the research procedure aimed at demonstrating the variability of the emotional value of the concept of {\textquotedblleft}the city{\textquotedblright} and {\textquotedblleft}the country{\textquotedblright} in the texts included in our corpus. The proposed method considers the complex socio-political nature of Central and Eastern Europe, especially the fact that there was no unified Polish state during this period. The method can be easily replicated in studies of the literature of countries with similar specificities.
Measuring Presence of Women and Men as Information Sources in News (2022)

In the news, statements from information sources are often quoted, made by individuals who interact in the news. Detecting those quotes and the gender of their sources is a key task when it comes to media analysis from a gender perspective. It is a challenging task: the structure of the quotes is variable, gender marks are not present in many languages, and quote authors are often omitted due to frequent use of coreferences. This paper proposes a strategy to measure the presence of women and men as information sources in news. We approach the problem of detecting sentences including quotes and the gender of the speaker as a joint task, by means of a supervised multiclass classifier of sentences. We have created the first datasets for Spanish and Basque by manually annotating quotes and the gender of the associated sources in news items. The results obtained show that BERT based approaches are significantly better than bag-of-words based classical ones, achieving accuracies close to 90{\%}. We also analyse a bilingual learning strategy and generating additional training examples synthetically; both provide improvements up to 3.4{\%} and 5.6{\%}, respectively.
Abstraction ou hallucination ? \'Etat des lieux et \'evaluation du risque pour les mod\`eles de g\'en\'eration de r\'esum\'es automatiques de type s\'equence-\`a-s\'equence (Abstraction or Hallucination ? Status and Risk assessment for sequence-to-sequence Automatic) (2022)

La g{\'e}n{\'e}ration de texte a r{\'e}cemment connu un tr{\`e}s fort int{\'e}r{\^e}t au vu des avanc{\'e}es notables dans le domaine des mod{\`e}les de langage neuronaux. Malgr{\'e} ces avanc{\'e}es, cette t{\^a}che reste difficile quand il s`agit d`un r{\'e}sum{\'e} automatique de texte par abstraction. Certains syst{\`e}mes de r{\'e}sum{\'e}s g{\'e}n{\`e}rent des textes qui ne sont pas forc{\'e}ment fid{\`e}les au document source. C`est sur cette th{\'e}matique que porte notre {\'e}tude. Nous pr{\'e}sentons une typologie d`erreurs pour les r{\'e}sum{\'e}s automatique et ainsi qu`une caract{\'e}risation du ph{\'e}nom{\`e}ne de l`abstraction pour les r{\'e}sum{\'e}s de r{\'e}f{\'e}rence afin de mieux comprendre l`ampleur de ces diff{\'e}rents ph{\'e}nom{\`e}nes sur les entit{\'e}s nomm{\'e}es. Nous proposons {\'e}galement une mesure d'{\'e}valuation du risque d`erreur lorsqu`un syst{\`e}me tente de faire des abstractions sur les entit{\'e}s nomm{\'e}es d`un document.
Choisir le bon co-\'equipier pour la g\'en\'eration coop\'erative de texte (Choosing The Right Teammate For Cooperative Text Generation) (2022)

Les mod{\`e}les de langue g{\'e}n{\`e}rent des textes en pr{\'e}disant successivement des distributions de probabilit{\'e} pour les prochains tokens en fonction des tokens pr{\'e}c{\'e}dents. Pour g{\'e}n{\'e}rer des textes avec des propri{\'e}t{\'e}s souhait{\'e}es (par ex. {\^e}tre plus naturels, non toxiques ou avoir un style d'{\'e}criture sp{\'e}cifique), une solution {---} le d{\'e}codage coop{\'e}ratif {---} consiste {\`a} utiliser un classifieur lors de la g{\'e}n{\'e}ration pour guider l'{\'e}chantillonnage de la distribution du mod{\`e}le de langue vers des textes ayant la propri{\'e}t{\'e} attendue. Dans cet article, nous examinons trois familles de discriminateurs (bas{\'e}s sur des transformers) pour cette t{\^a}che de d{\'e}codage coop{\'e}ratif : les discriminateurs bidirectionnels, unidirectionnels (de gauche {\`a} droite) et g{\'e}n{\'e}ratifs. Nous {\'e}valuons leurs avantages et inconv{\'e}nients, en explorant leur pr{\'e}cision respective sur des t{\^a}ches de classification, ainsi que leur impact sur la g{\'e}n{\'e}ration coop{\'e}rative et leur co{\^u}t de calcul, dans le cadre d`une strat{\'e}gie de d{\'e}codage {\'e}tat de l`art, bas{\'e}e sur une recherche arborescente de Monte-Carlo (MCTS). Nous fournissons {\'e}galement l`impl{\'e}mentation (batch{\'e}e) utilis{\'e}e pour nos exp{\'e}riences.
D\'ecodage guid\'e par un discriminateur avec le Monte Carlo Tree Search pour la g\'en\'eration de texte contrainte (Discriminator-guided decoding with Monte Carlo Tree Search for constrained text generation ) (2022)

Dans cet article, nous explorons comment contr{\^o}ler la g{\'e}n{\'e}ration de texte au moment du d{\'e}codage pour satisfaire certaines contraintes (e.g. {\^e}tre non toxique, transmettre certaines {\'e}motions...), sans n{\'e}cessiter de r{\'e}-entrainer le mod{\`e}le de langue. Pour cela, nous formalisons la g{\'e}n{\'e}ration sous contrainte comme un processus d`exploration d`arbre guid{\'e} par un discriminateur qui indique dans quelle mesure la s{\'e}quence associ{\'e}e respecte la contrainte. Nous proposons plusieurs m{\'e}thodes originales pour explorer cet arbre de g{\'e}n{\'e}ration, notamment le Monte Carlo Tree Search (MCTS) qui fournit des garanties th{\'e}oriques sur l`efficacit{\'e} de la recherche. Au travers d`exp{\'e}riences sur 3 jeux de donn{\'e}es et 2 langues, nous montrons que le d{\'e}codage par MCTS guid{\'e} par les discriminateurs permet d`obtenir des r{\'e}sultats {\`a} l'{\'e}tat-de-l`art. Nous d{\'e}montrons {\'e}galement que d`autres m{\'e}thodes de d{\'e}codage que nous proposons, bas{\'e}es sur le re-ordonnancement, peuvent {\^e}tre r{\'e}ellement efficaces lorsque la diversit{\'e} parmi les propositions g{\'e}n{\'e}r{\'e}es est encourag{\'e}e.
D\'etection d`anomalies textuelles \`a base de l`ing\'enierie d`invite (Prompt Engineering-Based Text Anomaly Detection ) (2022)

La d{\'e}tection d`anomalies textuelles est une t{\^a}che importante de la fouille de textes. Plusieurs approches g{\'e}n{\'e}rales, visant l`identification de points de donn{\'e}es aberrants, ont {\'e}t{\'e} appliqu{\'e}s dans ce domaine. N{\'e}anmoins, ces approches exploitent peu les nouvelles avanc{\'e}es du traitement automatique des langues naturelles (TALN). L`av{\`e}nement des mod{\`e}les de langage pr{\'e}-entra{\^i}n{\'e}s comme BERT et GPT-2 a donn{\'e} naissance {\`a} un nouveau paradigme de l`apprentissage automatique appel{\'e} ing{\'e}nierie d`invite (prompt engineering) qui a montr{\'e} de bonnes performances sur plusieurs t{\^a}ches du TALN. Cet article pr{\'e}sente un travail exploratoire visant {\`a} examiner la possibilit{\'e} de d{\'e}tecter des anomalies textuelles {\`a} l`aide de l`ing{\'e}nierie d`invite. Dans nos exp{\'e}rimentations, nous avons examin{\'e} la performance de diff{\'e}rents mod{\`e}les d`invite. Les r{\'e}sultats ont montr{\'e} que l`ing{\'e}nierie d`invite est une m{\'e}thode prometteuse pour la d{\'e}tection d`anomalies textuelles.
D\'etection des influenceurs dans des m\'edias sociaux par une approche hybride (Influencer detection in social media, a hybrid approach) (2022)

L`influence sociale est un ph{\'e}nom{\`e}ne important dans divers domaines, tels que l'{\'e}conomie et la politique, qui a gagn{\'e} en r{\'e}sonnance avec la popularit{\'e} des m{\'e}dias sociaux, notamment les r{\'e}seaux sociaux et les forums. La majorit{\'e} des travaux sur ce sujet propose des approches fond{\'e}es sur des th{\'e}ories en sciences humaines (sociologie, linguistique), et des techniques d`analyse de r{\'e}seau (mesures de propagation et de centralit{\'e}) ou de TAL. Dans cet article, nous pr{\'e}sentons un mod{\`e}le d`influence inspir{\'e} de travaux en psychologie sociale, sur lequel nous construisons un syst{\`e}me combinant un module de TAL pour d{\'e}tecter les messages refl{\'e}tant les processus d`influence, associ{\'e} {\`a} une analyse par centralit{\'e} de la transmission de ces messages. Nos exp{\'e}riences sur le forum de d{\'e}bats Change My View montrent que l`approche par hybridation, compar{\'e}e {\`a} la centralit{\'e} seule, aide {\`a} mieux d{\'e}tecter les influenceurs.
\'Etiquetage ou g\'en\'eration de s\'equences pour la compr\'ehension automatique du langage en contexte d`interaction? (Sequence tagging or sequence generation for Natural Language Understanding ?) (2022)

La t{\^a}che de compr{\'e}hension automatique du langage en contexte d`interaction (NLU pour Natural Language Understanding) est souvent r{\'e}duite {\`a} la d{\'e}tection d`intentions et de concepts sur des corpus mono-domaines annot{\'e}s avec une seule intention par {\'e}nonc{\'e}. Afin de d{\'e}passer ce paradigme, nous cherchons {\`a} aborder des r{\'e}f{\'e}rentiels plus complexes en visant des repr{\'e}sentations s{\'e}mantiques structur{\'e}es au-del{\`a} du simple mod{\`e}le intention/concept. Nous nous int{\'e}ressons au corpus MultiWOZ, couramment utilis{\'e} pour le suivi de l'{\'e}tat du dialogue. Nous questionnons la projection de ces annotations s{\'e}mantiques complexes pour le NLU, en comparant plusieurs approches d'{\'e}tiquetage de s{\'e}quence, puis en proposant un nouveau formalisme inspir{\'e} des m{\'e}thodes de g{\'e}n{\'e}ration de graphe pour la mod{\'e}lisation s{\'e}mantique AMR. Nous discutons enfin le potentiel des approches g{\'e}n{\'e}ratives.