pwc-archive/papers-with-abstracts huggingface dataset:
Investigating the Impact of ASR Errors on Spoken Implicit Discourse Relation Recognition
Authors: Linh The Nguyen, Dat Quoc Nguyen
Date: 2022-10-01 00:00:00
Proceeding: TU (COLING) 2022 10
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Relation, speech-recognition, Speech Recognition

We present an empirical study investigating the influence of automatic speech recognition (ASR) errors on the spoken implicit discourse relation recognition (IDRR) task. We construct a spoken dataset for this task based on the Penn Discourse Treebank 2.0. On this dataset, we conduct “Cascaded” experiments employing state-of-the-art ASR and text-based IDRR models and find that the ASR errors significantly decrease the IDRR performance. In addition, the “Cascaded” approach does remarkably better than an “End-to-End” one that directly predicts a relation label for each input argument speech pair.
Findings of the VarDial Evaluation Campaign 2022
Authors: Noëmi Aepli, Antonios Anastasopoulos, Adrian-Gabriel Chifu, William Domingues, Fahim Faisal, Mihaela Gaman, Radu Tudor Ionescu, Yves Scherrer
Date: 2022-10-01 00:00:00
Proceeding: VarDial (COLING) 2022 10
Tasks: Dialect Identification, Extractive Question-Answering, Question Answering

This report presents the results of the shared tasks organized as part of the VarDial Evaluation Campaign 2022. The campaign is part of the ninth workshop on Natural Language Processing (NLP) for Similar Languages, Varieties and Dialects (VarDial), co-located with COLING 2022. Three separate shared tasks were included this year: Identification of Languages and Dialects of Italy (ITDI), French Cross-Domain Dialect Identification (FDI), and Dialectal Extractive Question Answering (DialQA). All three tasks were organized for the first time this year.
Social Context and User Profiles of Linguistic Variation on a Micro Scale
Authors: Olga Kellert, Nicholas Hill Matlis
Date: 2022-10-01 00:00:00
Proceeding: VarDial (COLING) 2022 10

This paper presents a new tweet-based approach in geolinguistic analysis which combines geolocation, user IDs and textual features in order to identify patterns of linguistic variation on a sub-city scale. Sub-city variations can be connected to social drivers and thus open new opportunities for understanding the mechanisms of language variation and change. However, measuring linguistic variation on these scales is challenging due to the lack of highly-spatially-resolved data as well as to the daily movement or users’ “mobility” inside cities which can obscure the relation between the social context and linguistic variation. Here we demonstrate how combining geolocation with user IDs and textual analysis of tweets can yield information about the linguistic profiles of the users, the social context associated with specific locations and their connection to linguistic variation. We apply our methodology to analyze dialects in Buenos Aires and find evidence of socially-driven variation. Our methods will contribute to the identification of sociolinguistic patterns inside cities, which are valuable in social sciences and social services.
dialectR: Doing Dialectometry in R
Authors: Ryan Soh-Eun Shim, John Nerbonne
Date: 2022-10-01 00:00:00
Proceeding: VarDial (COLING) 2022 10
Tasks: Clustering

We present dialectR, an open-source R package for performing quantitative analyses of dialects based on categorical measures of difference and on variants of edit distance. dialectR stands as one of the first programmable toolkits that may freely be combined and extended by users with further statistical procedures. We describe implementational details of the package, and provide two examples of its use: one performing analyses based on multidimensional scaling and hierarchical clustering on a dataset of Dutch dialects, and another showing how an approximation of the acoustic vowel space may be achieved by performing an MFCC (Mel-Frequency Cepstral Coefficients)-based acoustic distance on audio recordings of vowels.
Low-Resource Neural Machine Translation: A Case Study of Cantonese
Authors: Evelyn Kai-Yan Liu
Date: 2022-10-01 00:00:00
Proceeding: VarDial (COLING) 2022 10
Tasks: Data Augmentation, Low Resource Neural Machine Translation, Low-Resource Neural Machine Translation, Machine Translation, NMT, Sentence, Translation

The development of Natural Language Processing (NLP) applications for Cantonese, a language with over 85 million speakers, is lagging compared to other languages with a similar number of speakers. In this paper, we present, to our best knowledge, the first benchmark of multiple neural machine translation (NMT) systems from Mandarin Chinese to Cantonese. Additionally, we performed parallel sentence mining (PSM) as data augmentation for the extremely low resource language pair and increased the number of sentence pairs from 1,002 to 35,877. Results show that with PSM, the best performing model (BPE-level bidirectional LSTM) scored 11.98 BLEU better than the vanilla baseline and 9.93 BLEU higher than our strong baseline. Our unsupervised NMT (UNMT) results also refuted previous assumption n (Rubino et al., 2020) that the poor performance was related to the lack of linguistic similarities between the target and source languages, particularly in the case of Cantonese and Mandarin. In the process of building the NMT system, we also created the first large-scale parallel training and evaluation datasets of the language pair. Codes and datasets are publicly available at https://github.com/evelynkyl/yue_nmt.
Phonetic, Semantic, and Articulatory Features in Assamese-Bengali Cognate Detection
Authors: Abhijnan Nath, Rahul Ghosh, Nikhil Krishnaswamy
Date: 2022-10-01 00:00:00
Proceeding: VarDial (COLING) 2022 10

In this paper, we propose a method to detect if words in two similar languages, Assamese and Bengali, are cognates. We mix phonetic, semantic, and articulatory features and use the cognate detection task to analyze the relative informational contribution of each type of feature to distinguish words in the two similar languages. In addition, since support for low-resourced languages like Assamese can be weak or nonexistent in some multilingual language models, we create a monolingual Assamese Transformer model and explore augmenting multilingual models with monolingual models using affine transformation techniques between vector spaces.
Mapping Phonology to Semantics: A Computational Model of Cross-Lingual Spoken-Word Recognition
Authors: Iuliia Zaitova, Badr Abdullah, Dietrich Klakow
Date: 2022-10-01 00:00:00
Proceeding: VarDial (COLING) 2022 10

Closely related languages are often mutually intelligible to various degrees. Therefore, speakers of closely related languages are usually capable of (partially) comprehending each other’s speech without explicitly learning the target, second language. The cross-linguistic intelligibility among closely related languages is mainly driven by linguistic factors such as lexical similarities. This paper presents a computational model of spoken-word recognition and investigates its ability to recognize word forms from different languages than its native, training language. Our model is based on a recurrent neural network that learns to map a word’s phonological sequence onto a semantic representation of the word. Furthermore, we present a case study on the related Slavic languages and demonstrate that the cross-lingual performance of our model not only predicts mutual intelligibility to a large extent but also reflects the genetic classification of the languages in our study.
OcWikiDisc: a Corpus of Wikipedia Talk Pages in Occitan
Authors: Aleksandra Miletic, Yves Scherrer
Date: 2022-10-01 00:00:00
Proceeding: VarDial (COLING) 2022 10
Tasks: 8k, Language Identification

This paper presents OcWikiDisc, a new freely available corpus in Occitan, as well as language identification experiments on Occitan done as part of the corpus building process. Occitan is a regional language spoken mainly in the south of France and in parts of Spain and Italy. It exhibits rich diatopic variation, it is not standardized, and it is still low-resourced, especially when it comes to large downloadable corpora. We introduce OcWikiDisc, a corpus extracted from the talk pages associated with the Occitan Wikipedia. The version of the corpus with the most restrictive language filtering contains 8K user messages for a total of 618K tokens. The language filtering is performed based on language identification experiments with five off-the-shelf tools, including the new fasttext’s language identification model from Meta AI’s No Language Left Behind initiative, released in July 2022.
Is Encoder-Decoder Transformer the Shiny Hammer?
Authors: Nat Gillin
Date: 2022-10-01 00:00:00
Proceeding: VarDial (COLING) 2022 10
Tasks: Classification, Decoder, Machine Translation, Multi-class Classification, Translation

We present an approach to multi-class classification using an encoder-decoder transformer model. We trained a network to identify French varieties using the same scripts we use to train an encoder-decoder machine translation model. With some slight modification to the data preparation and inference parameters, we showed that the same tools used for machine translation can be easily re-used to achieve competitive performance for classification. On the French Dialectal Identification (FDI) task, we scored 32.4 on weighted F1, but this is far from a simple naive bayes classifier that outperforms a neural encoder-decoder model at 41.27 weighted F1.
The Curious Case of Logistic Regression for Italian Languages and Dialects Identification
Authors: Giacomo Camposampiero, Quynh Anh Nguyen, Francesco Di Stefano
Date: 2022-10-01 00:00:00
Proceeding: VarDial (COLING) 2022 10
Tasks: Language Identification, Machine Translation, Opinion Mining, regression, Translation

Automatic Language Identification represents an important task for improving many real-world applications such as opinion mining and machine translation. In the case of closely-related languages such as regional dialects, this task is often challenging. In this paper, we propose an extensive evaluation of different approaches for the identification of Italian dialects and languages, spanning from classical machine learning models to more complex neural architectures and state-of-the-art pre-trained language models. Surprisingly, shallow machine learning models managed to outperform huge pre-trained language models in this specific task. This work was developed in the context of the Identification of Languages and Dialects of Italy (ITDI) task organised at VarDial 2022 Evaluation Campaign. Our best submission managed to achieve a weighted F1-score of 0.6880, ranking 5th out of 9 final submissions.
Neural Networks for Cross-domain Language Identification. Phlyers @Vardial 2022
Authors: Andrea Ceolin
Date: 2022-10-01 00:00:00
Proceeding: VarDial (COLING) 2022 10
Tasks: Language Identification

We present our contribution to the Identification of Languages and Dialects of Italy shared task (ITDI) proposed in the VarDial Evaluation Campaign 2022, which asked participants to automatically identify the language of a text associated to one of the language varieties of Italy. The method that yielded the best results in our experiments was a Deep Feedforward Neural Network (DNN) trained on character ngram counts, which provided a better performance compared to Naive Bayes methods and Convolutional Neural Networks (CNN). The system was among the best methods proposed for the ITDI shared task. The analysis of the results suggests that simple DNNs could be more efficient than CNNs to perform language identification of close varieties.
Transfer Learning Improves French Cross-Domain Dialect Identification: NRC @ VarDial 2022
Authors: Gabriel Bernier-Colborne, Serge Leger, Cyril Goutte
Date: 2022-10-01 00:00:00
Proceeding: VarDial (COLING) 2022 10
Tasks: Dialect Identification, Language Modeling, Language Modelling, Transfer Learning

We describe the systems developed by the National Research Council Canada for the French Cross-Domain Dialect Identification shared task at the 2022 VarDial evaluation campaign. We evaluated two different approaches to this task: SVM and probabilistic classifiers exploiting n-grams as features, and trained from scratch on the data provided; and a pre-trained French language model, CamemBERT, that we fine-tuned on the dialect identification task. The latter method turned out to improve the macro-F1 score on the test set from 0.344 to 0.430 (25% increase), which indicates that transfer learning can be helpful for dialect identification.
Italian Language and Dialect Identification and Regional French Variety Detection using Adaptive Naive Bayes
Authors: Tommi Jauhiainen, Heidi Jauhiainen, Krister Lindén
Date: 2022-10-01 00:00:00
Proceeding: VarDial (COLING) 2022 10
Tasks: Dialect Identification, Language Identification, Position

This article describes the language identification approach used by the SUKI team in the Identification of Languages and Dialects of Italy and the French Cross-Domain Dialect Identification shared tasks organized as part of the VarDial workshop 2022. We describe some experiments and the preprocessing techniques we used for the training data in preparation for the shared task submissions, which are also discussed. Our Naive Bayes-based adaptive system reached the first position in Italian language identification and came second in the French variety identification task.
IDN-Sum: A New Dataset for Interactive Digital Narrative Extractive Text Summarisation
Authors: Ashwathy T. Revi, Stuart E. Middleton, David E. Millard
Date: 2022-10-01 00:00:00
Proceeding: COLING (CreativeSumm) 2022 10
Tasks: Extractive Text Summarization, Text Summarization

Summarizing Interactive Digital Narratives (IDN) presents some unique challenges to existing text summarization models especially around capturing interactive elements in addition to important plot points. In this paper, we describe the first IDN dataset (IDN-Sum) designed specifically for training and testing IDN text summarization algorithms. Our dataset is generated using random playthroughs of 8 IDN episodes, taken from 2 different IDN games, and consists of 10,000 documents. Playthrough documents are annotated through automatic alignment with fan-sourced summaries using a commonly used alignment algorithm. We also report and discuss results from experiments applying common baseline extractive text summarization algorithms to this dataset. Qualitative analysis of the results reveals shortcomings in common annotation approaches and evaluation methods when applied to narrative and interactive narrative datasets. The dataset is released as open source for future researchers to train and test their own approaches for IDN text.
Summarization of Long Input Texts Using Multi-Layer Neural Network
Authors: Niladri Chatterjee, Aadyant Khatri, Raksha Agarwal
Date: 2022-10-01 00:00:00
Proceeding: COLING (CreativeSumm) 2022 10

This paper describes the architecture of a novel Multi-Layer Long Text Summarizer (MLLTS) system proposed for the task of creative writing summarization. Typically, such writings are very long, often spanning over 100 pages. Summarizers available online are either not equipped enough to handle long texts, or even if they are able to generate the summary, the quality is poor. The proposed MLLTS system handles the difficulty by splitting the text into several parts. Each part is then subjected to different existing summarizers. A multilayer network is constructed by establishing linkages between the different parts. During training phases, several hyperparameters are fine-tuned. The system achieved very good ROUGE scores on the test data supplied for the contest.
COLING 2022 Shared Task: LED Finteuning and Recursive Summary Generation for Automatic Summarization of Chapters from Novels
Authors: Prerna Kashyap
Date: 2022-10-01 00:00:00
Proceeding: COLING (CreativeSumm) 2022 10
Tasks: 16k

We present the results of the Workshop on Automatic Summarization for Creative Writing 2022 Shared Task on summarization of chapters from novels. In this task, we finetune a pretrained transformer model for long documents called LongformerEncoderDecoder which supports seq2seq tasks for long inputs which can be up to 16k tokens in length. We use the Booksum dataset for longform narrative summarization for training and validation, which maps chapters from novels, plays and stories to highly abstractive human written summaries. We use a summary of summaries approach to generate the final summaries for the blind test set, in which we recursively divide the text into paragraphs, summarize them, concatenate all resultant summaries and repeat this process until either a specified summary length is reached or there is no significant change in summary length in consecutive iterations. Our best model achieves a ROUGE-1 F-1 score of 29.75, a ROUGE-2 F-1 score of 7.89 and a BERT F-1 score of 54.10 on the shared task blind test dataset.
TEAM UFAL @ CreativeSumm 2022: BART and SamSum based few-shot approach for creative Summarization
Authors: Rishu Kumar, Rudolf Rosa
Date: 2022-10-01 00:00:00
Proceeding: COLING (CreativeSumm) 2022 10
Tasks: Few-Shot Learning

This system description paper details TEAM UFAL’s approach for the SummScreen, TVMegasite subtask of the CreativeSumm shared task. The subtask deals with creating summaries for dialogues from TV Soap operas. We utilized BART based pre-trained model fine-tuned on SamSum dialouge summarization dataset. Few examples from AutoMin dataset and the dataset provided by the organizers were also inserted into the data as a few-shot learning objective. The additional data was manually broken into chunks based on different boundaries in summary and the dialogue file. For inference we choose a similar strategy as the top-performing team at AutoMin 2021, where the data is split into chunks, either on [SCENE_CHANGE] or exceeding a pre-defined token length, to accommodate the maximum token possible in the pre-trained model for one example. The final training strategy was chosen based on how natural the responses looked instead of how well the model performed on an automated evaluation metrics such as ROGUE.
Long Input Dialogue Summarization with Sketch Supervision for Summarization of Primetime Television Transcripts
Authors: Nataliia Kees, Thien Nguyen, Tobias Eder, Georg Groh
Date: 2022-10-01 00:00:00
Proceeding: COLING (CreativeSumm) 2022 10

This paper presents our entry to the CreativeSumm 2022 shared task. Specifically tackling the problem of prime-time television screenplay summarization based on the SummScreen Forever Dreaming dataset. Our approach utilizes extended Longformers combined with sketch supervision including categories specifically for scene descriptions. Our system was able to produce the shortest summaries out of all submissions. While some problems with factual consistency still remain, the system was scoring highest among competitors in the ROUGE and BERTScore evaluation categories.
AMRTVSumm: AMR-augmented Hierarchical Network for TV Transcript Summarization
Authors: Yilun Hua, Zhaoyuan Deng, Zhijie Xu
Date: 2022-10-01 00:00:00
Proceeding: COLING (CreativeSumm) 2022 10
Tasks: Abstract Meaning Representation, Decoder

This paper describes our AMRTVSumm system for the SummScreen datasets in the Automatic Summarization for Creative Writing shared task (Creative-Summ 2022). In order to capture the complicated entity interactions and dialogue structures in transcripts of TV series, we introduce a new Abstract Meaning Representation (AMR) (Banarescu et al., 2013), particularly designed to represent individual scenes in an episode. We also propose a new cross-level cross-attention mechanism to incorporate these scene AMRs into a hierarchical encoder-decoder baseline. On both the ForeverDreaming and TVMegaSite datasets of SummScreen, our system consistently outperforms the hierarchical transformer baseline. Compared with the state-of-the-art DialogLM (Zhong et al., 2021), our system still has a lower performance primarily because it is pretrained only on out-of-domain news data, unlike DialogLM, which uses extensive in-domain pretraining on dialogue and TV show data. Overall, our work suggests a promising direction to capture complicated long dialogue structures through graph representations and the need to combine graph representations with powerful pretrained language models.
Automatic Summarization for Creative Writing: BART based Pipeline Method for Generating Summary of Movie Scripts
Authors: Aditya Upadhyay, Nidhir Bhavsar, Aakash Bhatnagar, Muskaan Singh, Petr Motlicek
Date: 2022-10-01 00:00:00
Proceeding: COLING (CreativeSumm) 2022 10
Tasks: Denoising

This paper documents our approach for the Creative-Summ 2022 shared task for Automatic Summarization of Creative Writing. For this purpose, we develop an automatic summarization pipeline where we leverage a denoising autoencoder for pretraining sequence-to-sequence models and fine-tune it on a large-scale abstractive screenplay summarization dataset to summarize TV transcripts from primetime shows. Our pipeline divides the input transcript into smaller conversational blocks, removes redundant text, summarises the conversational blocks, obtains the block-wise summaries, cleans, structures, and then integrates the summaries to create the meeting minutes. Our proposed system achieves some of the best scores across multiple metrics(lexical, semantical) in the Creative-Summ shared task.
The CreativeSumm 2022 Shared Task: A Two-Stage Summarization Model using Scene Attributes
Authors: Eunchong Kim, Taewoo Yoo, Gunhee Cho, Suyoung Bae, Yun-Gyung Cheong
Date: 2022-10-01 00:00:00
Proceeding: COLING (CreativeSumm) 2022 10
Tasks: Extractive Summarization

In this paper, we describe our work for the CreativeSumm 2022 Shared Task, Automatic Summarization for Creative Writing. The task is to summarize movie scripts, which is challenging due to their long length and complex format. To tackle this problem, we present a two-stage summarization approach using both the abstractive and an extractive summarization methods. In addition, we preprocess the script to enhance summarization performance. The results of our experiment demonstrate that the presented approach outperforms baseline models in terms of standard summarization evaluation metrics.
Two-Stage Movie Script Summarization: An Efficient Method For Low-Resource Long Document Summarization
Authors: Dongqi Pu, Xudong Hong, Pin-Jie Lin, Ernie Chang, Vera Demberg
Date: 2022-10-01 00:00:00
Proceeding: COLING (CreativeSumm) 2022 10
Tasks: 8k, Decoder, Document Summarization

The Creative Summarization Shared Task at COLING 2022 aspires to generate summaries given long-form texts from creative writing. This paper presents the system architecture and the results of our participation in the Scriptbase track that focuses on generating movie plots given movie scripts. The core innovation in our model employs a two-stage hierarchical architecture for movie script summarization. In the first stage, a heuristic extraction method is applied to extract actions and essential dialogues, which reduces the average length of input movie scripts by 66% from about 24K to 8K tokens. In the second stage, a state-of-the-art encoder-decoder model, Longformer-Encoder-Decoder (LED), is trained with effective fine-tuning methods, BitFit and NoisyTune. Evaluations on the unseen test set indicate that our system outperforms both zero-shot LED baselines as well as other participants on various automatic metrics and ranks 1st in the Scriptbase track.