pwc-archive/papers-with-abstracts huggingface dataset:
A Method to Automatically Identify Diachronic Variation in Collocations.
Authors: Marcos Garcia, Marcos Garc{\'\i}a Salido
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8

This paper introduces a novel method to track collocational variations in diachronic corpora that can identify several changes undergone by these phraseological combinations and to propose alternative solutions found in later periods. The strategy consists of extracting syntactically-related candidates of collocations and ranking them using statistical association measures. Then, starting from the first period of the corpus, the system tracks each combination over time, verifying different types of historical variation such as the loss of one or both lemmas, the disappearance of the collocation, or its diachronic frequency trend. Using a distributional semantics strategy, it also proposes other linguistic structures which convey similar meanings to those extinct collocations. A case study on historical corpora of Portuguese and Spanish shows that the system speeds up and facilitates the finding of some diachronic changes and phraseological shifts that are harder to identify without using automated methods.
Written on Leaves or in Stones?: Computational Evidence for the Era of Authorship of Old Thai Prose
Authors: Attapol Rutherford, Santhawat Thanyawong
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8

We aim to provide computational evidence for the era of authorship of two important old Thai texts: Traiphumikatha and Pumratchatham. The era of authorship of these two books is still an ongoing debate among Thai literature scholars. Analysis of old Thai texts present a challenge for standard natural language processing techniques, due to the lack of corpora necessary for building old Thai word and syllable segmentation. We propose an accurate and interpretable model to classify each segment as one of the three eras of authorship (Sukhothai, Ayuddhya, or Rattanakosin) without sophisticated linguistic preprocessing. Contrary to previous hypotheses, our model suggests that both books were written during the Sukhothai era. Moreover, the second half of the Pumratchtham is uncharacteristic of the Sukhothai era, which may have confounded literary scholars in the past. Further, our model reveals that the most indicative linguistic changes stem from unidirectional grammaticalized words and polyfunctional words, which show up as most dominant features in the model.
Identifying Temporal Trends Based on Perplexity and Clustering: Are We Looking at Language Change?
Authors: Sidsel Boldsen, Manex Agirrezabal, Patrizia Paggio
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Clustering

In this work we propose a data-driven methodology for identifying temporal trends in a corpus of medieval charters. We have used perplexities derived from RNNs as a distance measure between documents and then, performed clustering on those distances. We argue that perplexities calculated by such language models are representative of temporal trends. The clusters produced using the K-Means algorithm give an insight of the differences in language in different time periods at least partly due to language change. We suggest that the temporal distribution of the individual clusters might provide a more nuanced picture of temporal trends compared to discrete bins, thus providing better results when used in a classification task.
Predicting Historical Phonetic Features using Deep Neural Networks: A Case Study of the Phonetic System of Proto-Indo-European
Authors: Frederik Hartmann
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8

Traditional historical linguistics lacks the possibility to empirically assess its assumptions regarding the phonetic systems of past languages and language stages since most current methods rely on comparative tools to gain insights into phonetic features of sounds in proto- or ancestor languages. The paper at hand presents a computational method based on deep neural networks to predict phonetic features of historical sounds where the exact quality is unknown and to test the overall coherence of reconstructed historical phonetic features. The method utilizes the principles of coarticulation, local predictability and statistical phonological constraints to predict phonetic features by the features of their immediate phonetic environment. The validity of this method will be assessed using New High German phonetic data and its specific application to diachronic linguistics will be demonstrated in a case study of the phonetic system Proto-Indo-European.
ParHistVis: Visualization of Parallel Multilingual Historical Data
Authors: Aikaterini-Lida Kalouli, Rebecca Kehlbeck, Rita Sevastjanova, Katharina Kaiser, Georg A. Kaiser, Miriam Butt
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8

The study of language change through parallel corpora can be advantageous for the analysis of complex interactions between time, text domain and language. Often, those advantages cannot be fully exploited due to the sparse but high-dimensional nature of such historical data. To tackle this challenge, we introduce ParHistVis: a novel, free, easy-to-use, interactive visualization tool for parallel, multilingual, diachronic and synchronic linguistic data. We illustrate the suitability of the components of the tool based on a use case of word order change in Romance wh-interrogatives.
DiaHClust: an Iterative Hierarchical Clustering Approach for Identifying Stages in Language Change
Authors: Christin Sch{\"a}tzle, Hannah Booth
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Clustering

Language change is often assessed against a set of pre-determined time periods in order to be able to trace its diachronic trajectory. This is problematic, since a pre-determined periodization might obscure significant developments and lead to false assumptions about the data. Moreover, these time periods can be based on factors which are either arbitrary or non-linguistic, e.g., dividing the corpus data into equidistant stages or taking into account language-external events. Addressing this problem, in this paper we present a data-driven approach to periodization: {`}DiaHClust{'}. DiaHClust is based on iterative hierarchical clustering and offers a multi-layered perspective on change from text-level to broader time periods. We demonstrate the usefulness of DiaHClust via a case study investigating syntactic change in Icelandic, modelling the syntactic system of the language in terms of vectors of syntactic change.
Treat the Word As a Whole or Look Inside? Subword Embeddings Model Language Change and Typology
Authors: Yang Xu, Jiasheng Zhang, David Reitter
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8

We use a variant of word embedding model that incorporates subword information to characterize the degree of compositionality in lexical semantics. Our models reveal some interesting yet contrastive patterns of long-term change in multiple languages: Indo-European languages put more weight on subword units in newer words, while conversely Chinese puts less weights on the subwords, but more weight on the word as a whole. Our method provides novel evidence and methodology that enriches existing theories in evolutionary linguistics. The resulting word vectors also has decent performance in NLP-related tasks.
Times Are Changing: Investigating the Pace of Language Change in Diachronic Word Embeddings
Authors: Br, Stephanie l, David Lassner
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Diachronic Word Embeddings, Word Embeddings

We propose Word Embedding Networks, a novel method that is able to learn word embeddings of individual data slices while simultaneously aligning and ordering them without feeding temporal information a priori to the model. This gives us the opportunity to analyse the dynamics in word embeddings on a large scale in a purely data-driven manner. In experiments on two different newspaper corpora, the New York Times (English) and die Zeit (German), we were able to show that time actually determines the dynamics of semantic change. However, there is by no means a uniform evolution, but instead times of faster and times of slower change.
The Rationality of Semantic Change
Authors: Omer Korat
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8

This study investigates the mutual effects over time of semantically related function words on each other{'}s distribution over syntactic environments. Words that can have the same meaning are observed to have opposite trends of change in frequency across different syntactic structures which correspond to the shared meaning. This phenomenon is demonstrated to have a rational basis: it increases communicative efficiency by prioritizing words differently in the environments on which they compete.
Studying Laws of Semantic Divergence across Languages using Cognate Sets
Authors: Ana Uban, Alina Maria Ciobanu, Liviu P. Dinu
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8

Semantic divergence in related languages is a key concern of historical linguistics. Intra-lingual semantic shift has been previously studied in computational linguistics, but this can only provide a limited picture of the evolution of word meanings, which often develop in a multilingual environment. In this paper we investigate semantic change across languages by measuring the semantic distance of cognate words in multiple languages. By comparing current meanings of cognates in different languages, we hope to uncover information about their previous meanings, and about how they diverged in their respective languages from their common original etymon. We further study the properties of their semantic divergence, by analyzing how the features of words such as frequency and polysemy are related to the divergence in their meaning, and thus make the first steps towards formulating laws of cross-lingual semantic change.
Grammar and Meaning: Analysing the Topology of Diachronic Word Embeddings
Authors: Yuri Bizzoni, Stefania Degaetano-Ortlieb, Katrin Menzel, Pauline Krielke, Elke Teich
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Clustering, Diachronic Word Embeddings, Word Embeddings

The paper showcases the application of word embeddings to change in language use in the domain of science, focusing on the Late Modern English period (17-19th century). Historically, this is the period in which many registers of English developed, including the language of science. Our overarching interest is the linguistic development of scientific writing to a distinctive (group of) register(s). A register is marked not only by the choice of lexical words (discourse domain) but crucially by grammatical choices which indicate style. The focus of the paper is on the latter, tracing words with primarily grammatical functions (function words and some selected, poly-functional word forms) diachronically. To this end, we combine diachronic word embeddings with appropriate visualization and exploratory techniques such as clustering and relative entropy for meaningful aggregation of data and diachronic comparison.
Spatio-Temporal Prediction of Dialectal Variant Usage
Authors: P{\'e}ter Jeszenszky, Panote Siriaraya, Philipp Stoeckle, Adam Jatowt
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Diversity, Prediction, Survey

The distribution of most dialectal variants have not only spatial but also temporal patterns. Based on the {`}apparent time hypothesis{'}, much of dialect change is happening through younger speakers accepting innovations. Thus, synchronic diversity can be interpreted diachronically. With the assumption of the {`}contact effect{'}, i.e. contact possibility (contact and isolation) between speaker communities being responsible for language change, and the apparent time hypothesis, we aim to predict the usage of dialectal variants. In this paper we model the contact possibility based on two of the most important factors in sociolinguistics to be affecting language change: age and distance. The first steps of the approach involve modeling contact possibility using a logistic predictor, taking the age of respondents into account. We test the \textit{global}, and the \textit{local} role of age for variation where the local level means spatial subsets around each survey site, chosen based on \textit{k} nearest neighbors. The prediction approach is tested on Swiss German syntactic survey data, featuring multiple respondents from different age cohorts at survey sites. The results show the relative success of the logistic prediction approach and the limitations of the method, therefore further proposals are made to develop the methodology.
Measuring Diachronic Evolution of Evaluative Adjectives with Word Embeddings: the Case for English, Norwegian, and Russian
Authors: Julia Rodina, Baksh, Daria aeva, Vadim Fomin, Andrey Kutuzov, Samia Touileb, Erik Velldal
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Word Embeddings

We measure the intensity of diachronic semantic shifts in adjectives in English, Norwegian and Russian across 5 decades. This is done in order to test the hypothesis that evaluative adjectives are more prone to temporal semantic change. To this end, 6 different methods of quantifying semantic change are used. Frequency-controlled experimental results show that, depending on the particular method, evaluative adjectives either do not differ from other types of adjectives in terms of semantic change or appear to actually be less prone to shifting (particularly, to {`}jitter{'}-type shifting). Thus, in spite of many well-known examples of semantically changing evaluative adjectives (like {`}terrific{'} or {`}incredible{'}), it seems that such cases are not specific to this particular type of words.
Semantic Change in the Language of UK Parliamentary Debates
Authors: Gavin Abercrombie, Riza Batista-Navarro
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Word Embeddings

We investigate changes in the meanings of words used in the UK Parliament across two different epochs. We use word embeddings to explore changes in the distribution of words of interest and uncover words that appear to have undergone semantic transformation in the intervening period, and explore different ways of obtaining target words for this purpose. We find that semantic changes are generally in line with those found in other corpora, and little evidence that parliamentary language is more static than general English. It also seems that words with senses that have been recorded in the dictionary as having fallen into disuse do not undergo semantic changes in this domain.
Conceptual Change and Distributional Semantic Models: an Exploratory Study on Pitfalls and Possibilities
Authors: Pia Sommerauer, Antske Fokkens
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8

Studying conceptual change using embedding models has become increasingly popular in the Digital Humanities community while critical observations about them have received less attention. This paper investigates what the impact of known pitfalls can be on the conclusions drawn in a digital humanities study through the use case of {``}Racism{''}. In addition, we suggest an approach for modeling a complex concept in terms of words and relations representative of the conceptual system. Our results show that different models created from the same data yield different results, but also indicate that using different model architectures, comparing different corpora and comparing to control words and relations can help to identify which results are solid and which may be due to artefact. We propose guidelines to conduct similar studies, but also note that more work is needed to fully understand how we can distinguish artefacts from actual conceptual changes.
Towards Automatic Variant Analysis of Ancient Devotional Texts
Authors: Amir Hazem, B{\'e}atrice Daille, Dominique Stutzmann, Jacob Currie, Christine Jacquin
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8

We address in this paper the issue of text reuse in liturgical manuscripts of the middle ages. More specifically, we study variant readings of the Obsecro Te prayer, part of the devotional Books of Hours often used by Christians as guidance for their daily prayers. We aim at automatically extracting and categorising pairs of words and expressions that exhibit variant relations. For this purpose, we adopt a linguistic classification that allows to better characterize the variants than edit operations. Then, we study the evolution of Obsecro Te texts from a temporal and geographical axis. Finally, we contrast several unsupervised state-of-the-art approaches for the automatic extraction of Obsecro Te variants. Based on the manual observation of 772 Obsecro Te copies which show more than 21,000 variants, we show that the proposed methodology is helpful for an automatic study of variants and may serve as basis to analyze and to depict useful information from devotional texts.
Understanding the Evolution of Circular Economy through Language Change
Authors: Sampriti Mahanty, Frank Boons, H, Julia l, Riza Theresa Batista-Navarro
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8

In this study, we propose to focus on understanding the evolution of a specific scientific concept{---}that of Circular Economy (CE){---}by analysing how the language used in academic discussions has changed semantically. It is worth noting that the meaning and central theme of this concept has remained the same; however, we hypothesise that it has undergone semantic change by way of additional layers being added to the concept. We have shown that semantic change in language is a reflection of shifts in scientific ideas, which in turn help explain the evolution of a concept. Focusing on the CE concept, our analysis demonstrated that the change over time in the language used in academic discussions of CE is indicative of the way in which the concept evolved and expanded.
Gaussian Process Models of Sound Change in Indo-Aryan Dialectology
Authors: Chundra Cathcart
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Gaussian Processes

This paper proposes a Gaussian Process model of sound change targeted toward questions in Indo-Aryan dialectology. Gaussian Processes (GPs) provide a flexible means of expressing covariance between outcomes, and can be extended to a wide variety of probability distributions. We find that GP models fare better in terms of some key posterior predictive checks than models that do not express covariance between sound changes, and outline directions for future work.
Modeling a Historical Variety of a Low-Resource Language: Language Contact Effects in the Verbal Cluster of Early-Modern Frisian
Authors: Jelke Bloem, Arjen Versloot, Fred Weerman
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8

Certain phenomena of interest to linguists mainly occur in low-resource languages, such as contact-induced language change. We show that it is possible to study contact-induced language change computationally in a historical variety of a low-resource language, Early-Modern Frisian, by creating a model using features that were established to be relevant in a closely related language, modern Dutch. This allows us to test two hypotheses on two types of language contact that may have taken place between Frisian and Dutch during this time. Our model shows that Frisian verb cluster word orders are associated with different context features than Dutch verb orders, supporting the {`}learned borrowing{'} hypothesis.
Visualizing Linguistic Change as Dimension Interactions
Authors: Christin Sch{\"a}tzle, Frederik L. Dennig, Michael Blumenschein, Daniel A. Keim, Miriam Butt
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8

Historical change typically is the result of complex interactions between several linguistic factors. Identifying the relevant factors and understanding how they interact across the temporal dimension is the core remit of historical linguistics. With respect to corpus work, this entails a separate annotation, extraction and painstaking pair-wise comparison of the relevant bits of information. This paper presents a significant extension of HistoBankVis, a multilayer visualization system which allows a fast and interactive exploration of complex linguistic data. Linguistic factors can be understood as data dimensions which show complex interrelationships. We model these relationships with the Parallel Sets technique. We demonstrate the powerful potential of this technique by applying the system to understanding the interaction of case, grammatical relations and word order in the history of Icelandic.
The Meaning of ``Most'' for Visual Question Answering Models
Authors: Alex Kuhnle, er, Ann Copestake
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

The correct interpretation of quantifier statements in the context of a visual scene requires non-trivial inference mechanisms. For the example of {``}most{''}, we discuss two strategies which rely on fundamentally different cognitive concepts. Our aim is to identify what strategy deep learning models for visual question answering learn when trained on such questions. To this end, we carefully design data to replicate experiments from psycholinguistics where the same question was investigated for humans. Focusing on the FiLM visual question answering model, our experiments indicate that a form of approximate number system emerges whose performance declines with more difficult scenes as predicted by Weber{'}s law. Moreover, we identify confounding factors, like spatial arrangement of the scene, which impede the effectiveness of this system.
Detecting Political Bias in News Articles Using Headline Attention
Authors: Rama Rohit Reddy Gangula, Suma Reddy Duggenpudi, Radhika Mamidi
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8
Tasks: Articles, Bias Detection

Language is a powerful tool which can be used to state the facts as well as express our views and perceptions. Most of the times, we find a subtle bias towards or against someone or something. When it comes to politics, media houses and journalists are known to create bias by shrewd means such as misinterpreting reality and distorting viewpoints towards some parties. This misinterpretation on a large scale can lead to the production of biased news and conspiracy theories. Automating bias detection in newspaper articles could be a good challenge for research in NLP. We proposed a headline attention network for this bias detection. Our model has two distinctive characteristics: (i) it has a structure that mirrors a person{'}s way of reading a news article (ii) it has attention mechanism applied on the article based on its headline, enabling it to attend to more critical content to predict bias. As the required datasets were not available, we created a dataset comprising of 1329 news articles collected from various Telugu newspapers and marked them for bias towards a particular political party. The experiments conducted on it demonstrated that our model outperforms various baseline methods by a substantial margin.
Learning the Dyck Language with Attention-based Seq2Seq Models
Authors: Xiang Yu, Ngoc Thang Vu, Jonas Kuhn
Date: 2019-08-01 00:00:00
Proceeding: WS 2019 8

The generalized Dyck language has been used to analyze the ability of Recurrent Neural Networks (RNNs) to learn context-free grammars (CFGs). Recent studies draw conflicting conclusions on their performance, especially regarding the generalizability of the models with respect to the depth of recursion. In this paper, we revisit several common models and experimental settings, discuss the potential problems of the tasks and analyses. Furthermore, we explore the use of attention mechanisms within the seq2seq framework to learn the Dyck language, which could compensate for the limited encoding ability of RNNs. Our findings reveal that attention mechanisms still cannot truly generalize over the recursion depth, although they perform much better than other models on the closing bracket tagging task. Moreover, this also suggests that this commonly used task is not sufficient to test a model{'}s understanding of CFGs.