pwc-archive/papers-with-abstracts huggingface dataset:
Deception detection in Russian texts
Authors: Olga Litvinova, Pavel Seredin, Tatiana Litvinova, John Lyell
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Deception Detection, text-classification, Text Classification

Humans are known to detect deception in speech randomly and it is therefore important to develop tools to enable them to detect deception. The problem of deception detection has been studied for a significant amount of time, however the last 10-15 years have seen methods of computational linguistics being employed. Texts are processed using different NLP tools and then classified as deceptive/truthful using machine learning methods. While most research has been performed for English, Slavic languages have never been a focus of detection deception studies. The paper deals with deception detection in Russian narratives. It employs a specially designed corpus of truthful and deceptive texts on the same topic from each respondent, N = 113. The texts were processed using Linguistic Inquiry and Word Count software that is used in most studies of text-based deception detection. The list of parameters computed using the software was expanded due to the designed users{'} dictionaries. A variety of text classification methods was employed. The accuracy of the model was found to depend on the author{'}s gender and text type (deceptive/truthful).
A Computational Model of Human Preferences for Pronoun Resolution
Authors: Olga Seminck, Pascal Amsili
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Coreference Resolution

We present a cognitive computational model of pronoun resolution that reproduces the human interpretation preferences of the Subject Assignment Strategy and the Parallel Function Strategy. Our model relies on a probabilistic pronoun resolution system trained on corpus data. Factors influencing pronoun resolution are represented as features weighted by their relative importance. The importance the model gives to the preferences is in line with psycholinguistic studies. We demonstrate the cognitive plausibility of the model by running it on experimental items and simulating antecedent choice and reading times of human participants. Our model can be used as a new means to study pronoun resolution, because it captures the interaction of preferences.
Automatic Extraction of News Values from Headline Text
Authors: Alicja Piotrkowicz, Vania Dimitrova, Katja Markert
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Articles, Keyword Spotting, Recommendation Systems

Headlines play a crucial role in attracting audiences{'} attention to online artefacts (e.g. news articles, videos, blogs). The ability to carry out an automatic, large-scale analysis of headlines is critical to facilitate the selection and prioritisation of a large volume of digital content. In journalism studies news content has been extensively studied using manually annotated news values - factors used implicitly and explicitly when making decisions on the selection and prioritisation of news items. This paper presents the first attempt at a fully automatic extraction of news values from headline text. The news values extraction methods are applied on a large headlines corpus collected from The Guardian, and evaluated by comparing it with a manually annotated gold standard. A crowdsourcing survey indicates that news values affect people{'}s decisions to click on a headline, supporting the need for an automatic news values detection.
Assessing Convincingness of Arguments in Online Debates with Limited Number of Features
Authors: Lisa Andreevna Chalaguine, Claudia Schulz
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Argument Mining

We propose a new method in the field of argument analysis in social media to determining convincingness of arguments in online debates, following previous research by Habernal and Gurevych (2016). Rather than using argument specific feature values, we measure feature values relative to the average value in the debate, allowing us to determine argument convincingness with fewer features (between 5 and 35) than normally used for natural language processing tasks. We use a simple forward-feeding neural network for this task and achieve an accuracy of 0.77 which is comparable to the accuracy obtained using 64k features and a support vector machine by Habernal and Gurevych.
Zipf's and Benford's laws in Twitter hashtags
Authors: Jos{\'e} Alberto P{\'e}rez Meli{\'a}n, J. Alberto Conejero, C{\`e}sar Ferri Ram{\'\i}rez
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4

Social networks have transformed communication dramatically in recent years through the rise of new platforms and the development of a new language of communication. This landscape requires new forms to describe and predict the behaviour of users in networks. This paper presents an analysis of the frequency distribution of hashtag popularity in Twitter conversations. Our objective is to determine if these frequency distribution follow some well-known frequency distribution that many real-life sets of numerical data satisfy. In particular, we study the similarity of frequency distribution of hashtag popularity with respect to Zipf{'}s law, an empirical law referring to the phenomenon that many types of data in social sciences can be approximated with a Zipfian distribution. Additionally, we also analyse Benford{'}s law, is a special case of Zipf{'}s law, a common pattern about the frequency distribution of leading digits. In order to compute correctly the frequency distribution of hashtag popularity, we need to correct many spelling errors that Twitter{'}s users introduce. For this purpose we introduce a new filter to correct hashtag mistake based on string distances. The experiments obtained employing datasets of Twitter streams generated under controlled conditions show that Benford{'}s law and Zipf{'}s law can be used to model hashtag frequency distribution.
A Multi-aspect Analysis of Automatic Essay Scoring for Brazilian Portuguese
Authors: Evelin Amorim, Adriano Veloso
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4

Several methods for automatic essay scoring (AES) for English language have been proposed. However, multi-aspect AES systems for other languages are unusual. Therefore, we propose a multi-aspect AES system to apply on a dataset of Brazilian Portuguese essays, which human experts evaluated according to five aspects defined by Brazilian Government to the National Exam to High School Student (ENEM). These aspects are skills that student must master and every skill is assessed apart from each other. Besides the prediction of each aspect, the feature analysis also was performed for each aspect. The AES system proposed employs several features already employed by AES systems for English language. Our results show that predictions for some aspects performed well with the features we employed, while predictions for other aspects performed poorly. Also, it is possible to note the difference between the five aspects in the detailed feature analysis we performed. Besides these contributions, the eight millions of enrollments every year for ENEM raise some challenge issues for future directions in our research.
Literal or idiomatic? Identifying the reading of single occurrences of German multiword expressions using word embeddings
Authors: Rafael Ehren
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Machine Translation, Semantic Similarity, Semantic Textual Similarity, Word Embeddings

Non-compositional multiword expressions (MWEs) still pose serious issues for a variety of natural language processing tasks and their ubiquity makes it impossible to get around methods which automatically identify these kind of MWEs. The method presented in this paper was inspired by Sporleder and Li (2009) and is able to discriminate between the literal and non-literal use of an MWE in an unsupervised way. It is based on the assumption that words in a text form cohesive units. If the cohesion of these units is weakened by an expression, it is classified as literal, and otherwise as idiomatic. While Sporleder an Li used \textit{Normalized Google Distance} to modell semantic similarity, the present work examines the use of a variety of different word embeddings.
Evaluating the Reliability and Interaction of Recursively Used Feature Classes for Terminology Extraction
Authors: Anna H{\"a}tty, Michael Dorna, Sabine Schulte im Walde
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: BIG-bench Machine Learning, Classification, General Classification, Machine Translation, Text Generation

Feature design and selection is a crucial aspect when treating terminology extraction as a machine learning classification problem. We designed feature classes which characterize different properties of terms based on distributions, and propose a new feature class for components of term candidates. By using random forests, we infer optimal features which are later used to build decision tree classifiers. We evaluate our method using the ACL RD-TEC dataset. We demonstrate the importance of the novel feature class for downgrading termhood which exploits properties of term components. Furthermore, our classification suggests that the identification of reliable term candidates should be performed successively, rather than just once.
Please Clap: Modeling Applause in Campaign Speeches
Authors: Jon Gillick, David Bamman
Date: 2018-06-01 00:00:00
Proceeding: NAACL 2018 6

This work examines the rhetorical techniques that speakers employ during political campaigns. We introduce a new corpus of speeches from campaign events in the months leading up to the 2016 U.S. presidential election and develop new models for predicting moments of audience applause. In contrast to existing datasets, we tackle the challenge of working with transcripts that derive from uncorrected closed captioning, using associated audio recordings to automatically extract and align labels for instances of audience applause. In prediction experiments, we find that lexical features carry the most information, but that a variety of features are predictive, including prosody, long-term contextual dependencies, and theoretically motivated features designed to capture rhetorical techniques.
Automatic Focus Annotation: Bringing Formal Pragmatics Alive in Analyzing the Information Structure of Authentic Data
Authors: Ramon Ziai, Detmar Meurers
Date: 2018-06-01 00:00:00
Proceeding: NAACL 2018 6
Tasks: General Classification

Analyzing language in context, both from a theoretical and from a computational perspective, is receiving increased interest. Complementing the research in linguistics on discourse and information structure, in computational linguistics identifying discourse concepts was also shown to improve the performance of certain applications, for example, Short Answer Assessment systems (Ziai and Meurers, 2014). Building on the research that established detailed annotation guidelines for manual annotation of information structural concepts for written (Dipper et al., 2007; Ziai and Meurers, 2014) and spoken language data (Calhoun et al., 2010), this paper presents the first approach automating the analysis of focus in authentic written data. Our classification approach combines a range of lexical, syntactic, and semantic features to achieve an accuracy of 78.1{\%} for identifying focus.
A Melody-Conditioned Lyrics Language Model
Authors: Kento Watanabe, Yuichiroh Matsubayashi, Satoru Fukayama, Masataka Goto, Kentaro Inui, Tomoyasu Nakano
Date: 2018-06-01 00:00:00
Proceeding: NAACL 2018 6
Tasks: Language Modeling, Language Modelling, model, Sentence

This paper presents a novel, data-driven language model that produces entire lyrics for a given input melody. Previously proposed models for lyrics generation suffer from the inability of capturing the relationship between lyrics and melody partly due to the unavailability of lyrics-melody aligned data. In this study, we first propose a new practical method for creating a large collection of lyrics-melody aligned data and then create a collection of 1,000 lyrics-melody pairs augmented with precise syllable-note alignments and word/sentence/paragraph boundaries. We then provide a quantitative analysis of the correlation between word/sentence/paragraph boundaries in lyrics and melodies. We then propose an RNN-based lyrics language model conditioned on a featurized melody. Experimental results show that the proposed model generates fluent lyrics while maintaining the compatibility between boundaries of lyrics and melody structures.
Natural Answer Generation with Heterogeneous Memory
Authors: Yao Fu, Yansong Feng
Date: 2018-06-01 00:00:00
Proceeding: NAACL 2018 6
Tasks: Answer Generation, Decoder, Question Answering, Sentence, Text Generation

Memory augmented encoder-decoder framework has achieved promising progress for natural language generation tasks. Such frameworks enable a decoder to retrieve from a memory during generation. However, less research has been done to take care of the memory contents from different sources, which are often of heterogeneous formats. In this work, we propose a novel attention mechanism to encourage the decoder to actively interact with the memory by taking its heterogeneity into account. Our solution attends across the generated history and memory to explicitly avoid repetition, and introduce related knowledge to enrich our generated sentences. Experiments on the answer sentence generation task show that our method can effectively explore heterogeneous memory to produce readable and meaningful answer sentences while maintaining high coverage for given answer information.
Simplification Using Paraphrases and Context-Based Lexical Substitution
Authors: Reno Kriz, Eleni Miltsakaki, Marianna Apidianaki, Chris Callison-Burch
Date: 2018-06-01 00:00:00
Proceeding: NAACL 2018 6
Tasks: Complex Word Identification, Lexical Simplification, Text Simplification

Lexical simplification involves identifying complex words or phrases that need to be simplified, and recommending simpler meaning-preserving substitutes that can be more easily understood. We propose a complex word identification (CWI) model that exploits both lexical and contextual features, and a simplification mechanism which relies on a word-embedding lexical substitution model to replace the detected complex words with simpler paraphrases. We compare our CWI and lexical simplification models to several baselines, and evaluate the performance of our simplification system against human judgments. The results show that our models are able to detect complex words with higher accuracy than other commonly used methods, and propose good simplification substitutes in context. They also highlight the limited contribution of context features for CWI, which nonetheless improve simplification compared to context-unaware models.
Automated Essay Scoring in the Presence of Biased Ratings
Authors: Evelin Amorim, Marcia Can{\c{c}}ado, Adriano Veloso
Date: 2018-06-01 00:00:00
Proceeding: NAACL 2018 6
Tasks: Automated Essay Scoring

Studies in Social Sciences have revealed that when people evaluate someone else, their evaluations often reflect their biases. As a result, rater bias may introduce highly subjective factors that make their evaluations inaccurate. This may affect automated essay scoring models in many ways, as these models are typically designed to model (potentially biased) essay raters. While there is sizeable literature on rater effects in general settings, it remains unknown how rater bias affects automated essay scoring. To this end, we present a new annotated corpus containing essays and their respective scores. Different from existing corpora, our corpus also contains comments provided by the raters in order to ground their scores. We present features to quantify rater bias based on their comments, and we found that rater bias plays an important role in automated essay scoring. We investigated the extent to which rater bias affects models based on hand-crafted features. Finally, we propose to rectify the training set by removing essays associated with potentially biased scores while learning the scoring model.
Looking Beyond the Surface: A Challenge Set for Reading Comprehension over Multiple Sentences
Authors: Daniel Khashabi, Snigdha Chaturvedi, Michael Roth, Shyam Upadhyay, Dan Roth
Date: 2018-06-01 00:00:00
Proceeding: NAACL 2018 6
Tasks: Diversity, Natural Language Inference, Question Answering, Reading Comprehension, Sentence

We present a reading comprehension challenge in which questions can only be answered by taking into account information from multiple sentences. We solicit and verify questions and answers for this challenge through a 4-step crowdsourcing experiment. Our challenge dataset contains 6,500+ questions for 1000+ paragraphs across 7 different domains (elementary school science, news, travel guides, fiction stories, etc) bringing in linguistic diversity to the texts and to the questions wordings. On a subset of our dataset, we found human solvers to achieve an F1-score of 88.1{\%}. We analyze a range of baselines, including a recent state-of-art reading comprehension system, and demonstrate the difficulty of this challenge, despite a high human performance. The dataset is the first to study multi-sentence inference at scale, with an open-ended set of question types that requires reasoning skills.
QuickEdit: Editing Text \& Translations by Crossing Words Out
Authors: David Grangier, Michael Auli
Date: 2018-06-01 00:00:00
Proceeding: NAACL 2018 6
Tasks: Machine Translation, Sentence, Translation

We propose a framework for computer-assisted text editing. It applies to translation post-editing and to paraphrasing. Our proposal relies on very simple interactions: a human editor modifies a sentence by marking tokens they would like the system to change. Our model then generates a new sentence which reformulates the initial sentence by avoiding marked words. The approach builds upon neural sequence-to-sequence modeling and introduces a neural network which takes as input a sentence along with change markers. Our model is trained on translation bitext by simulating post-edits. We demonstrate the advantage of our approach for translation post-editing through simulated post-edits. We also evaluate our model for paraphrasing through a user study.
Tempo-Lexical Context Driven Word Embedding for Cross-Session Search Task Extraction
Authors: Procheta Sen, Debasis Ganguly, Gareth Jones
Date: 2018-06-01 00:00:00
Proceeding: NAACL 2018 6
Tasks: Information Retrieval, Session Search

Task extraction is the process of identifying search intents over a set of queries potentially spanning multiple search sessions. Most existing research on task extraction has focused on identifying tasks within a single session, where the notion of a session is defined by a fixed length time window. By contrast, in this work we seek to identify tasks that span across multiple sessions. To identify tasks, we conduct a global analysis of a query log in its entirety without restricting analysis to individual temporal windows. To capture inherent task semantics, we represent queries as vectors in an abstract space. We learn the embedding of query words in this space by leveraging the temporal and lexical contexts of queries. Embedded query vectors are then clustered into tasks. Experiments demonstrate that task extraction effectiveness is improved significantly with our proposed method of query vector embedding in comparison to existing approaches that make use of documents retrieved from a collection to estimate semantic similarities between queries.
Variable Typing: Assigning Meaning to Variables in Mathematical Text
Authors: Yiannos Stathopoulos, Simon Baker, Marek Rei, Simone Teufel
Date: 2018-06-01 00:00:00
Proceeding: NAACL 2018 6
Tasks: Information Retrieval, Retrieval, Sentence

Information about the meaning of mathematical variables in text is useful in NLP/IR tasks such as symbol disambiguation, topic modeling and mathematical information retrieval (MIR). We introduce \textit{variable typing}, the task of assigning one \textit{mathematical type} (multi-word technical terms referring to mathematical concepts) to each variable in a sentence of mathematical text. As part of this work, we also introduce a new annotated data set composed of 33,524 data points extracted from scientific documents published on arXiv. Our intrinsic evaluation demonstrates that our data set is sufficient to successfully train and evaluate current classifiers from three different model architectures. The best performing model is evaluated on an extrinsic task: MIR, by producing a \textit{typed formula index}. Our results show that the best performing MIR models make use of our typed index, compared to a formula index only containing raw symbols, thereby demonstrating the usefulness of variable typing.
Comparing Constraints for Taxonomic Organization
Authors: Anne Cocos, Marianna Apidianaki, Chris Callison-Burch
Date: 2018-06-01 00:00:00
Proceeding: NAACL 2018 6
Tasks: Entity Extraction using GAN

Building a taxonomy from the ground up involves several sub-tasks: selecting terms to include, predicting semantic relations between terms, and selecting a subset of relational instances to keep, given constraints on the taxonomy graph. Methods for this final step {--} taxonomic organization {--} vary both in terms of the constraints they impose, and whether they enable discovery of synonymous terms. It is hard to isolate the impact of these factors on the quality of the resulting taxonomy because organization methods are rarely compared directly. In this paper, we present a head-to-head comparison of six taxonomic organization algorithms that vary with respect to their structural and transitivity constraints, and treatment of synonymy. We find that while transitive algorithms out-perform their non-transitive counterparts, the top-performing transitive algorithm is prohibitively slow for taxonomies with as few as 50 entities. We propose a simple modification to a non-transitive optimum branching algorithm to explicitly incorporate synonymy, resulting in a method that is substantially faster than the best transitive algorithm while giving complementary performance.
Deep Dirichlet Multinomial Regression
Authors: Adrian Benton, Mark Dredze
Date: 2018-06-01 00:00:00
Proceeding: NAACL 2018 6
Tasks: Articles, regression, Topic Models

Dirichlet Multinomial Regression (DMR) and other supervised topic models can incorporate arbitrary document-level features to inform topic priors. However, their ability to model corpora are limited by the representation and selection of these features {--} a choice the topic modeler must make. Instead, we seek models that can learn the feature representations upon which to condition topic selection. We present deep Dirichlet Multinomial Regression (dDMR), a generative topic model that simultaneously learns document feature representations and topics. We evaluate dDMR on three datasets: New York Times articles with fine-grained tags, Amazon product reviews with product images, and Reddit posts with subreddit identity. dDMR learns representations that outperform DMR and LDA according to heldout perplexity and are more effective at downstream predictive tasks as the number of topics grows. Additionally, human subjects judge dDMR topics as being more representative of associated document features. Finally, we find that supervision leads to faster convergence as compared to an LDA baseline and that dDMR{'}s model fit is less sensitive to training parameters than DMR.