pwc-archive/papers-with-abstracts huggingface dataset:
Aligning Open IE Relations and KB Relations using a Siamese Network Based on Word Embedding
Authors: Rifki Afina Putri, Giwon Hong, Sung-Hyon Myaeng
Date: 2019-05-01 00:00:00
Proceeding: WS 2019 5
Tasks: Knowledge Graphs, Open Information Extraction, Relation, Word Embeddings

Open Information Extraction (Open IE) aims at generating entity-relation-entity triples from a large amount of text, aiming at capturing key semantics of the text. Given a triple, the relation expresses the type of semantic relation between the entities. Although relations from an Open IE system are more extensible than those used in a traditional Information Extraction system and a Knowledge Base (KB) such as Knowledge Graphs, the former lacks in semantics; an Open IE relation is simply a sequence of words, whereas a KB relation has a predefined meaning. As a way to provide a meaning to an Open IE relation, we attempt to align it with one of the predefined set of relations used in a KB. Our approach is to use a Siamese network that compares two sequences of word embeddings representing an Open IE relation and a predefined KB relation. In order to make the approach practical, we automatically generate a training dataset using a distant supervision approach instead of relying on a hand-labeled dataset. Our experiment shows that the proposed method can capture the relational semantics better than the recent approaches.
Language-Agnostic Model for Aspect-Based Sentiment Analysis
Authors: Md. Shad Akhtar, Abhishek Kumar, Asif Ekbal, Chris Biemann, Pushpak Bhattacharyya
Date: 2019-05-01 00:00:00
Proceeding: WS 2019 5
Tasks: Aspect-Based Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), model, Sentiment Analysis, Sentiment Classification, Term Extraction, Word Embeddings

In this paper, we propose a language-agnostic deep neural network architecture for aspect-based sentiment analysis. The proposed approach is based on Bidirectional Long Short-Term Memory (Bi-LSTM) network, which is further assisted with extra hand-crafted features. We define three different architectures for the successful combination of word embeddings and hand-crafted features. We evaluate the proposed approach for six languages (i.e. English, Spanish, French, Dutch, German and Hindi) and two problems (i.e. aspect term extraction and aspect sentiment classification). Experiments show that the proposed model attains state-of-the-art performance in most of the settings.
Predicting Word Concreteness and Imagery
Authors: Jean Charbonnier, Christian Wartena
Date: 2019-05-01 00:00:00
Proceeding: WS 2019 5
Tasks: regression, Word Embeddings

Concreteness of words has been studied extensively in psycholinguistic literature. A number of datasets have been created with average values for perceived concreteness of words. We show that we can train a regression model on these data, using word embeddings and morphological features, that can predict these concreteness values with high accuracy. We evaluate the model on 7 publicly available datasets. Only for a few small subsets of these datasets prediction of concreteness values are found in the literature. Our results clearly outperform the reported results for these datasets.
Fast and Discriminative Semantic Embedding
Authors: Rob Koopman, Sheng-Hui Wang, Gwenn Englebienne
Date: 2019-05-01 00:00:00
Proceeding: WS 2019 5
Tasks: STS, STS Benchmark

The embedding of words and documents in compact, semantically meaningful vector spaces is a crucial part of modern information systems. Deep Learning models are powerful but their hyperparameter selection is often complex and they are expensive to train, and while pre-trained models are available, embeddings trained on general corpora are not necessarily well-suited to domain specific tasks. We propose a novel embedding method which extends random projection by weighting and projecting raw term embeddings orthogonally to an average language vector, thus improving the discriminating power of resulting term embeddings, and build more meaningful document embeddings by assigning appropriate weights to individual terms. We describe how updating the term embeddings online as we process the training data results in an extremely efficient method, in terms of both computational and memory requirements. Our experiments show highly competitive results with various state-of-the-art embedding methods on different tasks, including the standard STS benchmark and a subject prediction task, at a fraction of the computational cost.
Using Wiktionary as a resource for WSD : the case of French verbs
Authors: Vincent Segonne, C, Marie ito, Beno{\^\i}t Crabb{\'e}
Date: 2019-05-01 00:00:00
Proceeding: WS 2019 5
Tasks: Word Sense Disambiguation, Word Sense Induction

As opposed to word sense induction, word sense disambiguation (WSD) has the advantage of us-ing interpretable senses, but requires annotated data, which are quite rare for most languages except English (Miller et al. 1993; Fellbaum, 1998). In this paper, we investigate which strategy to adopt to achieve WSD for languages lacking data that was annotated specifically for the task, focusing on the particular case of verb disambiguation in French. We first study the usability of Eurosense (Bovi et al. 2017) , a multilingual corpus extracted from Europarl (Kohen, 2005) and automatically annotated with BabelNet (Navigli and Ponzetto, 2010) senses. Such a resource opened up the way to supervised and semi-supervised WSD for resourceless languages like French. While this perspective looked promising, our evaluation on French verbs was inconclusive and showed the annotated senses{'} quality was not sufficient for supervised WSD on French verbs. Instead, we propose to use Wiktionary, a collaboratively edited, multilingual online dictionary, as a resource for WSD. Wiktionary provides both sense inventory and manually sense tagged examples which can be used to train supervised and semi-supervised WSD systems. Yet, because senses{'} distribution differ in lexicographic examples found in Wiktionary with respect to natural text, we then focus on studying the impact on WSD of the training data size and senses{'} distribution. Using state-of-the art semi-supervised systems, we report experiments of Wiktionary-based WSD for French verbs, evaluated on FrenchSemEval (FSE), a new dataset of French verbs manually annotated with wiktionary senses.
A Comparison of Context-sensitive Models for Lexical Substitution
Authors: Aina Gar{\'\i} Soler, Anne Cocos, Marianna Apidianaki, Chris Callison-Burch
Date: 2019-05-01 00:00:00
Proceeding: WS 2019 5
Tasks: Word Embeddings

Word embedding representations provide good estimates of word meaning and give state-of-the art performance in semantic tasks. Embedding approaches differ as to whether and how they account for the context surrounding a word. We present a comparison of different word and context representations on the task of proposing substitutes for a target word in context (lexical substitution). We also experiment with tuning contextualized word embeddings on a dataset of sense-specific instances for each target word. We show that powerful contextualized word representations, which give high performance in several semantics-related tasks, deal less well with the subtle in-context similarity relationships needed for substitution. This is better handled by models trained with this objective in mind, where the inter-dependence between word and context representations is explicitly modeled during training.
Natural Language Semantics With Pictures: Some Language \& Vision Datasets and Potential Uses for Computational Semantics
Authors: David Schlangen
Date: 2019-05-01 00:00:00
Proceeding: WS 2019 5
Tasks: Relation

Propelling, and propelled by, the {``}deep learning revolution{''}, recent years have seen the introduction of ever larger corpora of images annotated with natural language expressions. We survey some of these corpora, taking a perspective that reverses the usual directionality, as it were, by viewing the images as semantic annotation of the natural language expressions. We discuss datasets that can be derived from the corpora, and tasks of potential interest for computational semanticists that can be defined on those. In this, we make use of relations provided by the corpora (namely, the link between expression and image, and that between two expressions linked to the same image) and relations that we can add (similarity relations between expressions, or between images). Specifically, we show that in this way we can create data that can be used to learn and evaluate lexical and compositional grounded semantics, and we show that the {``}linked to same image{''} relation tracks a semantic implication relation that is recognisable to annotators even in the absence of the linking image as evidence. Finally, as an example of possible benefits of this approach, we show that an exemplar-model-based approach to implication beats a (simple) distributional space-based one on some derived datasets, while lending itself to explainability.
Frame Identification as Categorization: Exemplars vs Prototypes in Embeddingland
Authors: Jennifer Sikos, Sebastian Pad{\'o}
Date: 2019-05-01 00:00:00
Proceeding: WS 2019 5

Categorization is a central capability of human cognition, and a number of theories have been developed to account for properties of categorization. Even though many tasks in semantics also involve categorization of some kind, theories of categorization do not play a major role in contemporary research in computational linguistics. This paper follows the idea that embedding-based models of semantics lend themselves well to being formulated in terms of classical categorization theories. The benefit is a space of model families that enables (a) the formulation of hypotheses about the impact of major design decisions, and (b) a transparent assessment of these decisions. We instantiate this idea on the task of frame-semantic frame identification. We define four models that cross two design variables: (a) the choice of prototype vs. exemplar categorization, corresponding to different degrees of generalization applied to the input; and (b) the presence vs. absence of a fine-tuning step, corresponding to generic vs. task-adaptive categorization. We find that for frame identification, generalization and task-adaptive categorization both yield substantial benefits. Our prototype-based, fine-tuned model, which combines the best choices for these variables, establishes a new state of the art in frame identification.
A Distributional Model of Affordances in Semantic Type Coercion
Authors: Stephen McGregor, Elisabetta Jezek
Date: 2019-05-01 00:00:00
Proceeding: WS 2019 5
Tasks: Vocal Bursts Type Prediction

We explore a novel application for interpreting semantic type coercions, motivated by insight into the role that perceptual affordances play in the selection of the semantic roles of artefactual nouns which are observed as arguments for verbs which would stereotypically select for objects of a different type. In order to simulate affordances, which we take to be direct perceptions of context-specific opportunities for action, we preform a distributional analysis dependency relationships between target words and their modifiers and adjuncts. We use these relationships as the basis for generating on-line transformations which project semantic subspaces in which the interpretations of coercive compositions are expected to emerge as salient word-vectors. We offer some preliminary examples of how this model operates on a dataset of sentences involving coercive interactions between verbs and objects specifically designed to evaluate this work.
Natural Language Inference with Monotonicity
Authors: Hai Hu, Qi Chen, Larry Moss
Date: 2019-05-01 00:00:00
Proceeding: WS 2019 5
Tasks: Natural Language Inference

This paper describes a working system which performs natural language inference using polarity-marked parse trees. The system handles all of the instances of monotonicity inference in the FraCaS data set. Except for the initial parse, it is entirely deterministic. It handles multi-premise arguments, and the kind of inference performed is essentially {``}logical{''}, but it goes beyond what is representable in first-order logic. In any case, the system works on surface forms rather than on representations of any kind.
Distributional Semantics in the Real World: Building Word Vector Representations from a Truth-Theoretic Model
Authors: Elizaveta Kuzmenko, Aur{\'e}lie Herbelot
Date: 2019-05-01 00:00:00
Proceeding: WS 2019 5

Distributional semantics models (DSMs) are known to produce excellent representations of word meaning, which correlate with a range of behavioural data. As lexical representations, they have been said to be fundamentally different from truth-theoretic models of semantics, where meaning is defined as a correspondence relation to the world. There are two main aspects to this difference: a) DSMs are built over corpus data which may or may not reflect {`}what is in the world{'}; b) they are built from word co-occurrences, that is, from lexical types rather than entities and sets. In this paper, we inspect the properties of a distributional model built over a set-theoretic approximation of {`}the real world{'}. To achieve this, we take the annotation a large database of images marked with objects, attributes and relations, convert the data into a representation akin to first-order logic and build several distributional models using various combinations of features. We evaluate those models over both relatedness and similarity datasets, demonstrating their effectiveness in standard evaluations. This allows us to conclude that, despite prior claims, truth-theoretic models are good candidates for building graded lexical representations of meaning.
Linguistic Information in Neural Semantic Parsing with Multiple Encoders
Authors: Rik van Noord, Antonio Toral, Johan Bos
Date: 2019-05-01 00:00:00
Proceeding: WS 2019 5
Tasks: DRS Parsing, Machine Translation, Semantic Parsing, Translation

Recently, sequence-to-sequence models have achieved impressive performance on a number of semantic parsing tasks. However, they often do not exploit available linguistic resources, while these, when employed correctly, are likely to increase performance even further. Research in neural machine translation has shown that employing this information has a lot of potential, especially when using a multi-encoder setup. We employ a range of semantic and syntactic resources to improve performance for the task of Discourse Representation Structure Parsing. We show that (i) linguistic features can be beneficial for neural semantic parsing and (ii) the best method of adding these features is by using multiple encoders.
Making Sense of Conflicting (Defeasible) Rules in the Controlled Natural Language ACE: Design of a System with Support for Existential Quantification Using Skolemization
Authors: Martin Diller, Adam Wyner, Hannes Strass
Date: 2019-05-01 00:00:00
Proceeding: WS 2019 5
Tasks: Translation

We present the design of a system for making sense of conflicting rules expressed in a fragment of the prominent controlled natural language ACE, yet extended with means of expressing defeasible rules in the form of normality assumptions. The approach we describe is ultimately based on answer-set-programming (ASP); simulating existential quantification by using skolemization in a manner resembling a translation for ASP recently formalized in the context of âˆƒ-ASP. We discuss the advantages of this approach to building on the existing ACE interface to rule-systems, ACERules.
Distributional Interaction of Concreteness and Abstractness in Verb--Noun Subcategorisation
Authors: Diego Frassinelli, Sabine Schulte im Walde
Date: 2019-05-01 00:00:00
Proceeding: WS 2019 5
Tasks: Language Identification, Object

In recent years, both cognitive and computational research has provided empirical analyses of contextual co-occurrence of concrete and abstract words, partially resulting in inconsistent pictures. In this work we provide a more fine-grained description of the distributional nature in the corpus-based interaction of verbs and nouns within subcategorisation, by investigating the concreteness of verbs and nouns that are in a specific syntactic relationship with each other, i.e., subject, direct object, and prepositional object. Overall, our experiments show consistent patterns in the distributional representation of subcategorising and subcategorised concrete and abstract words. At the same time, the studies reveal empirical evidence why contextual abstractness represents a valuable indicator for automatic non-literal language identification.
Generating a Novel Dataset of Multimodal Referring Expressions
Authors: Nikhil Krishnaswamy, James Pustejovsky
Date: 2019-05-01 00:00:00
Proceeding: WS 2019 5

Referring expressions and definite descriptions of objects in space exploit information both about object characteristics and locations. To resolve potential ambiguity, referencing strategies in language can rely on increasingly abstract concepts to distinguish an object in a given location from similar ones elsewhere, yet the description of the intended location may still be imprecise or difficult to interpret. Meanwhile, modalities such as gesture may communicate spatial information such as locations in a more concise manner. In real peer-to-peer communication, humans use language and gesture together to reference entities, with a capacity for mixing and changing modalities where needed. While recent progress in AI and human-computer interaction has created systems where a human can interact with a computer multimodally, computers often lack the capacity to intelligently mix modalities when generating referring expressions. We present a novel dataset of referring expressions combining natural language and gesture, describe its creation and evaluation, and its uses to train computational models for generating and interpreting multimodal referring expressions.
On Learning Word Embeddings From Linguistically Augmented Text Corpora
Authors: Amila Silva, Chathurika Amarathunga
Date: 2019-05-01 00:00:00
Proceeding: WS 2019 5
Tasks: Learning Word Embeddings, Word Embeddings, Word Similarity

Word embedding is a technique in Natural Language Processing (NLP) to map words into vector space representations. Since it has boosted the performance of many NLP downstream tasks, the task of learning word embeddings has been addressing significantly. Nevertheless, most of the underlying word embedding methods such as word2vec and GloVe fail to produce high-quality embeddings if the text corpus is small and sparse. This paper proposes a method to generate effective word embeddings from limited data. Through experiments, we show that our proposed model outperforms existing works for the classical word similarity task and for a domain-specific application.
Sentiment Independent Topic Detection in Rated Hospital Reviews
Authors: Christian Wartena, S, Uwe er, Christiane Patzelt
Date: 2019-05-01 00:00:00
Proceeding: WS 2019 5

We present a simple method to find topics in user reviews that accompany ratings for products or services. Standard topic analysis will perform sub-optimal on such data since the word distributions in the documents are not only determined by the topics but by the sentiment as well. We reduce the influence of the sentiment on the topic selection by adding two explicit topics, representing positive and negative sentiment. We evaluate the proposed method on a set of over 15,000 hospital reviews. We show that the proposed method, Latent Semantic Analysis with explicit word features, finds topics with a much smaller bias for sentiments than other similar methods.
Investigating the Stability of Concrete Nouns in Word Embeddings
Authors: B{\'e}n{\'e}dicte Pierrejean, Ludovic Tanguy
Date: 2019-05-01 00:00:00
Proceeding: WS 2019 5
Tasks: Word Embeddings

We know that word embeddings trained using neural-based methods (such as word2vec SGNS) are sensitive to stability problems and that across two models trained using the exact same set of parameters, the nearest neighbors of a word are likely to change. All words are not equally impacted by this internal instability and recent studies have investigated features influencing the stability of word embeddings. This stability can be seen as a clue for the reliability of the semantic representation of a word. In this work, we investigate the influence of the degree of concreteness of nouns on the stability of their semantic representation. We show that for English generic corpora, abstract words are more affected by stability problems than concrete words. We also found that to a certain extent, the difference between the degree of concreteness of a noun and its nearest neighbors can partly explain the stability or instability of its neighbors.
A Dynamic Semantics for Causal Counterfactuals
Authors: Kenneth Lai, James Pustejovsky
Date: 2019-05-01 00:00:00
Proceeding: WS 2019 5
Tasks: counterfactual, Sentence

Under the standard approach to counterfactuals, to determine the meaning of a counterfactual sentence, we consider the {``}closest{''} possible world(s) where the antecedent is true, and evaluate the consequent. Building on the standard approach, some researchers have found that the set of worlds to be considered is dependent on context; it evolves with the discourse. Others have focused on how to define the {``}distance{''} between possible worlds, using ideas from causal modeling. This paper integrates the two ideas. We present a semantics for counterfactuals that uses a distance measure based on causal laws, that can also change over time. We show how our semantics can be implemented in the Haskell programming language.
Visual TTR - Modelling Visual Question Answering in Type Theory with Records
Authors: Ronja Utescher
Date: 2019-05-01 00:00:00
Proceeding: WS 2019 5
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA), Vocal Bursts Type Prediction

In this paper, I will describe a system that was developed for the task of Visual Question Answering. The system uses the rich type universe of Type Theory with Records (TTR) to model both the utterances about the image, the image itself and classifications made related to the two. At its most basic, the decision of whether any given predicate can be assigned to an object in the image is delegated to a CNN. Consequently, images can be judged as evidence for propositions. The end result is a model whose application of perceptual classifiers to a given image is guided by the accompanying utterance.
The Lexical Gap: An Improved Measure of Automated Image Description Quality
Authors: Austin Kershaw, Miroslaw Bober
Date: 2019-05-01 00:00:00
Proceeding: WS 2019 5
Tasks: 2k, Diversity, Image Description

The challenge of automatically describing images and videos has stimulated much research in Computer Vision and Natural Language Processing. In order to test the semantic abilities of new algorithms, we need reliable and objective ways of measuring progress. We show that standard evaluation measures do not take into account the semantic richness of a description, and give the impression that sparse machine descriptions outperform rich human descriptions. We introduce and test a new measure of semantic ability based on relative lexical diversity. We show how our measure can work alongside existing measures to achieve state of the art correlation with human judgement of quality. We also introduce a new dataset: Rich-Sparse Descriptions, which provides 2K human and machine descriptions to stimulate interest into the semantic evaluation of machine descriptions.