pwc-archive/papers-with-abstracts huggingface dataset:
HHU at SemEval-2018 Task 12: Analyzing an Ensemble-based Deep Learning Approach for the Argument Mining Task of Choosing the Correct Warrant
Authors: Matthias Liebeck, Andreas Funke, Stefan Conrad
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Argument Mining

This paper describes our participation in the SemEval-2018 Task 12 Argument Reasoning Comprehension Task which calls to develop systems that, given a reason and a claim, predict the correct warrant from two opposing options. We decided to use a deep learning architecture and combined 623 models with different hyperparameters into an ensemble. Our extensive analysis of our architecture and ensemble reveals that the decision to use an ensemble was suboptimal. Additionally, we benchmark a support vector machine as a baseline. Furthermore, we experimented with an alternative data split and achieved more stable results.
YNU Deep at SemEval-2018 Task 12: A BiLSTM Model with Neural Attention for Argument Reasoning Comprehension
Authors: Peng Ding, Xiaobing Zhou
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Constituency Parsing, Language Modeling, Language Modelling, Machine Translation, Reading Comprehension

This paper describes the system submitted to SemEval-2018 Task 12 (The Argument Reasoning Comprehension Task). Enabling a computer to understand a text so that it can answer comprehension questions is still a challenging goal of NLP. We propose a Bidirectional LSTM (BiLSTM) model that reads two sentences separated by a delimiter to determine which warrant is correct. We extend this model with a neural attention mechanism that encourages the model to make reasoning over the given claims and reasons. Officially released results show that our system ranks 6th among 22 submissions to this task.
UniMelb at SemEval-2018 Task 12: Generative Implication using LSTMs, Siamese Networks and Semantic Representations with Synonym Fuzzing
Authors: Anirudh Joshi, Tim Baldwin, Richard O. Sinnott, Cecile Paris
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: General Classification, Multi-Task Learning, Semantic Similarity, Semantic Textual Similarity, Word Embeddings

This paper describes a warrant classification system for SemEval 2018 Task 12, that attempts to learn semantic representations of reasons, claims and warrants. The system consists of 3 stacked LSTMs: one for the reason, one for the claim, and one shared Siamese Network for the 2 candidate warrants. Our main contribution is to force the embeddings into a shared feature space using vector operations, semantic similarity classification, Siamese networks, and multi-task learning. In doing so, we learn a form of generative implication, in encoding implication interrelationships between reasons, claims, and the associated correct and incorrect warrants. We augment the limited data in the task further by utilizing WordNet synonym {``}fuzzing{''}. When applied to SemEval 2018 Task 12, our system performs well on the development data, and officially ranked 8th among 21 teams.
Joker at SemEval-2018 Task 12: The Argument Reasoning Comprehension with Neural Attention
Authors: Guobin Sui, WenHan Chao, Zhunchen Luo
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Argument Mining, Stance Detection

This paper describes a classification system that participated in the SemEval-2018 Task 12: The Argument Reasoning Comprehension Task. Briefly the task can be described as that a natural language {``}argument{''} is what we have, with reason, claim, and correct and incorrect warrants, and we need to choose the correct warrant. In order to make fully understand of the semantic information of the sentences, we proposed a neural network architecture with attention mechanism to achieve this goal. Besides we try to introduce keywords into the model to improve accuracy. Finally the proposed system achieved 5th place among 22 participating systems
TakeLab at SemEval-2018 Task12: Argument Reasoning Comprehension with Skip-Thought Vectors
Authors: Ana Brassard, Tin Kuculo, Filip Boltu{\v{z}}i{\'c}, Jan {\v{S}}najder
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Common Sense Reasoning, Natural Language Inference, Sentence, Word Embeddings

This paper describes our system for the SemEval-2018 Task 12: Argument Reasoning Comprehension Task. We utilize skip-thought vectors, sentence-level distributional vectors inspired by the popular word embeddings and the skip-gram model. We encode preprocessed sentences from the dataset into vectors, then perform a binary supervised classification of the warrant that justifies the use of the reason as support for the claim. We explore a few variations of the model, reaching 54.1{\%} accuracy on the test set, which placed us 16th out of 22 teams participating in the task.
Lyb3b at SemEval-2018 Task 12: Ensemble-based Deep Learning Models for Argument Reasoning Comprehension Task
Authors: Yongbin Li, Xiaobing Zhou
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6

Reasoning is a crucial part of natural language argumentation. In order to comprehend an argument, we have to reconstruct and analyze its reasoning. In this task, given a natural language argument with a reason and a claim, the goal is to choose the correct implicit reasoning from two options, in order to form a reasonable structure of (Reason, Warrant, Claim). Our approach is to build distributed word embedding of reason, warrant and claim respectively, meanwhile, we use a series of frameworks such as CNN model, LSTM model, GRU with attention model and biLSTM with attention model for processing word vector. Finally, ensemble mechanism is used to integrate the results of each framework to improve the final accuracy. Experiments demonstrate superior performance of ensemble mechanism compared to each separate framework. We are the 11th in official results, the final model can reach a 0.568 accuracy rate on the test dataset.
TRANSRW at SemEval-2018 Task 12: Transforming Semantic Representations for Argument Reasoning Comprehension
Authors: Zhimin Chen, Wei Song, Lizhen Liu
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Semantic Composition

This paper describes our system in SemEval-2018 task 12: Argument Reasoning Comprehension. The task is to select the correct warrant that explains reasoning of a particular argument consisting of a claim and a reason. The main idea of our methods is based on the assumption that the semantic composition of the reason and the warrant should be close to the semantic representation of the corresponding claim. We propose two neural network models. The first one considers two warrant candidates simultaneously, while the second one processes each candidate separately and then chooses the best one. We also incorporate sentiment polarity by assuming that there are kinds of sentiment associations between the reason, the warrant and the claim. The experiments show that the first framework is more effective and sentiment polarity is useful.
Learning distributed event representations with a multi-task approach
Authors: Xudong Hong, Asad Sayeed, Vera Demberg
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Multi-Task Learning, World Knowledge

Human world knowledge contains information about prototypical events and their participants and locations. In this paper, we train the first models using multi-task learning that can both predict missing event participants and also perform semantic role classification based on semantic plausibility. Our best-performing model is an improvement over the previous state-of-the-art on thematic fit modelling tasks. The event embeddings learned by the model can additionally be used effectively in an event similarity task, also outperforming the state-of-the-art.
Assessing Meaning Components in German Complex Verbs: A Collection of Source-Target Domains and Directionality
Authors: Sabine Schulte im Walde, Maximilian K{\"o}per, Sylvia Springorum
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Sentence

This paper presents a collection to assess meaning components in German complex verbs, which frequently undergo meaning shifts. We use a novel strategy to obtain source and target domain characterisations via sentence generation rather than sentence annotation. A selection of arrows adds spatial directional information to the generated contexts. We provide a broad qualitative description of the dataset, and a series of standard classification experiments verifies the quantitative reliability of the presented resource. The setup for collecting the meaning components is applicable also to other languages, regarding complex verbs as well as other language-specific targets that involve meaning shifts.
Graph Algebraic Combinatory Categorial Grammar
Authors: Sebastian Beschke, Wolfgang Menzel
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: AMR Parsing, Machine Translation, Semantic Parsing

This paper describes CCG/AMR, a novel grammar for semantic parsing of Abstract Meaning Representations. CCG/AMR equips Combinatory Categorial Grammar derivations with graph semantics by assigning each CCG combinator an interpretation in terms of a graph algebra. We provide an algorithm that induces a CCG/AMR from a corpus and show that it creates a compact lexicon with low ambiguity and achieves a robust coverage of 78{\%} of the examined sentences under ideal conditions. We also identify several phenomena that affect any approach relying either on CCG or graph algebraic approaches for AMR parsing. This includes differences of representation between CCG and AMR, as well as non-compositional constructions that are not expressible through a monotonous construction process. To our knowledge, this paper provides the first analysis of these corpus issues.
Quantitative Semantic Variation in the Contexts of Concrete and Abstract Words
Authors: Daniela Naumann, Diego Frassinelli, Sabine Schulte im Walde
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Diversity

Across disciplines, researchers are eager to gain insight into empirical features of abstract vs. concrete concepts. In this work, we provide a detailed characterisation of the distributional nature of abstract and concrete words across 16,620 English nouns, verbs and adjectives. Specifically, we investigate the following questions: (1) What is the distribution of concreteness in the contexts of concrete and abstract target words? (2) What are the differences between concrete and abstract words in terms of contextual semantic diversity? (3) How does the entropy of concrete and abstract word contexts differ? Overall, our studies show consistent differences in the distributional representation of concrete and abstract words, thus challenging existing theories of cognition and providing a more fine-grained description of their nature.
EmoWordNet: Automatic Expansion of Emotion Lexicon Using English WordNet
Authors: Gilbert Badaro, Hussein Jundi, Hazem Hajj, Wassim El-Hajj
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Collaborative Filtering, Emotion Classification, Emotion Recognition, Recommendation Systems, Sentiment Analysis

Nowadays, social media have become a platform where people can easily express their opinions and emotions about any topic such as politics, movies, music, electronic products and many others. On the other hand, politicians, companies, and businesses are interested in analyzing automatically people{'}s opinions and emotions. In the last decade, a lot of efforts has been put into extracting sentiment polarity from texts. Recently, the focus has expanded to also cover emotion recognition from texts. In this work, we expand an existing emotion lexicon, DepecheMood, by leveraging semantic knowledge from English WordNet (EWN). We create an expanded lexicon, EmoWordNet, consisting of 67K terms aligned with EWN, almost 1.8 times the size of DepecheMood. We also evaluate EmoWordNet in an emotion recognition task using SemEval 2007 news headlines dataset and we achieve an improvement compared to the use of DepecheMood. EmoWordNet is publicly available to speed up research in the field on \url{http://oma-project.com}.
How Gender and Skin Tone Modifiers Affect Emoji Semantics in Twitter
Authors: Francesco Barbieri, Jose Camacho-Collados
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Word Embeddings

In this paper we analyze the use of emojis in social media with respect to gender and skin tone. By gathering a dataset of over twenty two million tweets from United States some findings are clearly highlighted after performing a simple frequency-based analysis. Moreover, we carry out a semantic analysis on the usage of emojis and their modifiers (e.g. gender and skin tone) by embedding all words, emojis and modifiers into the same vector space. Our analyses reveal that some stereotypes related to the skin color and gender seem to be reflected on the use of these modifiers. For example, emojis representing hand gestures are more widely utilized with lighter skin tones, and the usage across skin tones differs significantly. At the same time, the vector corresponding to the male modifier tends to be semantically close to emojis related to business or technology, whereas their female counterparts appear closer to emojis about love or makeup.
Element-wise Bilinear Interaction for Sentence Matching
Authors: Jihun Choi, Taeuk Kim, Sang-goo Lee
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Natural Language Inference, Paraphrase Identification, Sentence

When we build a neural network model predicting the relationship between two sentences, the most general and intuitive approach is to use a Siamese architecture, where the sentence vectors obtained from a shared encoder is given as input to a classifier. For the classifier to work effectively, it is important to extract appropriate features from the two vectors and feed them as input. There exist several previous works that suggest heuristic-based function for matching sentence vectors, however it cannot be said that the heuristics tailored for a specific task generalize to other tasks. In this work, we propose a new matching function, ElBiS, that learns to model element-wise interaction between two vectors. From experiments, we empirically demonstrate that the proposed ElBiS matching function outperforms the concatenation-based or heuristic-based matching functions on natural language inference and paraphrase identification, while maintaining the fused representation compact.
Named Graphs for Semantic Representation
Authors: Richard Crouch, Aikaterini-Lida Kalouli
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Knowledge Graphs, Negation, Position

A position paper arguing that purely graphical representations for natural language semantics lack a fundamental degree of expressiveness, and cannot deal with even basic Boolean operations like negation or disjunction. Moving from graphs to named graphs leads to representations that stand some chance of having sufficient expressive power. Named $\mathcal{FL}_0$ graphs are of particular interest.
Polarity Computations in Flexible Categorial Grammar
Authors: Hai Hu, Larry Moss
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6

This paper shows how to take parse trees in CCG and algorithmically find the polarities of all the constituents. Our work uses the well-known polarization principle corresponding to function application, and we have extended this with principles for type raising and composition. We provide an algorithm, extending the polarity marking algorithm of van Benthem. We discuss how our system works in practice, taking input from the C{\&}C parser.
Coarse Lexical Frame Acquisition at the Syntax--Semantics Interface Using a Latent-Variable PCFG Model
Authors: Laura Kallmeyer, Behrang Qasemizadeh, Jackie Chi Kit Cheung
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Clustering, Question Answering, Text Summarization

We present a method for unsupervised lexical frame acquisition at the syntax{--}semantics interface. Given a set of input strings derived from dependency parses, our method generates a set of clusters that resemble lexical frame structures. Our work is motivated not only by its practical applications (e.g., to build, or expand the coverage of lexical frame databases), but also to gain linguistic insight into frame structures with respect to lexical distributions in relation to grammatical structures. We model our task using a hierarchical Bayesian network and employ tools and methods from latent variable probabilistic context free grammars (L-PCFGs) for statistical inference and parameter fitting, for which we propose a new split and merge procedure. We show that our model outperforms several baselines on a portion of the Wall Street Journal sentences that we have newly annotated for evaluation purposes.
Exploiting Partially Annotated Data in Temporal Relation Extraction
Authors: Qiang Ning, Zhongzhi Yu, Chuchu Fan, Dan Roth
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Relation, Relation Extraction, Temporal Relation Extraction

Annotating temporal relations (TempRel) between events described in natural language is known to be labor intensive, partly because the total number of TempRels is quadratic in the number of events. As a result, only a small number of documents are typically annotated, limiting the coverage of various lexical/semantic phenomena. In order to improve existing approaches, one possibility is to make use of the readily available, partially annotated data (P as in partial) that cover more documents. However, missing annotations in P are known to hurt, rather than help, existing systems. This work is a case study in exploring various usages of P for TempRel extraction. Results show that despite missing annotations, P is still a useful supervision signal for this task within a constrained bootstrapping learning framework. The system described in this system is publicly available.
Predicting Word Embeddings Variability
Authors: B{\'e}n{\'e}dicte Pierrejean, Ludovic Tanguy
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Word Embeddings

Neural word embeddings models (such as those built with word2vec) are known to have stability problems: when retraining a model with the exact same hyperparameters, words neighborhoods may change. We propose a method to estimate such variation, based on the overlap of neighbors of a given word in two models trained with identical hyperparameters. We show that this inherent variation is not negligible, and that it does not affect every word in the same way. We examine the influence of several features that are intrinsic to a word, corpus or embedding model and provide a methodology that can predict the variability (and as such, reliability) of a word representation in a semantic vector space.
Deep Affix Features Improve Neural Named Entity Recognizers
Authors: Vikas Yadav, Rebecca Sharp, Steven Bethard
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Feature Engineering, Morphological Analysis, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

We propose a practical model for named entity recognition (NER) that combines word and character-level information with a specific learned representation of the prefixes and suffixes of the word. We apply this approach to multilingual and multi-domain NER and show that it achieves state of the art results on the CoNLL 2002 Spanish and Dutch and CoNLL 2003 German NER datasets, consistently achieving 1.5-2.3 percent over the state of the art without relying on any dictionary features. Additionally, we show improvement on SemEval 2013 task 9.1 DrugNER, achieving state of the art results on the MedLine dataset and the second best results overall (-1.3{\%} from state of the art). We also establish a new benchmark on the I2B2 2010 Clinical NER dataset with 84.70 F-score.
Quality Signals in Generated Stories
Authors: Manasvi Sagarkar, John Wieting, Lifu Tu, Kevin Gimpel
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Sentence, Story Continuation, Story Generation, Text Generation

We study the problem of measuring the quality of automatically-generated stories. We focus on the setting in which a few sentences of a story are provided and the task is to generate the next sentence ({``}continuation{''}) in the story. We seek to identify what makes a story continuation interesting, relevant, and have high overall quality. We crowdsource annotations along these three criteria for the outputs of story continuation systems, design features, and train models to predict the annotations. Our trained scorer can be used as a rich feature function for story generation, a reward function for systems that use reinforcement learning to learn to generate stories, and as a partial evaluation metric for story generation.
Agree or Disagree: Predicting Judgments on Nuanced Assertions
Authors: Michael Wojatzki, Torsten Zesch, Saif Mohammad, Svetlana Kiritchenko
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6

Being able to predict whether people agree or disagree with an assertion (i.e. an explicit, self-contained statement) has several applications ranging from predicting how many people will like or dislike a social media post to classifying posts based on whether they are in accordance with a particular point of view. We formalize this as two NLP tasks: predicting judgments of (i) individuals and (ii) groups based on the text of the assertion and previous judgments. We evaluate a wide range of approaches on a crowdsourced data set containing over 100,000 judgments on over 2,000 assertions. We find that predicting individual judgments is a hard task with our best results only slightly exceeding a majority baseline, but that judgments of groups can be more reliably predicted using a Siamese neural network, which outperforms all other approaches by a wide margin.
A Multimodal Translation-Based Approach for Knowledge Graph Representation Learning
Authors: Hatem Mousselly-Sergieh, Teresa Botschen, Iryna Gurevych, Stefan Roth
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Graph Representation Learning, Information Retrieval, Knowledge Graph Completion, Knowledge Graphs, Question Answering, Representation Learning, Translation, Triple Classification

Current methods for knowledge graph (KG) representation learning focus solely on the structure of the KG and do not exploit any kind of external information, such as visual and linguistic information corresponding to the KG entities. In this paper, we propose a multimodal translation-based approach that defines the energy of a KG triple as the sum of sub-energy functions that leverage both multimodal (visual and linguistic) and structural KG representations. Next, a ranking-based loss is minimized using a simple neural network architecture. Moreover, we introduce a new large-scale dataset for multimodal KG representation learning. We compared the performance of our approach to other baselines on two standard tasks, namely knowledge graph completion and triple classification, using our as well as the WN9-IMG dataset. The results demonstrate that our approach outperforms all baselines on both tasks and datasets.
Putting Semantics into Semantic Roles
Authors: James Allen, Choh Man Teng
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6

While there have been many proposals for theories of semantic roles over the years, these models are mostly justified by intuition and the only evaluation methods have been inter-annotator agreement. We explore three different ideas for providing more rigorous theories of semantic roles. These ideas give rise to more objective criteria for designing role sets, and lend themselves to some experimental evaluation. We illustrate the discussion by examining the semantic roles in TRIPS.