pwc-archive/papers-with-abstracts huggingface dataset:
ClaiRE at SemEval-2018 Task 7: Classification of Relations using Embeddings
Authors: Lena Hettinger, Alex Dallmann, er, Albin Zehe, Thomas Niebler, Andreas Hotho
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Classification, General Classification, Question Answering, Relation Classification, Relation Extraction, Sentiment Analysis, Word Embeddings

In this paper we describe our system for SemEval-2018 Task 7 on classification of semantic relations in scientific literature for clean (subtask 1.1) and noisy data (subtask 1.2). We compare two models for classification, a C-LSTM which utilizes only word embeddings and an SVM that also takes handcrafted features into account. To adapt to the domain of science we train word embeddings on scientific papers collected from arXiv.org. The hand-crafted features consist of lexical features to model the semantic relations as well as the entities between which the relation holds. Classification of Relations using Embeddings (ClaiRE) achieved an F1 score of 74.89{\%} for the first subtask and 78.39{\%} for the second.
TakeLab at SemEval-2018 Task 7: Combining Sparse and Dense Features for Relation Classification in Scientific Texts
Authors: Martin Gluhak, Maria Pia di Buono, Abbas Akkasi, Jan {\v{S}}najder
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: General Classification, Relation, Relation Classification, Relation Extraction

We describe two systems for semantic relation classification with which we participated in the SemEval 2018 Task 7, subtask 1 on semantic relation classification: an SVM model and a CNN model. Both models combine dense pretrained word2vec features and hancrafted sparse features. For training the models, we combine the two datasets provided for the subtasks in order to balance the under-represented classes. The SVM model performed better than CNN, achieving a F1-macro score of 69.98{\%} on subtask 1.1 and 75.69{\%} on subtask 1.2. The system ranked 7th on among 28 submissions on subtask 1.1 and 7th among 20 submissions on subtask 1.2.
NEUROSENT-PDI at SemEval-2018 Task 7: Discovering Textual Relations With a Neural Network Model
Authors: Mauro Dragoni
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Relation Extraction, Word Embeddings

Discovering semantic relations within textual documents is a timely topic worthy of investigation. Natural language processing strategies are generally used for linking chunks of text in order to extract information that can be exploited by semantic search engines for performing complex queries. The scientific domain is an interesting area where these techniques can be applied. In this paper, we describe a system based on neural networks applied to the SemEval 2018 Task 7. The system relies on the use of word embeddings for composing the vectorial representation of text chunks. Such representations are used for feeding a neural network aims to learn the structure of paths connecting chunks associated with a specific relation. Preliminary results demonstrated the suitability of the proposed approach encouraging the investigation of this research direction.
SciREL at SemEval-2018 Task 7: A System for Semantic Relation Extraction and Classification
Authors: Darshini Mahendran, Chathurika Brahmana, Bridget McInnes
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Feature Engineering, General Classification, Relation, Relation Classification, Relation Extraction

This paper describes our system, SciREL (Scientific abstract RELation extraction system), developed for the SemEval 2018 Task 7: Semantic Relation Extraction and Classification in Scientific Papers. We present a feature-vector based system to extract explicit semantic relation and classify them. Our system is trained in the ACL corpus (BIrd et al., 2008) that contains annotated abstracts given by the task organizers. When an abstract with annotated entities is given as the input into our system, it extracts the semantic relations through a set of defined features and classifies them into one of the given six categories of relations through feature engineering and a learned model. For the best combination of features, our system SciREL obtained an F-measure of 20.03 on the official test corpus which includes 150 abstracts in the relation classification Subtask 1.1. In this paper, we provide an in-depth error analysis of our results to prevent duplication of research efforts in the development of future systems
NTNU at SemEval-2018 Task 7: Classifier Ensembling for Semantic Relation Identification and Classification in Scientific Papers
Authors: Biswanath Barik, Utpal Kumar Sikdar, Bj{\"o}rn Gamb{\"a}ck
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Classification, feature selection, General Classification, Relation, Relation Classification, Relation Extraction

The paper presents NTNU{'}s contribution to SemEval-2018 Task 7 on relation identification and classification. The class weights and parameters of five alternative supervised classifiers were optimized through grid search and cross-validation. The outputs of the classifiers were combined through voting for the final prediction. A wide variety of features were explored, with the most informative identified by feature selection. The best setting achieved F1 scores of 47.4{\%} and 66.0{\%} in the relation classification subtasks 1.1 and 1.2. For relation identification and classification in subtask 2, it achieved F1 scores of 33.9{\%} and 17.0{\%},
Talla at SemEval-2018 Task 7: Hybrid Loss Optimization for Relation Classification using Convolutional Neural Networks
Authors: Bhanu Pratap, Daniel Shank, Oladipo Ositelu, Byron Galbraith
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Feature Engineering, General Classification, Question Answering, Relation, Relation Classification, Word Embeddings

This paper describes our approach to SemEval-2018 Task 7 {--} given an entity-tagged text from the ACL Anthology corpus, identify and classify pairs of entities that have one of six possible semantic relationships. Our model consists of a convolutional neural network leveraging pre-trained word embeddings, unlabeled ACL-abstracts, and multiple window sizes to automatically learn useful features from entity-tagged sentences. We also experiment with a hybrid loss function, a combination of cross-entropy loss and ranking loss, to boost the separation in classification scores. Lastly, we include WordNet-based features to further improve the performance of our model. Our best model achieves an F1(macro) score of 74.2 and 84.8 on subtasks 1.1 and 1.2, respectively.
TeamDL at SemEval-2018 Task 8: Cybersecurity Text Analysis using Convolutional Neural Network and Conditional Random Fields
Authors: Manik R, an, Krishna Madgula, Snehanshu Saha
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: General Classification, LEMMA, POS, Sentence, Sentence Classification, TAG, Text Classification, Word Embeddings

In this work we present our participation to SemEval-2018 Task 8 subtasks 1 {\&} 2 respectively. We developed Convolution Neural Network system for malware sentence classification (subtask 1) and Conditional Random Fields system for malware token label prediction (subtask 2). We experimented with couple of word embedding strategies, feature sets and achieved competitive performance across the two subtasks. For subtask 1 We experimented with two category of word embeddings namely native embeddings and task specific embedding using Word2vec and Glove algorithms. 1. Native Embeddings: All words including the unknown ones that are randomly initialized use embeddings from original Word2vec/Glove models. 2. Task specific : The embeddings are generated by training Word2vec/Glove algorithms on sentences from MalwareTextDB We found that glove outperforms rest of embeddings for subtask 1. For subtask 2, we used N-grams of size 6, previous, next tokens and labels, features giving disjunctions of words anywhere in the left or right, word shape features, word lemma of current, previous and next words, word-tag pair features, POS tags, prefix and suffixes.
HCCL at SemEval-2018 Task 8: An End-to-End System for Sequence Labeling from Cybersecurity Reports
Authors: Mingming Fu, Xuemin Zhao, Yonghong Yan
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Feature Engineering, Named Entity Recognition (NER), Structured Prediction

This paper describes HCCL team systems that participated in SemEval 2018 Task 8: SecureNLP (Semantic Extraction from cybersecurity reports using NLP). To solve the problem, our team applied a neural network architecture that benefits from both word and character level representaions automatically, by using combination of Bi-directional LSTM, CNN and CRF (Ma and Hovy, 2016). Our system is truly end-to-end, requiring no feature engineering or data preprocessing, and we ranked 4th in the subtask 1, 7th in the subtask2 and 3rd in the SubTask2-relaxed.
UMBC at SemEval-2018 Task 8: Understanding Text about Malware
Authors: Ankur Padia, Arpita Roy, Taneeya Satyapanich, Francis Ferraro, SHimei Pan, Youngja Park, Anupam Joshi, Tim Finin
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Attribute

We describe the systems developed by the UMBC team for 2018 SemEval Task 8, SecureNLP (Semantic Extraction from CybersecUrity REports using Natural Language Processing). We participated in three of the sub-tasks: (1) classifying sentences as being relevant or irrelevant to malware, (2) predicting token labels for sentences, and (4) predicting attribute labels from the Malware Attribute Enumeration and Characterization vocabulary for defining malware characteristics. We achieve F1 score of 50.34/18.0 (dev/test), 22.23 (test-data), and 31.98 (test-data) for Task1, Task2 and Task2 respectively. We also make our cybersecurity embeddings publicly available at \url{http://bit.ly/cyber2vec}.
Villani at SemEval-2018 Task 8: Semantic Extraction from Cybersecurity Reports using Representation Learning
Authors: Pablo Loyola, Kugamoorthy Gajananan, Yuji Watanabe, Fumiko Satoh
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Representation Learning

In this paper, we describe our proposal for the task of Semantic Extraction from Cybersecurity Reports. The goal is to explore if natural language processing methods can provide relevant and actionable knowledge to contribute to better understand malicious behavior. Our method consists of an attention-based Bi-LSTM which achieved competitive performance of 0.57 for the Subtask 1. In the due process we also present ablation studies across multiple embeddings and their level of representation and also report the strategies we used to mitigate the extreme imbalance between classes.
Flytxt\_NTNU at SemEval-2018 Task 8: Identifying and Classifying Malware Text Using Conditional Random Fields and Na\"\ive Bayes Classifiers
Authors: Utpal Kumar Sikdar, Biswanath Barik, Bj{\"o}rn Gamb{\"a}ck
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6

Cybersecurity risks such as malware threaten the personal safety of users, but to identify malware text is a major challenge. The paper proposes a supervised learning approach to identifying malware sentences given a document (subTask1 of SemEval 2018, Task 8), as well as to classifying malware tokens in the sentences (subTask2). The approach achieved good results, ranking second of twelve participants for both subtasks, with F-scores of 57{\%} for subTask1 and 28{\%} for subTask2.
Digital Operatives at SemEval-2018 Task 8: Using dependency features for malware NLP
Authors: Chris Brew
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Task 2, text-classification, Text Classification

The four sub-tasks of SecureNLP build towards a capability for quickly highlighting critical information from malware reports, such as the specific actions taken by a malware sample. Digital Operatives (DO) submitted to sub-tasks 1 and 2, using standard text analysis technology (text classification for sub-task 1, and a CRF for sub-task 2). Performance is broadly competitive with other submitted systems on sub-task 1 and weak on sub-task 2. The annotation guidelines for the intermediate sub-tasks create a linkage to the final task, which is both an annotation challenge and a potentially useful feature of the task. The methods that DO chose do not attempt to make use of this linkage, which may be a missed opportunity. This motivates a post-hoc error analysis. It appears that the annotation task is very hard, and that in some cases both deep conceptual knowledge and substantial surrounding context are needed in order to correctly classify sentences.
Apollo at SemEval-2018 Task 9: Detecting Hypernymy Relations Using Syntactic Dependencies
Authors: Mihaela Onofrei, Ionu{\textcommabelow{t}} Hulub, Tr, Diana ab{\u{a}}{\textcommabelow{t}}, Daniela G{\^\i}fu
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Hypernym Discovery

This paper presents the participation of Apollo{'}s team in the SemEval-2018 Task 9 {``}Hypernym Discovery{''}, Subtask 1: {``}General-Purpose Hypernym Discovery{''}, which tries to produce a ranked list of hypernyms for a specific term. We propose a novel approach for automatic extraction of hypernymy relations from a corpus by using dependency patterns. We estimated that the application of these patterns leads to a higher score than using the traditional lexical patterns.
NLP\_HZ at SemEval-2018 Task 9: a Nearest Neighbor Approach
Authors: Wei Qiu, Mosha Chen, Linlin Li, Luo Si
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Hypernym Discovery, Information Retrieval, Natural Language Inference, Question Answering, Word Sense Disambiguation

Hypernym discovery aims to discover the hypernym word sets given a hyponym word and proper corpus. This paper proposes a simple but effective method for the discovery of hypernym sets based on word embedding, which can be used to measure the contextual similarities between words. Given a test hyponym word, we get its hypernym lists by computing the similarities between the hyponym word and words in the training data, and fill the test word{'}s hypernym lists with the hypernym list in the training set of the nearest similarity distance to the test word. In SemEval 2018 task9, our results, achieve 1st on Spanish, 2nd on Italian, 6th on English in the metric of MAP.
UMDuluth-CS8761 at SemEval-2018 Task9: Hypernym Discovery using Hearst Patterns, Co-occurrence frequencies and Word Embeddings
Authors: Arshia Zernab Hassan, Manikya Swathi Vallabhajosyula, Ted Pedersen
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Hypernym Discovery, Word Embeddings

Hypernym Discovery is the task of identifying potential hypernyms for a given term. A hypernym is a more generalized word that is super-ordinate to more specific words. This paper explores several approaches that rely on co-occurrence frequencies of word pairs, Hearst Patterns based on regular expressions, and word embeddings created from the UMBC corpus. Our system Babbage participated in Subtask 1A for English and placed 6th of 19 systems when identifying concept hypernyms, and 12th of 18 systems for entity hypernyms.
EXPR at SemEval-2018 Task 9: A Combined Approach for Hypernym Discovery
Authors: Ahmad Issa Alaa Aldine, Mounira Harzallah, Giuseppe Berio, Nicolas B{\'e}chet, Ahmad Faour
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Hypernym Discovery, Information Retrieval, Machine Translation, Question Answering

In this paper, we present our proposed system (EXPR) to participate in the hypernym discovery task of SemEval 2018. The task addresses the challenge of discovering hypernym relations from a text corpus. Our proposal is a combined approach of path-based technique and distributional technique. We use dependency parser on a corpus to extract candidate hypernyms and represent their dependency paths as a feature vector. The feature vector is concatenated with a feature vector obtained using Wikipedia pre-trained term embedding model. The concatenated feature vector fits a supervised machine learning method to learn a classifier model. This model is able to classify new candidate hypernyms as hypernym or not. Our system performs well to discover new hypernyms not defined in gold hypernyms.
ADAPT at SemEval-2018 Task 9: Skip-Gram Word Embeddings for Unsupervised Hypernym Discovery in Specialised Corpora
Authors: Alfredo Maldonado, Filip Klubi{\v{c}}ka
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Hypernym Discovery, Word Embeddings

This paper describes a simple but competitive unsupervised system for hypernym discovery. The system uses skip-gram word embeddings with negative sampling, trained on specialised corpora. Candidate hypernyms for an input word are predicted based based on cosine similarity scores. Two sets of word embedding models were trained separately on two specialised corpora: a medical corpus and a music industry corpus. Our system scored highest in the medical domain among the competing unsupervised systems but performed poorly on the music industry domain. Our system does not depend on any external data other than raw specialised corpora.
300-sparsans at SemEval-2018 Task 9: Hypernymy as interaction of sparse attributes
Authors: G{\'a}bor Berend, M{\'a}rton Makrai, P{\'e}ter F{\"o}ldi{\'a}k
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Hypernym Discovery, Word Embeddings

This paper describes 300-sparsians{'}s participation in SemEval-2018 Task 9: Hypernym Discovery, with a system based on sparse coding and a formal concept hierarchy obtained from word embeddings. Our system took first place in subtasks (1B) Italian (all and entities), (1C) Spanish entities, and (2B) music entities.
UWB at SemEval-2018 Task 10: Capturing Discriminative Attributes from Word Distributions
Authors: Tom{\'a}{\v{s}} Brychc{\'\i}n, Tom{\'a}{\v{s}} Hercig, Josef Steinberger, Michal Konkol
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Attribute, Relation Extraction

We present our UWB system for the task of capturing discriminative attributes at SemEval 2018. Given two words and an attribute, the system decides, whether this attribute is discriminative between the words or not. Assuming Distributional Hypothesis, i.e., a word meaning is related to the distribution across contexts, we introduce several approaches to compare word contextual information. We experiment with state-of-the-art semantic spaces and with simple co-occurrence statistics. We show the word distribution in the corpus has potential for detecting discriminative attributes. Our system achieves F1 score 72.1{\%} and is ranked {\#}4 among 26 submitted systems.
Meaning\_space at SemEval-2018 Task 10: Combining explicitly encoded knowledge with information extracted from word embeddings
Authors: Pia Sommerauer, Antske Fokkens, Piek Vossen
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Semantic Textual Similarity, Word Embeddings

This paper presents the two systems submitted by the meaning space team in Task 10 of the SemEval competition 2018 entitled Capturing discriminative attributes. The systems consist of combinations of approaches exploiting explicitly encoded knowledge about concepts in WordNet and information encoded in distributional semantic vectors. Rather than aiming for high performance, we explore which kind of semantic knowledge is best captured by different methods. The results indicate that WordNet glosses on different levels of the hierarchy capture many attributes relevant for this task. In combination with exploiting word embedding similarities, this source of information yielded our best results. Our best performing system ranked 5th out of 13 final ranks. Our analysis yields insights into the different kinds of attributes represented by different sources of knowledge.
GHH at SemEval-2018 Task 10: Discovering Discriminative Attributes in Distributional Semantics
Authors: Mohammed Attia, Younes Samih, Manaal Faruqui, Wolfgang Maier
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Attribute, Word Embeddings

This paper describes our system submission to the SemEval 2018 Task 10 on Capturing Discriminative Attributes. Given two concepts and an attribute, the task is to determine whether the attribute is semantically related to one concept and not the other. In this work we assume that discriminative attributes can be detected by discovering the association (or lack of association) between a pair of words. The hypothesis we test in this contribution is whether the semantic difference between two pairs of concepts can be treated in terms of measuring the distance between words in a vector space, or can simply be obtained as a by-product of word co-occurrence counts.
CitiusNLP at SemEval-2018 Task 10: The Use of Transparent Distributional Models and Salient Contexts to Discriminate Word Attributes
Authors: Pablo Gamallo
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Attribute, Dimensionality Reduction

This article describes the unsupervised strategy submitted by the CitiusNLP team to the SemEval 2018 Task 10, a task which consists of predict whether a word is a discriminative attribute between two other words. Our strategy relies on the correspondence between discriminative attributes and relevant contexts of a word. More precisely, the method uses transparent distributional models to extract salient contexts of words which are identified as discriminative attributes. The system performance reaches about 70{\%} accuracy when it is applied on the development dataset, but its accuracy goes down (63{\%}) on the official test dataset.
THU\_NGN at SemEval-2018 Task 10: Capturing Discriminative Attributes with MLP-CNN model
Authors: Chuhan Wu, Fangzhao Wu, Sixing Wu, Zhigang Yuan, Yongfeng Huang
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Attribute, Semantic Similarity, Semantic Textual Similarity, Text Classification, Word Embeddings

Existing semantic models are capable of identifying the semantic similarity of words. However, it{'}s hard for these models to discriminate between a word and another similar word. Thus, the aim of SemEval-2018 Task 10 is to predict whether a word is a discriminative attribute between two concepts. In this task, we apply a multilayer perceptron (MLP)-convolutional neural network (CNN) model to identify whether an attribute is discriminative. The CNNs are used to extract low-level features from the inputs. The MLP takes both the flatten CNN maps and inputs to predict the labels. The evaluation F-score of our system on the test set is 0.629 (ranked 15th), which indicates that our system still needs to be improved. However, the behaviours of our system in our experiments provide useful information, which can help to improve the collective understanding of this novel task.
ALB at SemEval-2018 Task 10: A System for Capturing Discriminative Attributes
Authors: Bogdan Dumitru, Alina Maria Ciobanu, Liviu P. Dinu
Date: 2018-06-01 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Attribute, Dependency Parsing, Machine Translation, Semantic Parsing, Semantic Textual Similarity

Semantic difference detection attempts to capture whether a word is a discriminative attribute between two other words. For example, the discriminative feature red characterizes the first word from the (apple, banana) pair, but not the second. Modeling semantic difference is essential for language understanding systems, as it provides useful information for identifying particular aspects of word senses. This paper describes our system implementation (the ALB system of the NLP@Unibuc team) for the 10th task of the SemEval 2018 workshop, {``}Capturing Discriminative Attributes{''}. We propose a method for semantic difference detection that uses an SVM classifier with features based on co-occurrence counts and shallow semantic parsing, achieving 0.63 F1 score in the competition.