pwc-archive/papers-with-abstracts huggingface dataset:
Subcharacter Information in Japanese Embeddings: When Is It Worth It?
Authors: Marzena Karpinska, Bofang Li, Anna Rogers, Aleks Drozd, R
Date: 2018-07-01 00:00:00
Proceeding: WS 2018 7
Tasks: Text Classification

Languages with logographic writing systems present a difficulty for traditional character-level models. Leveraging the subcharacter information was recently shown to be beneficial for a number of intrinsic and extrinsic tasks in Chinese. We examine whether the same strategies could be applied for Japanese, and contribute a new analogy dataset for this language.
A neural parser as a direct classifier for head-final languages
Authors: Hiroshi Kanayama, Masayasu Muraoka, Ryosuke Kohita
Date: 2018-07-01 00:00:00
Proceeding: WS 2018 7
Tasks: Dependency Parsing

This paper demonstrates a neural parser implementation suitable for consistently head-final languages such as Japanese. Unlike the transition- and graph-based algorithms in most state-of-the-art parsers, our parser directly selects the head word of a dependent from a limited number of candidates. This method drastically simplifies the model so that we can easily interpret the output of the neural model. Moreover, by exploiting grammatical knowledge to restrict possible modification types, we can control the output of the parser to reduce specific errors without adding annotated corpora. The neural parser performed well both on conventional Japanese corpora and the Japanese version of Universal Dependency corpus, and the advantages of distributed representations were observed in the comparison with the non-neural conventional model.
Hierarchical Convolutional Attention Networks for Text Classification
Authors: Shang Gao, Arvind Ramanathan, Georgia Tourassi
Date: 2018-07-01 00:00:00
Proceeding: WS 2018 7
Tasks: Classification, Document Classification, General Classification, Machine Translation, Representation Learning, text-classification, Text Classification, Translation

Recent work in machine translation has demonstrated that self-attention mechanisms can be used in place of recurrent neural networks to increase training speed without sacrificing model accuracy. We propose combining this approach with the benefits of convolutional filters and a hierarchical structure to create a document classification model that is both highly accurate and fast to train {--} we name our method Hierarchical Convolutional Attention Networks. We demonstrate the effectiveness of this architecture by surpassing the accuracy of the current state-of-the-art on several classification tasks while being twice as fast to train.
Chat Discrimination for Intelligent Conversational Agents with a Hybrid CNN-LMTGRU Network
Authors: Dennis Singh Moirangthem, Minho Lee
Date: 2018-07-01 00:00:00
Proceeding: WS 2018 7
Tasks: Representation Learning

Recently, intelligent dialog systems and smart assistants have attracted the attention of many, and development of novel dialogue agents have become a research challenge. Intelligent agents that can handle both domain-specific task-oriented and open-domain chit-chat dialogs are one of the major requirements in the current systems. In order to address this issue and to realize such smart hybrid dialogue systems, we develop a model to discriminate user utterance between task-oriented and chit-chat conversations. We introduce a hybrid of convolutional neural network (CNN) and a lateral multiple timescale gated recurrent units (LMTGRU) that can represent multiple temporal scale dependencies for the discrimination task. With the help of the combined slow and fast units of the LMTGRU, our model effectively determines whether a user will have a chit-chat conversation or a task-specific conversation with the system. We also show that the LMTGRU structure helps the model to perform well on longer text inputs. We address the lack of dataset by constructing a dataset using Twitter and Maluuba Frames data. The results of the experiments demonstrate that the proposed hybrid network outperforms the conventional models on the chat discrimination task as well as performed comparable to the baselines on various benchmark datasets.
Text Completion using Context-Integrated Dependency Parsing
Authors: Amr Rekaby Salama, {\"O}zge Ala{\c{c}}am, Wolfgang Menzel
Date: 2018-07-01 00:00:00
Proceeding: WS 2018 7
Tasks: Decision Making, Dependency Parsing, Language Modeling, Language Modelling, Representation Learning

Incomplete linguistic input, i.e. due to a noisy environment, is one of the challenges that a successful communication system has to deal with. In this paper, we study text completion with a data set composed of sentences with gaps where a successful completion cannot be achieved through a uni-modal (language-based) approach. We present a solution based on a context-integrating dependency parser incorporating an additional non-linguistic modality. An incompleteness in one channel is compensated by information from another one and the parser learns the association between the two modalities from a multiple level knowledge representation. We examined several model variations by adjusting the degree of influence of different modalities in the decision making on possible filler words and their exact reference to a non-linguistic context element. Our model is able to fill the gap with 95.4{\%} word and 95.2{\%} exact reference accuracy hence the successful prediction can be achieved not only on the word level (such as mug) but also with respect to the correct identification of its context reference (such as mug 2 among several mug instances).
Natural Language Inference with Definition Embedding Considering Context On the Fly
Authors: Kosuke Nishida, Kyosuke Nishida, Hisako Asano, Junji Tomita
Date: 2018-07-01 00:00:00
Proceeding: WS 2018 7
Tasks: Domain Adaptation, Information Retrieval, Natural Language Inference, Question Answering, Representation Learning, Word Embeddings

Natural language inference (NLI) is one of the most important tasks in NLP. In this study, we propose a novel method using word dictionaries, which are pairs of a word and its definition, as external knowledge. Our neural definition embedding mechanism encodes input sentences with the definitions of each word of the sentences on the fly. It can encode the definition of words considering the context of input sentences by using an attention mechanism. We evaluated our method using WordNet as a dictionary and confirmed that our method performed better than baseline models when using the full or a subset of 100d GloVe as word embeddings.
Comparison of Representations of Named Entities for Document Classification
Authors: Lidia Pivovarova, Roman Yangarber
Date: 2018-07-01 00:00:00
Proceeding: WS 2018 7
Tasks: Classification, Document Classification, General Classification, Representation Learning, text-classification, Text Classification, Topic Classification, Word Embeddings

We explore representations for multi-word names in text classification tasks, on Reuters (RCV1) topic and sector classification. We find that: the best way to treat names is to split them into tokens and use each token as a separate feature; NEs have more impact on sector classification than topic classification; replacing NEs with entity types is not an effective strategy; representing tokens by different embeddings for proper names vs. common nouns does not improve results. We highlight the improvements over state-of-the-art results that our CNN models yield.
Connecting Supervised and Unsupervised Sentence Embeddings
Authors: Gil Levi
Date: 2018-07-01 00:00:00
Proceeding: WS 2018 7
Tasks: Denoising, Natural Language Inference, Representation Learning, Sentence, Sentence Embedding, Sentence Embeddings, Word Embeddings

Representing sentences as numerical vectors while capturing their semantic context is an important and useful intermediate step in natural language processing. Representations that are both general and discriminative can serve as a tool for tackling various NLP tasks. While common sentence representation methods are unsupervised in nature, recently, an approach for learning universal sentence representation in a supervised setting was presented in (Conneau et al.,2017). We argue that although promising results were obtained, an improvement can be reached by adding various unsupervised constraints that are motivated by auto-encoders and by language models. We show that by adding such constraints, superior sentence embeddings can be achieved. We compare our method with the original implementation and show improvements in several tasks.
A Hybrid Learning Scheme for Chinese Word Embedding
Authors: Wenfan Chen, Wei-Guo Sheng
Date: 2018-07-01 00:00:00
Proceeding: WS 2018 7
Tasks: Language Modeling, Language Modelling, Machine Translation, Question Answering, Representation Learning, Word Embeddings

To improve word embedding, subword information has been widely employed in state-of-the-art methods. These methods can be classified to either compositional or predictive models. In this paper, we propose a hybrid learning scheme, which integrates compositional and predictive model for word embedding. Such a scheme can take advantage of both models, thus effectively learning word embedding. The proposed scheme has been applied to learn word representation on Chinese. Our results show that the proposed scheme can significantly improve the performance of word embedding in terms of analogical reasoning and is robust to the size of training data.
Unsupervised Random Walk Sentence Embeddings: A Strong but Simple Baseline
Authors: Kawin Ethayarajh
Date: 2018-07-01 00:00:00
Proceeding: WS 2018 7
Tasks: Representation Learning, Sentence, Sentence Embeddings, Text Generation, Transfer Learning, Word Embeddings

Using a random walk model of text generation, Arora et al. (2017) proposed a strong baseline for computing sentence embeddings: take a weighted average of word embeddings and modify with SVD. This simple method even outperforms far more complex approaches such as LSTMs on textual similarity tasks. In this paper, we first show that word vector length has a confounding effect on the probability of a sentence being generated in Arora et al.{'}s model. We propose a random walk model that is robust to this confound, where the probability of word generation is inversely related to the angular distance between the word and sentence embeddings. Our approach beats Arora et al.{'}s by up to 44.4{\%} on textual similarity tasks and is competitive with state-of-the-art methods. Unlike Arora et al.{'}s method, ours requires no hyperparameter tuning, which means it can be used when there is no labelled data.
Exploiting Common Characters in Chinese and Japanese to Learn Cross-Lingual Word Embeddings via Matrix Factorization
Authors: Jilei Wang, Shiying Luo, Weiyan Shi, Tao Dai, Shu-Tao Xia
Date: 2018-07-01 00:00:00
Proceeding: WS 2018 7
Tasks: Cross-Lingual Word Embeddings, Machine Translation, Representation Learning, Sentence, Word Embeddings, Word Sense Disambiguation

Learning vector space representation of words (i.e., word embeddings) has recently attracted wide research interests, and has been extended to cross-lingual scenario. Currently most cross-lingual word embedding learning models are based on sentence alignment, which inevitably introduces much noise. In this paper, we show in Chinese and Japanese, the acquisition of semantic relation among words can benefit from the large number of common characters shared by both languages; inspired by this unique feature, we design a method named CJC targeting to generate cross-lingual context of words. We combine CJC with GloVe based on matrix factorization, and then propose an integrated model named CJ-Glo. Taking two sentence-aligned models and CJ-BOC (also exploits common characters but is based on CBOW) as baseline algorithms, we compare them with CJ-Glo on a series of NLP tasks including cross-lingual synonym, word analogy and sentence alignment. The result indicates CJ-Glo achieves the best performance among these methods, and is more stable in cross-lingual tasks; moreover, compared with CJ-BOC, CJ-Glo is less sensitive to the alteration of parameters.
WordNet Embeddings
Authors: Chakaveh Saedi, Ant{\'o}nio Branco, Jo{\~a}o Ant{\'o}nio Rodrigues, Jo{\~a}o Silva
Date: 2018-07-01 00:00:00
Proceeding: WS 2018 7
Tasks: Representation Learning, Semantic Similarity, Semantic Textual Similarity, Word Embeddings

Semantic networks and semantic spaces have been two prominent approaches to represent lexical semantics. While a unified account of the lexical meaning relies on one being able to convert between these representations, in both directions, the conversion direction from semantic networks into semantic spaces started to attract more attention recently. In this paper we present a methodology for this conversion and assess it with a case study. When it is applied over WordNet, the performance of the resulting embeddings in a mainstream semantic similarity task is very good, substantially superior to the performance of word embeddings based on very large collections of texts like word2vec.
Knowledge Graph Embedding with Numeric Attributes of Entities
Authors: Yanrong Wu, Zhichun Wang
Date: 2018-07-01 00:00:00
Proceeding: WS 2018 7
Tasks: Attribute, Graph Embedding, Knowledge Graph Embedding, Knowledge Graphs, Link Prediction, Representation Learning, Translation

Knowledge Graph (KG) embedding projects entities and relations into low dimensional vector space, which has been successfully applied in KG completion task. The previous embedding approaches only model entities and their relations, ignoring a large number of entities{'} numeric attributes in KGs. In this paper, we propose a new KG embedding model which jointly model entity relations and numeric attributes. Our approach combines an attribute embedding model with a translation-based structure embedding model, which learns the embeddings of entities, relations, and attributes simultaneously. Experiments of link prediction on YAGO and Freebase show that the performance is effectively improved by adding entities{'} numeric attributes in the embedding model.
Injecting Lexical Contrast into Word Vectors by Guiding Vector Space Specialisation
Authors: Ivan Vuli{\'c}
Date: 2018-07-01 00:00:00
Proceeding: WS 2018 7
Tasks: Dialogue State Tracking, Representation Learning, Semantic Similarity, Semantic Textual Similarity, Spoken Language Understanding, Text Generation

Word vector space specialisation models offer a portable, light-weight approach to fine-tuning arbitrary distributional vector spaces to discern between synonymy and antonymy. Their effectiveness is drawn from external linguistic constraints that specify the exact lexical relation between words. In this work, we show that a careful selection of the external constraints can steer and improve the specialisation. By simply selecting appropriate constraints, we report state-of-the-art results on a suite of tasks with well-defined benchmarks where modeling lexical contrast is crucial: 1) true semantic similarity, with highest reported scores on SimLex-999 and SimVerb-3500 to date; 2) detecting antonyms; and 3) distinguishing antonyms from synonyms.
Characters or Morphemes: How to Represent Words?
Authors: Ahmet {\"U}st{\"u}n, Murathan Kurfal{\i}, Burcu Can
Date: 2018-07-01 00:00:00
Proceeding: WS 2018 7
Tasks: Representation Learning, Semantic Textual Similarity

In this paper, we investigate the effects of using subword information in representation learning. We argue that using syntactic subword units effects the quality of the word representations positively. We introduce a morpheme-based model and compare it against to word-based, character-based, and character n-gram level models. Our model takes a list of candidate segmentations of a word and learns the representation of the word based on different segmentations that are weighted by an attention mechanism. We performed experiments on Turkish as a morphologically rich language and English with a comparably poorer morphology. The results show that morpheme-based models are better at learning word representations of morphologically complex languages compared to character-based and character n-gram level models since the morphemes help to incorporate more syntactic knowledge in learning, that makes morpheme-based models better at syntactic tasks.
Learning Hierarchical Structures On-The-Fly with a Recurrent-Recursive Model for Sequences
Authors: Athul Paul Jacob, Zhouhan Lin, Aless Sordoni, ro, Yoshua Bengio
Date: 2018-07-01 00:00:00
Proceeding: WS 2018 7
Tasks: Language Modeling, Language Modelling, Math, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Representation Learning

We propose a hierarchical model for sequential data that learns a tree on-the-fly, i.e. while reading the sequence. In the model, a recurrent network adapts its structure and reuses recurrent weights in a recursive manner. This creates adaptive skip-connections that ease the learning of long-term dependencies. The tree structure can either be inferred without supervision through reinforcement learning, or learned in a supervised manner. We provide preliminary experiments in a novel Math Expression Evaluation (MEE) task, which is created to have a hierarchical tree structure that can be used to study the effectiveness of our model. Additionally, we test our model in a well-known propositional logic and language modelling tasks. Experimental results have shown the potential of our approach.
Multilingual Seq2seq Training with Similarity Loss for Cross-Lingual Document Classification
Authors: Katherine Yu, Haoran Li, Barlas Oguz
Date: 2018-07-01 00:00:00
Proceeding: WS 2018 7
Tasks: Cross-Lingual Document Classification, Cross-Lingual Transfer, Document Classification, General Classification, Machine Translation, Representation Learning, Sentence, Sentence Embedding, Sentence Embeddings, Translation, Word Embeddings

In this paper we continue experiments where neural machine translation training is used to produce joint cross-lingual fixed-dimensional sentence embeddings. In this framework we introduce a simple method of adding a loss to the learning objective which penalizes distance between representations of bilingually aligned sentences. We evaluate cross-lingual transfer using two approaches, cross-lingual similarity search on an aligned corpus (Europarl) and cross-lingual document classification on a recently published benchmark Reuters corpus, and we find the similarity loss significantly improves performance on both. Furthermore, we notice that while our Reuters results are very competitive, our English results are not as competitive, showing room for improvement in the current cross-lingual state-of-the-art. Our results are based on a set of 6 European languages.
Learning Distributional Token Representations from Visual Features
Authors: Samuel Broscheit
Date: 2018-07-01 00:00:00
Proceeding: WS 2018 7
Tasks: Machine Translation, NMT, Part-Of-Speech Tagging, Representation Learning, Semantic Similarity, Semantic Textual Similarity, Translation

In this study, we compare token representations constructed from visual features (i.e., pixels) with standard lookup-based embeddings. Our goal is to gain insight about the challenges of encoding a text representation from low-level features, e.g. from characters or pixels. We focus on Chinese, which{---}as a logographic language{---}has properties that make a representation via visual features challenging and interesting. To train and evaluate different models for the token representation, we chose the task of character-based neural machine translation (NMT) from Chinese to English. We found that a token representation computed only from visual features can achieve competitive results to lookup embeddings. However, we also show different strengths and weaknesses in the models{'} performance in a part-of-speech tagging task and also a semantic similarity task. In summary, we show that it is possible to achieve a \textit{text representation} only from pixels. We hope that this is a useful stepping stone for future studies that exclusively rely on visual input, or aim at exploiting visual features of written language.
Predicting Concreteness and Imageability of Words Within and Across Languages via Word Embeddings
Authors: Nikola Ljube{\v{s}}i{\'c}, Darja Fi{\v{s}}er, Anita Peti-Stanti{\'c}
Date: 2018-07-01 00:00:00
Proceeding: WS 2018 7
Tasks: Cross-Lingual Transfer, Representation Learning, Word Embeddings

The notions of concreteness and imageability, traditionally important in psycholinguistics, are gaining significance in semantic-oriented natural language processing tasks. In this paper we investigate the predictability of these two concepts via supervised learning, using word embeddings as explanatory variables. We perform predictions both within and across languages by exploiting collections of cross-lingual embeddings aligned to a single vector space. We show that the notions of concreteness and imageability are highly predictable both within and across languages, with a moderate loss of up to 20{\%} in correlation when predicting across languages. We further show that the cross-lingual transfer via word embeddings is more efficient than the simple transfer via bilingual dictionaries.
Economic Event Detection in Company-Specific News Text
Authors: Gilles Jacobs, Els Lefever, V{\'e}ronique Hoste
Date: 2018-07-01 00:00:00
Proceeding: WS 2018 7
Tasks: Articles, BIG-bench Machine Learning, Classification, Event Detection, General Classification, Sentence, Sentiment Analysis

This paper presents a dataset and supervised classification approach for economic event detection in English news articles. Currently, the economic domain is lacking resources and methods for data-driven supervised event detection. The detection task is conceived as a sentence-level classification task for 10 different economic event types. Two different machine learning approaches were tested: a rich feature set Support Vector Machine (SVM) set-up and a word-vector-based long short-term memory recurrent neural network (RNN-LSTM) set-up. We show satisfactory results for most event types, with the linear kernel SVM outperforming the other experimental set-ups
Causality Analysis of Twitter Sentiments and Stock Market Returns
Authors: Narges Tabari, Piyusha Biswas, Bhanu Praneeth, Armin Seyeditabari, Mirsad Hadzikadic, Wlodek Zadrozny
Date: 2018-07-01 00:00:00
Proceeding: WS 2018 7
Tasks: General Classification, Sentiment Analysis, Twitter Sentiment Analysis

Sentiment analysis is the process of identifying the opinion expressed in text. Recently, it has been used to study behavioral finance, and in particular the effect of opinions and emotions on economic or financial decisions. In this paper, we use a public dataset of labeled tweets that has been labeled by Amazon Mechanical Turk and then we propose a baseline classification model. Then, by using Granger causality of both sentiment datasets with the different stocks, we shows that there is causality between social media and stock market returns (in both directions) for many stocks. Finally, We evaluate this causality analysis by showing that in the event of a specific news on certain dates, there are evidences of trending the same news on Twitter for that stock.