pwc-archive/papers-with-abstracts huggingface dataset:
Users and Data: The Two Neglected Children of Bilingual Natural Language Processing Research
Authors: Phillippe Langlais
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Machine Translation

Despite numerous studies devoted to mining parallel material from bilingual data, we have yet to see the resulting technologies wholeheartedly adopted by professional translators and terminologists alike. I argue that this state of affairs is mainly due to two factors: the emphasis published authors put on models (even though data is as important), and the conspicuous lack of concern for actual end-users.
Sentence Alignment using Unfolding Recursive Autoencoders
Authors: Jeenu Grover, Pabitra Mitra
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Information Retrieval, Machine Translation, Paraphrase Identification, Question Answering, Sentence, Text Summarization

In this paper, we propose a novel two step algorithm for sentence alignment in monolingual corpora using Unfolding Recursive Autoencoders. First, we use unfolding recursive auto-encoders (RAE) to learn feature vectors for phrases in syntactical tree of the sentence. To compare two sentences we use a similarity matrix which has dimensions proportional to the size of the two sentences. Since the similarity matrix generated to compare two sentences has varying dimension due to different sentence lengths, a dynamic pooling layer is used to map it to a matrix of fixed dimension. The resulting matrix is used to calculate the similarity scores between the two sentences. The second step of the algorithm captures the contexts in which the sentences occur in the document by using a dynamic programming algorithm for global alignment.
Toward a Comparable Corpus of Latvian, Russian and English Tweets
Authors: Dmitrijs Milajevs
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Information Retrieval, Machine Translation, Translation

Twitter has become a rich source for linguistic data. Here, a possibility of building a trilingual Latvian-Russian-English corpus of tweets from Riga, Latvia is investigated. Such a corpus, once constructed, might be of great use for multiple purposes including training machine translation models, examining cross-lingual phenomena and studying the population of Riga. This pilot study shows that it is feasible to build such a resource by collecting and analysing a pilot corpus, which is made publicly available and can be used to construct a large comparable corpus.
Automatic Extraction of Parallel Speech Corpora from Dubbed Movies
Authors: Alp {\"O}ktem, Mireia Farr{\'u}s, Leo Wanner
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Speech-to-Speech Translation, Translation

This paper presents a methodology to extract parallel speech corpora based on any language pair from dubbed movies, together with an application framework in which some corresponding prosodic parameters are extracted. The obtained parallel corpora are especially suitable for speech-to-speech translation applications when a prosody transfer between source and target languages is desired.
A parallel collection of clinical trials in Portuguese and English
Authors: Mariana Neves
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Machine Translation, Sentence, Translation

Parallel collections of documents are crucial resources for training and evaluating machine translation (MT) systems. Even though large collections are available for certain domains and language pairs, these are still scarce in the biomedical domain. We developed a parallel corpus of clinical trials in Portuguese and English. The documents are derived from the Brazilian Clinical Trials Registry and the corpus currently contains a total of 1188 documents. In this paper, we describe the corpus construction and discuss the quality of the translation and the sentence alignment that we obtained.
Weighted Set-Theoretic Alignment of Comparable Sentences
Authors: Andoni Azpeitia, Thierry Etchegoyhen, Eva Mart{\'\i}nez Garcia
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Machine Translation, Sentence

This article presents the STACCw system for the BUCC 2017 shared task on parallel sentence extraction from comparable corpora. The original STACC approach, based on set-theoretic operations over bags of words, had been previously shown to be efficient and portable across domains and alignment scenarios. Wedescribe an extension of this approach with a new weighting scheme and show that it provides significant improvements on the datasets provided for the shared task.
BUCC 2017 Shared Task: a First Attempt Toward a Deep Learning Framework for Identifying Parallel Sentences in Comparable Corpora
Authors: Francis Gr{\'e}goire, Philippe Langlais
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Feature Engineering, Language Modeling, Language Modelling, Machine Translation, Semantic Textual Similarity, Word Embeddings

This paper describes our participation in BUCC 2017 shared task: identifying parallel sentences in comparable corpora. Our goal is to leverage continuous vector representations and distributional semantics with a minimal use of external preprocessing and postprocessing tools. We report experiments that were conducted after transmitting our results.
zNLP: Identifying Parallel Sentences in Chinese-English Comparable Corpora
Authors: Zheng Zhang, Pierre Zweigenbaum
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Machine Translation, Sentence

This paper describes the zNLP system for the BUCC 2017 shared task. Our system identifies parallel sentence pairs in Chinese-English comparable corpora by translating word-by-word Chinese sentences into English, using the search engine Solr to select near-parallel sentences and then by using an SVM classifier to identify true parallel sentences from the previous results. It obtains an F1-score of 45{\%} (resp. 32{\%}) on the test (training) set.
BUCC2017: A Hybrid Approach for Identifying Parallel Sentences in Comparable Corpora
Authors: Sainik Mahata, Dipankar Das, B, Sivaji yopadhyay
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Machine Translation, Sentence, Translation

A Statistical Machine Translation (SMT) system is always trained using large parallel corpus to produce effective translation. Not only is the corpus scarce, it also involves a lot of manual labor and cost. Parallel corpus can be prepared by employing comparable corpora where a pair of corpora is in two different languages pointing to the same domain. In the present work, we try to build a parallel corpus for French-English language pair from a given comparable corpus. The data and the problem set are provided as part of the shared task organized by BUCC 2017. We have proposed a system that first translates the sentences by heavily relying on Moses and then group the sentences based on sentence length similarity. Finally, the one to one sentence selection was done based on Cosine Similarity algorithm.
Overview of the Second BUCC Shared Task: Spotting Parallel Sentences in Comparable Corpora
Authors: Pierre Zweigenbaum, Serge Sharoff, Reinhard Rapp
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Machine Translation, Sentence

This paper presents the BUCC 2017 shared task on parallel sentence extraction from comparable corpora. It recalls the design of the datasets, presents their final construction and statistics and the methods used to evaluate system results. 13 runs were submitted to the shared task by 4 teams, covering three of the four proposed language pairs: French-English (7 runs), German-English (3 runs), and Chinese-English (3 runs). The best F-scores as measured against the gold standard were 0.84 (German-English), 0.80 (French-English), and 0.43 (Chinese-English). Because of the design of the dataset, in which not all gold parallel sentence pairs are known, these are only minimum values. We examined manually a small sample of the false negative sentence pairs for the most precise French-English runs and estimated the number of parallel sentence pairs not yet in the provided gold standard. Adding them to the gold standard leads to revised estimates for the French-English F-scores of at most +1.5pt. This suggests that the BUCC 2017 datasets provide a reasonable approximate evaluation of the parallel sentence spotting task.
Sense Contextualization in a Dependency-Based Compositional Distributional Model
Authors: Pablo Gamallo
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: LEMMA, Representation Learning, Semantic Composition, Word Sense Disambiguation

Little attention has been paid to distributional compositional methods which employ syntactically structured vector models. As word vectors belonging to different syntactic categories have incompatible syntactic distributions, no trivial compositional operation can be applied to combine them into a new compositional vector. In this article, we generalize the method described by Erk and Pad{\'o} (2009) by proposing a dependency-base framework that contextualize not only lemmas but also selectional preferences. The main contribution of the article is to expand their model to a fully compositional framework in which syntactic dependencies are put at the core of semantic composition. We claim that semantic composition is mainly driven by syntactic dependencies. Each syntactic dependency generates two new compositional vectors representing the contextualized sense of the two related lemmas. The sequential application of the compositional operations associated to the dependencies results in as many contextualized vectors as lemmas the composite expression contains. At the end of the semantic process, we do not obtain a single compositional vector representing the semantic denotation of the whole composite expression, but one contextualized vector for each lemma of the whole expression. Our method avoids the troublesome high-order tensor representations by defining lemmas and selectional restrictions as first-order tensors (i.e. standard vectors). A corpus-based experiment is performed to both evaluate the quality of the compositional vectors built with our strategy, and to compare them to other approaches on distributional compositional semantics. The experiments show that our dependency-based compositional method performs as (or even better than) the state-of-the-art.
Towards Harnessing Memory Networks for Coreference Resolution
Authors: Joe Cheri, Pushpak Bhattacharyya
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: coreference-resolution, Coreference Resolution, Question Answering, Representation Learning

Coreference resolution task demands comprehending a discourse, especially for anaphoric mentions which require semantic information for resolving antecedents. We investigate into how memory networks can be helpful for coreference resolution when posed as question answering problem. The comprehension capability of memory networks assists coreference resolution, particularly for the mentions that require semantic and context information. We experiment memory networks for coreference resolution, with 4 synthetic datasets generated for coreference resolution with varying difficulty levels. Our system{'}s performance is compared with a traditional coreference resolution system to show why memory network can be promising for coreference resolution.
Combining Word-Level and Character-Level Representations for Relation Classification of Informal Text
Authors: Dongyun Liang, Weiran Xu, Yinge Zhao
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Classification, General Classification, Relation, Relation Classification, Representation Learning, Slot Filling

Word representation models have achieved great success in natural language processing tasks, such as relation classification. However, it does not always work on informal text, and the morphemes of some misspelling words may carry important short-distance semantic information. We propose a hybrid model, combining the merits of word-level and character-level representations to learn better representations on informal text. Experiments on two dataset of relation classification, SemEval-2010 Task8 and a large-scale one we compile from informal text, show that our model achieves a competitive result in the former and state-of-the-art with the other.
Modeling Large-Scale Structured Relationships with Shared Memory for Knowledge Base Completion
Authors: Yelong Shen, Po-Sen Huang, Ming-Wei Chang, Jianfeng Gao
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Knowledge Base Completion, Question Answering, Representation Learning

Recent studies on knowledge base completion, the task of recovering missing relationships based on recorded relations, demonstrate the importance of learning embeddings from multi-step relations. However, due to the size of knowledge bases, learning multi-step relations directly on top of observed triplets could be costly. Hence, a manually designed procedure is often used when training the models. In this paper, we propose Implicit ReasoNets (IRNs), which is designed to perform multi-step inference implicitly through a controller and shared memory. Without a human-designed inference procedure, IRNs use training data to learn to perform multi-step inference in an embedding neural space through the shared memory and controller. While the inference procedure does not explicitly operate on top of observed triplets, our proposed model outperforms all previous approaches on the popular FB15k benchmark by more than 5.7{\%}.
Semantic Vector Encoding and Similarity Search Using Fulltext Search Engines
Authors: Jan Rygl, Jan Pomik{\'a}lek, Radim {\v{R}}eh{\r{u}}{\v{r}}ek, Michal R{\r{u}}{\v{z}}i{\v{c}}ka, V{\'\i}t Novotn{\'y}, Petr Sojka
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Information Retrieval, Representation Learning

Vector representations and vector space modeling (VSM) play a central role in modern machine learning. We propose a novel approach to {`}vector similarity searching{'} over dense semantic representations of words and documents that can be deployed on top of traditional inverted-index-based fulltext engines, taking advantage of their robustness, stability, scalability and ubiquity. We show that this approach allows the indexing and querying of dense vectors in text domains. This opens up exciting avenues for major efficiency gains, along with simpler deployment, scaling and monitoring. The end result is a fast and scalable vector database with a tunable trade-off between vector search performance and quality, backed by a standard fulltext engine such as Elasticsearch. We empirically demonstrate its querying performance and quality by applying this solution to the task of semantic searching over a dense vector representation of the entire English Wikipedia.
Representing Compositionality based on Multiple Timescales Gated Recurrent Neural Networks with Adaptive Temporal Hierarchy for Character-Level Language Models
Authors: Dennis Singh Moirangthem, Jegyung Son, Minho Lee
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Language Modeling, Language Modelling, Representation Learning

A novel character-level neural language model is proposed in this paper. The proposed model incorporates a biologically inspired temporal hierarchy in the architecture for representing multiple compositions of language in order to handle longer sequences for the character-level language model. The temporal hierarchy is introduced in the language model by utilizing a Gated Recurrent Neural Network with multiple timescales. The proposed model incorporates a timescale adaptation mechanism for enhancing the performance of the language model. We evaluate our proposed model using the popular Penn Treebank and Text8 corpora. The experiments show that the use of multiple timescales in a Neural Language Model (NLM) enables improved performance despite having fewer parameters and with no additional computation requirements. Our experiments also demonstrate the ability of the adaptive temporal hierarchies to represent multiple compositonality without the help of complex hierarchical architectures and shows that better representation of the longer sequences lead to enhanced performance of the probabilistic language model.
Learning Bilingual Projections of Embeddings for Vocabulary Expansion in Machine Translation
Authors: Pranava Swaroop Madhyastha, Cristina Espa{\~n}a-Bonet
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Machine Translation, Representation Learning, Translation, Transliteration, Word Embeddings

We propose a simple log-bilinear softmax-based model to deal with vocabulary expansion in machine translation. Our model uses word embeddings trained on significantly large unlabelled monolingual corpora and learns over a fairly small, word-to-word bilingual dictionary. Given an out-of-vocabulary source word, the model generates a probabilistic list of possible translations in the target language using the trained bilingual embeddings. We integrate these translation options into a standard phrase-based statistical machine translation system and obtain consistent improvements in translation quality on the English{--}Spanish language pair. When tested over an out-of-domain testset, we get a significant improvement of 3.9 BLEU points.
Prediction of Frame-to-Frame Relations in the FrameNet Hierarchy with Frame Embeddings
Authors: Teresa Botschen, Hatem Mousselly-Sergieh, Iryna Gurevych
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Natural Language Inference, Representation Learning, Semantic Role Labeling

Automatic completion of frame-to-frame (F2F) relations in the FrameNet (FN) hierarchy has received little attention, although they incorporate meta-level commonsense knowledge and are used in downstream approaches. We address the problem of sparsely annotated F2F relations. First, we examine whether the manually defined F2F relations emerge from text by learning text-based frame embeddings. Our analysis reveals insights about the difficulty of reconstructing F2F relations purely from text. Second, we present different systems for predicting F2F relations; our best-performing one uses the FN hierarchy to train on and to ground embeddings in. A comparison of systems and embeddings exposes the crucial influence of knowledge-based embeddings to a system{'}s performance in predicting F2F relations.
Gradual Learning of Matrix-Space Models of Language for Sentiment Analysis
Authors: Shima Asaadi, Sebastian Rudolph
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Representation Learning, Sentiment Analysis

Learning word representations to capture the semantics and compositionality of language has received much research interest in natural language processing. Beyond the popular vector space models, matrix representations for words have been proposed, since then, matrix multiplication can serve as natural composition operation. In this work, we investigate the problem of learning matrix representations of words. We present a learning approach for compositional matrix-space models for the task of sentiment analysis. We show that our approach, which learns the matrices gradually in two steps, outperforms other approaches and a gradient-descent baseline in terms of quality and computational cost.
Intrinsic and Extrinsic Evaluation of Spatiotemporal Text Representations in Twitter Streams
Authors: Lawrence Phillips, Kyle Shaffer, Dustin Arendt, Nathan Hodas, Svitlana Volkova
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Representation Learning, Type prediction

Language in social media is a dynamic system, constantly evolving and adapting, with words and concepts rapidly emerging, disappearing, and changing their meaning. These changes can be estimated using word representations in context, over time and across locations. A number of methods have been proposed to track these spatiotemporal changes but no general method exists to evaluate the quality of these representations. Previous work largely focused on qualitative evaluation, which we improve by proposing a set of visualizations that highlight changes in text representation over both space and time. We demonstrate usefulness of novel spatiotemporal representations to explore and characterize specific aspects of the corpus of tweets collected from European countries over a two-week period centered around the terrorist attacks in Brussels in March 2016. In addition, we quantitatively evaluate spatiotemporal representations by feeding them into a downstream classification task {--} event type prediction. Thus, our work is the first to provide both intrinsic (qualitative) and extrinsic (quantitative) evaluation of text representations for spatiotemporal trends.
Plan, Attend, Generate: Character-Level Neural Machine Translation with Planning
Authors: Caglar Gulcehre, Francis Dutil, Adam Trischler, Yoshua Bengio
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Decoder, Hierarchical Reinforcement Learning, Machine Translation, Reinforcement Learning, Reinforcement Learning (RL), Representation Learning, Translation

We investigate the integration of a planning mechanism into an encoder-decoder architecture with attention. We develop a model that can plan ahead when it computes alignments between the source and target sequences not only for a single time-step but for the next k time-steps as well by constructing a matrix of proposed future alignments and a commitment vector that governs whether to follow or recompute the plan. This mechanism is inspired by strategic attentive reader and writer (STRAW) model, a recent neural architecture for planning with hierarchical reinforcement learning that can also learn higher level temporal abstractions. Our proposed model is end-to-end trainable with differentiable operations. We show that our model outperforms strong baselines on character-level translation task from WMT{'}15 with fewer parameters and computes alignments that are qualitatively intuitive.
Does the Geometry of Word Embeddings Help Document Classification? A Case Study on Persistent Homology-Based Representations
Authors: Paul Michel, Ravich, Abhilasha er, Shruti Rijhwani
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Classification, Clustering, Document Classification, Document Embedding, General Classification, Representation Learning, Sentiment Analysis, Sentiment Classification, Text Classification, Word Embeddings

We investigate the pertinence of methods from algebraic topology for text data analysis. These methods enable the development of mathematically-principled isometric-invariant mappings from a set of vectors to a document embedding, which is stable with respect to the geometry of the document in the selected metric space. In this work, we evaluate the utility of these topology-based document representations in traditional NLP tasks, specifically document clustering and sentiment classification. We find that the embeddings do not benefit text analysis. In fact, performance is worse than simple techniques like tf-idf, indicating that the geometry of the document does not provide enough variability for classification on the basis of topic or sentiment in the chosen datasets.
newsLens: building and visualizing long-ranging news stories
Authors: Philippe Laban, Marti Hearst
Date: 2017-08-01 00:00:00
Proceeding: WS 2017 8
Tasks: Articles, Navigate

We propose a method to aggregate and organize a large, multi-source dataset of news articles into a collection of major stories, and automatically name and visualize these stories in a working system. The approach is able to run online, as new articles are added, processing 4 million news articles from 20 news sources, and extracting 80000 major stories, some of which span several years. The visual interface consists of lanes of timelines, each annotated with information that is deemed important for the story, including extracted quotations. The working system allows a user to search and navigate 8 years of story information.