pwc-archive/papers-with-abstracts huggingface dataset:
TwistBytes - Identification of Cuneiform Languages and German Dialects at VarDial 2019
Authors: Fern Benites, o, Pius von D{\"a}niken, Mark Cieliebak
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Dialect Identification, Language Identification

We describe our approaches for the German Dialect Identification (GDI) and the Cuneiform Language Identification (CLI) tasks at the VarDial Evaluation Campaign 2019. The goal was to identify dialects of Swiss German in GDI and Sumerian and Akkadian in CLI. In GDI, the system should distinguish four dialects from the German-speaking part of Switzerland. Our system for GDI achieved third place out of 6 teams, with a macro averaged F-1 of 74.6{\%}. In CLI, the system should distinguish seven languages written in cuneiform script. Our system achieved third place out of 8 teams, with a macro averaged F-1 of 74.7{\%}.
DTeam @ VarDial 2019: Ensemble based on skip-gram and triplet loss neural networks for Moldavian vs. Romanian cross-dialect topic identification
Authors: Diana Tudoreanu
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: General Classification, Triplet

This paper presents the solution proposed by DTeam in the VarDial 2019 Evaluation Campaign for the Moldavian vs. Romanian cross-topic identification task. The solution proposed is a Support Vector Machines (SVM) ensemble composed of a two character-level neural networks. The first network is a skip-gram classification model formed of an embedding layer, three convolutional layers and two fully-connected layers. The second network has a similar architecture, but is trained using the triplet loss function.
Comparing Pipelined and Integrated Approaches to Dialectal Arabic Neural Machine Translation
Authors: Pamela Shapiro, Kevin Duh
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Dialect Identification, Machine Translation, Translation

When translating diglossic languages such as Arabic, situations may arise where we would like to translate a text but do not know which dialect it is. A traditional approach to this problem is to design dialect identification systems and dialect-specific machine translation systems. However, under the recent paradigm of neural machine translation, shared multi-dialectal systems have become a natural alternative. Here we explore under which conditions it is beneficial to perform dialect identification for Arabic neural machine translation versus using a general system for all dialects.
Cross-lingual Annotation Projection Is Effective for Neural Part-of-Speech Tagging
Authors: Matthias Huck, Diana Dutka, Alex Fraser, er
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Part-Of-Speech Tagging, POS, POS Tagging

We tackle the important task of part-of-speech tagging using a neural model in the zero-resource scenario, where we have no access to gold-standard POS training data. We compare this scenario with the low-resource scenario, where we have access to a small amount of gold-standard POS training data. Our experiments focus on Ukrainian as a representative of under-resourced languages. Russian is highly related to Ukrainian, so we exploit gold-standard Russian POS tags. We consider four techniques to perform Ukrainian POS tagging: zero-shot tagging and cross-lingual annotation projection (for the zero-resource scenario), and compare these with self-training and multilingual learning (for the low-resource scenario). We find that cross-lingual annotation projection works particularly well in the zero-resource scenario.
SPARSE: Structured Prediction using Argument-Relative Structured Encoding
Authors: Rishi Bommasani, Arzoo Katiyar, Claire Cardie
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Attribute, Prediction, Relation, Sentiment Analysis, Structured Prediction

We propose structured encoding as a novel approach to learning representations for relations and events in neural structured prediction. Our approach explicitly leverages the structure of available relation and event metadata to generate these representations, which are parameterized by both the attribute structure of the metadata as well as the learned representation of the arguments of the relations and events. We consider affine, biaffine, and recurrent operators for building hierarchical representations and modelling underlying features. We apply our approach to the second-order structured prediction task studied in the 2016/2017 Belief and Sentiment analysis evaluations (BeSt): given a document and its entities, relations, and events (including metadata and mentions), determine the sentiment of each entity towards every relation and event in the document. Without task-specific knowledge sources or domain engineering, we significantly improve over systems and baselines that neglect the available metadata or its hierarchical structure. We observe across-the-board improvements on the BeSt 2016/2017 sentiment analysis task of at least 2.3 (absolute) and 10.6{\%} (relative) F-measure over the previous state-of-the-art.
Semi-Supervised Teacher-Student Architecture for Relation Extraction
Authors: Fan Luo, Ajay Nagesh, Rebecca Sharp, Mihai Surdeanu
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Binary Relation Extraction, Denoising, Relation, Relation Extraction

Generating a large amount of training data for information extraction (IE) is either costly (if annotations are created manually), or runs the risk of introducing noisy instances (if distant supervision is used). On the other hand, semi-supervised learning (SSL) is a cost-efficient solution to combat lack of training data. In this paper, we adapt Mean Teacher (Tarvainen and Valpola, 2017), a denoising SSL framework to extract semantic relations between pairs of entities. We explore the sweet spot of amount of supervision required for good performance on this binary relation extraction task. Additionally, different syntax representations are incorporated into our models to enhance the learned representation of sentences. We evaluate our approach on the Google-IISc Distant Supervision (GDS) dataset, which removes test data noise present in all previous distance supervision datasets, which makes it a reliable evaluation benchmark (Jat et al., 2017). Our results show that the SSL Mean Teacher approach nears the performance of fully-supervised approaches even with only 10{\%} of the labeled corpus. Further, the syntax-aware model outperforms other syntax-free approaches across all levels of supervision.
Corpus of Multimodal Interaction for Collaborative Planning
Authors: Miltiadis Marios Katsakioris, Helen Hastie, Ioannis Konstas, Atanas Laskov
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: multimodal interaction

As autonomous systems become more commonplace, we need a way to easily and naturally communicate to them our goals and collaboratively come up with a plan on how to achieve these goals. To this end, we conducted a Wizard of Oz study to gather data and investigate the way operators would collaboratively make plans via a conversational {`}planning assistant{'} for remote autonomous systems. We present here a corpus of 22 dialogs from expert operators, which can be used to train such a system. Data analysis shows that multimodality is key to successful interaction, measured both quantitatively and qualitatively via user feedback.
?`Es un pl\'atano? Exploring the Application of a Physically Grounded Language Acquisition System to Spanish
Authors: Caroline Kery, Francis Ferraro, Cynthia Matuszek
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Grounded language learning, Language Acquisition

In this paper we describe a multilingual grounded language learning system adapted from an English-only system. This system learns the meaning of words used in crowd-sourced descriptions by grounding them in the physical representations of the objects they are describing. Our work presents a framework to compare the performance of the system when applied to a new language and to identify modifications necessary to attain equal performance, with the goal of enhancing the ability of robots to learn language from a more diverse range of people. We then demonstrate this system with Spanish, through first analyzing the performance of translated Spanish, and then extending this analysis to a new corpus of crowd-sourced Spanish language data. We find that with small modifications, the system is able to learn color, object, and shape words with comparable performance between languages.
From Virtual to Real: A Framework for Verbal Interaction with Robots
Authors: Eugene Joseph
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Natural Language Understanding, Object Recognition, Task Planning

A Natural Language Understanding (NLU) pipeline integrated with a 3D physics-based scene is a flexible way to develop and test language-based human-robot interaction, by virtualizing people, robot hardware and the target 3D environment. Here, interaction means both controlling robots using language and conversing with them about the user{'}s physical environment and her daily life. Such a virtual development framework was initially developed for the Bot Colony videogame launched on Steam in June 2014, and has been undergoing improvements since. The framework is focused of developing intuitive verbal interaction with various types of robots. Key robot functions (robot vision and object recognition, path planning and obstacle avoidance, task planning and constraints, grabbing and inverse kinematics), the human participants in the interaction, and the impact of gravity and other forces on the environment are all simulated using commercial 3D tools. The framework can be used as a robotics testbed: the results of our simulations can be compared with the output of algorithms in real robots, to validate such algorithms. A novelty of our framework is support for social interaction with robots - enabling robots to converse about people and objects in the user{'}s environment, as well as learning about human needs and everyday life topics from their owner.
Semantic Spatial Representation: a unique representation of an environment based on an ontology for robotic applications
Authors: Guillaume Sarthou, Aur{\'e}lie Clodic, Rachid Alami
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6

It is important, for human-robot interaction, to endow the robot with the knowledge necessary to understand human needs and to be able to respond to them. We present a formalized and unified representation for indoor environments using an ontology devised for a route description task in which a robot must provide explanations to a person. We show that this representation can be used to choose a route to explain to a human as well as to verbalize it using a route perspective. Based on ontology, this representation has a strong possibility of evolution to adapt to many other applications. With it, we get the semantics of the environment elements while keeping a description of the known connectivity of the environment. This representation and the illustration algorithms, to find and verbalize a route, have been tested in two environments of different scales.
SpatialNet: A Declarative Resource for Spatial Relations
Authors: Morgan Ulinski, Bob Coyne, Julia Hirschberg
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Automatic Speech Recognition (ASR)

This paper introduces SpatialNet, a novel resource which links linguistic expressions to actual spatial configurations. SpatialNet is based on FrameNet (Ruppenhofer et al., 2016) and VigNet (Coyne et al., 2011), two resources which use frame semantics to encode lexical meaning. SpatialNet uses a deep semantic representation of spatial relations to provide a formal description of how a language expresses spatial information. This formal representation of the lexical semantics of spatial language also provides a consistent way to represent spatial meaning across multiple languages. In this paper, we describe the structure of SpatialNet, with examples from English and German. We also show how SpatialNet can be combined with other existing NLP tools to create a text-to-scene system for a language.
What a neural language model tells us about spatial relations
Authors: Mehdi Ghanimifard, Simon Dobnik
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Image Description, Language Modeling, Language Modelling

Understanding and generating spatial descriptions requires knowledge about what objects are related, their functional interactions, and where the objects are geometrically located. Different spatial relations have different functional and geometric bias. The wide usage of neural language models in different areas including generation of image description motivates the study of what kind of knowledge is encoded in neural language models about individual spatial relations. With the premise that the functional bias of relations is expressed in their word distributions, we construct multi-word distributional vector representations and show that these representations perform well on intrinsic semantic reasoning tasks, thus confirming our premise. A comparison of our vector representations to human semantic judgments indicates that different bias (functional or geometric) is captured in different data collection tasks which suggests that the contribution of the two meaning modalities is dynamic, related to the context of the task.
A user study to compare two conversational assistants designed for people with hearing impairments
Authors: Anja Virkkunen, Juri Lukkarila, Kalle Palom{\"a}ki, Mikko Kurimo
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Speaker Identification, speech-recognition, Speech Recognition

Participating in conversations can be difficult for people with hearing loss, especially in acoustically challenging environments. We studied the preferences the hearing impaired have for a personal conversation assistant based on automatic speech recognition (ASR) technology. We created two prototypes which were evaluated by hearing impaired test users. This paper qualitatively compares the two based on the feedback obtained from the tests. The first prototype was a proof-of-concept system running real-time ASR on a laptop. The second prototype was developed for a mobile device with the recognizer running on a separate server. In the mobile device, augmented reality (AR) was used to help the hearing impaired observe gestures and lip movements of the speaker simultaneously with the transcriptions. Several testers found the systems useful enough to use in their daily lives, with majority preferring the mobile AR version. The biggest concern of the testers was the accuracy of the transcriptions and the lack of speaker identification.
Permanent Magnetic Articulograph (PMA) vs Electromagnetic Articulograph (EMA) in Articulation-to-Speech Synthesis for Silent Speech Interface
Authors: Beiming Cao, Nordine Sebkhi, Ted Mau, Omer T. Inan, Jun Wang
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Speech Synthesis

Silent speech interfaces (SSIs) are devices that enable speech communication when audible speech is unavailable. Articulation-to-speech (ATS) synthesis is a software design in SSI that directly converts articulatory movement information into audible speech signals. Permanent magnetic articulograph (PMA) is a wireless articulator motion tracking technology that is similar to commercial, wired Electromagnetic Articulograph (EMA). PMA has shown great potential for practical SSI applications, because it is wireless. The ATS performance of PMA, however, is unknown when compared with current EMA. In this study, we compared the performance of ATS using a PMA we recently developed and a commercially available EMA (NDI Wave system). Datasets with same stimuli and size that were collected from tongue tip were used in the comparison. The experimental results indicated the performance of PMA was close to, although not as equally good as that of EMA. Furthermore, in PMA, converting the raw magnetic signals to positional signals did not significantly affect the performance of ATS, which support the future direction in PMA-based ATS can be focused on the use of positional signals to maximize the benefit of spatial analysis.
Speech-based Estimation of Bulbar Regression in Amyotrophic Lateral Sclerosis
Authors: Alan Wisler, Kristin Teplansky, Jordan Green, Yana Yunusova, Thomas Campbell, Daragh Heitzman, Jun Wang
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: regression

Amyotrophic Lateral Sclerosis (ALS) is a progressive neurological disease that leads to degeneration of motor neurons and, as a result, inhibits the ability of the brain to control muscle movements. Monitoring the progression of ALS is of fundamental importance due to the wide variability in disease outlook that exists across patients. This progression is typically tracked using the ALS functional rating scale - revised (ALSFRS-R), which is the current clinical assessment of a patient{'}s level of functional impairment including speech and other motor tasks. In this paper, we investigated automatic estimation of the ALSFRS-R bulbar subscore from acoustic and articulatory movement samples. Experimental results demonstrated the AFSFRS-R bulbar subscore can be predicted from speech samples, which has clinical implication for automatic monitoring of the disease progression of ALS using speech information.
A Blissymbolics Translation System
Authors: Usman Sohail, David Traum
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Translation

Blissymbolics (Bliss) is a pictographic writing system that is used by people with communication disorders. Bliss attempts to create a writing system that makes words easier to distinguish by using pictographic symbols that encapsulate meaning rather than sound, as the English alphabet does for example. Users of Bliss rely on human interpreters to use Bliss. We created a translation system from Bliss to natural English with the hopes of decreasing the reliance on human interpreters by the Bliss community. We first discuss the basic rules of Blissymbolics. Then we point out some of the challenges associated with developing computer assisted tools for Blissymbolics. Next we talk about our ongoing work in developing a translation system, including current limitations, and future work. We conclude with a set of examples showing the current capabilities of our translation system.
Investigating Speech Recognition for Improving Predictive AAC
Authors: Jiban Adhikary, Robbie Watling, Crystal Fletcher, Alex Stanage, Keith Vertanen
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Language Modeling, Language Modelling, speech-recognition, Speech Recognition

Making good letter or word predictions can help accelerate the communication of users of high-tech AAC devices. This is particularly important for real-time person-to-person conversations. We investigate whether per forming speech recognition on the speaking-side of a conversation can improve language model based predictions. We compare the accuracy of three plausible microphone deployment options and the accuracy of two commercial speech recognition engines (Google and IBM Watson). We found that despite recognition word error rates of 7-16{\%}, our ensemble of N-gram and recurrent neural network language models made predictions nearly as good as when they used the reference transcripts.
Noisy Neural Language Modeling for Typing Prediction in BCI Communication
Authors: Rui Dong, David Smith, Shiran Dudy, Steven Bedrick
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Brain Computer Interface, Language Modeling, Language Modelling

Language models have broad adoption in predictive typing tasks. When the typing history contains numerous errors, as in open-vocabulary predictive typing with brain-computer interface (BCI) systems, we observe significant performance degradation in both n-gram and recurrent neural network language models trained on clean text. In evaluations of ranking character predictions, training recurrent LMs on noisy text makes them much more robust to noisy histories, even when the error model is misspecified. We also propose an effective strategy for combining evidence from multiple ambiguous histories of BCI electroencephalogram measurements.
``Caption'' as a Coherence Relation: Evidence and Implications
Authors: Malihe Alikhani, Matthew Stone
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Diversity, Image Retrieval, Relation, Retrieval

We study verbs in image{--}text corpora, contrasting \textit{caption} corpora, where texts are explicitly written to characterize image content, with \textit{depiction} corpora, where texts and images may stand in more general relations. Captions show a distinctively limited distribution of verbs, with strong preferences for specific tense, aspect, lexical aspect, and semantic field. These limitations, which appear in data elicited by a range of methods, restrict the utility of caption corpora to inform image retrieval, multimodal document generation, and perceptually-grounded semantic models. We suggest that these limitations reflect the discourse constraints in play when subjects write texts to accompany imagery, so we argue that future development of image{--}text corpora should work to increase the diversity of event descriptions, while looking explicitly at the different ways text and imagery can be coherently related.
Grounded Word Sense Translation
Authors: Chiraag Lala, Pranava Madhyastha, Lucia Specia
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Grounded language learning, Machine Translation, Multimodal Machine Translation, Question Answering, Translation, Visual Question Answering, Visual Question Answering (VQA)

Recent work on visually grounded language learning has focused on broader applications of grounded representations, such as visual question answering and multimodal machine translation. In this paper we consider grounded word sense translation, i.e. the task of correctly translating an ambiguous source word given the corresponding textual and visual context. Our main objective is to investigate the extent to which images help improve word-level (lexical) translation quality. We do so by first studying the dataset for this task to understand the scope and challenges of the task. We then explore different data settings, image features, and ways of grounding to investigate the gain from using images in each of the combinations. We find that grounding on the image is specially beneficial in weaker unidirectional recurrent translation models. We observe that adding structured image information leads to stronger gains in lexical translation accuracy.
Extracting Adverse Drug Event Information with Minimal Engineering
Authors: Timothy Miller, Alon Geva, Dmitriy Dligach
Date: 2019-06-01 00:00:00
Proceeding: WS 2019 6
Tasks: Attribute, Relation, Relation Extraction

In this paper we describe an evaluation of the potential of classical information extraction methods to extract drug-related attributes, including adverse drug events, and compare to more recently developed neural methods. We use the 2018 N2C2 shared task data as our gold standard data set for training. We train support vector machine classifiers to detect drug and drug attribute spans, and pair these detected entities as training instances for an SVM relation classifier, with both systems using standard features. We compare to baseline neural methods that use standard contextualized embedding representations for entity and relation extraction. The SVM-based system and a neural system obtain comparable results, with the SVM system doing better on concepts and the neural system performing better on relation extraction tasks. The neural system obtains surprisingly strong results compared to the system based on years of research in developing features for information extraction.