pwc-archive/papers-with-abstracts huggingface dataset:
Contextualized Sparse Representations for Real-Time Open-Domain Question Answering
Authors: Jinhyuk Lee, Minjoon Seo, Hannaneh Hajishirzi, Jaewoo Kang
Date: 2019-11-07 00:00:00
Proceeding: ACL 2020 6
Tasks: Information Retrieval, Open-Domain Question Answering, Question Answering, Retrieval

Open-domain question answering can be formulated as a phrase retrieval problem, in which we can expect huge scalability and speed benefit but often suffer from low accuracy due to the limitation of existing phrase representation models. In this paper, we aim to improve the quality of each phrase embedding by augmenting it with a contextualized sparse representation (Sparc). Unlike previous sparse vectors that are term-frequency-based (e.g., tf-idf) or directly learned (only few thousand dimensions), we leverage rectified self-attention to indirectly learn sparse vectors in n-gram vocabulary space. By augmenting the previous phrase retrieval model (Seo et al., 2019) with Sparc, we show 4%+ improvement in CuratedTREC and SQuAD-Open. Our CuratedTREC score is even better than the best known retrieve & read model with at least 45x faster inference speed.
Improving Joint Training of Inference Networks and Structured Prediction Energy Networks
Authors: Lifu Tu, Richard Yuanzhe Pang, Kevin Gimpel
Date: 2019-11-07 00:00:00
Proceeding: EMNLP (spnlp) 2020 11
Tasks: Structured Prediction

Deep energy-based models are powerful, but pose challenges for learning and inference (Belanger and McCallum, 2016). Tu and Gimpel (2018) developed an efficient framework for energy-based models by training "inference networks" to approximate structured inference instead of using gradient descent. However, their alternating optimization approach suffers from instabilities during training, requiring additional loss terms and careful hyperparameter tuning. In this paper, we contribute several strategies to stabilize and improve this joint training of energy functions and inference networks for structured prediction. We design a compound objective to jointly train both cost-augmented and test-time inference networks along with the energy function. We propose joint parameterizations for the inference networks that encourage them to capture complementary functionality during learning. We empirically validate our strategies on two sequence labeling tasks, showing easier paths to strong performance than prior work, as well as further improvements with global energy terms.
This dataset does not exist: training models from generated images
Authors: Victor Besnier, Himalaya Jain, Andrei Bursuc, Matthieu Cord, Patrick Pérez
Date: 2019-11-07 00:00:00

Current generative networks are increasingly proficient in generating high-resolution realistic images. These generative networks, especially the conditional ones, can potentially become a great tool for providing new image datasets. This naturally brings the question: Can we train a classifier only on the generated data? This potential availability of nearly unlimited amounts of training data challenges standard practices for training machine learning models, which have been crafted across the years for limited and fixed size datasets. In this work we investigate this question and its related challenges. We identify ways to improve significantly the performance over naive training on randomly generated images with regular heuristics. We propose three standalone techniques that can be applied at different stages of the pipeline, i.e., data generation, training on generated data, and deploying on real data. We evaluate our proposed approaches on a subset of the ImageNet dataset and show encouraging results compared to classifiers trained on real images.
Hierarchical Finite State Controllers for Generalized Planning
Authors: Javier Segovia-Aguas, Sergio Jiménez, Anders Jonsson
Date: 2019-11-07 00:00:00

Finite State Controllers (FSCs) are an effective way to represent sequential plans compactly. By imposing appropriate conditions on transitions, FSCs can also represent generalized plans that solve a range of planning problems from a given domain. In this paper we introduce the concept of {\it hierarchical FSCs} for planning by allowing controllers to call other controllers. We show that hierarchical FSCs can represent generalized plans more compactly than individual FSCs. Moreover, our call mechanism makes it possible to generate hierarchical FSCs in a modular fashion, or even to apply recursion. We also introduce a compilation that enables a classical planner to generate hierarchical FSCs that solve challenging generalized planning problems. The compilation takes as input a set of planning problems from a given domain and outputs a single classical planning problem, whose solution corresponds to a hierarchical FSC.
Graph Domain Adaptation with Localized Graph Signal Representations
Authors: Yusuf Yigit Pilavci, Eylem Tugce Guneyi, Cemil Cengiz, Elif Vural
Date: 2019-11-07 00:00:00
Tasks: Domain Adaptation, GRAPH DOMAIN ADAPTATION

In this paper we propose a domain adaptation algorithm designed for graph domains. Given a source graph with many labeled nodes and a target graph with few or no labeled nodes, we aim to estimate the target labels by making use of the similarity between the characteristics of the variation of the label functions on the two graphs. Our assumption about the source and the target domains is that the local behaviour of the label function, such as its spread and speed of variation on the graph, bears resemblance between the two graphs. We estimate the unknown target labels by solving an optimization problem where the label information is transferred from the source graph to the target graph based on the prior that the projections of the label functions onto localized graph bases be similar between the source and the target graphs. In order to efficiently capture the local variation of the label functions on the graphs, spectral graph wavelets are used as the graph bases. Experimentation on various data sets shows that the proposed method yields quite satisfactory classification accuracy compared to reference domain adaptation methods.
Deep neural network Grad-Shafranov solver constrained with measured magnetic signals
Authors: Semin Joung, Jaewook Kim, Sehyun Kwak, J. G. Bak, S. G. Lee, H. S. Han, H. S. Kim, Geunho Lee, Daeho Kwon, Y. -c. Ghim
Date: 2019-11-07 00:00:00
Tasks: Imputation

A neural network solving Grad-Shafranov equation constrained with measured magnetic signals to reconstruct magnetic equilibria in real time is developed. Database created to optimize the neural network's free parameters contain off-line EFIT results as the output of the network from $1,118$ KSTAR experimental discharges of two different campaigns. Input data to the network constitute magnetic signals measured by a Rogowski coil (plasma current), magnetic pick-up coils (normal and tangential components of magnetic fields) and flux loops (poloidal magnetic fluxes). The developed neural networks fully reconstruct not only the poloidal flux function $\psi\left( R, Z\right)$ but also the toroidal current density function $j_\phi\left( R, Z\right)$ with the off-line EFIT quality. To preserve robustness of the networks against a few missing input data, an imputation scheme is utilized to eliminate the required additional training sets with large number of possible combinations of the missing inputs.
$H_\infty$ Model-free Reinforcement Learning with Robust Stability Guarantee
Authors: Minghao Han, Yuan Tian, Lixian Zhang, Jun Wang, Wei Pan
Date: 2019-11-07 00:00:00
Tasks: Autonomous Driving, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Robot Manipulation

Reinforcement learning is showing great potentials in robotics applications, including autonomous driving, robot manipulation and locomotion. However, with complex uncertainties in the real-world environment, it is difficult to guarantee the successful generalization and sim-to-real transfer of learned policies theoretically. In this paper, we introduce and extend the idea of robust stability and $H_\infty$ control to design policies with both stability and robustness guarantee. Specifically, a sample-based approach for analyzing the Lyapunov stability and performance robustness of a learning-based control system is proposed. Based on the theoretical results, a maximum entropy algorithm is developed for searching Lyapunov function and designing a policy with provable robust stability guarantee. Without any specific domain knowledge, our method can find a policy that is robust to various uncertainties and generalizes well to different test environments. In our experiments, we show that our method achieves better robustness to both large impulsive disturbances and parametric variations in the environment than the state-of-art results in both robust and generic RL, as well as classic control. Anonymous code is available to reproduce the experimental results at https://github.com/RobustStabilityGuaranteeRL/RobustStabilityGuaranteeRL.
Dice Loss for Data-imbalanced NLP Tasks
Authors: Xiaoya Li, Xiaofei Sun, Yuxian Meng, Junjun Liang, Fei Wu, Jiwei Li
Date: 2019-11-07 00:00:00
Proceeding: ACL 2020 6
Tasks: Chinese Named Entity Recognition, Machine Reading Comprehension, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Paraphrase Identification, Part-Of-Speech Tagging, Question Answering, Reading Comprehension

Many NLP tasks such as tagging and machine reading comprehension are faced with the severe data imbalance issue: negative examples significantly outnumber positive examples, and the huge number of background examples (or easy-negative examples) overwhelms the training. The most commonly used cross entropy (CE) criteria is actually an accuracy-oriented objective, and thus creates a discrepancy between training and test: at training time, each training instance contributes equally to the objective function, while at test time F1 score concerns more about positive examples. In this paper, we propose to use dice loss in replacement of the standard cross-entropy objective for data-imbalanced NLP tasks. Dice loss is based on the Sorensen-Dice coefficient or Tversky index, which attaches similar importance to false positives and false negatives, and is more immune to the data-imbalance issue. To further alleviate the dominating influence from easy-negative examples in training, we propose to associate training examples with dynamically adjusted weights to deemphasize easy-negative examples.Theoretical analysis shows that this strategy narrows down the gap between the F1 score in evaluation and the dice loss in training. With the proposed training objective, we observe significant performance boost on a wide range of data imbalanced NLP tasks. Notably, we are able to achieve SOTA results on CTB5, CTB6 and UD1.4 for the part of speech tagging task; SOTA results on CoNLL03, OntoNotes5.0, MSRA and OntoNotes4.0 for the named entity recognition task; along with competitive results on the tasks of machine reading comprehension and paraphrase identification.
Dependency and Span, Cross-Style Semantic Role Labeling on PropBank and NomBank
Authors: Zuchao Li, Hai Zhao, Junru Zhou, Kevin Parnow, Shexia He
Date: 2019-11-07 00:00:00
Tasks: Semantic Role Labeling

The latest developments in neural semantic role labeling (SRL) have shown great performance improvements with both the dependency and span formalisms/styles. Although the two styles share many similarities in linguistic meaning and computation, most previous studies focus on a single style. In this paper, we define a new cross-style semantic role label convention and propose a new cross-style joint optimization model designed around the most basic linguistic meaning of a semantic role, providing a solution to make the results of the two styles more comparable and allowing both formalisms of SRL to benefit from their natural connections in both linguistics and computation. Our model learns a general semantic argument structure and is capable of outputting in either style. Additionally, we propose a syntax-aided method to uniformly enhance the learning of both dependency and span representations. Experiments show that the proposed methods are effective on both span and dependency SRL benchmarks.
CROWN: Conversational Passage Ranking by Reasoning over Word Networks
Authors: Magdalena Kaiser, Rishiraj Saha Roy, Gerhard Weikum
Date: 2019-11-07 00:00:00
Tasks: Passage Ranking

Information needs around a topic cannot be satisfied in a single turn; users typically ask follow-up questions referring to the same theme and a system must be capable of understanding the conversational context of a request to retrieve correct answers. In this paper, we present our submission to the TREC Conversational Assistance Track 2019, in which such a conversational setting is explored. We propose a simple unsupervised method for conversational passage ranking by formulating the passage score for a query as a combination of similarity and coherence. To be specific, passages are preferred that contain words semantically similar to the words used in the question, and where such words appear close by. We built a word-proximity network (WPN) from a large corpus, where words are nodes and there is an edge between two nodes if they co-occur in the same passages in a statistically significant way, within a context window. Our approach, named CROWN, improved nDCG scores over a provided Indri baseline on the CAsT training data. On the evaluation data for CAsT, our best run submission achieved above-average performance with respect to AP@5 and nDCG@1000.
Explicit Pairwise Word Interaction Modeling Improves Pretrained Transformers for English Semantic Similarity Tasks
Authors: Yinan Zhang, Raphael Tang, Jimmy Lin
Date: 2019-11-07 00:00:00
Tasks: Semantic Similarity, Semantic Textual Similarity, Sentence

In English semantic similarity tasks, classic word embedding-based approaches explicitly model pairwise "interactions" between the word representations of a sentence pair. Transformer-based pretrained language models disregard this notion, instead modeling pairwise word interactions globally and implicitly through their self-attention mechanism. In this paper, we hypothesize that introducing an explicit, constrained pairwise word interaction mechanism to pretrained language models improves their effectiveness on semantic similarity tasks. We validate our hypothesis using BERT on four tasks in semantic textual similarity and answer sentence selection. We demonstrate consistent improvements in quality by adding an explicit pairwise word interaction module to BERT.
Teacher-Student Training for Robust Tacotron-based TTS
Authors: Rui Liu, Berrak Sisman, Jingdong Li, Feilong Bao, Guanglai Gao, Haizhou Li
Date: 2019-11-07 00:00:00
Tasks: Decoder, Knowledge Distillation, text-to-speech, Text to Speech

While neural end-to-end text-to-speech (TTS) is superior to conventional statistical methods in many ways, the exposure bias problem in the autoregressive models remains an issue to be resolved. The exposure bias problem arises from the mismatch between the training and inference process, that results in unpredictable performance for out-of-domain test data at run-time. To overcome this, we propose a teacher-student training scheme for Tacotron-based TTS by introducing a distillation loss function in addition to the feature loss function. We first train a Tacotron2-based TTS model by always providing natural speech frames to the decoder, that serves as a teacher model. We then train another Tacotron2-based model as a student model, of which the decoder takes the predicted speech frames as input, similar to how the decoder works during run-time inference. With the distillation loss, the student model learns the output probabilities from the teacher model, that is called knowledge distillation. Experiments show that our proposed training scheme consistently improves the voice quality for out-of-domain test data both in Chinese and English systems.
Post-mortem Iris Decomposition and its Dynamics in Morgue Conditions
Authors: Mateusz Trokielewicz, Adam Czajka, Piotr Maciejewicz
Date: 2019-11-07 00:00:00
Tasks: Iris Recognition

With increasing interest in employing iris biometrics as a forensic tool for identification by investigation authorities, there is a need for a thorough examination and understanding of post-mortem decomposition processes that take place within the human eyeball, especially the iris. This can prove useful for fast and accurate matching of ante-mortem with post-mortem data acquired at crime scenes or mass casualties, as well as for ensuring correct dispatching of bodies from the incident scene to a mortuary or funeral homes. Following these needs of forensic community, this paper offers an analysis of the coarse effects of eyeball decay done from a perspective of automatic iris recognition point of view. Therefore, we analyze post-mortem iris images acquired in both visible light as well as in near-infrared light (860 nm), as the latter wavelength is used in commercial iris recognition systems. Conclusions and suggestions are provided that may aid forensic examiners in successfully utilizing iris patterns in post-mortem identification of deceased subjects. Initial guidelines regarding the imaging process, types of illumination, resolution are also given, together with expectations with respect to the iris features decomposition rates.
ViSTRA2: Video Coding using Spatial Resolution and Effective Bit Depth Adaptation
Authors: Fan Zhang, Mariana Afonso, David R. Bull
Date: 2019-11-07 00:00:00
Tasks: Decoder, Video Compression

We present a new video compression framework (ViSTRA2) which exploits adaptation of spatial resolution and effective bit depth, down-sampling these parameters at the encoder based on perceptual criteria, and up-sampling at the decoder using a deep convolution neural network. ViSTRA2 has been integrated with the reference software of both the HEVC (HM 16.20) and VVC (VTM 4.01), and evaluated under the Joint Video Exploration Team Common Test Conditions using the Random Access configuration. Our results show consistent and significant compression gains against HM and VVC based on Bj{\o}negaard Delta measurements, with average BD-rate savings of 12.6% (PSNR) and 19.5% (VMAF) over HM and 5.5% (PSNR) and 8.6% (VMAF) over VTM.
Improving Grammatical Error Correction with Machine Translation Pairs
Authors: Wangchunshu Zhou, Tao Ge, Chang Mu, Ke Xu, Furu Wei, Ming Zhou
Date: 2019-11-07 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Grammatical Error Correction, Language Modeling, Language Modelling, Machine Translation, Sentence, Translation

We propose a novel data synthesis method to generate diverse error-corrected sentence pairs for improving grammatical error correction, which is based on a pair of machine translation models of different qualities (i.e., poor and good). The poor translation model resembles the ESL (English as a second language) learner and tends to generate translations of low quality in terms of fluency and grammatical correctness, while the good translation model generally generates fluent and grammatically correct translations. We build the poor and good translation model with phrase-based statistical machine translation model with decreased language model weight and neural machine translation model respectively. By taking the pair of their translations of the same sentences in a bridge language as error-corrected sentence pairs, we can construct unlimited pseudo parallel data. Our approach is capable of generating diverse fluency-improving patterns without being limited by the pre-defined rule set and the seed error-corrected data. Experimental results demonstrate the effectiveness of our approach and show that it can be combined with other synthetic data sources to yield further improvements.
Enhancing Pre-trained Chinese Character Representation with Word-aligned Attention
Authors: Yanzeng Li, Bowen Yu, Mengge Xue, Tingwen Liu
Date: 2019-11-07 00:00:00
Proceeding: ACL 2020 6

Most Chinese pre-trained models take character as the basic unit and learn representation according to character's external contexts, ignoring the semantics expressed in the word, which is the smallest meaningful utterance in Chinese. Hence, we propose a novel word-aligned attention to exploit explicit word information, which is complementary to various character-based Chinese pre-trained language models. Specifically, we devise a pooling mechanism to align the character-level attention to the word level and propose to alleviate the potential issue of segmentation error propagation by multi-source information fusion. As a result, word and character information are explicitly integrated at the fine-tuning procedure. Experimental results on five Chinese NLP benchmark tasks demonstrate that our model could bring another significant gain over several pre-trained models.
Transition-Based Deep Input Linearization
Authors: Ratish Puduppully, Yue Zhang, Manish Shrivastava
Date: 2019-11-07 00:00:00
Proceeding: EACL 2017 4
Tasks: Data-to-Text Generation, Machine Translation

Traditional methods for deep NLG adopt pipeline approaches comprising stages such as constructing syntactic input, predicting function words, linearizing the syntactic input and generating the surface forms. Though easier to visualize, pipeline approaches suffer from error propagation. In addition, information available across modules cannot be leveraged by all modules. We construct a transition-based model to jointly perform linearization, function word prediction and morphological generation, which considerably improves upon the accuracy compared to a pipelined baseline system. On a standard deep input linearization shared task, our system achieves the best results reported so far.
Improving Human Annotation in Single Object Tracking
Authors: Yu Pang, Xinyi Li, Lin Yuan, Haibin Ling
Date: 2019-11-07 00:00:00
Tasks: Object, Object Tracking, Video Object Tracking

Human annotation is always considered as ground truth in video object tracking tasks. It is used in both training and evaluation purposes. Thus, ensuring its high quality is an important task for the success of trackers and evaluations between them. In this paper, we give a qualitative and quantitative analysis of the existing human annotations. We show that human annotation tends to be non-smooth and is prone to partial visibility and deformation. We propose a smoothing trajectory strategy with the ability to handle moving scenes. We use a two-step adaptive image alignment algorithm to find the canonical view of the video sequence. We then use different techniques to smooth the trajectories at certain degree. Once we convert back to the original image coordination, we can compare with the human annotation. With the experimental results, we can get more consistent trajectories. At a certain degree, it can also slightly improve the trained model. If go beyond a certain threshold, the smoothing error will start eating up the benefit. Overall, our method could help extrapolate the missing annotation frames or identify and correct human annotation outliers as well as help improve the training data quality.