pwc-archive/papers-with-abstracts huggingface dataset:
Enhancing Machine Translation with Dependency-Aware Self-Attention
Authors: Emanuele Bugliarello, Naoaki Okazaki
Date: 2019-09-06 00:00:00
Proceeding: ACL 2020 6
Tasks: Machine Translation, Translation

Most neural machine translation models only rely on pairs of parallel sentences, assuming syntactic information is automatically learned by an attention mechanism. In this work, we investigate different approaches to incorporate syntactic knowledge in the Transformer model and also propose a novel, parameter-free, dependency-aware self-attention mechanism that improves its translation quality, especially for long sentences and in low-resource scenarios. We show the efficacy of each approach on WMT English-German and English-Turkish, and WAT English-Japanese translation tasks.
Self Learning from Large Scale Code Corpus to Infer Structure of Method Invocations
Authors: Hung Phan
Date: 2019-09-06 00:00:00
Tasks: Machine Translation, Self-Learning, Translation

Automatically generating code from a textual description of method invocation confronts challenges. There were two current research directions for this problem. One direction focuses on considering a textual description of method invocations as a separate Natural Language query and do not consider the surrounding context of the code. Another direction takes advantage of a practical large scale code corpus for providing a Machine Translation model to generate code. However, this direction got very low accuracy. In this work, we tried to improve these drawbacks by proposing MethodInfoToCode, an approach that embeds context information and optimizes the ability of learning of original Phrase-based Statistical Machine Translation (PBMT) in NLP to infer implementation of method invocation given method name and other context information. We conduct an expression prediction models learned from 2.86 million method invocations from the practical data of high qualities corpus on Github that used 6 popular libraries: JDK, Android, GWT, Joda-Time, Hibernate, and Xstream. By the evaluation, we show that if the developers only write the method name of a method invocation in a body of a method, MethodInfoToCode can predict the generated expression correctly at 73% in F1 score.
Geometry-Aware Video Object Detection for Static Cameras
Authors: Dan Xu, Weidi Xie, Andrew Zisserman
Date: 2019-09-06 00:00:00
Tasks: Autonomous Driving, feature selection, Human Detection, Object, object-detection, Object Detection, Video Object Detection

In this paper we propose a geometry-aware model for video object detection. Specifically, we consider the setting that cameras can be well approximated as static, e.g. in video surveillance scenarios, and scene pseudo depth maps can therefore be inferred easily from the object scale on the image plane. We make the following contributions: First, we extend the recent anchor-free detector (CornerNet [17]) to video object detections. In order to exploit the spatial-temporal information while maintaining high efficiency, the proposed model accepts video clips as input, and only makes predictions for the starting and the ending frames, i.e. heatmaps of object bounding box corners and the corresponding embeddings for grouping. Second, to tackle the challenge from scale variations in object detection, scene geometry information, e.g. derived depth maps, is explicitly incorporated into deep networks for multi-scale feature selection and for the network prediction. Third, we validate the proposed architectures on an autonomous driving dataset generated from the Carla simulator [5], and on a real dataset for human detection (DukeMTMC dataset [28]). When comparing with the existing competitive single-stage or two-stage detectors, the proposed geometry-aware spatio-temporal network achieves significantly better results.
To lemmatize or not to lemmatize: how word normalisation affects ELMo performance in word sense disambiguation
Authors: Andrey Kutuzov, Elizaveta Kuzmenko
Date: 2019-09-06 00:00:00
Proceeding: WS 2019 9
Tasks: Lemmatization, Word Sense Disambiguation

We critically evaluate the widespread assumption that deep learning NLP models do not require lemmatized input. To test this, we trained versions of contextualised word embedding ELMo models on raw tokenized corpora and on the corpora with word tokens replaced by their lemmas. Then, these models were evaluated on the word sense disambiguation task. This was done for the English and Russian languages. The experiments showed that while lemmatization is indeed not necessary for English, the situation is different for Russian. It seems that for rich-morphology languages, using lemmatized training and testing data yields small but consistent improvements: at least for word sense disambiguation. This means that the decisions about text pre-processing before training ELMo should consider the linguistic nature of the language in question.
DeepInSAR: A Deep Learning Framework for SAR Interferometric Phase Restoration and Coherence Estimation
Authors: Xinyao Sun, Aaron Zimmer, Subhayan Mukherjee, Navaneeth Kamballur Kottayil, Parwant Ghuman, Irene Cheng
Date: 2019-09-06 00:00:00
Tasks: 3D Plane Detection

Over the past decade, Interferometric Synthetic Aperture Radar (InSAR) has become a successful remote sensing technique. However, during the acquisition step, microwave reflections received at satellite are usually disturbed by strong noise, leading to a noisy single-look complex (SLC) SAR image. The quality of their interferometric phase is even worse. InSAR phase filtering is an ill-posed problem and plays a key role in subsequent processing. However, most of existing methods usually require expert supervision or heavy runtime, which limits the usability and scalability for practical usages such as wide-area monitoring and forecasting. In this work, we propose a deep convolutional neural network (CNN) based model DeepInSAR to intelligently solve both the phase filtering and coherence estimation problems. We demonstrate our DeepInSAR using both simulated and real data. A teacher-student framework is proposed to deal with the issue that there is no ground truth sample for real-world InSAR data. Quantitative and qualitative comparisons show that DeepInSAR achieves comparable or even better results than its stacked-based teacher method on new test datasets but requiring fewer pairs of SLCs as well as outperforms three other established non-stack based methods with less running time and no human supervision.
Trading-Off Static and Dynamic Regret in Online Least-Squares and Beyond
Authors: Jianjun Yuan, Andrew Lamperski
Date: 2019-09-06 00:00:00

Recursive least-squares algorithms often use forgetting factors as a heuristic to adapt to non-stationary data streams. The first contribution of this paper rigorously characterizes the effect of forgetting factors for a class of online Newton algorithms. For exp-concave and strongly convex objectives, the algorithms achieve the dynamic regret of $\max\{O(\log T),O(\sqrt{TV})\}$, where $V$ is a bound on the path length of the comparison sequence. In particular, we show how classic recursive least-squares with a forgetting factor achieves this dynamic regret bound. By varying $V$, we obtain a trade-off between static and dynamic regret. In order to obtain more computationally efficient algorithms, our second contribution is a novel gradient descent step size rule for strongly convex functions. Our gradient descent rule recovers the order optimal dynamic regret bounds described above. For smooth problems, we can also obtain static regret of $O(T^{1-\beta})$ and dynamic regret of $O(T^\beta V^*)$, where $\beta \in (0,1)$ and $V^*$ is the path length of the sequence of minimizers. By varying $\beta$, we obtain a trade-off between static and dynamic regret.
Machine Learning Approaches for Detecting the Depression from Resting-State Electroencephalogram (EEG): A Review Study
Authors: Milena Čukić Radenković, Victoria Lopez Lopez
Date: 2019-09-06 00:00:00
Tasks: BIG-bench Machine Learning, Depression Detection, Diagnostic, EEG, Electroencephalogram (EEG)

In this paper, we aimed at reviewing present literature on employing nonlinear analysis in combination with machine learning methods, in depression detection or prediction task. We are focusing on an affordable data-driven approach, applicable for everyday clinical practice, and in particular, those based on electroencephalographic (EEG) recordings. Among those studies utilizing EEG, we are discussing a group of applications used for detecting the depression based on the resting state EEG (detection studies) and interventional studies (using stimulus in their protocols or aiming to predict the outcome of therapy). We conclude with a discussion and review of guidelines to improve the reliability of developed models that could serve the improvement of diagnostic and more accurate treatment of depression.
High Resolution Medical Image Analysis with Spatial Partitioning
Authors: Le Hou, Youlong Cheng, Noam Shazeer, Niki Parmar, Yeqing Li, Panagiotis Korfiatis, Travis M. Drucker, Daniel J. Blezek, Xiaodan Song
Date: 2019-09-06 00:00:00
Tasks: GPU, Medical Image Analysis, Vocal Bursts Intensity Prediction

Medical images such as 3D computerized tomography (CT) scans and pathology images, have hundreds of millions or billions of voxels/pixels. It is infeasible to train CNN models directly on such high resolution images, because neural activations of a single image do not fit in the memory of a single GPU/TPU, and naive data and model parallelism approaches do not work. Existing image analysis approaches alleviate this problem by cropping or down-sampling input images, which leads to complicated implementation and sub-optimal performance due to information loss. In this paper, we implement spatial partitioning, which internally distributes the input and output of convolutional layers across GPUs/TPUs. Our implementation is based on the Mesh-TensorFlow framework and the computation distribution is transparent to end users. With this technique, we train a 3D Unet on up to 512 by 512 by 512 resolution data. To the best of our knowledge, this is the first work for handling such high resolution images end-to-end.
Efficient Sentence Embedding using Discrete Cosine Transform
Authors: Nada Almarwani, Hanan Aldarmaki, Mona Diab
Date: 2019-09-06 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Classification, General Classification, Sentence, Sentence Embedding, Sentence-Embedding

Vector averaging remains one of the most popular sentence embedding methods in spite of its obvious disregard for syntactic structure. While more complex sequential or convolutional networks potentially yield superior classification performance, the improvements in classification accuracy are typically mediocre compared to the simple vector averaging. As an efficient alternative, we propose the use of discrete cosine transform (DCT) to compress word sequences in an order-preserving manner. The lower order DCT coefficients represent the overall feature patterns in sentences, which results in suitable embeddings for tasks that could benefit from syntactic features. Our results in semantic probing tasks demonstrate that DCT embeddings indeed preserve more syntactic information compared with vector averaging. With practically equivalent complexity, the model yields better overall performance in downstream classification tasks that correlate with syntactic features, which illustrates the capacity of DCT to preserve word order information.
Self-supervised Dense 3D Reconstruction from Monocular Endoscopic Video
Authors: Xingtong Liu, Ayushi Sinha, Masaru Ishii, Gregory D. Hager, Russell H. Taylor, Mathias Unberath
Date: 2019-09-06 00:00:00
Tasks: 3D Reconstruction, Anatomy, Self-Supervised Learning

We present a self-supervised learning-based pipeline for dense 3D reconstruction from full-length monocular endoscopic videos without a priori modeling of anatomy or shading. Our method only relies on unlabeled monocular endoscopic videos and conventional multi-view stereo algorithms, and requires neither manual interaction nor patient CT in both training and application phases. In a cross-patient study using CT scans as groundtruth, we show that our method is able to produce photo-realistic dense 3D reconstructions with submillimeter mean residual errors from endoscopic videos from unseen patients and scopes.
Attending the Emotions to Detect Online Abusive Language
Authors: Niloofar Safi Samghabadi, Afsheen Hatami, Mahsa Shafaei, Sudipta Kar, Thamar Solorio
Date: 2019-09-06 00:00:00
Proceeding: EMNLP (ALW) 2020 11
Tasks: Abusive Language

In recent years, abusive behavior has become a serious issue in online social networks. In this paper, we present a new corpus from a semi-anonymous social media platform, which contains the instances of offensive and neutral classes. We introduce a single deep neural architecture that considers both local and sequential information from the text in order to detect abusive language. Along with this model, we introduce a new attention mechanism called emotion-aware attention. This mechanism utilizes the emotions behind the text to find the most important words within that text. We experiment with this model on our dataset and later present the analysis. Additionally, we evaluate our proposed method on different corpora and show new state-of-the-art results with respect to offensive language detection.
Abductive Reasoning as Self-Supervision for Common Sense Question Answering
Authors: Sathyanarayanan N. Aakur, Sudeep Sarkar
Date: 2019-09-06 00:00:00
Tasks: Common Sense Reasoning, Domain Adaptation, Question Answering

Question answering has seen significant advances in recent times, especially with the introduction of increasingly bigger transformer-based models pre-trained on massive amounts of data. While achieving impressive results on many benchmarks, their performances appear to be proportional to the amount of training data available in the target domain. In this work, we explore the ability of current question-answering models to generalize - to both other domains as well as with restricted training data. We find that large amounts of training data are necessary, both for pre-training as well as fine-tuning to a task, for the models to perform well on the designated task. We introduce a novel abductive reasoning approach based on Grenander's Pattern Theory framework to provide self-supervised domain adaptation cues or "pseudo-labels," which can be used instead of expensive human annotations. The proposed self-supervised training regimen allows for effective domain adaptation without losing performance compared to fully supervised baselines. Extensive experiments on two publicly available benchmarks show the efficacy of the proposed approach. We show that neural networks models trained using self-labeled data can retain up to $75\%$ of the performance of models trained on large amounts of human-annotated training data.
Automatic Critical Mechanic Discovery Using Playtraces in Video Games
Authors: Michael Cerny Green, Ahmed Khalifa, Gabriella A. B. Barros, Tiago Machado, Julian Togelius
Date: 2019-09-06 00:00:00

We present a new method of automatic critical mechanic discovery for video games using a combination of game description parsing and playtrace information. This method is applied to several games within the General Video Game Artificial Intelligence (GVG-AI) framework. In a user study, human-identified mechanics are compared against system-identified critical mechanics to verify alignment between humans and the system. The results of the study demonstrate that the new method is able to match humans with higher consistency than baseline. Our system is further validated by comparing MCTS agents augmented with critical mechanics and vanilla MCTS agents on $4$ games from GVG-AI. Our new playtrace method shows a significant performance improvement over the baseline for all 4 tested games. The proposed method also shows either matched or improved performance over the old method, demonstrating that playtrace information is responsible for more complete critical mechanic discovery.
Solving Interpretable Kernel Dimension Reduction
Authors: Chieh Wu, Jared Miller, Yale Chang, Mario Sznaier, Jennifer Dy
Date: 2019-09-06 00:00:00
Tasks: Clustering, Dimensionality Reduction

Kernel dimensionality reduction (KDR) algorithms find a low dimensional representation of the original data by optimizing kernel dependency measures that are capable of capturing nonlinear relationships. The standard strategy is to first map the data into a high dimensional feature space using kernels prior to a projection onto a low dimensional space. While KDR methods can be easily solved by keeping the most dominant eigenvectors of the kernel matrix, its features are no longer easy to interpret. Alternatively, Interpretable KDR (IKDR) is different in that it projects onto a subspace \textit{before} the kernel feature mapping, therefore, the projection matrix can indicate how the original features linearly combine to form the new features. Unfortunately, the IKDR objective requires a non-convex manifold optimization that is difficult to solve and can no longer be solved by eigendecomposition. Recently, an efficient iterative spectral (eigendecomposition) method (ISM) has been proposed for this objective in the context of alternative clustering. However, ISM only provides theoretical guarantees for the Gaussian kernel. This greatly constrains ISM's usage since any kernel method using ISM is now limited to a single kernel. This work extends the theoretical guarantees of ISM to an entire family of kernels, thereby empowering ISM to solve any kernel method of the same objective. In identifying this family, we prove that each kernel within the family has a surrogate $\Phi$ matrix and the optimal projection is formed by its most dominant eigenvectors. With this extension, we establish how a wide range of IKDR applications across different learning paradigms can be solved by ISM. To support reproducible results, the source code is made publicly available on \url{https://github.com/chieh-neu/ISM_supervised_DR}.
A framework for seizure detection using effective connectivity, graph theory and deep modular neural networks
Authors: Behnaz Akbarian, Abbas Erfanian
Date: 2019-09-06 00:00:00
Tasks: EEG, Electroencephalogram (EEG), General Classification, Seizure Detection

Objective The electrical characteristics of the EEG signals can be used for seizure detection. Statistical independence between different brain regions is measured by functional brain connectivity (FBC). Specific directional effects can't consider by FBC and thus effective brain connectivity (EBC) is used to measure causal intervention between one neuronal region and the rest of the neuronal regions. Our main purpose is to provide a reliable automatic seizure detection approach. Methods In this study, three new methods are provided. Deep modular neural network (DMNN) is developed based on a combination of various EBC classification results in the different frequencies. Another method is named "modular effective neural networks (MENN)". This method combines the classification results of the three different EBC in the specific frequency. "Modular frequency neural networks (MFNN)" is another method that combines the classification results of the specific EBC in the seven different frequencies. Results The mean accuracy of the MFNN are 97.14%, 98.53%, and 97.91% using directed transfer function, directed coherence, and generalized partial directed coherence, respectively. Using the MENN, the highest mean accuracy is 98.34%. Finally, DMNN has the highest mean accuracy which is equal to 99.43. To our best knowledge, the proposed method is a new method that provides the high accuracy in comparison to other studies which used MIT-CHB database. Conclusion and significance The knowledge of structure-function relationships between different areas of the brain is necessary for characterizing the underlying dynamics. Hence, features based on EBC can provide a reliable automatic seizure detection approach.
ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons
Authors: Margaret Li, Jason Weston, Stephen Roller
Date: 2019-09-06 00:00:00
Tasks: Dialogue Evaluation

While dialogue remains an important end-goal of natural language research, the difficulty of evaluation is an oft-quoted reason why it remains troublesome to make real progress towards its solution. Evaluation difficulties are actually two-fold: not only do automatic metrics not correlate well with human judgments, but also human judgments themselves are in fact difficult to measure. The two most used human judgment tests, single-turn pairwise evaluation and multi-turn Likert scores, both have serious flaws as we discuss in this work. We instead provide a novel procedure involving comparing two full dialogues, where a human judge is asked to pay attention to only one speaker within each, and make a pairwise judgment. The questions themselves are optimized to maximize the robustness of judgments across different annotators, resulting in better tests. We also show how these tests work in self-play model chat setups, resulting in faster, cheaper tests. We hope these tests become the de facto standard, and will release open-source code to that end.
Learning to Discriminate Perturbations for Blocking Adversarial Attacks in Text Classification
Authors: Yichao Zhou, Jyun-Yu Jiang, Kai-Wei Chang, Wei Wang
Date: 2019-09-06 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Blocking, General Classification, Sentiment Analysis, text-classification, Text Classification

Adversarial attacks against machine learning models have threatened various real-world applications such as spam filtering and sentiment analysis. In this paper, we propose a novel framework, learning to DIScriminate Perturbations (DISP), to identify and adjust malicious perturbations, thereby blocking adversarial attacks for text classification models. To identify adversarial attacks, a perturbation discriminator validates how likely a token in the text is perturbed and provides a set of potential perturbations. For each potential perturbation, an embedding estimator learns to restore the embedding of the original word based on the context and a replacement token is chosen based on approximate kNN search. DISP can block adversarial attacks for any NLP model without modifying the model structure or training procedure. Extensive experiments on two benchmark datasets demonstrate that DISP significantly outperforms baseline methods in blocking adversarial attacks for text classification. In addition, in-depth analysis shows the robustness of DISP across different situations.
One Size Does Not Fit All: Multi-Scale, Cascaded RNNs for Radar Classification
Authors: Dhrubojyoti Roy, Sangeeta Srivastava, Aditya Kusupati, Pranshu Jain, Manik Varma, Anish Arora
Date: 2019-09-06 00:00:00
Tasks: All, Feature Engineering, General Classification

Edge sensing with micro-power pulse-Doppler radars is an emergent domain in monitoring and surveillance with several smart city applications. Existing solutions for the clutter versus multi-source radar classification task are limited in terms of either accuracy or efficiency, and in some cases, struggle with a trade-off between false alarms and recall of sources. We find that this problem can be resolved by learning the classifier across multiple time-scales. We propose a multi-scale, cascaded recurrent neural network architecture, MSC-RNN, comprised of an efficient multi-instance learning (MIL) Recurrent Neural Network (RNN) for clutter discrimination at a lower tier, and a more complex RNN classifier for source classification at the upper tier. By controlling the invocation of the upper RNN with the help of the lower tier conditionally, MSC-RNN achieves an overall accuracy of 0.972. Our approach holistically improves the accuracy and per-class recalls over ML models suitable for radar inferencing. Notably, we outperform cross-domain handcrafted feature engineering with time-domain deep feature learning, while also being up to $\sim$3$\times$ more efficient than a competitive solution.