pwc-archive/papers-with-abstracts huggingface dataset:
Investigating the Effectiveness of Representations Based on Pretrained Transformer-based Language Models in Active Learning for Labelling Text Datasets
Authors: Jinghui Lu, Brian MacNamee
Date: 2020-04-21 00:00:00
Tasks: Active Learning, Word Embeddings

Active learning has been shown to be an effective way to alleviate some of the effort required in utilising large collections of unlabelled data for machine learning tasks without needing to fully label them. The representation mechanism used to represent text documents when performing active learning, however, has a significant influence on how effective the process will be. While simple vector representations such as bag-of-words and embedding-based representations based on techniques such as word2vec have been shown to be an effective way to represent documents during active learning, the emergence of representation mechanisms based on the pre-trained transformer-based neural network models popular in natural language processing research (e.g. BERT) offer a promising, and as yet not fully explored, alternative. This paper describes a comprehensive evaluation of the effectiveness of representations based on pre-trained transformer-based language models for active learning. This evaluation shows that transformer-based models, especially BERT-like models, that have not yet been widely used in active learning, achieve a significant improvement over more commonly used vector representations like bag-of-words or other classical word embeddings like word2vec. This paper also investigates the effectiveness of representations based on variants of BERT such as Roberta, Albert as well as comparing the effectiveness of the [CLS] token representation and the aggregated representation that can be generated using BERT-like models. Finally, we propose an approach Adaptive Tuning Active Learning. Our experiments show that the limited label information acquired in active learning can not only be used for training a classifier but can also adaptively improve the embeddings generated by the BERT-like language models as well.
A Scalable Feature Selection and Opinion Miner Using Whale Optimization Algorithm
Authors: Amir Javadpour, Samira Rezaei, Kuan-Ching Li, Guojun Wang
Date: 2020-04-21 00:00:00
Tasks: feature selection, General Classification

Due to the fast-growing volume of text documents and reviews in recent years, current analyzing techniques are not competent enough to meet the users' needs. Using feature selection techniques not only support to understand data better but also lead to higher speed and also accuracy. In this article, the Whale Optimization algorithm is considered and applied to the search for the optimum subset of features. As known, F-measure is a metric based on precision and recall that is very popular in comparing classifiers. For the evaluation and comparison of the experimental results, PART, random tree, random forest, and RBF network classification algorithms have been applied to the different number of features. Experimental results show that the random forest has the best accuracy on 500 features. Keywords: Feature selection, Whale Optimization algorithm, Selecting optimal, Classification algorithm
Colon Shape Estimation Method for Colonoscope Tracking using Recurrent Neural Networks
Authors: Masahiro Oda, Holger R. Roth, Takayuki Kitasaka, Kazuhiro Furukawa, Ryoji Miyahara, Yoshiki Hirooka, Hidemi Goto, Nassir Navab, Kensaku MORI
Date: 2020-04-20 00:00:00

We propose an estimation method using a recurrent neural network (RNN) of the colon's shape where deformation was occurred by a colonoscope insertion. Colonoscope tracking or a navigation system that navigates physician to polyp positions is needed to reduce such complications as colon perforation. Previous tracking methods caused large tracking errors at the transverse and sigmoid colons because these areas largely deform during colonoscope insertion. Colon deformation should be taken into account in tracking processes. We propose a colon deformation estimation method using RNN and obtain the colonoscope shape from electromagnetic sensors during its insertion into the colon. This method obtains positional, directional, and an insertion length from the colonoscope shape. From its shape, we also calculate the relative features that represent the positional and directional relationships between two points on a colonoscope. Long short-term memory is used to estimate the current colon shape from the past transition of the features of the colonoscope shape. We performed colon shape estimation in a phantom study and correctly estimated the colon shapes during colonoscope insertion with 12.39 (mm) estimation error.
Data Augmentation Imbalance For Imbalanced Attribute Classification
Authors: Yang Hu, Xiaying Bai, Pan Zhou, Fanhua Shang, ShengMei Shen
Date: 2020-04-19 00:00:00
Tasks: Attribute, Classification, Data Augmentation, General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Pedestrian Attribute Recognition

Pedestrian attribute recognition is an important multi-label classification problem. Although the convolutional neural networks are prominent in learning discriminative features from images, the data imbalance in multi-label setting for fine-grained tasks remains an open problem. In this paper, we propose a new re-sampling algorithm called: data augmentation imbalance (DAI) to explicitly enhance the ability to discriminate the fewer attributes via increasing the proportion of labels accounting for a small part. Fundamentally, by applying over-sampling and under-sampling on the multi-label dataset at the same time, the thought of robbing the rich attributes and helping the poor makes a significant contribution to DAI. Extensive empirical evidence shows that our DAI algorithm achieves state-of-the-art results, based on pedestrian attribute datasets, i.e. standard PA-100K and PETA datasets.
The Effect of the Multi-Layer Text Summarization Model on the Efficiency and Relevancy of the Vector Space-based Information Retrieval
Authors: Ahmad Hussein Ababneh, Joan Lu, Qiang Xu
Date: 2020-04-18 00:00:00
Tasks: Information Retrieval, Retrieval, Text Summarization

The massive upload of text on the internet creates a huge inverted index in information retrieval systems, which hurts their efficiency. The purpose of this research is to measure the effect of the Multi-Layer Similarity model of the automatic text summarization on building an informative and condensed invert index in the IR systems. To achieve this purpose, we summarized a considerable number of documents using the Multi-Layer Similarity model, and we built the inverted index from the automatic summaries that were generated from this model. A series of experiments were held to test the performance in terms of efficiency and relevancy. The experiments include comparisons with three existing text summarization models; the Jaccard Coefficient Model, the Vector Space Model, and the Latent Semantic Analysis model. The experiments examined three groups of queries with manual and automatic relevancy assessment. The positive effect of the Multi-Layer Similarity in the efficiency of the IR system was clear without noticeable loss in the relevancy results. However, the evaluation showed that the traditional statistical models without semantic investigation failed to improve the information retrieval efficiency. Comparing with the previous publications that addressed the use of summaries as a source of the index, the relevancy assessment of our work was higher, and the Multi-Layer Similarity retrieval constructed an inverted index that was 58% smaller than the main corpus inverted index.
Convolutional Framework for Accelerated Magnetic Resonance Imaging
Authors: Shen Zhao, Lee C. Potter, Kiryung Lee, Rizwan Ahmad
Date: 2020-02-08 00:00:00
Tasks: Image Reconstruction

Magnetic Resonance Imaging (MRI) is a noninvasive imaging technique that provides exquisite soft-tissue contrast without using ionizing radiation. The clinical application of MRI may be limited by long data acquisition times; therefore, MR image reconstruction from highly undersampled k-space data has been an active area of research. Many works exploit rank deficiency in a Hankel data matrix to recover unobserved k-space samples; the resulting problem is non-convex, so the choice of numerical algorithm can significantly affect performance, computation, and memory. We present a simple, scalable approach called Convolutional Framework (CF). We demonstrate the feasibility and versatility of CF using measured data from 2D, 3D, and dynamic applications.
High-dimensional Fast Convolutional Framework (HICU) for Calibrationless MRI
Authors: Shen Zhao, Lee C. Potter, Rizwan Ahmad
Date: 2020-04-19 00:00:00
Tasks: compressed sensing, MRI Reconstruction, Vocal Bursts Intensity Prediction

Purpose: To present a computational procedure for accelerated, calibrationless magnetic resonance image (Cl-MRI) reconstruction that is fast, memory efficient, and scales to high-dimensional imaging. Theory and Methods: Cl-MRI methods can enable high acceleration rates and flexible sampling patterns, but their clinical application is limited by computational complexity and large memory footprint. The proposed computational procedure, HIgh-dimensional fast ConvolU-tional framework (HICU), provides fast, memory-efficient recovery of unsampled k-space points. For demonstration, HICU is applied to six 2D T2-weighted brain, seven 2D cardiac cine, five 3D knee, and one multi-shot diffusion weighted imaging (MSDWI) datasets. Results: The 2D imaging results show that HICU can offer one to two orders of magnitude computation speedup compared to other Cl-MRI methods without sacrificing imaging quality. The 2D cine and 3D imaging results show that the computational acceleration techniques included in HICU yield computing time on par with SENSE-based compressed sensing methods with up to 3 dB improvement in signal-to-error ratio and better perceptual quality. The MSDWI results demonstrate the feasibility of HICU for a challenging multi-shot echo-planar imaging application. Conclusions: The presented method, HICU, offers efficient computation and scalability as well as extendibility to a wide variety of MRI applications.
Towards Faster Reasoners By Using Transparent Huge Pages
Authors: Johannes K. Fichte, Norbert Manthey, Julian Stecklina, André Schidler
Date: 2020-04-29 00:00:00
Tasks: Management

Various state-of-the-art automated reasoning (AR) tools are widely used as backend tools in research of knowledge representation and reasoning as well as in industrial applications. In testing and verification, those tools often run continuously or nightly. In this work, we present an approach to reduce the runtime of AR tools by 10% on average and up to 20% for long running tasks. Our improvement addresses the high memory usage that comes with the data structures used in AR tools, which are based on conflict driven no-good learning. We establish a general way to enable faster memory access by using the memory cache line of modern hardware more effectively. Therefore, we extend the standard C library (glibc) by dynamically allowing to use a memory management feature called huge pages. Huge pages allow to reduce the overhead that is required to translate memory addresses between the virtual memory of the operating system and the physical memory of the hardware. In that way, we can reduce runtime, costs, and energy consumption of AR tools and applications with similar memory access patterns simply by linking the tool against this new glibc library when compiling it. In every day industrial applications this easily allows to be more eco-friendly in computation. To back up the claimed speed-up, we present experimental results for tools that are commonly used in the AR community, including the domains ASP, BMC, MaxSAT, SAT, and SMT.
ToTTo: A Controlled Table-To-Text Generation Dataset
Authors: Ankur P. Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, Dipanjan Das
Date: 2020-04-29 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Conditional Text Generation, Data-to-Text Generation, Sentence, Table-to-Text Generation, Text Generation

We present ToTTo, an open-domain English table-to-text dataset with over 120,000 training examples that proposes a controlled generation task: given a Wikipedia table and a set of highlighted table cells, produce a one-sentence description. To obtain generated targets that are natural but also faithful to the source table, we introduce a dataset construction process where annotators directly revise existing candidate sentences from Wikipedia. We present systematic analyses of our dataset and annotation process as well as results achieved by several state-of-the-art baselines. While usually fluent, existing methods often hallucinate phrases that are not supported by the table, suggesting that this dataset can serve as a useful research benchmark for high-precision conditional text generation.
VGGSound: A Large-scale Audio-Visual Dataset
Authors: Honglie Chen, Weidi Xie, Andrea Vedaldi, Andrew Zisserman
Date: 2020-04-29 00:00:00
Tasks: image-classification, Image Classification

Our goal is to collect a large-scale audio-visual dataset with low label noise from videos in the wild using computer vision techniques. The resulting dataset can be used for training and evaluating audio recognition models. We make three contributions. First, we propose a scalable pipeline based on computer vision techniques to create an audio dataset from open-source media. Our pipeline involves obtaining videos from YouTube; using image classification algorithms to localize audio-visual correspondence; and filtering out ambient noise using audio verification. Second, we use this pipeline to curate the VGGSound dataset consisting of more than 210k videos for 310 audio classes. Third, we investigate various Convolutional Neural Network~(CNN) architectures and aggregation approaches to establish audio recognition baselines for our new dataset. Compared to existing audio datasets, VGGSound ensures audio-visual correspondence and is collected under unconstrained conditions. Code and the dataset are available at http://www.robots.ox.ac.uk/~vgg/data/vggsound/
Editing in Style: Uncovering the Local Semantics of GANs
Authors: Edo Collins, Raja Bala, Bob Price, Sabine Süsstrunk
Date: 2020-04-29 00:00:00
Proceeding: CVPR 2020 6
Tasks: Disentanglement, Image Generation

While the quality of GAN image synthesis has improved tremendously in recent years, our ability to control and condition the output is still limited. Focusing on StyleGAN, we introduce a simple and effective method for making local, semantically-aware edits to a target output image. This is accomplished by borrowing elements from a source image, also a GAN output, via a novel manipulation of style vectors. Our method requires neither supervision from an external model, nor involves complex spatial morphing operations. Instead, it relies on the emergent disentanglement of semantic objects that is learned by StyleGAN during its training. Semantic editing is demonstrated on GANs producing human faces, indoor scenes, cats, and cars. We measure the locality and photorealism of the edits produced by our method, and find that it accomplishes both.
Elastic weight consolidation for better bias inoculation
Authors: James Thorne, Andreas Vlachos
Date: 2020-04-29 00:00:00
Proceeding: EACL 2021 2
Tasks: Fact Verification, General Classification, Natural Language Inference, Sentence, Sentence-Pair Classification

The biases present in training datasets have been shown to affect models for sentence pair classification tasks such as natural language inference (NLI) and fact verification. While fine-tuning models on additional data has been used to mitigate them, a common issue is that of catastrophic forgetting of the original training dataset. In this paper, we show that elastic weight consolidation (EWC) allows fine-tuning of models to mitigate biases while being less susceptible to catastrophic forgetting. In our evaluation on fact verification and NLI stress tests, we show that fine-tuning with EWC dominates standard fine-tuning, yielding models with lower levels of forgetting on the original (biased) dataset for equivalent gains in accuracy on the fine-tuning (unbiased) dataset.
Informed Sampling for Diversity in Concept-to-Text NLG
Authors: Giulio Zhou, Gerasimos Lampouras
Date: 2020-04-29 00:00:00
Proceeding: Findings (EMNLP) 2021 11
Tasks: Concept-To-Text Generation, Diversity, Imitation Learning, Text Generation

Deep-learning models for language generation tasks tend to produce repetitive output. Various methods have been proposed to encourage lexical diversity during decoding, but this often comes at a cost to the perceived fluency and adequacy of the output. In this work, we propose to ameliorate this cost by using an Imitation Learning approach to explore the level of diversity that a language generation model can reliably produce. Specifically, we augment the decoding process with a meta-classifier trained to distinguish which words at any given timestep will lead to high-quality output. We focus our experiments on concept-to-text generation where models are sensitive to the inclusion of irrelevant words due to the strict relation between input and output. Our analysis shows that previous methods for diversity underperform in this setting, while human evaluation suggests that our proposed method achieves a high level of diversity with minimal effect to the output's fluency and adequacy.
TS-MPC for Autonomous Vehicle using a Learning Approach
Authors: Eugenio Alcalá, Olivier Sename, Vicenç Puig, Joseba Quevedo
Date: 2020-04-29 00:00:00
Tasks: Autonomous Driving, Model Predictive Control

In this paper, the Model Predictive Control (MPC) and Moving Horizon Estimator (MHE) strategies using a data-driven approach to learn a Takagi-Sugeno (TS) representation of the vehicle dynamics are proposed to solve autonomous driving control problems in real-time. To address the TS modeling, we use the Adaptive Neuro-Fuzzy Inference System (ANFIS) approach to obtain a set of polytopic-based linear representations as well as a set of membership functions relating in a non-linear way the different linear subsystems. The proposed control approach is provided by racing-based references of an external planner and estimations from the MHE offering a high driving performance in racing mode. The control-estimation scheme is tested in a simulated racing environment to show the potential of the presented approaches.
Detecting Domain Polarity-Changes of Words in a Sentiment Lexicon
Authors: Shuai Wang, Guangyi Lv, Sahisnu Mazumder, Bing Liu
Date: 2020-04-29 00:00:00
Proceeding: Findings (ACL) 2021 8
Tasks: Sentiment Analysis, Sentiment Classification

Sentiment lexicons are instrumental for sentiment analysis. One can use a set of sentiment words provided in a sentiment lexicon and a lexicon-based classifier to perform sentiment classification. One major issue with this approach is that many sentiment words are domain dependent. That is, they may be positive in some domains but negative in some others. We refer to this problem as domain polarity-changes of words. Detecting such words and correcting their sentiment for an application domain is very important. In this paper, we propose a graph-based technique to tackle this problem. Experimental results show its effectiveness on multiple real-world datasets.
AxCell: Automatic Extraction of Results from Machine Learning Papers
Authors: Marcin Kardas, Piotr Czapla, Pontus Stenetorp, Sebastian Ruder, Sebastian Riedel, Ross Taylor, Robert Stojnic
Date: 2020-04-29 00:00:00
Proceeding: EMNLP 2020 11
Tasks: BIG-bench Machine Learning, Scientific Results Extraction

Tracking progress in machine learning has become increasingly difficult with the recent explosion in the number of papers. In this paper, we present AxCell, an automatic machine learning pipeline for extracting results from papers. AxCell uses several novel components, including a table segmentation subtask, to learn relevant structural knowledge that aids extraction. When compared with existing methods, our approach significantly improves the state of the art for results extraction. We also release a structured, annotated dataset for training models for results extraction, and a dataset for evaluating the performance of models on this task. Lastly, we show the viability of our approach enables it to be used for semi-automated results extraction in production, suggesting our improvements make this task practically viable for the first time. Code is available on GitHub.
Learning to Learn to Disambiguate: Meta-Learning for Few-Shot Word Sense Disambiguation
Authors: Nithin Holla, Pushkar Mishra, Helen Yannakoudakis, Ekaterina Shutova
Date: 2020-04-29 00:00:00
Proceeding: Findings of the Association for Computational Linguistics 2020
Tasks: Few-Shot Learning, Meta-Learning, Word Sense Disambiguation

The success of deep learning methods hinges on the availability of large training datasets annotated for the task of interest. In contrast to human intelligence, these methods lack versatility and struggle to learn and adapt quickly to new tasks, where labeled data is scarce. Meta-learning aims to solve this problem by training a model on a large number of few-shot tasks, with an objective to learn new tasks quickly from a small number of examples. In this paper, we propose a meta-learning framework for few-shot word sense disambiguation (WSD), where the goal is to learn to disambiguate unseen words from only a few labeled instances. Meta-learning approaches have so far been typically tested in an $N$-way, $K$-shot classification setting where each task has $N$ classes with $K$ examples per class. Owing to its nature, WSD deviates from this controlled setup and requires the models to handle a large number of highly unbalanced classes. We extend several popular meta-learning approaches to this scenario, and analyze their strengths and weaknesses in this new challenging setting.
End-to-End Slot Alignment and Recognition for Cross-Lingual NLU
Authors: Weijia Xu, Batool Haider, Saab Mansour
Date: 2020-04-29 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Cross-Lingual Transfer, Goal-Oriented Dialog, intent-classification, Intent Classification, Machine Translation, Natural Language Understanding, Translation, Word Alignment

Natural language understanding (NLU) in the context of goal-oriented dialog systems typically includes intent classification and slot labeling tasks. Existing methods to expand an NLU system to new languages use machine translation with slot label projection from source to the translated utterances, and thus are sensitive to projection errors. In this work, we propose a novel end-to-end model that learns to align and predict target slot labels jointly for cross-lingual transfer. We introduce MultiATIS++, a new multilingual NLU corpus that extends the Multilingual ATIS corpus to nine languages across four language families, and evaluate our method using the corpus. Results show that our method outperforms a simple label projection method using fast-align on most languages, and achieves competitive performance to the more complex, state-of-the-art projection method with only half of the training time. We release our MultiATIS++ corpus to the community to continue future research on cross-lingual NLU.