pwc-archive/papers-with-abstracts huggingface dataset:
Bregman-divergence-guided Legendre exponential dispersion model with finite cumulants (K-LED)
Authors: Hyenkyun Woo
Date: 2019-10-04 00:00:00

Exponential dispersion model is a useful framework in machine learning and statistics. Primarily, thanks to the additive structure of the model, it can be achieved without difficulty to estimate parameters including mean. However, tight conditions on cumulant function, such as analyticity, strict convexity, and steepness, reduce the class of exponential dispersion model. In this work, we present relaxed exponential dispersion model K-LED (Legendre exponential dispersion model with K cumulants). The cumulant function of the proposed model is a convex function of Legendre type having continuous partial derivatives of K-th order on the interior of a convex domain. Most of the K-LED models are developed via Bregman-divergence-guided log-concave density function with coercivity shape constraints. The main advantage of the proposed model is that the first cumulant (or the mean parameter space) of the 1-LED model is easily computed through the extended global optimum property of Bregman divergence. An extended normal distribution is introduced as an example of 1-LED based on Tweedie distribution. On top of that, we present 2-LED satisfying mean-variance relation of quasi-likelihood function. There is an equivalence between a subclass of quasi-likelihood function and a regular 2-LED model, of which the canonical parameter space is open. A typical example is a regular 2-LED model with power variance function, i.e., a variance is in proportion to the power of the mean of observations. This model is equivalent to a subclass of beta-divergence (or a subclass of quasi-likelihood function with power variance function). Furthermore, a new parameterized K-LED model, the cumulant function of which is the convex extended logistic loss function, is proposed. This model includes Bernoulli distribution and Poisson distribution.
Investigating the Effectiveness of Representations Based on Word-Embeddings in Active Learning for Labelling Text Datasets
Authors: Jinghui Lu, Maeve Henchion, Brian Mac Namee
Date: 2019-10-04 00:00:00
Tasks: Active Learning, BIG-bench Machine Learning, Word Embeddings

Manually labelling large collections of text data is a time-consuming, expensive, and laborious task, but one that is necessary to support machine learning based on text datasets. Active learning has been shown to be an effective way to alleviate some of the effort required in utilising large collections of unlabelled data for machine learning tasks without needing to fully label them. The representation mechanism used to represent text documents when performing active learning, however, has a significant influence on how effective the process will be. While simple vector representations such as bag of words have been shown to be an effective way to represent documents during active learning, the emergence of representation mechanisms based on the word embeddings prevalent in neural network research (e.g. word2vec and transformer-based models like BERT) offer a promising, and as yet not fully explored, alternative. This paper describes a large-scale evaluation of the effectiveness of different text representation mechanisms for active learning across 8 datasets from varied domains. This evaluation shows that using representations based on modern word embeddings---especially BERT---, which have not yet been widely used in active learning, achieves a significant improvement over more commonly used vector-based methods like bag of words.
Fine-grained Sentiment Classification using BERT
Authors: Manish Munikar, Sushil Shakya, Aakash Shrestha
Date: 2019-10-04 00:00:00
Tasks: Chinese Sentiment Analysis, Classification, General Classification, Sentiment Analysis, Sentiment Classification, Transfer Learning

Sentiment classification is an important process in understanding people's perception towards a product, service, or topic. Many natural language processing models have been proposed to solve the sentiment classification problem. However, most of them have focused on binary sentiment classification. In this paper, we use a promising deep learning model called BERT to solve the fine-grained sentiment classification task. Experiments show that our model outperforms other popular models for this task without sophisticated architecture. We also demonstrate the effectiveness of transfer learning in natural language processing in the process.
Correlation of Auroral Dynamics and GNSS Scintillation with an Autoencoder
Authors: Kara Lamb, Garima Malhotra, Athanasios Vlontzos, Edward Wagstaff, Atılım Günes Baydin, Anahita Bhiwandiwalla, Yarin Gal, Alfredo Kalaitzis, Anthony Reina, Asti Bhatt
Date: 2019-10-04 00:00:00

High energy particles originating from solar activity travel along the the Earth's magnetic field and interact with the atmosphere around the higher latitudes. These interactions often manifest as aurora in the form of visible light in the Earth's ionosphere. These interactions also result in irregularities in the electron density, which cause disruptions in the amplitude and phase of the radio signals from the Global Navigation Satellite Systems (GNSS), known as 'scintillation'. In this paper we use a multi-scale residual autoencoder (Res-AE) to show the correlation between specific dynamic structures of the aurora and the magnitude of the GNSS phase scintillations ($\sigma_{\phi}$). Auroral images are encoded in a lower dimensional feature space using the Res-AE, which in turn are clustered with t-SNE and UMAP. Both methods produce similar clusters, and specific clusters demonstrate greater correlations with observed phase scintillations. Our results suggest that specific dynamic structures of auroras are highly correlated with GNSS phase scintillations.
A Rademacher Complexity Based Method fo rControlling Power and Confidence Level in Adaptive Statistical Analysis
Authors: Lorenzo De Stefani, Eli Upfal
Date: 2019-10-04 00:00:00
Tasks: BIG-bench Machine Learning, Generalization Bounds

While standard statistical inference techniques and machine learning generalization bounds assume that tests are run on data selected independently of the hypotheses, practical data analysis and machine learning are usually iterative and adaptive processes where the same holdout data is often used for testing a sequence of hypotheses (or models), which may each depend on the outcome of the previous tests on the same data. In this work, we present RadaBound a rigorous, efficient and practical procedure for controlling the generalization error when using a holdout sample for multiple adaptive testing. Our solution is based on a new application of the Rademacher Complexity generalization bounds, adapted to dependent tests. We demonstrate the statistical power and practicality of our method through extensive simulations and comparisons to alternative approaches.
Synthesizing Credit Card Transactions
Authors: Erik R. Altman
Date: 2019-10-04 00:00:00
Tasks: Cloud Computing

Two elements have been essential to AI's recent boom: (1) deep neural nets and the theory and practice behind them; and (2) cloud computing with its abundant labeled data and large computing resources. Abundant labeled data is available for key domains such as images, speech, natural language processing, and recommendation engines. However, there are many other domains where such data is not available, or access to it is highly restricted for privacy reasons, as with health and financial data. Even when abundant data is available, it is often not labeled. Doing such labeling is labor-intensive and non-scalable. As a result, to the best of our knowledge, key domains still lack labeled data or have at most toy data; or the synthetic data must have access to real data from which it can mimic new data. This paper outlines work to generate realistic synthetic data for an important domain: credit card transactions. Some challenges: there are many patterns and correlations in real purchases. There are millions of merchants and innumerable locations. Those merchants offer a wide variety of goods. Who shops where and when? How much do people pay? What is a realistic fraudulent transaction? We use a mixture of technical approaches and domain knowledge including mechanics of credit card processing, a broad set of consumer domains: electronics, clothing, hair styling, etc. Connecting everything is a virtual world. This paper outlines some of our key techniques and provides evidence that the data generated is indeed realistic. Beyond the scope of this paper: (1) use of our data to develop and train models to predict fraud; (2) coupling models and the synthetic dataset to assess performance in designing accelerators such as GPUs and TPUs.
IRF: Interactive Recommendation through Dialogue
Authors: Oznur Alkan, Massimiliano Mattetti, Elizabeth M. Daly, Adi Botea, Inge Vejsbjerg
Date: 2019-10-03 00:00:00
Tasks: Interactive Recommendation

Recent research focuses beyond recommendation accuracy, towards human factors that influence the acceptance of recommendations, such as user satisfaction, trust, transparency and sense of control.We present a generic interactive recommender framework that can add interaction functionalities to non-interactive recommender systems.We take advantage of dialogue systems to interact with the user and we design a middleware layer to provide the interaction functions, such as providing explanations for the recommendations, managing users preferences learnt from dialogue, preference elicitation and refining recommendations based on learnt preferences.
Mental Task Classification Using Electroencephalogram Signal
Authors: Zeyu Bai, Ruizhi Yang, Youzhi Liang
Date: 2019-10-02 00:00:00
Tasks: Classification, Decoder, EEG, Electroencephalogram (EEG), General Classification, Hyperparameter Optimization

This paper studies the classification problem on electroencephalogram (EEG) data of mental tasks, using standard architecture of three-layer CNN, stacked LSTM, stacked GRU. We further propose a novel classifier - a mixed LSTM model with a CNN decoder. A hyperparameter optimization on CNN shows validation accuracy of 72% and testing accuracy of 62%. The stacked LSTM and GRU models with FFT preprocessing and downsampling on data achieve 55% and 51% testing accuracy respectively. As for the mixed LSTM model with CNN decoder, validation accuracy of 75% and testing accuracy of 70% are obtained. We believe the mixed model is more robust and accurate than both CNN and LSTM individually, by using the CNN layer as a decoder for following LSTM layers. The code is completed in the framework of Pytorch and Keras. Results and code can be found at https://github.com/theyou21/BigProject.
INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps
Authors: Wei Zhan, Liting Sun, Di Wang, Haojie Shi, Aubrey Clausse, Maximilian Naumann, Julius Kummerle, Hendrik Konigshof, Christoph Stiller, Arnaud de La Fortelle, Masayoshi Tomizuka
Date: 2019-09-30 00:00:00
Tasks: Imitation Learning, motion prediction

Behavior-related research areas such as motion prediction/planning, representation/imitation learning, behavior modeling/generation, and algorithm testing, require support from high-quality motion datasets containing interactive driving scenarios with different driving cultures. In this paper, we present an INTERnational, Adversarial and Cooperative moTION dataset (INTERACTION dataset) in interactive driving scenarios with semantic maps. Five features of the dataset are highlighted. 1) The interactive driving scenarios are diverse, including urban/highway/ramp merging and lane changes, roundabouts with yield/stop signs, signalized intersections, intersections with one/two/all-way stops, etc. 2) Motion data from different countries and different continents are collected so that driving preferences and styles in different cultures are naturally included. 3) The driving behavior is highly interactive and complex with adversarial and cooperative motions of various traffic participants. Highly complex behavior such as negotiations, aggressive/irrational decisions and traffic rule violations are densely contained in the dataset, while regular behavior can also be found from cautious car-following, stop, left/right/U-turn to rational lane-change and cycling and pedestrian crossing, etc. 4) The levels of criticality span wide, from regular safe operations to dangerous, near-collision maneuvers. Real collision, although relatively slight, is also included. 5) Maps with complete semantic information are provided with physical layers, reference lines, lanelet connections and traffic rules. The data is recorded from drones and traffic cameras. Statistics of the dataset in terms of number of entities and interaction density are also provided, along with some utilization examples in a variety of behavior-related research areas. The dataset can be downloaded via https://interaction-dataset.com.
Towards Controllable and Personalized Review Generation
Authors: Pan Li, Alexander Tuzhilin
Date: 2019-09-30 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Review Generation, Sentence

In this paper, we propose a novel model RevGAN that automatically generates controllable and personalized user reviews based on the arbitrarily given sentimental and stylistic information. RevGAN utilizes the combination of three novel components, including self-attentive recursive autoencoders, conditional discriminators, and personalized decoders. We test its performance on the several real-world datasets, where our model significantly outperforms state-of-the-art generation models in terms of sentence quality, coherence, personalization and human evaluations. We also empirically show that the generated reviews could not be easily distinguished from the organically produced reviews and that they follow the same statistical linguistics laws.
Instagram Fake and Automated Account Detection
Authors: Fatih Cagatay Akyon, Esat Kalfaoglu
Date: 2019-09-13 00:00:00

Fake engagement is one of the significant problems in Online Social Networks (OSNs) which is used to increase the popularity of an account in an inorganic manner. The detection of fake engagement is crucial because it leads to loss of money for businesses, wrong audience targeting in advertising, wrong product predictions systems, and unhealthy social network environment. This study is related with the detection of fake and automated accounts which leads to fake engagement on Instagram. Prior to this work, there were no publicly available dataset for fake and automated accounts. For this purpose, two datasets have been published for the detection of fake and automated accounts. For the detection of these accounts, machine learning algorithms like Naive Bayes, Logistic Regression, Support Vector Machines and Neural Networks are applied. Additionally, for the detection of automated accounts, cost sensitive genetic algorithm is proposed to handle the unnatural bias in the dataset. To deal with the unevenness problem in the fake dataset, Smote-nc algorithm is implemented. For the automated and fake account detection datasets, 86% and 96% classification accuracies are obtained, respectively.
End-to-End Resume Parsing and Finding Candidates for a Job Description using BERT
Authors: Vedant Bhatia, Prateek Rawat, Ajit Kumar, Rajiv Ratn Shah
Date: 2019-09-30 00:00:00
Tasks: Sentence, Sentence-Pair Classification

The ever-increasing number of applications to job positions presents a challenge for employers to find suitable candidates manually. We present an end-to-end solution for ranking candidates based on their suitability to a job description. We accomplish this in two stages. First, we build a resume parser which extracts complete information from candidate resumes. This parser is made available to the public in the form of a web application. Second, we use BERT sentence pair classification to perform ranking based on their suitability to the job description. To approximate the job description, we use the description of past job experiences by a candidate as mentioned in his resume. Our dataset comprises resumes in LinkedIn format and general non-LinkedIn formats. We parse the LinkedIn resumes with 100\% accuracy and establish a strong baseline of 73\% accuracy for candidate suitability.
Named Entity Recognition System for Sindhi Language
Authors: Awais Khan Jumani, Mashooque Ahmed Memon, Fida Hussain Khoso, Anwar Ali Sanjrani, Safeeullah Soomro
Date: 2019-09-28 00:00:00
Tasks: Information Retrieval, Machine Translation, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Question Answering, Retrieval, TAG, Translation

Named Entity Recognition (NER) System aims to extract the existing information into the following categories such as: Persons Name, Organization, Location, Date and Time, Term, Designation and Short forms. Now, it is considered to be important aspect for many natural languages processing (NLP) tasks such as: information retrieval system, machine translation system, information extraction system and question answering. Even at a surface level, the understanding of the named entities involved in a document gives richer analytical framework and cross referencing. It has been used for different Arabic Script-Based languages like, Arabic, Persian and Urdu but, Sindhi could not come into being yet. This paper explains the problem of NER in the framework of Sindhi Language and provides relevant solution. The system is developed to tag ten different Named Entities. We have used Ruled based approach for NER system of Sindhi Language. For the training and testing, 936 words were used and calculated performance accuracy of 98.71%.
Semi-Supervised Neural Text Generation by Joint Learning of Natural Language Generation and Natural Language Understanding Models
Authors: Raheel Qader, François Portet, Cyril Labbé
Date: 2019-09-29 00:00:00
Proceeding: WS 2019 10
Tasks: Natural Language Understanding, Text Generation

In Natural Language Generation (NLG), End-to-End (E2E) systems trained through deep learning have recently gained a strong interest. Such deep models need a large amount of carefully annotated data to reach satisfactory performance. However, acquiring such datasets for every new NLG application is a tedious and time-consuming task. In this paper, we propose a semi-supervised deep learning scheme that can learn from non-annotated data and annotated data when available. It uses an NLG and a Natural Language Understanding (NLU) sequence-to-sequence models which are learned jointly to compensate for the lack of annotation. Experiments on two benchmark datasets show that, with limited amount of annotated data, the method can achieve very competitive results while not using any pre-processing or re-scoring tricks. These findings open the way to the exploitation of non-annotated datasets which is the current bottleneck for the E2E NLG system development to new applications.
Fake news detection using Deep Learning
Authors: Álvaro Ibrain Rodríguez, Lara Lloret Iglesias
Date: 2019-09-29 00:00:00
Tasks: Deep Learning, Fake News Detection, Language Modeling, Language Modelling

The evolution of the information and communication technologies has dramatically increased the number of people with access to the Internet, which has changed the way the information is consumed. As a consequence of the above, fake news have become one of the major concerns because its potential to destabilize governments, which makes them a potential danger to modern society. An example of this can be found in the US. electoral campaign, where the term "fake news" gained great notoriety due to the influence of the hoaxes in the final result of these. In this work the feasibility of applying deep learning techniques to discriminate fake news on the Internet using only their text is studied. In order to accomplish that, three different neural network architectures are proposed, one of them based on BERT, a modern language model created by Google which achieves state-of-the-art results.
Sleep stage classification from heart-rate variability using long short-term memory neural networks
Authors: Mustafa Radha, Pedro Fonseca, Arnaud Moreau, Marco Ross, Andreas Cerny, Peter Anderer, Xi Long, Ronald M. Aarts
Date: 2019-10-02 00:00:00
Proceeding: Scientific Reports 2019 10
Tasks: Electrocardiography (ECG), General Classification, Heart Rate Variability, Sleep Stage Detection

Automated sleep stage classification using heart rate variability (HRV) may provide an ergonomic and low-cost alternative to gold standard polysomnography, creating possibilities for unobtrusive home-based sleep monitoring. Current methods however are limited in their ability to take into account long-term sleep architectural patterns. A long short-term memory (LSTM) network is proposed as a solution to model long-term cardiac sleep architecture information and validated on a comprehensive data set (292 participants, 584 nights, 541.214 annotated 30 s sleep segments) comprising a wide range of ages and pathological profiles, annotated according to the Rechtschaffen and Kales (R&K) annotation standard. It is shown that the model outperforms state-of-the-art approaches which were often limited to non-temporal or short-term recurrent classifiers. The model achieves a Cohen’s k of 0.61 ± 0.15 and accuracy of 77.00 ± 8.90% across the entire database. Further analysis revealed that the performance for individuals aged 50 years and older may decline. These results demonstrate the merit of deep temporal modelling using a diverse data set and advance the state-of-the-art for HRV-based sleep stage classification. Further research is warranted into individuals over the age of 50 as performance tends to worsen in this sub-population.
Unaligned Image-to-Sequence Transformation with Loop Consistency
Authors: Siyang Wang, Justin Lazarow, Kwonjoon Lee, Zhuowen Tu
Date: 2019-10-09 00:00:00
Proceeding: ICLR 2020 1

We tackle the problem of modeling sequential visual phenomena. Given examples of a phenomena that can be divided into discrete time steps, we aim to take an input from any such time and realize this input at all other time steps in the sequence. Furthermore, we aim to do this without ground-truth aligned sequences -- avoiding the difficulties needed for gathering aligned data. This generalizes the unpaired image-to-image problem from generating pairs to generating sequences. We extend cycle consistency to loop consistency and alleviate difficulties associated with learning in the resulting long chains of computation. We show competitive results compared to existing image-to-image techniques when modeling several different data sets including the Earth's seasons and aging of human faces.