pwc-archive/papers-with-abstracts huggingface dataset:
EEG Channel Interpolation Using Deep Encoder-decoder Netwoks
Authors: Sari Saba-Sadiya, Tuka Alhanai, Taosheng Liu, Mohammad M. Ghassemi
Date: 2020-09-21 00:00:00
Tasks: Decoder, EEG, Electroencephalogram (EEG), Self-Learning, Transfer Learning

Electrode "pop" artifacts originate from the spontaneous loss of connectivity between a surface and an electrode. Electroencephalography (EEG) uses a dense array of electrodes, hence "popped" segments are among the most pervasive type of artifact seen during the collection of EEG data. In many cases, the continuity of EEG data is critical for downstream applications (e.g. brain machine interface) and requires that popped segments be accurately interpolated. In this paper we frame the interpolation problem as a self-learning task using a deep encoder-decoder network. We compare our approach against contemporary interpolation methods on a publicly available EEG data set. Our approach exhibited a minimum of ~15% improvement over contemporary approaches when tested on subjects and tasks not used during model training. We demonstrate how our model's performance can be enhanced further on novel subjects and tasks using transfer learning. All code and data associated with this study is open-source to enable ease of extension and practical use. To our knowledge, this work is the first solution to the EEG interpolation problem that uses deep learning.
Generate Novel Molecules With Target Properties Using Conditional Generative Models
Authors: Abhinav Sagar
Date: 2020-09-15 00:00:00
Tasks: Decoder, Drug Discovery, Property Prediction

Drug discovery using deep learning has attracted a lot of attention of late as it has obvious advantages like higher efficiency, less manual guessing and faster process time. In this paper, we present a novel neural network for generating small molecules similar to the ones in the training set. Our network consists of an encoder made up of bi-GRU layers for converting the input samples to a latent space, predictor for enhancing the capability of encoder made up of 1D-CNN layers and a decoder comprised of uni-GRU layers for reconstructing the samples from the latent space representation. Condition vector in latent space is used for generating molecules with the desired properties. We present the loss functions used for training our network, experimental details and property prediction metrics. Our network outperforms previous methods using Molecular weight, LogP and Quantitative Estimation of Drug-likeness as the evaluation metrics.
Stock Price Prediction Using Convolutional Neural Networks on a Multivariate Timeseries
Authors: Sidra Mehtab, Jaydip Sen
Date: 2020-01-10 00:00:00
Tasks: BIG-bench Machine Learning, regression, Stock Price Prediction

Prediction of future movement of stock prices has been a subject matter of many research work. In this work, we propose a hybrid approach for stock price prediction using machine learning and deep learning-based methods. We select the NIFTY 50 index values of the National Stock Exchange of India, over a period of four years, from January 2015 till December 2019. Based on the NIFTY data during the said period, we build various predictive models using machine learning approaches, and then use those models to predict the Close value of NIFTY 50 for the year 2019, with a forecast horizon of one week. For predicting the NIFTY index movement patterns, we use a number of classification methods, while for forecasting the actual Close values of NIFTY index, various regression models are built. We, then, augment our predictive power of the models by building a deep learning-based regression model using Convolutional Neural Network with a walk-forward validation. The CNN model is fine-tuned for its parameters so that the validation loss stabilizes with increasing number of iterations, and the training and validation accuracies converge. We exploit the power of CNN in forecasting the future NIFTY index values using three approaches which differ in number of variables used in forecasting, number of sub-models used in the overall models and, size of the input data for training the models. Extensive results are presented on various metrics for all classification and regression models. The results clearly indicate that CNN-based multivariate forecasting model is the most effective and accurate in predicting the movement of NIFTY index values with a weekly forecast horizon.
Goal-Directed Occupancy Prediction for Lane-Following Actors
Authors: Poornima Kaniarasu, Galen Clark Haynes, Micol Marchetti-Bowick
Date: 2020-09-06 00:00:00
Tasks: Autonomous Driving, Prediction

Predicting the possible future behaviors of vehicles that drive on shared roads is a crucial task for safe autonomous driving. Many existing approaches to this problem strive to distill all possible vehicle behaviors into a simplified set of high-level actions. However, these action categories do not suffice to describe the full range of maneuvers possible in the complex road networks we encounter in the real world. To combat this deficiency, we propose a new method that leverages the mapped road topology to reason over possible goals and predict the future spatial occupancy of dynamic road actors. We show that our approach is able to accurately predict future occupancy that remains consistent with the mapped lane geometry and naturally captures multi-modality based on the local scene context while also not suffering from the mode collapse problem observed in prior work.
Predicting COVID-19 cases using Bidirectional LSTM on multivariate time series
Authors: Ahmed Ben Said, Abdelkarim Erradi, Hussein Aly, Abdelmonem Mohamed
Date: 2020-09-10 00:00:00
Tasks: Clustering, Time Series, Time Series Analysis

Background: To assist policy makers in taking adequate decisions to stop the spread of COVID-19 pandemic, accurate forecasting of the disease propagation is of paramount importance. Materials and Methods: This paper presents a deep learning approach to forecast the cumulative number of COVID-19 cases using Bidirectional Long Short-Term Memory (Bi-LSTM) network applied to multivariate time series. Unlike other forecasting techniques, our proposed approach first groups the countries having similar demographic and socioeconomic aspects and health sector indicators using K-Means clustering algorithm. The cumulative cases data for each clustered countries enriched with data related to the lockdown measures are fed to the Bidirectional LSTM to train the forecasting model. Results: We validate the effectiveness of the proposed approach by studying the disease outbreak in Qatar. Quantitative evaluation, using multiple evaluation metrics, shows that the proposed technique outperforms state-of-art forecasting approaches. Conclusion: Using data of multiple countries in addition to lockdown measures improve accuracy of the forecast of daily cumulative COVID-19 cases.
Image Augmentation for Object Image Classification Based On Combination of PreTrained CNN and SVM
Authors: Yoshihiro Shima
Date: 2018-01-01 00:00:00
Tasks: 8k, Data Augmentation, General Classification, Image Augmentation, image-classification, Image Classification, Object

Neural networks are a powerful means of classifying object images. The proposed
image category classification method for object images combines convolutional neural
networks (CNNs) and support vector machines (SVMs). A pre-trained CNN, called Alex-Net,
is used as a pattern-feature extractor. Alex-Net is pre-trained for the large-scale object-image
dataset ImageNet. Instead of training, Alex-Net, pre-trained for ImageNet is used. An SVM is
used as trainable classifier. The feature vectors are passed to the SVM from Alex-Net. The
STL-10 dataset are used as object images. The number of classes is ten. Training and test
samples are clearly split. STL-10 object images are trained by the SVM with data
augmentation. We use the pattern transformation method with the cosine function. We also
apply some augmentation method such as rotation, skewing and elastic distortion. By using the
cosine function, the original patterns were left-justified, right-justified, top-justified, or bottomjustified. Patterns were also center-justified and enlarged. Test error rate is decreased by 0.435
percentage points from 16.055% by augmentation with cosine transformation. Error rates are
increased by other augmentation method such as rotation, skewing and elastic distortion,
compared without augmentation . Number of augmented data is 30 times that of the original
STL-10 5K training samples. Experimental test error rate for the test 8k STL-10 object images
was 15.620%, which shows that image augmentation is effective for image category
classification.
Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect
Authors: Kaihua Tang, Jianqiang Huang, Hanwang Zhang
Date: 2020-09-28 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Causal Inference, counterfactual, Counterfactual Reasoning, General Classification, image-classification, Image Classification, Instance Segmentation, Long-tail Learning, Representation Learning, Semantic Segmentation

As the class size grows, maintaining a balanced dataset across many classes is challenging because the data are long-tailed in nature; it is even impossible when the sample-of-interest co-exists with each other in one collectable unit, e.g., multiple visual instances in one image. Therefore, long-tailed classification is the key to deep learning at scale. However, existing methods are mainly based on re-weighting/re-sampling heuristics that lack a fundamental theory. In this paper, we establish a causal inference framework, which not only unravels the whys of previous methods, but also derives a new principled solution. Specifically, our theory shows that the SGD momentum is essentially a confounder in long-tailed classification. On one hand, it has a harmful causal effect that misleads the tail prediction biased towards the head. On the other hand, its induced mediation also benefits the representation learning and head prediction. Our framework elegantly disentangles the paradoxical effects of the momentum, by pursuing the direct causal effect caused by an input sample. In particular, we use causal intervention in training, and counterfactual reasoning in inference, to remove the "bad" while keep the "good". We achieve new state-of-the-arts on three long-tailed visual recognition benchmarks: Long-tailed CIFAR-10/-100, ImageNet-LT for image classification and LVIS for instance segmentation.
Causal Intervention for Weakly-Supervised Semantic Segmentation
Authors: Dong Zhang, Hanwang Zhang, Jinhui Tang, Xian-Sheng Hua, Qianru Sun
Date: 2020-09-26 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Attribute, Causal Inference, Segmentation, Semantic Segmentation, Weakly supervised Semantic Segmentation, Weakly-Supervised Semantic Segmentation

We present a causal inference framework to improve Weakly-Supervised Semantic Segmentation (WSSS). Specifically, we aim to generate better pixel-level pseudo-masks by using only image-level labels -- the most crucial step in WSSS. We attribute the cause of the ambiguous boundaries of pseudo-masks to the confounding context, e.g., the correct image-level classification of "horse" and "person" may be not only due to the recognition of each instance, but also their co-occurrence context, making the model inspection (e.g., CAM) hard to distinguish between the boundaries. Inspired by this, we propose a structural causal model to analyze the causalities among images, contexts, and class labels. Based on it, we develop a new method: Context Adjustment (CONTA), to remove the confounding bias in image-level classification and thus provide better pseudo-masks as ground-truth for the subsequent segmentation model. On PASCAL VOC 2012 and MS-COCO, we show that CONTA boosts various popular WSSS methods to new state-of-the-arts.
DCASENET: A joint pre-trained deep neural network for detecting and classifying acoustic scenes and events
Authors: Jee-weon Jung, Hye-jin Shim, Ju-ho Kim, Ha-Jin Yu
Date: 2020-09-21 00:00:00
Tasks: Acoustic Scene Classification, Audio Tagging, Event Detection, Scene Classification, Sound Event Detection

Single task deep neural networks that perform a target task among diverse cross-related tasks in the acoustic scene and event literature are being developed. Few studies exist that investigate to combine such tasks, however, the work is at its preliminary stage. In this study, we propose an integrated deep neural network that can perform three tasks: acoustic scene classification, audio tagging, and sound event detection. Through vast experiments using three datasets, we show that the proposed system, DCASENet, itself can be directly used for any tasks with competitive results, or it can be further fine-tuned for the target task.
FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
Authors: Kihyuk Sohn∗ David Berthelot∗ Chun-Liang Li Zizhao Zhang Nicholas Carlini Ekin D. Cubuk Alex Kurakin Han Zhang Colin Raffel
Date: 2020-01-21 00:00:00
Tasks: Image Classification, Pseudo Label

Semi-supervised learning (SSL) provides an effective
means of leveraging unlabeled data to improve a model’s
performance. In this paper, we demonstrate the power of a
simple combination of two common SSL methods: consistency regularization and pseudo-labeling. Our algorithm,
FixMatch, first generates pseudo-labels using the model’s
predictions on weakly-augmented unlabeled images. For a
given image, the pseudo-label is only retained if the model
produces a high-confidence prediction. The model is then
trained to predict the pseudo-label when fed a stronglyaugmented version of the same image. Despite its simplicity, we show that FixMatch achieves state-of-the-art performance across a variety of standard semi-supervised learning benchmarks, including 94.93% accuracy on CIFAR-10
with 250 labels and 88.61% accuracy with 40 – just 4 labels per class. Since FixMatch bears many similarities
to existing SSL methods that achieve worse performance,
we carry out an extensive ablation study to tease apart
the experimental factors that are most important to FixMatch’s success. We make our code available at https:
//github.com/google-research/fixmatch.
iNLTK: Natural Language Toolkit for Indic Languages
Authors: Gaurav Arora
Date: 2020-09-26 00:00:00
Proceeding: EMNLP (NLPOSS) 2020 11
Tasks: Data Augmentation, Paraphrase Generation, Sentence, Sentence Embeddings, text-classification, Text Classification, Text Generation, Word Embeddings

We present iNLTK, an open-source NLP library consisting of pre-trained language models and out-of-the-box support for Data Augmentation, Textual Similarity, Sentence Embeddings, Word Embeddings, Tokenization and Text Generation in 13 Indic Languages. By using pre-trained models from iNLTK for text classification on publicly available datasets, we significantly outperform previously reported results. On these datasets, we also show that by using pre-trained models and data augmentation from iNLTK, we can achieve more than 95% of the previous best performance by using less than 10% of the training data. iNLTK is already being widely used by the community and has 40,000+ downloads, 600+ stars and 100+ forks on GitHub. The library is available at https://github.com/goru001/inltk.
Interpretable Detail-Fidelity Attention Network for Single Image Super-Resolution
Authors: Yuanfei Huang, Jie Li, Xinbo Gao, Yanting Hu, Wen Lu
Date: 2020-09-28 00:00:00
Tasks: Image Super-Resolution, Super-Resolution

Benefiting from the strong capabilities of deep CNNs for feature representation and nonlinear mapping, deep-learning-based methods have achieved excellent performance in single image super-resolution. However, most existing SR methods depend on the high capacity of networks which is initially designed for visual recognition, and rarely consider the initial intention of super-resolution for detail fidelity. Aiming at pursuing this intention, there are two challenging issues to be solved: (1) learning appropriate operators which is adaptive to the diverse characteristics of smoothes and details; (2) improving the ability of model to preserve the low-frequency smoothes and reconstruct the high-frequency details. To solve them, we propose a purposeful and interpretable detail-fidelity attention network to progressively process these smoothes and details in divide-and-conquer manner, which is a novel and specific prospect of image super-resolution for the purpose on improving the detail fidelity, instead of blindly designing or employing the deep CNNs architectures for merely feature representation in local receptive fields. Particularly, we propose a Hessian filtering for interpretable feature representation which is high-profile for detail inference, a dilated encoder-decoder and a distribution alignment cell to improve the inferred Hessian features in morphological manner and statistical manner respectively. Extensive experiments demonstrate that the proposed methods achieve superior performances over the state-of-the-art methods quantitatively and qualitatively. Code is available at https://github.com/YuanfeiHuang/DeFiAN.
LiPo-LCD: Combining Lines and Points for Appearance-based Loop Closure Detection
Authors: Joan P. Company-Corcoles, Emilio Garcia-Fidalgo, Alberto Ortiz
Date: 2020-09-03 00:00:00
Tasks: Loop Closure Detection

Visual SLAM approaches typically depend on loop closure detection to correct the inconsistencies that may arise during the map and camera trajectory calculations, typically making use of point features for detecting and closing the existing loops. In low-textured scenarios, however, it is difficult to find enough point features and, hence, the performance of these solutions drops drastically. An alternative for human-made scenarios, due to their structural regularity, is the use of geometrical cues such as straight segments, frequently present within these environments. Under this context, in this paper we introduce LiPo-LCD, a novel appearance-based loop closure detection method that integrates lines and points. Adopting the idea of incremental Bag-of-Binary-Words schemes, we build separate BoW models for each feature, and use them to retrieve previously seen images using a late fusion strategy. Additionally, a simple but effective mechanism, based on the concept of island, groups similar images close in time to reduce the image candidate search effort. A final step validates geometrically the loop candidates by incorporating the detected lines by means of a process comprising a line feature matching stage, followed by a robust spatial verification stage, now combining both lines and points. As it is reported in the paper, LiPo-LCD compares well with several state-of-the-art solutions for a number of datasets involving different environmental conditions.
SIMILARITY LEARNING FOR COVER SONG IDENTIFICATION USING CROSS-SIMILARITY MATRICES OF MULTI-LEVEL DEEP SEQUENCES
Authors: Chaoya Jiang, Deshun Yang, Xiaoou Chen
Date: 2020-05-14 00:00:00
Tasks: Cover song identification, Metric Learning

In recent years, several deep learning models have been proposed for cover song identification and they have been designed to learn fixed-length feature vectors for music tracks. However, the aspect of temporal progression of music, which is important for measuring the melody similarity between two tracks, is not well represented by fixed-length vectors. In this paper, we propose a new Siamese network architecture for music melody similarity metric learning. The architecture consists of two parts. One part is a network for learn- ing the deep sequence representation of music tracks, and the other is a similarity estimation network which takes as input the cross- similarity matrices calculated from the deep sequences of a pair of tracks. The two networks are jointly trained and optimized to achieve high melody similarity prediction accuracy. Experiments conducted on several public datasets demonstrate the superiority of the proposed architecture.
A Robust graph attention network with dynamic adjusted Graph
Authors: Xianchen Zhou, Yaoyun Zeng, Hongxia Wang
Date: 2020-09-28 00:00:00
Tasks: Graph Attention

Graph Attention Networks(GATs) are useful deep learning models to deal with the graph data. However, recent works show that the classical GAT is vulnerable to adversarial attacks. It degrades dramatically with slight perturbations. Therefore, how to enhance the robustness of GAT is a critical problem. Robust GAT(RoGAT) is proposed in this paper to improve the robustness of GAT based on the revision of the attention mechanism. Different from the original GAT, which uses the attention mechanism for different edges but is still sensitive to the perturbation, RoGAT adds an extra dynamic attention score progressively and improves the robustness. Firstly, RoGAT revises the edges weight based on the smoothness assumption which is quite common for ordinary graphs. Secondly, RoGAT further revises the features to suppress features' noise. Then, an extra attention score is generated by the dynamic edge's weight and can be used to reduce the impact of adversarial attacks. Different experiments against targeted and untargeted attacks on citation data on citation data demonstrate that RoGAT outperforms most of the recent defensive methods.
Information Obfuscation of Graph Neural Networks
Authors: Peiyuan Liao, Han Zhao, Keyulu Xu, Tommi Jaakkola, Geoffrey Gordon, Stefanie Jegelka, Ruslan Salakhutdinov
Date: 2020-09-28 00:00:00
Tasks: Adversarial Defense, Graph Representation Learning, Knowledge Graphs, Recommendation Systems, Representation Learning

While the advent of Graph Neural Networks (GNNs) has greatly improved node and graph representation learning in many applications, the neighborhood aggregation scheme exposes additional vulnerabilities to adversaries seeking to extract node-level information about sensitive attributes. In this paper, we study the problem of protecting sensitive attributes by information obfuscation when learning with graph structured data. We propose a framework to locally filter out pre-determined sensitive attributes via adversarial training with the total variation and the Wasserstein distance. Our method creates a strong defense against inference attacks, while only suffering small loss in task performance. Theoretically, we analyze the effectiveness of our framework against a worst-case adversary, and characterize an inherent trade-off between maximizing predictive accuracy and minimizing information leakage. Experiments across multiple datasets from recommender systems, knowledge graphs and quantum chemistry demonstrate that the proposed approach provides a robust defense across various graph structures and tasks, while producing competitive GNN encoders for downstream tasks.