pwc-archive/papers-with-abstracts huggingface dataset:
A Deep Cybersickness Predictor Based on Brain Signal Analysis for Virtual Reality Contents
Authors: Jinwoo Kim,  Woojae Kim,  Heeseok Oh,  Seongmin Lee,  Sanghoon Lee
Date: 2019-10-01 00:00:00
Proceeding: ICCV 2019 10
Tasks: EEG, Electroencephalogram (EEG), Prediction

What if we could interpret the cognitive state of a user while experiencing a virtual reality (VR) and estimate the cognitive state from a visual stimulus? In this paper, we address the above question by developing an electroencephalography (EEG) driven VR cybersickness prediction model. The EEG data has been widely utilized to learn the cognitive representation of brain activity. In the first stage, to fully exploit the advantages of the EEG data, it is transformed into the multi-channel spectrogram which enables to account for the correlation of spectral and temporal coefficient. Then, a convolutional neural network (CNN) is applied to encode the cognitive representation of the EEG spectrogram. In the second stage, we train a cybersickness prediction model on the VR video sequence by designing a Recurrent Neural Network (RNN). Here, the encoded cognitive representation is transferred to the model to train the visual and cognitive features for cybersickness prediction. Through the proposed framework, it is possible to predict the cybersickness level that reflects brain activity automatically. We use 8-channels EEG data to record brain activity while more than 200 subjects experience 44 different VR contents. After rigorous training, we demonstrate that the proposed framework reliably estimates cognitive states without the EEG data. Furthermore, it achieves state-of-the-art performance comparing to existing VR cybersickness prediction models.
Learning With Unsure Data for Medical Image Diagnosis
Authors: Botong Wu,  Xinwei Sun,  Lingjing Hu,  Yizhou Wang
Date: 2019-10-01 00:00:00
Proceeding: ICCV 2019 10
Tasks: Disease Prediction

In image-based disease prediction, it can be hard to give certain cases a deterministic "disease/normal" label due to lack of enough information, e.g., at its early stage. We call such cases "unsure" data. Labeling such data as unsure suggests follow-up examinations so as to avoid irreversible medical accident/loss in contrast to incautious prediction. This is a common practice in clinical diagnosis, however, mostly neglected by existing methods. Learning with unsure data also interweaves with two other practical issues: (i) data imbalance issue that may incur model-bias towards the majority class, and (ii) conservative/aggressive strategy consideration, i.e., the negative (normal) samples and positive (disease) samples should NOT be treated equally \-- the former should be detected with a high precision (conservativeness) and the latter should be detected with a high recall (aggression) to avoid missing opportunity for treatment. Mixed with these issues, learning with unsure data becomes particularly challenging. In this paper, we raise "learning with unsure data" problem and formulate it as an ordinal regression and propose a unified end-to-end learning framework, which also considers the aforementioned two issues: (i) incorporate cost-sensitive parameters to alleviate the data imbalance problem, and (ii) execute the conservative and aggressive strategies by introducing two parameters in the training procedure. The benefits of learning with unsure data and validity of our models are demonstrated on the prediction of Alzheimer's Disease and lung nodules.
Align, Attend and Locate: Chest X-Ray Diagnosis via Contrast Induced Attention Network With Limited Supervision
Authors: Jingyu Liu,  Gangming Zhao,  Yu Fei,  Ming Zhang,  Yizhou Wang,  Yizhou Yu
Date: 2019-10-01 00:00:00
Proceeding: ICCV 2019 10
Tasks: Contrastive Learning

Obstacles facing accurate identification and localization of diseases in chest X-ray images lie in the lack of high-quality images and annotations. In this paper, we propose a Contrast Induced Attention Network (CIA-Net), which exploits the highly structured property of chest X-ray images and localizes diseases via contrastive learning on the aligned positive and negative samples. To force the attention module to focus only on sites of abnormalities, we also introduce a learnable alignment module to adjust all the input images, which eliminates variations of scales, angles, and displacements of X-ray images generated under bad scan conditions. We show that the use of contrastive attention and alignment module allows the model to learn rich identification and localization information using only a small amount of location annotations, resulting in state-of-the-art performance in NIH chest X-ray dataset.
Joint Acne Image Grading and Counting via Label Distribution Learning
Authors: Xiaoping Wu,  Ni Wen,  Jie Liang,  Yu-Kun Lai,  Dongyu She,  Ming-Ming Cheng,  Jufeng Yang
Date: 2019-10-01 00:00:00
Proceeding: ICCV 2019 10
Tasks: Acne Severity Grading, Medical Image Classification, Multi-Task Learning

Accurate grading of skin disease severity plays a crucial role in precise treatment for patients. Acne vulgaris, the most common skin disease in adolescence, can be graded by evidence-based lesion counting as well as experience-based global estimation in the medical field. However, due to the appearance similarity of acne with close severity, it is challenging to count and grade acne accurately. In this paper, we address the problem of acne image analysis via Label Distribution Learning (LDL) considering the ambiguous information among acne severity. Based on the professional grading criterion, we generate two acne label distributions considering the relationship between the similar number of lesions and severity of acne, respectively. We also propose a unified framework for joint acne image grading and counting, which is optimized by the multi-task learning loss. In addition, we further build the ACNE04 dataset with annotations of acne severity and lesion number of each image for evaluation. Experiments demonstrate that our proposed framework performs favorably against state-of-the-art methods. We make the code and dataset publicly available at https://github.com/xpwu95/ldl.
HistoSegNet: Semantic Segmentation of Histological Tissue Type in Whole Slide Images
Authors: Lyndon Chan,  Mahdi S. Hosseini,  Corwyn Rowsell,  Konstantinos N. Plataniotis,  Savvas Damaskinos
Date: 2019-10-01 00:00:00
Proceeding: ICCV 2019 10
Tasks: Diagnostic, Medical Image Segmentation, Segmentation, Semantic Segmentation, Vocal Bursts Type Prediction, Weakly supervised Semantic Segmentation, Weakly-Supervised Semantic Segmentation, whole slide images

In digital pathology, tissue slides are scanned into Whole Slide Images (WSI) and pathologists first screen for diagnostically-relevant Regions of Interest (ROIs) before reviewing them. Screening for ROIs is a tedious and time-consuming visual recognition task which can be exhausting. The cognitive workload could be reduced by developing a visual aid to narrow down the visual search area by highlighting (or segmenting) regions of diagnostic relevance, enabling pathologists to spend more time diagnosing relevant ROIs. In this paper, we propose HistoSegNet, a method for semantic segmentation of histological tissue type (HTT). Using the HTT-annotated Atlas of Digital Pathology (ADP) database, we train a Convolutional Neural Network on the patch annotations, infer Gradient-Weighted Class Activation Maps, average overlapping predictions, and post-process the segmentation with a fully-connected Conditional Random Field. Our method out-performs more complicated weakly-supervised semantic segmentation methods and can generalize to other datasets without retraining.
DEDUCE: A pattern matching method for automatic de-identification of Dutch medical text
Authors: Vincent Menger, Floor Scheepers, Lisette Maria van Wijk, Marco Spruit
Date: 2018-07-01 00:00:00
Proceeding: Telematics and Informatics 2018 7
Tasks: De-identification

In order to use medical text for research purposes, it is necessary to de-identify the text for legal and privacy reasons. We report on a pattern matching method to automatically de-identify medical text written in Dutch, which requires a low amount of effort to be hand tailored. First, a selection of Protected Health Information (PHI) categories is determined in cooperation with medical staff. Then, we devise a method for de-identifying all information in one of these PHI categories, that relies on lookup tables, decision rules and fuzzy string matching. Our de-identification method DEDUCE is validated on a test corpus of 200 nursing notes and 200 treatment plans obtained from the University Medical Center Utrecht (UMCU) in the Netherlands, achieving a total micro-averaged precision of 0.814, a recall of 0.916 and a F1-score of 0.862. For person names, a recall of 0.964 was achieved, while no names of patients were missed.
Conversational Recommender System
Authors: Yueming Sun, Yi Zhang
Date: 2018-06-08 00:00:00
Tasks: Conversational Recommendation, Deep Reinforcement Learning, Recommendation Systems, Reinforcement Learning

A personalized conversational sales agent could have much commercial potential. E-commerce companies such as Amazon, eBay, JD, Alibaba etc. are piloting such kind of agents with their users. However, the research on this topic is very limited and existing solutions are either based on single round adhoc search engine or traditional multi round dialog system. They usually only utilize user inputs in the current session, ignoring users' long term preferences. On the other hand, it is well known that sales conversion rate can be greatly improved based on recommender systems, which learn user preferences based on past purchasing behavior and optimize business oriented metrics such as conversion rate or expected revenue. In this work, we propose to integrate research in dialog systems and recommender systems into a novel and unified deep reinforcement learning framework to build a personalized conversational recommendation agent that optimizes a per session based utility function.
NLTK: The Natural Language Toolkit
Authors: Edward Loper, Steven Bird
Date: 2002-05-17 00:00:00
Tasks: Multi-Label Text Classification

NLTK, the Natural Language Toolkit, is a suite of open source program modules, tutorials and problem sets, providing ready-to-use computational linguistics courseware. NLTK covers symbolic and statistical natural language processing, and is interfaced to annotated corpora. Students augment and replace existing components, learn structured programming by example, and manipulate sophisticated models from the outset.
Computing recommendations via a Knowledge Graph-aware Autoencoder
Authors: Vito Bellini, Angelo Schiavone, Tommaso Di Noia, Azzurra Ragone, Eugenio Di Sciascio
Date: 2018-07-13 00:00:00
Tasks: Diversity

In the last years, deep learning has shown to be a game-changing technology in artificial intelligence thanks to the numerous successes it reached in diverse application fields. Among others, the use of deep learning for the recommendation problem, although new, looks quite promising due to its positive performances in terms of accuracy of recommendation results. In a recommendation setting, in order to predict user ratings on unknown items a possible configuration of a deep neural network is that of autoencoders tipically used to produce a lower dimensionality representation of the original data. In this paper we present KG-AUTOENCODER, an autoencoder that bases the structure of its neural network on the semanticsaware topology of a knowledge graph thus providing a label for neurons in the hidden layer that are eventually used to build a user profile and then compute recommendations. We show the effectiveness of KG-AUTOENCODER in terms of accuracy, diversity and novelty by comparing with state of the art recommendation algorithms.
A data-driven approach to predict the success of bank telemarketing
Authors: Sérgio Moro, Paulo Cortez, Paulo Rita
Date: 2014-06-01 00:00:00
Proceeding: Decision Support Systems 2014 6
Tasks: feature selection

We propose a data mining (DM) approach to predict the success of telemarketing calls for selling bank long-term deposits. A Portuguese retail bank was addressed, with data collected from 2008 to 2013, thus including the effects of the recent financial crisis. We analyzed a large set of 150 features related with bank client, product and social-economic attributes. A semi-automatic feature selection was explored in the modeling phase, performed with the data prior to July 2012 and that allowed to select a reduced set of 22 features. We also compared four DM models: logistic regression, decision trees (DTs), neural network (NN) and support vector machine. Using two metrics, area of the receiver operating characteristic curve (AUC) and area of the LIFT cumulative curve (ALIFT), the four models were tested on an evaluation set, using the most recent data (after July 2012) and a rolling window scheme. The NN presented the best results (AUC = 0.8 and ALIFT = 0.7), allowing to reach 79% of the subscribers by selecting the half better classified clients. Also, two knowledge extraction methods, a sensitivity analysis and a DT, were applied to the NN model and revealed several key attributes (e.g., Euribor rate, direction of the call and bank agent experience). Such knowledge extraction confirmed the obtained model as credible and valuable for telemarketing campaign managers.
Contrastive Representation Distillation
Authors: Yonglong Tian, Dilip Krishnan, Phillip Isola
Date: 2019-10-23 00:00:00
Proceeding: ICLR 2020 1
Tasks: Contrastive Learning, Knowledge Distillation, Model Compression, Transfer Learning

Often we wish to transfer representational knowledge from one neural network to another. Examples include distilling a large network into a smaller one, transferring knowledge from one sensory modality to a second, or ensembling a collection of models into a single estimator. Knowledge distillation, the standard approach to these problems, minimizes the KL divergence between the probabilistic outputs of a teacher and student network. We demonstrate that this objective ignores important structural knowledge of the teacher network. This motivates an alternative objective by which we train a student to capture significantly more information in the teacher's representation of the data. We formulate this objective as contrastive learning. Experiments demonstrate that our resulting new objective outperforms knowledge distillation and other cutting-edge distillers on a variety of knowledge transfer tasks, including single model compression, ensemble distillation, and cross-modal transfer. Our method sets a new state-of-the-art in many transfer tasks, and sometimes even outperforms the teacher network when combined with knowledge distillation. Code: http://github.com/HobbitLong/RepDistiller.
Correction of Automatic Speech Recognition with Transformer Sequence-to-sequence Model
Authors: Oleksii Hrinchuk, Mariya Popova, Boris Ginsburg
Date: 2019-10-23 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Data Augmentation, Decoder, Language Modeling, Language Modelling, speech-recognition, Speech Recognition

In this work, we introduce a simple yet efficient post-processing model for automatic speech recognition (ASR). Our model has Transformer-based encoder-decoder architecture which "translates" ASR model output into grammatically and semantically correct text. We investigate different strategies for regularizing and optimizing the model and show that extensive data augmentation and the initialization with pre-trained weights are required to achieve good performance. On the LibriSpeech benchmark, our method demonstrates significant improvement in word error rate over the baseline acoustic model with greedy decoding, especially on much noisier dev-other and test-other portions of the evaluation dataset. Our model also outperforms baseline with 6-gram language model re-scoring and approaches the performance of re-scoring with Transformer-XL neural language model.
Deterministic tensor completion with hypergraph expanders
Authors: Kameron Decker Harris, Yizhe Zhu
Date: 2019-10-23 00:00:00
Tasks: LEMMA

We provide a novel analysis of low-rank tensor completion based on hypergraph expanders. As a proxy for rank, we minimize the max-quasinorm of the tensor, which generalizes the max-norm for matrices. Our analysis is deterministic and shows that the number of samples required to approximately recover an order-$t$ tensor with at most $n$ entries per dimension is linear in $n$, under the assumption that the rank and order of the tensor are $O(1)$. As steps in our proof, we find a new expander mixing lemma for a $t$-partite, $t$-uniform regular hypergraph model, and prove several new properties about tensor max-quasinorm. To the best of our knowledge, this is the first deterministic analysis of tensor completion. We develop a practical algorithm that solves a relaxed version of the max-quasinorm minimization problem, and we demonstrate its efficacy with numerical experiments.
Context-Aware Sentence/Passage Term Importance Estimation For First Stage Retrieval
Authors: Zhuyun Dai, Jamie Callan
Date: 2019-10-23 00:00:00
Tasks: Passage Retrieval, Retrieval, Sentence

Term frequency is a common method for identifying the importance of a term in a query or document. But it is a weak signal, especially when the frequency distribution is flat, such as in long queries or short documents where the text is of sentence/passage-length. This paper proposes a Deep Contextualized Term Weighting framework that learns to map BERT's contextualized text representations to context-aware term weights for sentences and passages. When applied to passages, DeepCT-Index produces term weights that can be stored in an ordinary inverted index for passage retrieval. When applied to query text, DeepCT-Query generates a weighted bag-of-words query. Both types of term weight can be used directly by typical first-stage retrieval algorithms. This is novel because most deep neural network based ranking models have higher computational costs, and thus are restricted to later-stage rankers. Experiments on four datasets demonstrate that DeepCT's deep contextualized text understanding greatly improves the accuracy of first-stage retrieval algorithms.
Machine Learning for Scent: Learning Generalizable Perceptual Representations of Small Molecules
Authors: Benjamin Sanchez-Lengeling, Jennifer N. Wei, Brian K. Lee, Richard C. Gerkin, Alán Aspuru-Guzik, Alexander B. Wiltschko
Date: 2019-10-23 00:00:00
Tasks: BIG-bench Machine Learning, Nutrition, Transfer Learning

Predicting the relationship between a molecule's structure and its odor remains a difficult, decades-old task. This problem, termed quantitative structure-odor relationship (QSOR) modeling, is an important challenge in chemistry, impacting human nutrition, manufacture of synthetic fragrance, the environment, and sensory neuroscience. We propose the use of graph neural networks for QSOR, and show they significantly out-perform prior methods on a novel data set labeled by olfactory experts. Additional analysis shows that the learned embeddings from graph neural networks capture a meaningful odor space representation of the underlying relationship between structure and odor, as demonstrated by strong performance on two challenging transfer learning tasks. Machine learning has already had a large impact on the senses of sight and sound. Based on these early results with graph neural networks for molecular properties, we hope machine learning can eventually do for olfaction what it has already done for vision and hearing.
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Authors: Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu
Date: 2019-10-23 00:00:00
Proceeding: arXiv 2019 10
Tasks: Answer Generation, Common Sense Reasoning, Coreference Resolution, Diversity, Linguistic Acceptability, Machine Translation, Multimodal Intent Recognition, Natural Language Inference, Poll Generation, Question Answering, Question Generation, Semantic Parsing, Semantic Textual Similarity, Sentiment Analysis, Transfer Learning, Word Sense Disambiguation

Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts all text-based language problems into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled data sets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new ``Colossal Clean Crawled Corpus'', we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our data set, pre-trained models, and code.
Feature Selection and Extraction for Graph Neural Networks
Authors: Deepak Bhaskar Acharya, Dr. Huaming Zhang
Date: 2019-10-23 00:00:00
Tasks: Clustering, feature selection, General Classification

Graph Neural Networks (GNNs) have been a latest hot research topic in data science, due to the fact that they use the ubiquitous data structure graphs as the underlying elements for constructing and training neural networks. In a GNN, each node has numerous features associated with it. The entire task (for example, classification, or clustering) utilizes the features of the nodes to make decisions, at node level or graph level. In this paper, (1) we extend the feature selection algorithm presented in via Gumbel Softmax to GNNs. We conduct a series of experiments on our feature selection algorithms, using various benchmark datasets: Cora, Citeseer and Pubmed. (2) We implement a mechanism to rank the extracted features. We demonstrate the effectiveness of our algorithms, for both feature selection and ranking. For the Cora dataset, (1) we use the algorithm to select 225 features out of 1433 features. Our experimental results demonstrate their effectiveness for the same classification problem. (2) We extract features such that they are linear combinations of the original features, where the coefficients for each extracted features are non-negative and sum up to one. We propose an algorithm to rank the extracted features in the sense that when using them for the same classification problem, the accuracy goes down gradually for the extracted features within the rank 1 - 50, 51 - 100, 100 - 150, and 151 - 200.
A Useful Taxonomy for Adversarial Robustness of Neural Networks
Authors: Leslie N. Smith
Date: 2019-10-23 00:00:00
Tasks: Adversarial Robustness

Adversarial attacks and defenses are currently active areas of research for the deep learning community. A recent review paper divided the defense approaches into three categories; gradient masking, robust optimization, and adversarial example detection. We divide gradient masking and robust optimization differently: (1) increasing intra-class compactness and inter-class separation of the feature vectors improves adversarial robustness, and (2) marginalization or removal of non-robust image features also improves adversarial robustness. By reframing these topics differently, we provide a fresh perspective that provides insight into the underlying factors that enable training more robust networks and can help inspire novel solutions. In addition, there are several papers in the literature of adversarial defenses that claim there is a cost for adversarial robustness, or a trade-off between robustness and accuracy but, under this proposed taxonomy, we hypothesis that this is not universal. We follow up on our taxonomy with several challenges to the deep learning research community that builds on the connections and insights in this paper.