pwc-archive/papers-with-abstracts huggingface dataset:
Multilevel Initialization for Layer-Parallel Deep Neural Network Training
Authors: Eric C. Cyr, Stefanie Günther, Jacob B. Schroder
Date: 2019-12-19 00:00:00

This paper investigates multilevel initialization strategies for training very deep neural networks with a layer-parallel multigrid solver. The scheme is based on the continuous interpretation of the training problem as a problem of optimal control, in which neural networks are represented as discretizations of time-dependent ordinary differential equations. A key goal is to develop a method able to intelligently initialize the network parameters for the very deep networks enabled by scalable layer-parallel training. To do this, we apply a refinement strategy across the time domain, that is equivalent to refining in the layer dimension. The resulting refinements create deep networks, with good initializations for the network parameters coming from the coarser trained networks. We investigate the effectiveness of such multilevel "nested iteration" strategies for network training, showing supporting numerical evidence of reduced run time for equivalent accuracy. In addition, we study whether the initialization strategies provide a regularizing effect on the overall training process and reduce sensitivity to hyperparameters and randomness in initial network parameters.
Learning a Spatio-Temporal Embedding for Video Instance Segmentation
Authors: Anthony Hu, Alex Kendall, Roberto Cipolla
Date: 2019-12-19 00:00:00
Tasks: Instance Segmentation, Semantic Segmentation, Video Instance Segmentation

We present a novel embedding approach for video instance segmentation. Our method learns a spatio-temporal embedding integrating cues from appearance, motion, and geometry; a 3D causal convolutional network models motion, and a monocular self-supervised depth loss models geometry. In this embedding space, video-pixels of the same instance are clustered together while being separated from other instances, to naturally track instances over time without any complex post-processing. Our network runs in real-time as our architecture is entirely causal - we do not incorporate information from future frames, contrary to previous methods. We show that our model can accurately track and segment instances, even with occlusions and missed detections, advancing the state-of-the-art on the KITTI Multi-Object and Tracking Dataset.
Fashion Outfit Complementary Item Retrieval
Authors: Yen-Liang Lin, Son Tran, Larry S. Davis
Date: 2019-12-19 00:00:00
Proceeding: CVPR 2020 6
Tasks: Prediction, Retrieval

Complementary fashion item recommendation is critical for fashion outfit completion. Existing methods mainly focus on outfit compatibility prediction but not in a retrieval setting. We propose a new framework for outfit complementary item retrieval. Specifically, a category-based subspace attention network is presented, which is a scalable approach for learning the subspace attentions. In addition, we introduce an outfit ranking loss that better models the item relationships of an entire outfit. We evaluate our method on the outfit compatibility, FITB and new retrieval tasks. Experimental results demonstrate that our approach outperforms state-of-the-art methods in both compatibility prediction and complementary item retrieval
Exploring AI Futures Through Role Play
Authors: Shahar Avin, Ross Gruetzemacher, James Fox
Date: 2019-12-19 00:00:00

We present an innovative methodology for studying and teaching the impacts of AI through a role play game. The game serves two primary purposes: 1) training AI developers and AI policy professionals to reflect on and prepare for future social and ethical challenges related to AI and 2) exploring possible futures involving AI technology development, deployment, social impacts, and governance. While the game currently focuses on the inter relations between short --, mid and long term impacts of AI, it has potential to be adapted for a broad range of scenarios, exploring in greater depths issues of AI policy research and affording training within organizations. The game presented here has undergone two years of development and has been tested through over 30 events involving between 3 and 70 participants. The game is under active development, but preliminary findings suggest that role play is a promising methodology for both exploring AI futures and training individuals and organizations in thinking about, and reflecting on, the impacts of AI and strategic mistakes that can be avoided today.
Going Beneath the Surface: Evaluating Image Captioning for Grammaticality, Truthfulness and Diversity
Authors: Huiyuan Xie, Tom Sherborne, Alexander Kuhnle, Ann Copestake
Date: 2019-12-19 00:00:00
Tasks: Diagnostic, Diversity, Image Captioning

Image captioning as a multimodal task has drawn much interest in recent years. However, evaluation for this task remains a challenging problem. Existing evaluation metrics focus on surface similarity between a candidate caption and a set of reference captions, and do not check the actual relation between a caption and the underlying visual content. We introduce a new diagnostic evaluation framework for the task of image captioning, with the goal of directly assessing models for grammaticality, truthfulness and diversity (GTD) of generated captions. We demonstrate the potential of our evaluation framework by evaluating existing image captioning models on a wide ranging set of synthetic datasets that we construct for diagnostic evaluation. We empirically show how the GTD evaluation framework, in combination with diagnostic datasets, can provide insights into model capabilities and limitations to supplement standard evaluations.
Optimization for deep learning: theory and algorithms
Authors: Ruoyu Sun
Date: 2019-12-19 00:00:00
Tasks: Deep Learning, Learning Theory

When and why can a neural network be successfully trained? This article provides an overview of optimization algorithms and theory for training neural networks. First, we discuss the issue of gradient explosion/vanishing and the more general issue of undesirable spectrum, and then discuss practical solutions including careful initialization and normalization methods. Second, we review generic optimization methods used in training neural networks, such as SGD, adaptive gradient methods and distributed methods, and theoretical results for these algorithms. Third, we review existing research on the global issues of neural network training, including results on bad local minima, mode connectivity, lottery ticket hypothesis and infinite-width analysis.
One-Shot Decision-Making with and without Surrogates
Authors: Jakob Bossek, Pascal Kerschke, Aneta Neumann, Frank Neumann, Carola Doerr
Date: 2019-12-19 00:00:00
Tasks: Decision Making, regression

One-shot decision making is required in situations in which we can evaluate a fixed number of solution candidates but do not have any possibility for further, adaptive sampling. Such settings are frequently encountered in neural network design, hyper-parameter optimization, and many simulation-based real-world optimization tasks, in which evaluations are costly and time sparse. It seems intuitive that well-distributed samples should be more meaningful in one-shot decision making settings than uniform or grid-based samples, since they show a better coverage of the decision space. In practice, quasi-random designs such as Latin Hypercube Samples and low-discrepancy point sets form indeed the state of the art, as confirmed by a number of recent studies and competitions. In this work we take a closer look into the correlation between the distribution of the quasi-random designs and their performance in one-shot decision making tasks, with the goal to investigate whether the assumed correlation between uniform distribution and performance can be confirmed. We study three different decision tasks: classic one-shot optimization (only the best sample matters), one-shot optimization with surrogates (allowing to use surrogate models for selecting a design that need not necessarily be one of the evaluated samples), and one-shot regression (i.e., function approximation, with minimization of mean squared error as objective). Our results confirm an advantage of low-discrepancy designs for all three settings. The overall correlation, however, is rather weak. We complement our study by evolving problem-specific samples that show significantly better performance for the regression task than the standard approaches based on low-discrepancy sequences, giving strong indication that significant performance gains over state-of-the-art one-shot sampling techniques are possible.
An Adversarial Perturbation Oriented Domain Adaptation Approach for Semantic Segmentation
Authors: Jihan Yang, Ruijia Xu, Ruiyu Li, Xiaojuan Qi, Xiaoyong Shen, Guanbin Li, Liang Lin
Date: 2019-12-18 00:00:00
Tasks: Domain Adaptation, Position, Segmentation, Semantic Segmentation, Unsupervised Domain Adaptation

We focus on Unsupervised Domain Adaptation (UDA) for the task of semantic segmentation. Recently, adversarial alignment has been widely adopted to match the marginal distribution of feature representations across two domains globally. However, this strategy fails in adapting the representations of the tail classes or small objects for semantic segmentation since the alignment objective is dominated by head categories or large objects. In contrast to adversarial alignment, we propose to explicitly train a domain-invariant classifier by generating and defensing against pointwise feature space adversarial perturbations. Specifically, we firstly perturb the intermediate feature maps with several attack objectives (i.e., discriminator and classifier) on each individual position for both domains, and then the classifier is trained to be invariant to the perturbations. By perturbing each position individually, our model treats each location evenly regardless of the category or object size and thus circumvents the aforementioned issue. Moreover, the domain gap in feature space is reduced by extrapolating source and target perturbed features towards each other with attack on the domain discriminator. Our approach achieves the state-of-the-art performance on two challenging domain adaptation tasks for semantic segmentation: GTA5 -> Cityscapes and SYNTHIA -> Cityscapes.
The power of synergy in differential privacy: Combining a small curator with local randomizers
Authors: Amos Beimel, Aleksandra Korolova, Kobbi Nissim, Or Sheffet, Uri Stemmer
Date: 2019-12-18 00:00:00
Tasks: Two-sample testing

Motivated by the desire to bridge the utility gap between local and trusted curator models of differential privacy for practical applications, we initiate the theoretical study of a hybrid model introduced by "Blender" [Avent et al.,\ USENIX Security '17], in which differentially private protocols of n agents that work in the local-model are assisted by a differentially private curator that has access to the data of m additional users. We focus on the regime where m << n and study the new capabilities of this (m,n)-hybrid model. We show that, despite the fact that the hybrid model adds no significant new capabilities for the basic task of simple hypothesis-testing, there are many other tasks (under a wide range of parameters) that can be solved in the hybrid model yet cannot be solved either by the curator or by the local-users separately. Moreover, we exhibit additional tasks where at least one round of interaction between the curator and the local-users is necessary -- namely, no hybrid model protocol without such interaction can solve these tasks. Taken together, our results show that the combination of the local model with a small curator can become part of a promising toolkit for designing and implementing differential privacy.
Enhancing streamflow forecast and extracting insights using long-short term memory networks with data integration at continental scales
Authors: Dapeng Feng, Kuai Fang, Chaopeng Shen
Date: 2019-12-18 00:00:00
Tasks: Data Integration

Recent observations with varied schedules and types (moving average, snapshot, or regularly spaced) can help to improve streamflow forecasts, but it is challenging to integrate them effectively. Based on a long short-term memory (LSTM) streamflow model, we tested multiple versions of a flexible procedure we call data integration (DI) to leverage recent discharge measurements to improve forecasts. DI accepts lagged inputs either directly or through a convolutional neural network (CNN) unit. DI ubiquitously elevated streamflow forecast performance to unseen levels, reaching a record continental-scale median Nash-Sutcliffe Efficiency coefficient value of 0.86. Integrating moving-average discharge, discharge from the last few days, or even average discharge from the previous calendar month could all improve daily forecasts. Directly using lagged observations as inputs was comparable in performance to using the CNN unit. Importantly, we obtained valuable insights regarding hydrologic processes impacting LSTM and DI performance. Before applying DI, the base LSTM model worked well in mountainous or snow-dominated regions, but less well in regions with low discharge volumes (due to either low precipitation or high precipitation-energy synchronicity) and large inter-annual storage variability. DI was most beneficial in regions with high flow autocorrelation: it greatly reduced baseflow bias in groundwater-dominated western basins and also improved peak prediction for basins with dynamical surface water storage, such as the Prairie Potholes or Great Lakes regions. However, even DI cannot elevate high-aridity basins with one-day flash peaks. Despite this limitation, there is much promise for a deep-learning-based forecast paradigm due to its performance, automation, efficiency, and flexibility.
Counterfactual thinking in cooperation dynamics
Authors: Luis Moniz Pereira, Francisco C. Santos
Date: 2019-12-18 00:00:00
Tasks: counterfactual, Counterfactual Reasoning

Counterfactual Thinking is a human cognitive ability studied in a wide variety of domains. It captures the process of reasoning about a past event that did not occur, namely what would have happened had this event occurred, or, otherwise, to reason about an event that did occur but what would ensue had it not. Given the wide cognitive empowerment of counterfactual reasoning in the human individual, the question arises of how the presence of individuals with this capability may improve cooperation in populations of self-regarding individuals. Here we propose a mathematical model, grounded on Evolutionary Game Theory, to examine the population dynamics emerging from the interplay between counterfactual thinking and social learning (i.e., individuals that learn from the actions and success of others) whenever the individuals in the population face a collective dilemma. Our results suggest that counterfactual reasoning fosters coordination in collective action problems occurring in large populations, and has a limited impact on cooperation dilemmas in which coordination is not required. Moreover, we show that a small prevalence of individuals resorting to counterfactual thinking is enough to nudge an entire population towards highly cooperative standards.
Pathomic Fusion: An Integrated Framework for Fusing Histopathology and Genomic Features for Cancer Diagnosis and Prognosis
Authors: Richard J. Chen, Ming Y. Lu, Jingwen Wang, Drew F. K. Williamson, Scott J. Rodig, Neal I. Lindeman, Faisal Mahmood
Date: 2019-12-18 00:00:00
Tasks: Feature Importance, Prognosis

Cancer diagnosis, prognosis, and therapeutic response predictions are based on morphological information from histology slides and molecular profiles from genomic data. However, most deep learning-based objective outcome prediction and grading paradigms are based on histology or genomics alone and do not make use of the complementary information in an intuitive manner. In this work, we propose Pathomic Fusion, an interpretable strategy for end-to-end multimodal fusion of histology image and genomic (mutations, CNV, RNA-Seq) features for survival outcome prediction. Our approach models pairwise feature interactions across modalities by taking the Kronecker product of unimodal feature representations and controls the expressiveness of each representation via a gating-based attention mechanism. Following supervised learning, we are able to interpret and saliently localize features across each modality, and understand how feature importance shifts when conditioning on multimodal input. We validate our approach using glioma and clear cell renal cell carcinoma datasets from the Cancer Genome Atlas (TCGA), which contains paired whole-slide image, genotype, and transcriptome data with ground truth survival and histologic grade labels. In a 15-fold cross-validation, our results demonstrate that the proposed multimodal fusion paradigm improves prognostic determinations from ground truth grading and molecular subtyping, as well as unimodal deep networks trained on histology and genomic data alone. The proposed method establishes insight and theory on how to train deep networks on multimodal biomedical data in an intuitive manner, which will be useful for other problems in medicine that seek to combine heterogeneous data streams for understanding diseases and predicting response and resistance to treatment.
One-Shot Weakly Supervised Video Object Segmentation
Authors: Mennatullah Siam, Naren Doraiswamy, Boris N. Oreshkin, Hengshuai Yao, Martin Jagersand
Date: 2019-12-18 00:00:00
Tasks: Object, Segmentation, Semantic Segmentation, Video Object Segmentation, Video Semantic Segmentation, Word Embeddings

Conventional few-shot object segmentation methods learn object segmentation from a few labelled support images with strongly labelled segmentation masks. Recent work has shown to perform on par with weaker levels of supervision in terms of scribbles and bounding boxes. However, there has been limited attention given to the problem of few-shot object segmentation with image-level supervision. We propose a novel multi-modal interaction module for few-shot object segmentation that utilizes a co-attention mechanism using both visual and word embeddings. It enables our model to achieve 5.1% improvement over previously proposed image-level few-shot object segmentation. Our method compares relatively close to the state of the art methods that use strong supervision, while ours use the least possible supervision. We further propose a novel setup for few-shot weakly supervised video object segmentation(VOS) that relies on image-level labels for the first frame. The proposed setup uses weak annotation unlike semi-supervised VOS setting that utilizes strongly labelled segmentation masks. The setup evaluates the effectiveness of generalizing to novel classes in the VOS setting. The setup splits the VOS data into multiple folds with different categories per fold. It provides a potential setup to evaluate how few-shot object segmentation methods can benefit from additional object poses, or object interactions that is not available in static frames as in PASCAL-5i benchmark.
An Adaptive Similarity Measure to Tune Trust Influence in Memory-Based Collaborative Filtering
Authors: Mohammad Reza Zarei, Mohammad R. Moosavi
Date: 2019-12-18 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

The aim of the recommender systems is to provide relevant and potentially interesting information to each user. This is fulfilled by utilizing the already recorded tendencies of similar users or detecting items similar to interested items of the user. Challenges such as data sparsity and cold start problem are addressed in recent studies. Utilizing social information not only enhances the prediction accuracy but also tackles the data sparseness challenges. In this paper, we investigate the impact of using direct and indirect trust information in a memory-based collaborative filtering recommender system. An adaptive similarity measure is proposed and the contribution of social information is tuned using two learning schemes, greedy and gradient-based optimization. The results of the proposed method are compared with state-of-the-art memory-based and model-based CF approaches on two real-world datasets, Epinions and FilmTrust. The experiments show that our method is quite effective in designing an accurate and comprehensive recommender system.
Collaborative Filtering vs. Content-Based Filtering: differences and similarities
Authors: Rafael Glauber, Angelo Loula
Date: 2019-12-18 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

Recommendation Systems (SR) suggest items exploring user preferences, helping them with the information overload problem. Two approaches to SR have received more prominence, Collaborative Filtering, and Content-Based Filtering. Moreover, even though studies are indicating their advantages and disadvantages, few results empirically prove their characteristics, similarities, and differences. In this work, an experimental methodology is proposed to perform comparisons between recommendation algorithms for different approaches going beyond the "precision of the predictions". For the experiments, three algorithms of recommendation were tested: a baseline for Collaborative Filtration and two algorithms for Content-based Filtering that were developed for this evaluation. The experiments demonstrate the behavior of these systems in different data sets, its main characteristics and especially the complementary aspect of the two main approaches.
On the Metrics and Adaptation Methods for Domain Divergences of sEMG-based Gesture Recognition
Authors: István Ketykó, Ferenc Kovács
Date: 2019-12-18 00:00:00
Tasks: Domain Adaptation, Gesture Recognition, Time Series, Time Series Analysis, Time Series Classification

We propose a new metric to measure domain divergence and a new domain adaptation method for time-series classification. The metric belongs to the class of probability distributions-based metrics, is transductive, and does not assume the presence of source data samples. The 2-stage method utilizes an improved autoregressive, RNN-based architecture with deep/non-linear transformation. We assess our metric and the performance of our model in the context of sEMG/EMG-based gesture recognition under inter-session and inter-subject domain shifts.
Enabling Smartphone-based Estimation of Heart Rate
Authors: Nutta Homdee, Mehdi Boukhechba, Yixue W. Feng, Natalie Kramer, John Lach, Laura E. Barnes
Date: 2019-12-18 00:00:00

Continuous, ubiquitous monitoring through wearable sensors has the potential to collect useful information about users' context. Heart rate is an important physiologic measure used in a wide variety of applications, such as fitness tracking and health monitoring. However, wearable sensors that monitor heart rate, such as smartwatches and electrocardiogram (ECG) patches, can have gaps in their data streams because of technical issues (e.g., bad wireless channels, battery depletion, etc.) or user-related reasons (e.g. motion artifacts, user compliance, etc.). The ability to use other available sensor data (e.g., smartphone data) to estimate missing heart rate readings is useful to cope with any such gaps, thus improving data quality and continuity. In this paper, we test the feasibility of estimating raw heart rate using smartphone sensor data. Using data generated by 12 participants in a one-week study period, we were able to build both personalized and generalized models using regression, SVM, and random forest algorithms. All three algorithms outperformed the baseline moving-average interpolation method for both personalized and generalized settings. Moreover, our findings suggest that personalized models outperformed the generalized models, which speaks to the importance of considering personal physiology, behavior, and life style in the estimation of heart rate. The promising results provide preliminary evidence of the feasibility of combining smartphone sensor data with wearable sensor data for continuous heart rate monitoring.