pwc-archive/papers-with-abstracts huggingface dataset:
A Deep Learning Pipeline for Patient Diagnosis Prediction Using Electronic Health Records
Authors: Leopold Franz, Yash Raj Shrestha, Bibek Paudel
Date: 2020-06-23 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making

Augmentation of disease diagnosis and decision-making in healthcare with machine learning algorithms is gaining much impetus in recent years. In particular, in the current epidemiological situation caused by COVID-19 pandemic, swift and accurate prediction of disease diagnosis with machine learning algorithms could facilitate identification and care of vulnerable clusters of population, such as those having multi-morbidity conditions. In order to build a useful disease diagnosis prediction system, advancement in both data representation and development of machine learning architectures are imperative. First, with respect to data collection and representation, we face severe problems due to multitude of formats and lack of coherency prevalent in Electronic Health Records (EHRs). This causes hindrance in extraction of valuable information contained in EHRs. Currently, no universal global data standard has been established. As a useful solution, we develop and publish a Python package to transform public health dataset into an easy to access universal format. This data transformation to an international health data format facilitates researchers to easily combine EHR datasets with clinical datasets of diverse formats. Second, machine learning algorithms that predict multiple disease diagnosis categories simultaneously remain underdeveloped. We propose two novel model architectures in this regard. First, DeepObserver, which uses structured numerical data to predict the diagnosis categories and second, ClinicalBERT_Multi, that incorporates rich information available in clinical notes via natural language processing methods and also provides interpretable visualizations to medical practitioners. We show that both models can predict multiple diagnoses simultaneously with high accuracy.
Neural Network Middle-Term Probabilistic Forecasting of Daily Power Consumption
Authors: Michele Azzone, Roberto Baviera
Date: 2020-06-05 00:00:00

Middle-term horizon (months to a year) power consumption prediction is a main challenge in the energy sector, in particular when probabilistic forecasting is considered. We propose a new modelling approach that incorporates trend, seasonality and weather conditions, as explicative variables in a shallow Neural Network with an autoregressive feature. We obtain excellent results for density forecast on the one-year test set applying it to the daily power consumption in New England U.S.A.. The quality of the achieved power consumption probabilistic forecasting has been verified, on the one hand, comparing the results to other standard models for density forecasting and, on the other hand, considering measures that are frequently used in the energy sector as pinball loss and CI backtesting.
Validation and generalization of pixel-wise relevance in convolutional neural networks trained for face classification
Authors: Jñani Crawford, Eshed Margalit, Kalanit Grill-Spector, Sonia Poltoratski
Date: 2020-06-16 00:00:00
Tasks: Classification, Face Recognition, General Classification

The increased use of convolutional neural networks for face recognition in science, governance, and broader society has created an acute need for methods that can show how these 'black box' decisions are made. To be interpretable and useful to humans, such a method should convey a model's learned classification strategy in a way that is robust to random initializations or spurious correlations in input data. To this end, we applied the decompositional pixel-wise attribution method of layer-wise relevance propagation (LRP) to resolve the decisions of several classes of VGG-16 models trained for face recognition. We then quantified how these relevance measures vary with and generalize across key model parameters, such as the pretraining dataset (ImageNet or VGGFace), the finetuning task (gender or identity classification), and random initializations of model weights. Using relevance-based image masking, we find that relevance maps for face classification prove generally stable across random initializations, and can generalize across finetuning tasks. However, there is markedly less generalization across pretraining datasets, indicating that ImageNet- and VGGFace-trained models sample face information differently even as they achieve comparably high classification performance. Fine-grained analyses of relevance maps across models revealed asymmetries in generalization that point to specific benefits of choice parameters, and suggest that it may be possible to find an underlying set of important face image pixels that drive decisions across convolutional neural networks and tasks. Finally, we evaluated model decision weighting against human measures of similarity, providing a novel framework for interpreting face recognition decisions across human and machine.
Large image datasets: A pyrrhic win for computer vision?
Authors: Vinay Uday Prabhu, Abeba Birhane
Date: 2020-06-24 00:00:00

In this paper we investigate problematic practices and consequences of large scale vision datasets. We examine broad issues such as the question of consent and justice as well as specific concerns such as the inclusion of verifiably pornographic images in datasets. Taking the ImageNet-ILSVRC-2012 dataset as an example, we perform a cross-sectional model-based quantitative census covering factors such as age, gender, NSFW content scoring, class-wise accuracy, human-cardinality-analysis, and the semanticity of the image class information in order to statistically investigate the extent and subtleties of ethical transgressions. We then use the census to help hand-curate a look-up-table of images in the ImageNet-ILSVRC-2012 dataset that fall into the categories of verifiably pornographic: shot in a non-consensual setting (up-skirt), beach voyeuristic, and exposed private parts. We survey the landscape of harm and threats both society broadly and individuals face due to uncritical and ill-considered dataset curation practices. We then propose possible courses of correction and critique the pros and cons of these. We have duly open-sourced all of the code and the census meta-datasets generated in this endeavor for the computer vision community to build on. By unveiling the severity of the threats, our hope is to motivate the constitution of mandatory Institutional Review Boards (IRB) for large scale dataset curation processes.
We Should at Least Be Able to Design Molecules That Dock Well
Authors: Tobiasz Cieplinski, Tomasz Danel, Sabina Podlewska, Stanislaw Jastrzebski
Date: 2020-06-20 00:00:00
Tasks: Drug Design, Drug Discovery

Designing compounds with desired properties is a key element of the drug discovery process. However, measuring progress in the field has been challenging due to the lack of realistic retrospective benchmarks, and the large cost of prospective validation. To close this gap, we propose a benchmark based on docking, a popular computational method for assessing molecule binding to a protein. Concretely, the goal is to generate drug-like molecules that are scored highly by SMINA, a popular docking software. We observe that popular graph-based generative models fail to generate molecules with a high docking score when trained using a realistically sized training set. This suggests a limitation of the current incarnation of models for de novo drug design. Finally, we propose a simplified version of the benchmark based on a simpler scoring function, and show that the tested models are able to partially solve it. We release the benchmark as an easy to use package available at https://github.com/cieplinski-tobiasz/smina-docking-benchmark. We hope that our benchmark will serve as a stepping stone towards the goal of automatically generating promising drug candidates.
QPSO-CD: Quantum-behaved Particle Swarm Optimization Algorithm with Cauchy Distribution
Authors: Amandeep Singh Bhatia, Mandeep Kaur Saggi, Shenggen Zheng, Soumya Ranjan Nayak
Date: 2020-06-27 00:00:00

Motivated by particle swarm optimization (PSO) and quantum computing theory, we have presented a quantum variant of PSO (QPSO) mutated with Cauchy operator and natural selection mechanism (QPSO-CD) from evolutionary computations. The performance of proposed hybrid quantum-behaved particle swarm optimization with Cauchy distribution (QPSO-CD) is investigated and compared with its counterparts based on a set of benchmark problems. Moreover, QPSO-CD is employed in well-studied constrained engineering problems to investigate its applicability. Further, the correctness and time complexity of QPSO-CD are analysed and compared with the classical PSO. It has been proven that QPSO-CD handles such real-life problems efficiently and can attain superior solutions in most of the problems. The experimental results showed that QPSO associated with Cauchy distribution and natural selection strategy outperforms other variants in the context of stability and convergence.
An energy efficient service composition mechanism using a hybrid meta-heuristic algorithm in a mobile cloud environment
Authors: Godar J. Ibrahim, Tarik A. Rashid, Mobayode O. Akinsolu
Date: 2020-05-19 00:00:00
Tasks: Service Composition

By increasing mobile devices in technology and human life, using a runtime and mobile services has gotten more complex along with the composition of a large number of atomic services. Different services are provided by mobile cloud components to represent the non-functional properties as Quality of Service (QoS), which is applied by a set of standards. On the other hand, the growth of the energy-source heterogeneity in mobile clouds is an emerging challenge according to the energy-saving problem in mobile nodes. To mobile cloud service composition as an NP-Hard problem, an efficient selection method should be taken by problem using optimal energy-aware methods that can extend the deployment and interoperability of mobile cloud components. Also, an energy-aware service composition mechanism is required to preserve high energy saving scenarios for mobile cloud components. In this paper, an energy-aware mechanism is applied to optimize mobile cloud service composition using a hybrid Shuffled Frog Leaping Algorithm and Genetic Algorithm (SFGA). Experimental results capture that the proposed mechanism improves the feasibility of the service composition with minimum energy consumption, response time, and cost for mobile cloud components against some current algorithms.
Applying Machine Learning Techniques for Caching in Edge Networks: A Comprehensive Survey
Authors: Junaid Shuja, Kashif Bilal, Waleed Alasmary, Hassan Sinky, Eisa Alanazi
Date: 2020-06-21 00:00:00
Tasks: BIG-bench Machine Learning, Community Detection

Edge networking is a complex and dynamic computing paradigm that aims to push cloud resources closer to the end user improving responsiveness and reducing backhaul traffic. User mobility, preferences, and content popularity are the dominant dynamic features of edge networks. Temporal and social features of content, such as the number of views and likes are leveraged to estimate the popularity of content from a global perspective. However, such estimates should not be mapped to an edge network with particular social and geographic characteristics. In next generation edge networks, i.e., 5G and beyond 5G, machine learning techniques can be applied to predict content popularity based on user preferences, cluster users based on similar content interests, and optimize cache placement and replacement strategies provided a set of constraints and predictions about the state of the network. These applications of machine learning can help identify relevant content for an edge network. This article investigates the application of machine learning techniques for in-network caching in edge networks. We survey recent state-of-the-art literature and formulate a comprehensive taxonomy based on (a) machine learning technique (method, objective, and features), (b) caching strategy (policy, location, and replacement), and (c) edge network (type and delivery strategy). A comparative analysis of the state-of-the-art literature is presented with respect to the parameters identified in the taxonomy. Moreover, we debate research challenges and future directions for optimal caching decisions and the application of machine learning in edge networks.
Rapid Response Crop Maps in Data Sparse Regions
Authors: Hannah Kerner, Gabriel Tseng, Inbal Becker-Reshef, Catherine Nakalembe, Brian Barker, Blake Munshell, Madhava Paliyam, Mehdi Hosseini
Date: 2020-06-23 00:00:00
Tasks: Humanitarian

Spatial information on cropland distribution, often called cropland or crop maps, are critical inputs for a wide range of agriculture and food security analyses and decisions. However, high-resolution cropland maps are not readily available for most countries, especially in regions dominated by smallholder farming (e.g., sub-Saharan Africa). These maps are especially critical in times of crisis when decision makers need to rapidly design and enact agriculture-related policies and mitigation strategies, including providing humanitarian assistance, dispersing targeted aid, or boosting productivity for farmers. A major challenge for developing crop maps is that many regions do not have readily accessible ground truth data on croplands necessary for training and validating predictive models, and field campaigns are not feasible for collecting labels for rapid response. We present a method for rapid mapping of croplands in regions where little to no ground data is available. We present results for this method in Togo, where we delivered a high-resolution (10 m) cropland map in under 10 days to facilitate rapid response to the COVID-19 pandemic by the Togolese government. This demonstrated a successful transition of machine learning applications research to operational rapid response in a real humanitarian crisis. All maps, data, and code are publicly available to enable future research and operational systems in data-sparse regions.
An alternative to synthetic control for models with many covariates under sparsity
Authors: Marianne Bléhaut, Xavier D'Haultfoeuille, Jérémy L'Hour, Alexandre B. Tsybakov
Date: 2020-05-25 00:00:00

The synthetic control method is a an econometric tool to evaluate causal effects when only one unit is treated. While initially aimed at evaluating the effect of large-scale macroeconomic changes with very few available control units, it has increasingly been used in place of more well-known microeconometric tools in a broad range of applications, but its properties in this context are unknown. This paper introduces an alternative to the synthetic control method, which is developed both in the usual asymptotic framework and in the high-dimensional scenario. We propose an estimator of average treatment effect that is doubly robust, consistent and asymptotically normal. It is also immunized against first-step selection mistakes. We illustrate these properties using Monte Carlo simulations and applications to both standard and potentially high-dimensional settings, and offer a comparison with the synthetic control method.
Mixed Logit Models and Network Formation
Authors: Harsh Gupta, Mason A. Porter
Date: 2020-06-30 00:00:00
Tasks: Discrete Choice Models, Sociology

The study of network formation is pervasive in economics, sociology, and many other fields. In this paper, we model network formation as a `choice' that is made by nodes in a network to connect to other nodes. We study these `choices' using discrete-choice models, in which an agent chooses between two or more discrete alternatives. We employ the `repeated-choice' (RC) model to study network formation. We argue that the RC model overcomes important limitations of the multinomial logit (MNL) model, which gives one framework for studying network formation, and that it is well-suited to study network formation. We also illustrate how to use the RC model to accurately study network formation using both synthetic and real-world networks. Using edge-independent synthetic networks, we also compare the performance of the MNL model and the RC model. We find that the RC model estimates the data-generation process of our synthetic networks more accurately than the MNL model. In a patent citation network, which forms sequentially, we present a case study of a qualitatively interesting scenario -- the fact that new patents are more likely to cite older, more cited, and similar patents -- for which employing the RC model yields interesting insights.
Swapping Autoencoder for Deep Image Manipulation
Authors: Taesung Park, Jun-Yan Zhu, Oliver Wang, Jingwan Lu, Eli Shechtman, Alexei A. Efros, Richard Zhang
Date: 2020-07-01 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Image Manipulation

Deep generative models have become increasingly effective at producing realistic images from randomly sampled seeds, but using such models for controllable manipulation of existing images remains challenging. We propose the Swapping Autoencoder, a deep model designed specifically for image manipulation, rather than random sampling. The key idea is to encode an image with two independent components and enforce that any swapped combination maps to a realistic image. In particular, we encourage the components to represent structure and texture, by enforcing one component to encode co-occurrent patch statistics across different parts of an image. As our method is trained with an encoder, finding the latent codes for a new input image becomes trivial, rather than cumbersome. As a result, it can be used to manipulate real input images in various ways, including texture swapping, local and global editing, and latent code vector arithmetic. Experiments on multiple datasets show that our model produces better results and is substantially more efficient compared to recent generative models.
Group Ensemble: Learning an Ensemble of ConvNets in a single ConvNet
Authors: Hao Chen, Abhinav Shrivastava
Date: 2020-07-01 00:00:00
Tasks: Action Recognition, Ensemble Learning, object-detection, Object Detection

Ensemble learning is a general technique to improve accuracy in machine learning. However, the heavy computation of a ConvNets ensemble limits its usage in deep learning. In this paper, we present Group Ensemble Network (GENet), an architecture incorporating an ensemble of ConvNets in a single ConvNet. Through a shared-base and multi-head structure, GENet is divided into several groups to make explicit ensemble learning possible in a single ConvNet. Owing to group convolution and the shared-base, GENet can fully leverage the advantage of explicit ensemble learning while retaining the same computation as a single ConvNet. Additionally, we present Group Averaging, Group Wagging and Group Boosting as three different strategies to aggregate these ensemble members. Finally, GENet outperforms larger single networks, standard ensembles of smaller networks, and other recent state-of-the-art methods on CIFAR and ImageNet. Specifically, group ensemble reduces the top-1 error by 1.83% for ResNeXt-50 on ImageNet. We also demonstrate its effectiveness on action recognition and object detection tasks.
Measuring Robustness to Natural Distribution Shifts in Image Classification
Authors: Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, Ludwig Schmidt
Date: 2020-07-01 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Classification, Domain Generalization, General Classification, image-classification, Image Classification

We study how robust current ImageNet models are to distribution shifts arising from natural variations in datasets. Most research on robustness focuses on synthetic image perturbations (noise, simulated weather artifacts, adversarial examples, etc.), which leaves open how robustness on synthetic distribution shift relates to distribution shift arising in real data. Informed by an evaluation of 204 ImageNet models in 213 different test conditions, we find that there is often little to no transfer of robustness from current synthetic to natural distribution shift. Moreover, most current techniques provide no robustness to the natural distribution shifts in our testbed. The main exception is training on larger and more diverse datasets, which in multiple cases increases robustness, but is still far from closing the performance gaps. Our results indicate that distribution shifts arising in real data are currently an open research problem. We provide our testbed and data as a resource for future work at https://modestyachts.github.io/imagenet-testbed/ .
Object Goal Navigation using Goal-Oriented Semantic Exploration
Authors: Devendra Singh Chaplot, Dhiraj Gandhi, Abhinav Gupta, Ruslan Salakhutdinov
Date: 2020-07-01 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Object, Robot Navigation

This work studies the problem of object goal navigation which involves navigating to an instance of the given object category in unseen environments. End-to-end learning-based navigation methods struggle at this task as they are ineffective at exploration and long-term planning. We propose a modular system called, `Goal-Oriented Semantic Exploration' which builds an episodic semantic map and uses it to explore the environment efficiently based on the goal object category. Empirical results in visually realistic simulation environments show that the proposed model outperforms a wide range of baselines including end-to-end learning-based methods as well as modular map-based methods and led to the winning entry of the CVPR-2020 Habitat ObjectNav Challenge. Ablation analysis indicates that the proposed model learns semantic priors of the relative arrangement of objects in a scene, and uses them to explore efficiently. Domain-agnostic module design allow us to transfer our model to a mobile robot platform and achieve similar performance for object goal navigation in the real-world.
All in the Exponential Family: Bregman Duality in Thermodynamic Variational Inference
Authors: Rob Brekelmans, Vaden Masrani, Frank Wood, Greg Ver Steeg, Aram Galstyan
Date: 2020-07-01 00:00:00
Tasks: All, Variational Inference

The recently proposed Thermodynamic Variational Objective (TVO) leverages thermodynamic integration to provide a family of variational inference objectives, which both tighten and generalize the ubiquitous Evidence Lower Bound (ELBO). However, the tightness of TVO bounds was not previously known, an expensive grid search was used to choose a "schedule" of intermediate distributions, and model learning suffered with ostensibly tighter bounds. In this work, we propose an exponential family interpretation of the geometric mixture curve underlying the TVO and various path sampling methods, which allows us to characterize the gap in TVO likelihood bounds as a sum of KL divergences. We propose to choose intermediate distributions using equal spacing in the moment parameters of our exponential family, which matches grid search performance and allows the schedule to adaptively update over the course of training. Finally, we derive a doubly reparameterized gradient estimator which improves model learning and allows the TVO to benefit from more refined bounds. To further contextualize our contributions, we provide a unified framework for understanding thermodynamic integration and the TVO using Taylor series remainders.