OpenRaiser/Intern-Atlas huggingface dataset:
MDLdroid: a ChainSGD-reduce Approach to Mobile Deep Learning for Personal Mobile Sensing
Venue: arXiv cs.LG 2020
Year: 2020

Personal mobile sensing is fast permeating our daily lives to enable activity monitoring, healthcare and rehabilitation. Combined with deep learning, these applications have achieved significant success in recent years. Different from conventional cloud-based paradigms, running deep learning on devices offers several advantages including data privacy preservation and low-latency response for both model inference and update. Since data collection is costly in reality, Google's Federated Learning offers not only complete data privacy but also better model robustness based on multiple user data. However, personal mobile sensing applications are mostly user-specific and highly affected by environment. As a result, continuous local changes may seriously affect the performance of a global model generated by Federated Learning. In addition, deploying Federated Learning on a local server, e.g., edge server, may quickly reach the bottleneck due to resource constraint and serious failure by attacks. Towards pushing deep learning on devices, we present MDLdroid, a novel decentralized mobile deep learning framework to enable resource-aware on-device collaborative learning for personal mobile sensing applications. To address resource limitation, we propose a ChainSGD-reduce approach which includes a novel chain-directed Synchronous Stochastic Gradient Descent algorithm to effectively reduce overhead among multiple devices. We also design an agent-based multi-goal reinforcement learning mechanism to balance resources in a fair and efficient manner. Our evaluations show that our model training on off-the-shelf mobile devices achieves 2x to 3.5x faster than single-device training, and 1.5x faster than the master-slave approach.
Domain Embedded Multi-model Generative Adversarial Networks for Image-based Face Inpainting
Venue: arXiv cs.CV 2020
Year: 2020

Prior knowledge of face shape and structure plays an important role in face inpainting. However, traditional face inpainting methods mainly focus on the generated image resolution of the missing portion without consideration of the special particularities of the human face explicitly and generally produce discordant facial parts. To solve this problem, we present a domain embedded multi-model generative adversarial model for inpainting of face images with large cropped regions. We firstly represent only face regions using the latent variable as the domain knowledge and combine it with the non-face parts textures to generate high-quality face images with plausible contents. Two adversarial discriminators are finally used to judge whether the generated distribution is close to the real distribution or not. It can not only synthesize novel image structures but also explicitly utilize the embedded face domain knowledge to generate better predictions with consistency on structures and appearance. Experiments on both CelebA and CelebA-HQ face datasets demonstrate that our proposed approach achieved state-of-the-art performance and generates higher quality inpainting results than existing ones.
SPN-CNN: Boosting Sensor-Based Source Camera Attribution With Deep Learning
Venue: arXiv cs.CV 2020
Year: 2020

We explore means to advance source camera identification based on sensor noise in a data-driven framework. Our focus is on improving the sensor pattern noise (SPN) extraction from a single image at test time. Where existing works suppress nuisance content with denoising filters that are largely agnostic to the specific SPN signal of interest, we demonstrate that a~deep learning approach can yield a more suitable extractor that leads to improved source attribution. A series of extensive experiments on various public datasets confirms the feasibility of our approach and its applicability to image manipulation localization and video source attribution. A critical discussion of potential pitfalls completes the text.
A deep-learning view of chemical space designed to facilitate drug discovery
Venue: arXiv cs.LG 2020
Year: 2020

Drug discovery projects entail cycles of design, synthesis, and testing that yield a series of chemically related small molecules whose properties, such as binding affinity to a given target protein, are progressively tailored to a particular drug discovery goal. The use of deep learning technologies could augment the typical practice of using human intuition in the design cycle, and thereby expedite drug discovery projects. Here we present DESMILES, a deep neural network model that advances the state of the art in machine learning approaches to molecular design. We applied DESMILES to a previously published benchmark that assesses the ability of a method to modify input molecules to inhibit the dopamine receptor D2, and DESMILES yielded a 77% lower failure rate compared to state-of-the-art models. To explain the ability of DESMILES to hone molecular properties, we visualize a layer of the DESMILES network, and further demonstrate this ability by using DESMILES to tailor the same molecules used in the D2 benchmark test to dock more potently against seven different receptors.
$M^3$T: Multi-Modal Continuous Valence-Arousal Estimation in the Wild
Venue: arXiv cs.CV 2020
Year: 2020

This report describes a multi-modal multi-task ($M^3$T) approach underlying our submission to the valence-arousal estimation track of the Affective Behavior Analysis in-the-wild (ABAW) Challenge, held in conjunction with the IEEE International Conference on Automatic Face and Gesture Recognition (FG) 2020. In the proposed $M^3$T framework, we fuse both visual features from videos and acoustic features from the audio tracks to estimate the valence and arousal. The spatio-temporal visual features are extracted with a 3D convolutional network and a bidirectional recurrent neural network. Considering the correlations between valence / arousal, emotions, and facial actions, we also explores mechanisms to benefit from other tasks. We evaluated the $M^3$T framework on the validation set provided by ABAW and it significantly outperforms the baseline method.
Local Facial Attribute Transfer through Inpainting
Venue: arXiv cs.CV 2020
Year: 2020

The term attribute transfer refers to the tasks of altering images in such a way, that the semantic interpretation of a given input image is shifted towards an intended direction, which is quantified by semantic attributes. Prominent example applications are photo realistic changes of facial features and expressions, like changing the hair color, adding a smile, enlarging the nose or altering the entire context of a scene, like transforming a summer landscape into a winter panorama. Recent advances in attribute transfer are mostly based on generative deep neural networks, using various techniques to manipulate images in the latent space of the generator. In this paper, we present a novel method for the common sub-task of local attribute transfers, where only parts of a face have to be altered in order to achieve semantic changes (e.g. removing a mustache). In contrast to previous methods, where such local changes have been implemented by generating new (global) images, we propose to formulate local attribute transfers as an inpainting problem. Removing and regenerating only parts of images, our Attribute Transfer Inpainting Generative Adversarial Network (ATI-GAN) is able to utilize local context information to focus on the attributes while keeping the background unmodified resulting in visually sound results.
Bayesian Residual Policy Optimization: Scalable Bayesian Reinforcement Learning with Clairvoyant Experts
Venue: arXiv cs.RO 2020
Year: 2020

Informed and robust decision making in the face of uncertainty is critical for robots that perform physical tasks alongside people. We formulate this as Bayesian Reinforcement Learning over latent Markov Decision Processes (MDPs). While Bayes-optimality is theoretically the gold standard, existing algorithms do not scale well to continuous state and action spaces. Our proposal builds on the following insight: in the absence of uncertainty, each latent MDP is easier to solve. We first obtain an ensemble of experts, one for each latent MDP, and fuse their advice to compute a baseline policy. Next, we train a Bayesian residual policy to improve upon the ensemble's recommendation and learn to reduce uncertainty. Our algorithm, Bayesian Residual Policy Optimization (BRPO), imports the scalability of policy gradient methods and task-specific expert skills. BRPO significantly improves the ensemble of experts and drastically outperforms existing adaptive RL methods.
Assistive robots for the social management of health: a framework for robot design and human-robot interaction research
Venue: arXiv cs.HC 2020
Year: 2020

There is a close connection between health and the quality of one's social life. Strong social bonds are essential for health and wellbeing, but often health conditions can detrimentally affect a person's ability to interact with others. This can become a vicious cycle resulting in further decline in health. For this reason, the social management of health is an important aspect of healthcare. We propose that socially assistive robots (SARs) could help people with health conditions maintain positive social lives by supporting them in social interactions. This paper makes three contributions, as detailed below. We develop a framework of social mediation functions that robots could perform, motivated by the special social needs that people with health conditions have. In this framework we identify five types of functions that SARs could perform: a) changing how the person is perceived, b) enhancing the social behavior of the person, c) modifying the social behavior of others, d) providing structure for interactions, and e) changing how the person feels. We thematically organize and review the existing literature on robots supporting human-human interactions, in both clinical and non-clinical settings, and explain how the findings and design ideas from these studies can be applied to the functions identified in the framework. Finally, we point out and discuss challenges in designing SARs for supporting social interactions, and highlight opportunities for future robot design and HRI research on the mediator role of robots.
Paving the way for Distributed Non-Blocking Algorithms and Data Structures in the Partitioned Global Address Space
Venue: arXiv cs.DC 2020
Year: 2020

The partitioned global address space has bridged the gap between shared and distributed memory, and with this bridge comes the ability to adapt shared memory concepts, such as non-blocking programming, to distributed systems such as supercomputers. To enable non-blocking algorithms, we present ways to perform scalable atomic operations on objects in remote memory via remote direct memory address and pointer compression. As a solution to the problem of concurrent-safe reclamation of memory in a distributed system, we adapt Epoch-Based Memory Reclamation to distributed memory and implement it in such a way to support global-view programming. This construct is designed and implemented for the Chapel programming language, but can be adapted and generalized to work on other languages and libraries.
Blank Language Models
Venue: arXiv cs.CL 2020
Year: 2020

We propose Blank Language Model (BLM), a model that generates sequences by dynamically creating and filling in blanks. The blanks control which part of the sequence to expand, making BLM ideal for a variety of text editing and rewriting tasks. The model can start from a single blank or partially completed text with blanks at specified locations. It iteratively determines which word to place in a blank and whether to insert new blanks, and stops generating when no blanks are left to fill. BLM can be efficiently trained using a lower bound of the marginal data likelihood. On the task of filling missing text snippets, BLM significantly outperforms all other baselines in terms of both accuracy and fluency. Experiments on style transfer and damaged ancient text restoration demonstrate the potential of this framework for a wide range of applications.
On Probabilistic Byzantine Fault Tolerance
Venue: arXiv cs.DC 2020
Year: 2020

Byzantine fault tolerance (BFT) has been extensively studied in distributed trustless systems to guarantee system's functioning when up to 1/3 Byzantine processes exist. Despite a plethora of previous work in BFT systems, they are mainly concerned about common knowledge deducible from the states of all participant processes. In BFT systems, it is crucial to know about which knowledge a process knows about the states of other processes and the global state of the system. However, there is a lack of studies about common knowledge of Byzantine faults, such as, whether existence of a Byzantine node is known by all honest nodes. In a dynamic setting, processes may fail or get compromised unexpectedly and unpredictably. It is critical to reason about which processes know about the faulty processes of the network. In this paper, we are interested in studying BFT systems in which Byzantine processes may misbehave randomly, possibly at some random periods of time. The problem of \emph{probabilistic Byzantine} (PB) processes studied in this paper is more general than the problems previously studied in existing work. We propose an approach that allows us to formulate and reason about the concurrent knowledge of the PB processes by all processes. We present our study of the proposed approach in both synchronous and asynchronous systems.
BitPruning: Learning Bitlengths for Aggressive and Accurate Quantization
Venue: arXiv cs.LG 2020
Year: 2020

Neural networks have demonstrably achieved state-of-the art accuracy using low-bitlength integer quantization, yielding both execution time and energy benefits on existing hardware designs that support short bitlengths. However, the question of finding the minimum bitlength for a desired accuracy remains open. We introduce a training method for minimizing inference bitlength at any granularity while maintaining accuracy. Namely, we propose a regularizer that penalizes large bitlength representations throughout the architecture and show how it can be modified to minimize other quantifiable criteria, such as number of operations or memory footprint. We demonstrate that our method learns thrifty representations while maintaining accuracy. With ImageNet, the method produces an average per layer bitlength of 4.13, 3.76 and 4.36 bits on AlexNet, ResNet18 and MobileNet V2 respectively, remaining within 2.0%, 0.5% and 0.5% of the base TOP-1 accuracy.
Inferential Induction: A Novel Framework for Bayesian Reinforcement Learning
Venue: arXiv cs.LG 2020
Year: 2020

Bayesian reinforcement learning (BRL) offers a decision-theoretic solution for reinforcement learning. While "model-based" BRL algorithms have focused either on maintaining a posterior distribution on models or value functions and combining this with approximate dynamic programming or tree search, previous Bayesian "model-free" value function distribution approaches implicitly make strong assumptions or approximations. We describe a novel Bayesian framework, Inferential Induction, for correctly inferring value function distributions from data, which leads to the development of a new class of BRL algorithms. We design an algorithm, Bayesian Backwards Induction, with this framework. We experimentally demonstrate that the proposed algorithm is competitive with respect to the state of the art.
Shipper Cooperation in Stochastic Drone Delivery: A Dynamic Bayesian Game Approach
Venue: arXiv cs.GT 2020
Year: 2020

With the recent technological innovation, unmanned aerial vehicles, known as drones, have found numerous applications including package and parcel delivery for shippers. Drone delivery offers benefits over conventional ground-based vehicle delivery in terms of faster speed, lower cost, more environment-friendly, and less manpower needed. However, most of existing studies on drone delivery planning and scheduling focus on a single shipper and ignore uncertainty factors. As such, in this paper, we consider a scenario that multiple shippers can cooperate to minimize their drone delivery cost. We propose the Bayesian Shipper Cooperation in Stochastic Drone Delivery (BCoSDD) framework. The framework is composed of three functions, i.e., package assignment, shipper cooperation formation and cost management. The uncertainties of drone breakdown and misbehavior of cooperative shippers are taken into account by using multistage stochastic programming optimization and dynamic Bayesian coalition formation game. We conduct extensive performance evaluation of the BCoSDD framework by using customer locations from Solomon benchmark suite and a real Singapore logistics industry. As a result, the framework can help the shippers plan and schedule their drone delivery effectively.
Variable-Viewpoint Representations for 3D Object Recognition
Venue: arXiv cs.CV 2020
Year: 2020

For the problem of 3D object recognition, researchers using deep learning methods have developed several very different input representations, including "multi-view" snapshots taken from discrete viewpoints around an object, as well as "spherical" representations consisting of a dense map of essentially ray-traced samples of the object from all directions. These representations offer trade-offs in terms of what object information is captured and to what degree of detail it is captured, but it is not clear how to measure these information trade-offs since the two types of representations are so different. We demonstrate that both types of representations in fact exist at two extremes of a common representational continuum, essentially choosing to prioritize either the number of views of an object or the pixels (i.e., field of view) allotted per view. We identify interesting intermediate representations that lie at points in between these two extremes, and we show, through systematic empirical experiments, how accuracy varies along this continuum as a function of input information as well as the particular deep learning architecture that is used.
HHH: An Online Medical Chatbot System based on Knowledge Graph and Hierarchical Bi-Directional Attention
Venue: arXiv cs.CL 2020
Year: 2020

This paper proposes a chatbot framework that adopts a hybrid model which consists of a knowledge graph and a text similarity model. Based on this chatbot framework, we build HHH, an online question-and-answer (QA) Healthcare Helper system for answering complex medical questions. HHH maintains a knowledge graph constructed from medical data collected from the Internet. HHH also implements a novel text representation and similarity deep learning model, Hierarchical BiLSTM Attention Model (HBAM), to find the most similar question from a large QA dataset. We compare HBAM with other state-of-the-art language models such as bidirectional encoder representation from transformers (BERT) and Manhattan LSTM Model (MaLSTM). We train and test the models with a subset of the Quora duplicate questions dataset in the medical area. The experimental results show that our model is able to achieve a superior performance than these existing methods.
Multi-Label Class Balancing Algorithm for Action Unit Detection
Venue: arXiv cs.CV 2020
Year: 2020

Isolated facial movements, so-called Action Units, can describe combined emotions or physical states such as pain. As datasets are limited and mostly imbalanced, we present an approach incorporating a multi-label class balancing algorithm. This submission is subject to the Action Unit detection task of the Affective Behavior Analysis in-the-wild (ABAW) challenge at the IEEE Conference on Face and Gesture Recognition 2020.
GradMix: Multi-source Transfer across Domains and Tasks
Venue: arXiv cs.CV 2020
Year: 2020

The computer vision community is witnessing an unprecedented rate of new tasks being proposed and addressed, thanks to the deep convolutional networks' capability to find complex mappings from X to Y. The advent of each task often accompanies the release of a large-scale annotated dataset, for supervised training of deep network. However, it is expensive and time-consuming to manually label sufficient amount of training data. Therefore, it is important to develop algorithms that can leverage off-the-shelf labeled dataset to learn useful knowledge for the target task. While previous works mostly focus on transfer learning from a single source, we study multi-source transfer across domains and tasks (MS-DTT), in a semi-supervised setting. We propose GradMix, a model-agnostic method applicable to any model trained with gradient-based learning rule, to transfer knowledge via gradient descent by weighting and mixing the gradients from all sources during training. GradMix follows a meta-learning objective, which assigns layer-wise weights to the source gradients, such that the combined gradient follows the direction that minimize the loss for a small set of samples from the target dataset. In addition, we propose to adaptively adjust the learning rate for each mini-batch based on its importance to the target task, and a pseudo-labeling method to leverage the unlabeled samples in the target domain. We conduct MS-DTT experiments on two tasks: digit recognition and action recognition, and demonstrate the advantageous performance of the proposed method against multiple baselines.
Learning efficient structured dictionary for image classification
Venue: arXiv cs.CV 2020
Year: 2020

Recent years have witnessed the success of dictionary learning (DL) based approaches in the domain of pattern classification. In this paper, we present an efficient structured dictionary learning (ESDL) method which takes both the diversity and label information of training samples into account. Specifically, ESDL introduces alternative training samples into the process of dictionary learning. To increase the discriminative capability of representation coefficients for classification, an ideal regularization term is incorporated into the objective function of ESDL. Moreover, in contrast with conventional DL approaches which impose computationally expensive L1-norm constraint on the coefficient matrix, ESDL employs L2-norm regularization term. Experimental results on benchmark databases (including four face databases and one scene dataset) demonstrate that ESDL outperforms previous DL approaches. More importantly, ESDL can be applied in a wide range of pattern classification tasks.
Learning High Order Feature Interactions with Fine Control Kernels
Venue: arXiv cs.LG 2020
Year: 2020

We provide a methodology for learning sparse statistical models that use as features all possible multiplicative interactions among an underlying atomic set of features. While the resulting optimization problems are exponentially sized, our methodology leads to algorithms that can often solve these problems exactly or provide approximate solutions based on combining highly correlated features. We also introduce an algorithmic paradigm, the Fine Control Kernel framework, so named because it is based on Fenchel Duality and is reminiscent of kernel methods. Its theory is tailored to large sparse learning problems, and it leads to efficient feature screening rules for interactions. These rules are inspired by the Apriori algorithm for market basket analysis -- which also falls under the purview of Fine Control Kernels, and can be applied to a plurality of learning problems including the Lasso and sparse matrix estimation. Experiments on biomedical datasets demonstrate the efficacy of our methodology in deriving algorithms that efficiently produce interactions models which achieve state-of-the-art accuracy and are interpretable.