OpenRaiser/Intern-Atlas huggingface dataset:
Anytime MiniBatch: Exploiting Stragglers in Online Distributed Optimization
Venue: arXiv cs.LG 2020
Year: 2020

Distributed optimization is vital in solving large-scale machine learning problems. A widely-shared feature of distributed optimization techniques is the requirement that all nodes complete their assigned tasks in each computational epoch before the system can proceed to the next epoch. In such settings, slow nodes, called stragglers, can greatly slow progress. To mitigate the impact of stragglers, we propose an online distributed optimization method called Anytime Minibatch. In this approach, all nodes are given a fixed time to compute the gradients of as many data samples as possible. The result is a variable per-node minibatch size. Workers then get a fixed communication time to average their minibatch gradients via several rounds of consensus, which are then used to update primal variables via dual averaging. Anytime Minibatch prevents stragglers from holding up the system without wasting the work that stragglers can complete. We present a convergence analysis and analyze the wall time performance. Our numerical results show that our approach is up to 1.5 times faster in Amazon EC2 and it is up to five times faster when there is greater variability in compute node performance.
Self-Supervised Reinforcement Learning for Recommender Systems
Venue: arXiv cs.LG 2020
Year: 2020

In session-based or sequential recommendation, it is important to consider a number of factors like long-term user engagement, multiple types of user-item interactions such as clicks, purchases etc. The current state-of-the-art supervised approaches fail to model them appropriately. Casting sequential recommendation task as a reinforcement learning (RL) problem is a promising direction. A major component of RL approaches is to train the agent through interactions with the environment. However, it is often problematic to train a recommender in an on-line fashion due to the requirement to expose users to irrelevant recommendations. As a result, learning the policy from logged implicit feedback is of vital importance, which is challenging due to the pure off-policy setting and lack of negative rewards (feedback). In this paper, we propose self-supervised reinforcement learning for sequential recommendation tasks. Our approach augments standard recommendation models with two output layers: one for self-supervised learning and the other for RL. The RL part acts as a regularizer to drive the supervised layer focusing on specific rewards(e.g., recommending items which may lead to purchases rather than clicks) while the self-supervised layer with cross-entropy loss provides strong gradient signals for parameter updates. Based on such an approach, we propose two frameworks namely Self-Supervised Q-learning(SQN) and Self-Supervised Actor-Critic(SAC). We integrate the proposed frameworks with four state-of-the-art recommendation models. Experimental results on two real-world datasets demonstrate the effectiveness of our approach.
Embedding Task Knowledge into 3D Neural Networks via Self-supervised Learning
Venue: arXiv cs.CV 2020
Year: 2020

Deep learning highly relies on the amount of annotated data. However, annotating medical images is extremely laborious and expensive. To this end, self-supervised learning (SSL), as a potential solution for deficient annotated data, attracts increasing attentions from the community. However, SSL approaches often design a proxy task that is not necessarily related to target task. In this paper, we propose a novel SSL approach for 3D medical image classification, namely Task-related Contrastive Prediction Coding (TCPC), which embeds task knowledge into training 3D neural networks. The proposed TCPC first locates the initial candidate lesions via supervoxel estimation using simple linear iterative clustering. Then, we extract features from the sub-volume cropped around potential lesion areas, and construct a calibrated contrastive predictive coding scheme for self-supervised learning. Extensive experiments are conducted on public and private datasets. The experimental results demonstrate the effectiveness of embedding lesion-related prior-knowledge into neural networks for 3D medical image classification.
A Separation-Based Methodology to Consensus Tracking of Switched High-Order Nonlinear Multi-Agent Systems
Venue: arXiv cs.MA 2020
Year: 2020

This work investigates a reduced-complexity adaptive methodology to consensus tracking for a team of uncertain high-order nonlinear systems with switched (possibly asynchronous) dynamics. It is well known that high-order nonlinear systems are intrinsically challenging as feedback linearization and backstepping methods successfully developed for low-order systems fail to work. At the same time, even the adding-one power-integrator methodology, well explored for the single-agent high-order case, presents some complexity issues and is unsuited for distributed control. At the core of the proposed distributed methodology is a newly proposed definition for separable functions: this definition allows the formulation of a separation-based lemma to handle the high-order terms with reduced complexity in the control design. Complexity is reduced in a twofold sense: the control gain of each virtual control law does not have to be incorporated in the next virtual control law iteratively, thus leading to a simpler expression of the control laws; the order of the virtual control gains increases only proportionally (rather than exponentially) with the order of the systems, dramatically reducing high-gain issues.
To Regularize or Not To Regularize? The Bias Variance Trade-off in Regularized AEs
Venue: arXiv cs.LG 2020
Year: 2020

Regularized Auto-Encoders (RAEs) form a rich class of neural generative models. They effectively model the joint-distribution between the data and the latent space using an Encoder-Decoder combination, with regularization imposed in terms of a prior over the latent space. Despite their advantages, such as stability in training, the performance of AE based models has not reached the superior standards as that of the other generative models such as Generative Adversarial Networks (GANs). Motivated by this, we examine the effect of the latent prior on the generation quality of deterministic AE models in this paper. Specifically, we consider the class of RAEs with deterministic Encoder-Decoder pairs, Wasserstein Auto-Encoders (WAE), and show that having a fixed prior distribution, \textit{a priori}, oblivious to the dimensionality of the `true' latent space, will lead to the infeasibility of the optimization problem considered. Further, we show that, in the finite data regime, despite knowing the correct latent dimensionality, there exists a bias-variance trade-off with any arbitrary prior imposition. As a remedy to both the issues mentioned above, we introduce an additional state space in the form of flexibly learnable latent priors, in the optimization objective of the WAEs. We implicitly learn the distribution of the latent prior jointly with the AE training, which not only makes the learning objective feasible but also facilitates operation on different points of the bias-variance curve. We show the efficacy of our model, called FlexAE, through several experiments on multiple datasets, and demonstrate that it is the new state-of-the-art for the AE based generative models.
Speech Fusion to Face: Bridging the Gap Between Human's Vocal Characteristics and Facial Imaging
Venue: arXiv cs.CV 2020
Year: 2020

While deep learning technologies are now capable of generating realistic images confusing humans, the research efforts are turning to the synthesis of images for more concrete and application-specific purposes. Facial image generation based on vocal characteristics from speech is one of such important yet challenging tasks. It is the key enabler to influential use cases of image generation, especially for business in public security and entertainment. Existing solutions to the problem of speech2face renders limited image quality and fails to preserve facial similarity due to the lack of quality dataset for training and appropriate integration of vocal features. In this paper, we investigate these key technical challenges and propose Speech Fusion to Face, or SF2F in short, attempting to address the issue of facial image quality and the poor connection between vocal feature domain and modern image generation models. By adopting new strategies on data model and training, we demonstrate dramatic performance boost over state-of-the-art solution, by doubling the recall of individual identity, and lifting the quality score from 15 to 19 based on the mutual information score with VGGFace classifier.
Learning to Play Table Tennis From Scratch using Muscular Robots
Venue: arXiv cs.RO 2020
Year: 2020

Dynamic tasks like table tennis are relatively easy to learn for humans but pose significant challenges to robots. Such tasks require accurate control of fast movements and precise timing in the presence of imprecise state estimation of the flying ball and the robot. Reinforcement Learning (RL) has shown promise in learning of complex control tasks from data. However, applying step-based RL to dynamic tasks on real systems is safety-critical as RL requires exploring and failing safely for millions of time steps in high-speed regimes. In this paper, we demonstrate that safe learning of table tennis using model-free Reinforcement Learning can be achieved by using robot arms driven by pneumatic artificial muscles (PAMs). Softness and back-drivability properties of PAMs prevent the system from leaving the safe region of its state space. In this manner, RL empowers the robot to return and smash real balls with 5 m\s and 12m\s on average to a desired landing point. Our setup allows the agent to learn this safety-critical task (i) without safety constraints in the algorithm, (ii) while maximizing the speed of returned balls directly in the reward function (iii) using a stochastic policy that acts directly on the low-level controls of the real system and (iv) trains for thousands of trials (v) from scratch without any prior knowledge. Additionally, we present HYSR, a practical hybrid sim and real training that avoids playing real balls during training by randomly replaying recorded ball trajectories in simulation and applying actions to the real robot. This work is the first to (a) fail-safe learn of a safety-critical dynamic task using anthropomorphic robot arms, (b) learn a precision-demanding problem with a PAM-driven system despite the control challenges and (c) train robots to play table tennis without real balls. Videos and datasets are available at muscularTT.embodied.ml.
Learning normalizing flows from Entropy-Kantorovich potentials
Venue: arXiv cs.LG 2020
Year: 2020

We approach the problem of learning continuous normalizing flows from a dual perspective motivated by entropy-regularized optimal transport, in which continuous normalizing flows are cast as gradients of scalar potential functions. This formulation allows us to train a dual objective comprised only of the scalar potential functions, and removes the burden of explicitly computing normalizing flows during training. After training, the normalizing flow is easily recovered from the potential functions.
Efficient Partial Snapshot Implementations
Venue: arXiv cs.DC 2020
Year: 2020

In this work, we propose the $λ$-scanner snapshot, a variation of the snapshot object, which supports any fixed amount of $0 < λ\leq n$ different $SCAN$ operations being active at any given time. Whenever $λ$ is equal to the number of processes $n$ in the system, the $λ$-scanner object implements a multi-scanner object, while in case that $λ$ is equal to $1$, the $λ$-scanner object implements a single-scanner object. We present the $λ-Snap$ snapshot object, a wait-free $λ$-scanner snapshot implementation that has a step complexity of $O(λ)$ for $UPDATE$ operations and $O(λm)$ for $SCAN$ operations. The space complexity of $λ-Snap$ is $O(λm)$. $λ-Snap$ provides a trade-off between the step/space complexity and the maximum number of $SCAN$ operations that the system can afford to be active on any given point in time. The low space complexity that our implementations provide makes them more appealing in real system applications. Moreover, we provide a slightly modified version of the $λ-Snap$ implementation, which is called partial $λ-Snap$, that is able to support dynamic partial scan operations. In such an object, processes can execute modified $SCAN$ operations called $PARTIAL\_SCAN$ that could obtain a part of the snapshot object avoiding to read the whole set of components. In this work, we first provide a simple single-scanner version of $λ-Snap$, which is called $1-Snap$. We provide $1-Snap$ just for presentation purposes, since it is simpler than $λ-Snap$. The $UPDATE$ in $1-Snap$ has a step complexity of $O(1)$, while the $SCAN$ has a step complexity of $O(m)$. This implementation uses $O(m)$ $CAS$ registers.
Accelerating linear solvers for Stokes problems with C++ metaprogramming
Venue: arXiv cs.MS 2020
Year: 2020

The efficient solution of large sparse saddle point systems is very important in computational fluid mechanics. The discontinuous Galerkin finite element methods have become increasingly popular for incompressible flow problems but their application is limited due to high computational cost. We describe the C++ programming techniques that may help to accelerate linear solvers for such problems. The approach is based on the policy-based design pattern and partial template specialization, and is implemented in the open source AMGCL library. The efficiency is demonstrated with the example of accelerating an iterative solver of a discontinuous Galerkin finite element method for the Stokes problem. The implementation allows selecting algorithmic components of the solver by adjusting template parameters without any changes to the codebase. It is possible to switch the system matrix to use small statically sized blocks to store the nonzero values, or use a mixed precision solution, which results in up to 4 times speedup, and reduces the memory footprint of the algorithm by about 40\%. We evaluate both monolithic and composite preconditioning strategies for the 3 benchmark problems. The performance of the proposed solution is compared with a multithreaded direct Pardiso solver and a parallel iterative PETSc solver.
PeopleMap: Visualization Tool for Mapping Out Researchers using Natural Language Processing
Venue: arXiv cs.DL 2020
Year: 2020

Discovering research expertise at institutions can be a difficult task. Manually curated university directories easily become out of date and they often lack the information necessary for understanding a researcher's interests and past work, making it harder to explore the diversity of research at an institution and identify research talents. This results in lost opportunities for both internal and external entities to discover new connections and nurture research collaboration. To solve this problem, we have developed PeopleMap, the first interactive, open-source, web-based tool that visually "maps out" researchers based on their research interests and publications by leveraging embeddings generated by natural language processing (NLP) techniques. PeopleMap provides a new engaging way for institutions to summarize their research talents and for people to discover new connections. The platform is developed with ease-of-use and sustainability in mind. Using only researchers' Google Scholar profiles as input, PeopleMap can be readily adopted by any institution using its publicly-accessible repository and detailed documentation.
Image Deconvolution via Noise-Tolerant Self-Supervised Inversion
Venue: arXiv cs.CV 2020
Year: 2020

We propose a general framework for solving inverse problems in the presence of noise that requires no signal prior, no noise estimate, and no clean training data. We only require that the forward model be available and that the noise be statistically independent across measurement dimensions. We build upon the theory of $\mathcal{J}$-invariant functions (Batson & Royer 2019, arXiv:1901.11365) and show how self-supervised denoising \emph{à la} Noise2Self is a special case of learning a noise-tolerant pseudo-inverse of the identity. We demonstrate our approach by showing how a convolutional neural network can be taught in a self-supervised manner to deconvolve images and surpass in image quality classical inversion schemes such as Lucy-Richardson deconvolution.
Automatic Photo to Ideophone Manga Matching
Venue: arXiv cs.HC 2020
Year: 2020

Photo applications offer tools for annotation via text and stickers. Ideophones, mimetic and onomatopoeic words, which are common in graphic novels, have yet to be explored for photo annotation use. We present a method for automatic ideophone recommendation and positioning of the text on photos. These annotations are accomplished by obtaining a list of ideophones with English definitions and applying a suite of visual object detectors to the image. Next, a semantic embedding maps the visual objects to the possible relevant ideophones. Our system stands in contrast to traditional computer vision-based annotation systems, which stop at recommending object and scene-level annotation, by providing annotations that are communicative, fun, and engaging. We test these annotations in Japanese and find they carry a strong preference and increase enjoyment and sharing likelihood when compared to unannotated and object-based annotated photos.
Recovery and Generalization in Over-Realized Dictionary Learning
Venue: arXiv cs.LG 2020
Year: 2020

In over two decades of research, the field of dictionary learning has gathered a large collection of successful applications, and theoretical guarantees for model recovery are known only whenever optimization is carried out in the same model class as that of the underlying dictionary. This work characterizes the surprising phenomenon that dictionary recovery can be facilitated by searching over the space of larger over-realized models. This observation is general and independent of the specific dictionary learning algorithm used. We thoroughly demonstrate this observation in practice and provide an analysis of this phenomenon by tying recovery measures to generalization bounds. In particular, we show that model recovery can be upper-bounded by the empirical risk, a model-dependent quantity and the generalization gap, reflecting our empirical findings. We further show that an efficient and provably correct distillation approach can be employed to recover the correct atoms from the over-realized model. As a result, our meta-algorithm provides dictionary estimates with consistently better recovery of the ground-truth model.
JIT-Masker: Efficient Online Distillation for Background Matting
Venue: arXiv cs.CV 2020
Year: 2020

We design a real-time portrait matting pipeline for everyday use, particularly for "virtual backgrounds" in video conferences. Existing segmentation and matting methods prioritize accuracy and quality over throughput and efficiency, and our pipeline enables trading off a controllable amount of accuracy for better throughput by leveraging online distillation on the input video stream. We construct our own dataset of simulated video calls in various scenarios, and show that our approach delivers a 5x speedup over a saliency detection based pipeline in a non-GPU accelerated setting while delivering higher quality results. We demonstrate that an online distillation approach can feasibly work as part of a general, consumer level product as a "virtual background" tool. Our public implementation is at https://github.com/josephch405/jit-masker.
Fall Detector Adapted to Nursing Home Needs through an Optical-Flow based CNN
Venue: arXiv cs.CV 2020
Year: 2020

Fall detection in specialized homes for the elderly is challenging. Vision-based fall detection solutions have a significant advantage over sensor-based ones as they do not instrument the resident who can suffer from mental diseases. This work is part of a project intended to deploy fall detection solutions in nursing homes. The proposed solution, based on Deep Learning, is built on a Convolutional Neural Network (CNN) trained to maximize a sensitivity-based metric. This work presents the requirements from the medical side and how it impacts the tuning of a CNN. Results highlight the importance of the temporal aspect of a fall. Therefore, a custom metric adapted to this use case and an implementation of a decision-making process are proposed in order to best meet the medical teams requirements. Clinical relevance This work presents a fall detection solution enabled to detect 86.2% of falls while producing only 11.6% of false alarms in average on the considered databases.
SECure: A Social and Environmental Certificate for AI Systems
Venue: arXiv cs.CY 2020
Year: 2020

In a world increasingly dominated by AI applications, an understudied aspect is the carbon and social footprint of these power-hungry algorithms that require copious computation and a trove of data for training and prediction. While profitable in the short-term, these practices are unsustainable and socially extractive from both a data-use and energy-use perspective. This work proposes an ESG-inspired framework combining socio-technical measures to build eco-socially responsible AI systems. The framework has four pillars: compute-efficient machine learning, federated learning, data sovereignty, and a LEEDesque certificate. Compute-efficient machine learning is the use of compressed network architectures that show marginal decreases in accuracy. Federated learning augments the first pillar's impact through the use of techniques that distribute computational loads across idle capacity on devices. This is paired with the third pillar of data sovereignty to ensure the privacy of user data via techniques like use-based privacy and differential privacy. The final pillar ties all these factors together and certifies products and services in a standardized manner on their environmental and social impacts, allowing consumers to align their purchase with their values.
Avoiding abelian powers cyclically
Venue: arXiv cs.FL 2020
Year: 2020

We study a new notion of cyclic avoidance of abelian powers. A finite word $w$ avoids abelian $N$-powers cyclically if for each abelian $N$-power of period $m$ occurring in the infinite word $w^ω$, we have $m \geq |w|$. Let $\mathcal{A}(k)$ be the least integer $N$ such that for all $n$ there exists a word of length $n$ over a $k$-letter alphabet that avoids abelian $N$-powers cyclically. Let $\mathcal{A}_\infty(k)$ be the least integer $N$ such that there exist arbitrarily long words over a $k$-letter alphabet that avoid abelian $N$-powers cyclically. We prove that $5 \leq \mathcal{A}(2) \leq 8$, $3 \leq \mathcal{A}(3) \leq 4$, $2 \leq \mathcal{A}(4) \leq 3$, and $\mathcal{A}(k) = 2$ for $k \geq 5$. Moreover, we show that $\mathcal{A}_\infty(2) = 4$, $\mathcal{A}_\infty(3) = 3$, and $\mathcal{A}_\infty(4) = 2$.