OpenRaiser/Intern-Atlas huggingface dataset:
Detector-in-Detector: Multi-Level Analysis for Human-Parts
Venue: arXiv cs.CV 2019
Year: 2019

Vision-based person, hand or face detection approaches have achieved incredible success in recent years with the development of deep convolutional neural network (CNN). In this paper, we take the inherent correlation between the body and body parts into account and propose a new framework to boost up the detection performance of the multi-level objects. In particular, we adopt a region-based object detection structure with two carefully designed detectors to separately pay attention to the human body and body parts in a coarse-to-fine manner, which we call Detector-in-Detector network (DID-Net). The first detector is designed to detect human body, hand, and face. The second detector, based on the body detection results of the first detector, mainly focus on the detection of small hand and face inside each body. The framework is trained in an end-to-end way by optimizing a multi-task loss. Due to the lack of human body, face and hand detection dataset, we have collected and labeled a new large dataset named Human-Parts with 14,962 images and 106,879 annotations. Experiments show that our method can achieve excellent performance on Human-Parts.
Design and Control of a Quasi-Direct Drive Soft Exoskeleton for Knee Injury Prevention during Squatting
Venue: arXiv cs.RO 2019
Year: 2019

This paper presents design and control innovations of wearable robots that tackle two barriers to widespread adoption of powered exoskeletons, namely restriction of human movement and versatile control of wearable co-robot systems. First, the proposed quasi-direct drive actuation comprising of our customized high torque density motors and low ratio transmission mechanism significantly reduces the mass of the robot and produces high backdrivability. Second, we derive a biomechanics model-based control that generates biological torque profile for versatile control of both squat and stoop lifting assistance. The control algorithm detects lifting postures using compact inertial measurement unit (IMU) sensors to generate an assistive profile that is proportional to the biological torque produced from our model. Experimental results demonstrate that the robot exhibits low mechanical impedance (1.5 Nm resistive torque) when it is unpowered and 0.5 Nm resistive torque with zero-torque tracking control. Root mean square (RMS) error of torque tracking is less than 0.29 Nm (1.21% error of 24 Nm peak torque). Compared with squatting without the exoskeleton, the controller reduces 87.5%, 80% and 75% of the of three knee extensor muscles (average peak EMG of 3 healthy subjects) during squat with 50% of biological torque assistance.
Global Convergence of Adaptive Gradient Methods for An Over-parameterized Neural Network
Venue: arXiv cs.LG 2019
Year: 2019

Adaptive gradient methods like AdaGrad are widely used in optimizing neural networks. Yet, existing convergence guarantees for adaptive gradient methods require either convexity or smoothness, and, in the smooth setting, only guarantee convergence to a stationary point. We propose an adaptive gradient method and show that for two-layer over-parameterized neural networks -- if the width is sufficiently large (polynomially) -- then the proposed method converges \emph{to the global minimum} in polynomial time, and convergence is robust, \emph{ without the need to fine-tune hyper-parameters such as the step-size schedule and with the level of over-parametrization independent of the training error}. Our analysis indicates in particular that over-parametrization is crucial for the harnessing the full potential of adaptive gradient methods in the setting of neural networks.
A Soft High Force Hand Exoskeleton for Rehabilitation and Assistance of Spinal Cord Injury and Stroke Individuals
Venue: arXiv cs.RO 2019
Year: 2019

Individuals with spinal cord injury (SCI) and stroke who is lack of manipulation capability have a particular need for robotic hand exoskeletons. Among assistive and rehabilitative medical exoskeletons, there exists a sharp trade-off between device power on the one hand and ergonomics and portability on other, devices that provide stronger grasping assistance do so at the cost of patient comfort. This paper proposes using fin-ray inspired, cable-driven finger orthoses to generate high fingertip forces without the painful compressive and shear stresses commonly associated with conventional cable-drive exoskeletons. With combination cable-driven transmission and segmented-finger orthoses, the exoskeleton transmitted larger forces and applied torques discretely to the fingers, leading to strong fingertip forces. A prototype of the finger orthoses and associated cable transmission was fabricated, and force transmission tests of the prototype in the finger flexion mode demonstrated a 2:1 input-output ratio between cable tension and fingertip force, with a maximum fingertip force of 22 N. Moreover, the proposed design provides a comfortable experience for wearers thanks to its lightweight and conformal properties to the hands.
Sequential Synthesis of Distributed Controllers for Cascade Interconnected Systems
Venue: arXiv cs.SY 2019
Year: 2019

We consider the problem of designing distributed controllers to ensure passivity of a large-scale interconnection of linear subsystems connected in a cascade topology. The control design process needs to be carried out at the subsystem-level with no direct knowledge of the dynamics of other subsystems in the interconnection. We present a distributed approach to solve this problem, where subsystem-level controllers are locally designed in a sequence starting at one end of the cascade using only the dynamics of the particular subsystem, coupling with the immediately preceding subsystem and limited information from the preceding subsystem in the cascade to ensure passivity of the interconnected system up to that point. We demonstrate that this design framework also allows for new subsystems to be compositionally added to the interconnection without requiring redesign of the pre-existing controllers.
Recovery of a mixture of Gaussians by sum-of-norms clustering
Venue: arXiv cs.LG 2019
Year: 2019

Sum-of-norms clustering is a method for assigning $n$ points in $\mathbb{R}^d$ to $K$ clusters, $1\le K\le n$, using convex optimization. Recently, Panahi et al.\ proved that sum-of-norms clustering is guaranteed to recover a mixture of Gaussians under the restriction that the number of samples is not too large. The purpose of this note is to lift this restriction, i.e., show that sum-of-norms clustering with equal weights can recover a mixture of Gaussians even as the number of samples tends to infinity. Our proof relies on an interesting characterization of clusters computed by sum-of-norms clustering that was developed inside a proof of the agglomeration conjecture by Chiquet et al. Because we believe this theorem has independent interest, we restate and reprove the Chiquet et al.\ result herein.
An ultra-low-power sigma-delta neuron circuit
Venue: arXiv cs.ET 2019
Year: 2019

Neural processing systems typically represent data using leaky integrate and fire (LIF) neuron models that generate spikes or pulse trains at a rate proportional to their input amplitudes. This mechanism requires high firing rates when encoding time-varying signals, leading to increased power consumption. Neuromorphic systems that use adaptive LIF neuron models overcome this problem by encoding signals in the relative timing of their output spikes rather than their rate. In this paper, we analyze recent adaptive LIF neuron circuit implementations and highlight the analogies and differences between them and a first-order sigma-delta feedback loop. We propose a new sigma-delta neuron circuit that addresses some of the limitations in existing implementations and present simulation results that quantify the improvements. We show that the new circuit, implemented in a 1.8 V, 180 nm CMOS process, offers up to 42 dB signal-to-distortion ratio and consumes orders of magnitude lower energy. Finally, we also demonstrate how the sigma-delta interpretation enables mapping of real-valued recurrent neural network to the spiking framework to emphasize the envisioned application of the proposed circuit.
Comparing Apples and Oranges: Measuring Differences between Exploratory Data Mining Results
Venue: arXiv cs.DB 2019
Year: 2019

Deciding whether the results of two different mining algorithms provide significantly different information is an important, yet understudied, open problem in exploratory data mining. Whether the goal is to select the most informative result for analysis, or to decide which mining approach will most likely provide the most novel insight, it is essential that we can tell how different the information is that different results by possibly different methods provide. In this paper we take a first step towards comparing exploratory data mining results on binary data. We propose to meaningfully convert results into sets of noisy tiles, and compare between these sets by Maximum Entropy modelling and Kullback-Leibler divergence, well-founded notions from Information Theory. We so construct a measure that is highly flexible, and allows us to naturally include background knowledge, such that differences in results can be measured from the perspective of what a user already knows. Furthermore, adding to its interpretability, it coincides with Jaccard dissimilarity when we only consider exact tiles. Our approach provides a means to study and tell differences between results of different exploratory data mining methods. As an application, we show that our measure can also be used to identify which parts of results best redescribe other results. Furthermore, we study its use for iterative data mining, where one iteratively wants to find that result that will provide maximal novel information. Experimental evaluation shows our measure gives meaningful results, correctly identifies methods that are similar in nature, automatically provides sound redescriptions of results, and is highly applicable for iterative data mining.
An application of communication complexity, Kolmogorov complexity and extremal combinatorics to parity games
Venue: arXiv cs.FL 2019
Year: 2019

So-called separation automata are in the core of several recently invented quasi-polynomial time algorithms for parity games. An explicit $q$-state separation automaton implies an algorithm for parity games with running time polynomial in $q$. It is open whether a polynomial-state separation automaton exists. A positive answer will lead to a polynomial-time algorithm for parity games, while a negative answer will at least demonstrate impossibility to construct such an algorithm using separation approach. In this work we prove exponential lower bound for a restricted class of separation automata. Our technique combines communication complexity and Kolmogorov complexity. One of our technical contributions belongs to extremal combinatorics. Namely, we prove a new upper bound on the product of sizes of two families of sets with small pairwise intersection.
Identifying nonlinear dynamical systems via generative recurrent neural networks with applications to fMRI
Venue: arXiv cs.LG 2019
Year: 2019

A major tenet in theoretical neuroscience is that cognitive and behavioral processes are ultimately implemented in terms of the neural system dynamics. Accordingly, a major aim for the analysis of neurophysiological measurements should lie in the identification of the computational dynamics underlying task processing. Here we advance a state space model (SSM) based on generative piecewise-linear recurrent neural networks (PLRNN) to assess dynamics from neuroimaging data. In contrast to many other nonlinear time series models which have been proposed for reconstructing latent dynamics, our model is easily interpretable in neural terms, amenable to systematic dynamical systems analysis of the resulting set of equations, and can straightforwardly be transformed into an equivalent continuous-time dynamical system. The major contributions of this paper are the introduction of a new observation model suitable for functional magnetic resonance imaging (fMRI) coupled to the latent PLRNN, an efficient stepwise training procedure that forces the latent model to capture the 'true' underlying dynamics rather than just fitting (or predicting) the observations, and of an empirical measure based on the Kullback-Leibler divergence to evaluate from empirical time series how well this goal of approximating the underlying dynamics has been achieved. We validate and illustrate the power of our approach on simulated 'ground-truth' dynamical (benchmark) systems as well as on actual experimental fMRI time series, and demonstrate that the latent dynamics harbors task-related nonlinear structure that a linear dynamical model fails to capture. Given that fMRI is one of the most common techniques for measuring brain activity non-invasively in human subjects, this approach may provide a novel step toward analyzing aberrant (nonlinear) dynamics for clinical assessment or neuroscientific research.
BusyHands: A Hand-Tool Interaction Database for Assembly Tasks Semantic Segmentation
Venue: arXiv cs.CV 2019
Year: 2019

Visual segmentation has seen tremendous advancement recently with ready solutions for a wide variety of scene types, including human hands and other body parts. However, focus on segmentation of human hands while performing complex tasks, such as manual assembly, is still severely lacking. Segmenting hands from tools, work pieces, background and other body parts is extremely difficult because of self-occlusions and intricate hand grips and poses. In this paper we introduce BusyHands, a large open dataset of pixel-level annotated images of hands performing 13 different tool-based assembly tasks, from both real-world captures and virtual-world renderings. A total of 7906 samples are included in our first-in-kind dataset, with both RGB and depth images as obtained from a Kinect V2 camera and Blender. We evaluate several state-of-the-art semantic segmentation methods on our dataset as a proposed performance benchmark.
Dataplant: Enhancing System Security with Low-Cost In-DRAM Value Generation Primitives
Venue: arXiv cs.CR 2019
Year: 2019

DRAM manufacturers have been prioritizing memory capacity, yield, and bandwidth for years, while trying to keep the design complexity as simple as possible. DRAM chips do not carry out any computation or other important functions, such as security. Processors implement most of the existing security mechanisms that protect the system against security threats, because 1) executing security mechanisms usually require non-trivial computational capabilities (e.g., encryption), and 2) commodity DRAM chips are not designed to perform computations or tasks other than data storage. In this work, we advocate for DRAM as a key component for providing security mechanisms to the system. To this end, we propose Dataplant, a new class of low-cost, high-performance, and reliable security primitives that can be integrated in commodity DRAM chips with minimal changes. The main idea of Dataplant is to slightly modify the internal DRAM timing signals to expose the inherent process variation found in all DRAM chips for generating unpredictable but reproducible values (e.g., keys) within DRAM. We use Dataplant to build two new security mechanisms. First, a new Dataplant-based physical unclonable function (PUF) with non-destructive read-out, low evaluation latency, robust responses, resiliency to temperature changes, and data-independent responses. Second, a new cold boot attack prevention mechanism that automatically destroys all data within DRAM on every power cycle with zero run-time energy and latency overheads. Using a combination of detailed simulations and experiments with 136 real commodity DRAM chips, we show that our Dataplant-based PUF has 1.8x higher throughput than the best state-of-the-art DRAM PUFs. We also demonstrate that our Dataplant-based cold boot attack protection mechanism is 19.5x faster and consumes 2.54x less energy when compared to existing mechanisms.
Touchdown on the Cloud: The impact of the Super Bowl on Cloud
Venue: arXiv cs.NI 2019
Year: 2019

The Super Bowl is the world's biggest televised sporting event. We examine the impact of the increasing online activities during the Super Bowl of year 2015-2017 on various types of Cloud systems, including the Cloud infrastructure service, the Cloud content delivery networks for live streaming, and popular web services such as on-demand video streaming service and social network applications. We probe these systems from agents deployed around the world to compare their load variations and performance changes during and after the game. Through our studies of three consecutive years, we find that Super Bowl events have impacts on the Cloud. However, the current Cloud system is still able to ensure capacity to cope with the challenges brought by traffic changes during such massive events.
DNNVM : End-to-End Compiler Leveraging Heterogeneous Optimizations on FPGA-based CNN Accelerators
Venue: arXiv cs.DC 2019
Year: 2019

The convolutional neural network (CNN) has become a state-of-the-art method for several artificial intelligence domains in recent years. The increasingly complex CNN models are both computation-bound and I/O-bound. FPGA-based accelerators driven by custom instruction set architecture (ISA) achieve a balance between generality and efficiency, but there is much on them left to be optimized. We propose the full-stack compiler DNNVM, which is an integration of optimizers for graphs, loops and data layouts, and an assembler, a runtime supporter and a validation environment. The DNNVM works in the context of deep learning frameworks and transforms CNN models into the directed acyclic graph: XGraph. Based on XGraph, we transform the optimization challenges for both the data layout and pipeline into graph-level problems. DNNVM enumerates all potentially profitable fusion opportunities by a heuristic subgraph isomorphism algorithm to leverage pipeline and data layout optimizations, and searches for the best choice of execution strategies of the whole computing graph. On the Xilinx ZU2 @330 MHz and ZU9 @330 MHz, we achieve equivalently state-of-the-art performance on our benchmarks by naïve implementations without optimizations, and the throughput is further improved up to 1.26x by leveraging heterogeneous optimizations in DNNVM. Finally, with ZU9 @330 MHz, we achieve state-of-the-art performance for VGG and ResNet50. We achieve a throughput of 2.82 TOPs/s and an energy efficiency of 123.7 GOPs/s/W for VGG. Additionally, we achieve 1.38 TOPs/s for ResNet50 and 1.41 TOPs/s for GoogleNet.
Patch-based Output Space Adversarial Learning for Joint Optic Disc and Cup Segmentation
Venue: arXiv cs.CV 2019
Year: 2019

Glaucoma is a leading cause of irreversible blindness. Accurate segmentation of the optic disc (OD) and cup (OC) from fundus images is beneficial to glaucoma screening and diagnosis. Recently, convolutional neural networks demonstrate promising progress in joint OD and OC segmentation. However, affected by the domain shift among different datasets, deep networks are severely hindered in generalizing across different scanners and institutions. In this paper, we present a novel patchbased Output Space Adversarial Learning framework (pOSAL) to jointly and robustly segment the OD and OC from different fundus image datasets. We first devise a lightweight and efficient segmentation network as a backbone. Considering the specific morphology of OD and OC, a novel morphology-aware segmentation loss is proposed to guide the network to generate accurate and smooth segmentation. Our pOSAL framework then exploits unsupervised domain adaptation to address the domain shift challenge by encouraging the segmentation in the target domain to be similar to the source ones. Since the whole-segmentationbased adversarial loss is insufficient to drive the network to capture segmentation details, we further design the pOSAL in a patch-based fashion to enable fine-grained discrimination on local segmentation details. We extensively evaluate our pOSAL framework and demonstrate its effectiveness in improving the segmentation performance on three public retinal fundus image datasets, i.e., Drishti-GS, RIM-ONE-r3, and REFUGE. Furthermore, our pOSAL framework achieved the first place in the OD and OC segmentation tasks in MICCAI 2018 Retinal Fundus Glaucoma Challenge.
From Sharma-Mittal to von-Neumann Entropy of a Graph
Venue: arXiv cs.DM 2019
Year: 2019

In this article, we introduce the Sharma-Mittal entropy of a graph, which is a generalization of the existing idea of the von-Neumann entropy. The well-known R{é}nyi, Thallis, and von-Neumann entropies can be expressed as limiting cases of Sharma-Mittal entropy. We have explicitly calculated them for cycle, path, and complete graphs. Also, we have proposed a number of bounds for these entropies. In addition, we have also discussed the entropy of product graphs, such as Cartesian, Kronecker, Lexicographic, Strong, and Corona products. The change in entropy can also be utilized in the analysis of growing network models (Corona graphs), useful in generating complex networks.
Opportunities and Enabling Technologies for 5G Spectrum Sharing
Venue: arXiv cs.NI 2019
Year: 2019

In this paper an overview is given of the current status of 5G industry standards, spectrum allocation and use cases, followed by initial investigations of new opportunities for spectrum sharing in 5G and the underlying technologies to enable efficient sharing, considering both licensed and unlicensed scenarios and spectrum both below 6 GHz and in the millimeter-wave frequency range
TomoGAN: Low-Dose Synchrotron X-Ray Tomography with Generative Adversarial Networks
Venue: arXiv cs.CV 2019
Year: 2019

Synchrotron-based x-ray tomography is a noninvasive imaging technique that allows for reconstructing the internal structure of materials at high spatial resolutions from tens of micrometers to a few nanometers. In order to resolve sample features at smaller length scales, however, a higher radiation dose is required. Therefore, the limitation on the achievable resolution is set primarily by noise at these length scales. We present \TOMOGAN{}, a denoising technique based on generative adversarial networks, for improving the quality of reconstructed images for low-dose imaging conditions. We evaluate our approach in two photon-budget-limited experimental conditions: (1) sufficient number of low-dose projections (based on Nyquist sampling), and (2) insufficient or limited number of high-dose projections. In both cases the angular sampling is assumed to be isotropic, and the photon budget throughout the experiment is fixed based on the maximum allowable radiation dose on the sample. Evaluation with both simulated and experimental datasets shows that our approach can significantly reduce noise in reconstructed images, improving the structural similarity score of simulation and experimental data from 0.18 to 0.9 and from 0.18 to 0.41, respectively. Furthermore, the quality of the reconstructed images with filtered back projection followed by our denoising approach exceeds that of reconstructions with the simultaneous iterative reconstruction technique, showing the computational superiority of our approach.