OpenRaiser/Intern-Atlas huggingface dataset:
Dataflow Aware Mapping of Convolutional Neural Networks Onto Many-Core Platforms With Network-on-Chip Interconnect
Venue: arXiv cs.DC 2020
Year: 2020

Machine intelligence, especially using convolutional neural networks (CNNs), has become a large area of research over the past years. Increasingly sophisticated hardware accelerators are proposed that exploit e.g. the sparsity in computations and make use of reduced precision arithmetic to scale down the energy consumption. However, future platforms require more than just energy efficiency: Scalability is becoming an increasingly important factor. The required effort for physical implementation grows with the size of the accelerator making it more difficult to meet target constraints. Using many-core platforms consisting of several homogeneous cores can alleviate the aforementioned limitations with regard to physical implementation at the expense of an increased dataflow mapping effort. While the dataflow in CNNs is deterministic and can therefore be optimized offline, the problem of finding a suitable scheme that minimizes both runtime and off-chip memory accesses is a challenging task which becomes even more complex if an interconnect system is involved. This work presents an automated mapping strategy starting at the single-core level with different optimization targets for minimal runtime and minimal off-chip memory accesses. The strategy is then extended towards a suitable many-core mapping scheme and evaluated using a scalable system-level simulation with a network-on-chip interconnect. Design space exploration is performed by mapping the well-known CNNs AlexNet and VGG-16 to platforms of different core counts and computational power per core in order to investigate the trade-offs. Our mapping strategy and system setup is scaled starting from the single core level up to 128 cores, thereby showing the limits of the selected approach.
Human-Expert-Level Brain Tumor Detection Using Deep Learning with Data Distillation and Augmentation
Venue: arXiv cs.CV 2020
Year: 2020

The application of Deep Learning (DL) for medical diagnosis is often hampered by two problems. First, the amount of training data may be scarce, as it is limited by the number of patients who have acquired the condition to be diagnosed. Second, the training data may be corrupted by various types of noise. Here, we study the problem of brain tumor detection from magnetic resonance spectroscopy (MRS) data, where both types of problems are prominent. To overcome these challenges, we propose a new method for training a deep neural network that distills particularly representative training examples and augments the training data by mixing these samples from one class with those from the same and other classes to create additional training samples. We demonstrate that this technique substantially improves performance, allowing our method to reach human-expert-level accuracy with just a few thousand training examples. Interestingly, the network learns to rely on features of the data that are usually ignored by human experts, suggesting new directions for future research.
Musical Smart City: Perspectives on Ubiquitous Sonification
Venue: arXiv cs.SD 2020
Year: 2020

Smart cities are urban areas with sensor networks that collect data used towards efficient management. As a source of ubiquitous data, smart city initiatives present opportunities to enhance inhabitants' urban awareness. However, making sense of smart city data is challenging and there is a gap between available data and end-user applications. Sonification emerges as a promising method for the interpretation of smart city data and the production of novel musical experiences. In this paper, we first present the smart city paradigm. We then cover the topics of ubiquitous and mobile music, followed by an overview of sonification research. Finally, we propose an approach entitled ubiquitous sonification and present the initial design of a speculative use case for musical smart city systems, leveraging user and urban data to inform behaviour.
Binary linear codes with few weights from two-to-one functions
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we apply two-to-one functions over $\mathbb{F}_{2^n}$ in two generic constructions of binary linear codes. We consider two-to-one functions in two forms: (1) generalized quadratic functions; and (2) $\left(x^{2^t}+x\right)^e$ with $\gcd(t, n)=1$ and $\gcd\left(e, 2^n-1\right)=1$. Based on the study of the Walsh transforms of those functions or their related-ones, we present many classes of linear codes with few nonzero weights, including one weight, three weights, four weights and five weights. The weight distributions of the proposed codes with one weight and with three weights are determined. In addition, we discuss the minimum distance of the dual of the constructed codes and show that some of them achieve the sphere packing bound. { Moreover, several examples show that some of our codes are optimal and some have the best known parameters.}
On the alpha-loss Landscape in the Logistic Model
Venue: arXiv cs.LG 2020
Year: 2020

We analyze the optimization landscape of a recently introduced tunable class of loss functions called $α$-loss, $α\in (0,\infty]$, in the logistic model. This family encapsulates the exponential loss ($α= 1/2$), the log-loss ($α= 1$), and the 0-1 loss ($α= \infty$) and contains compelling properties that enable the practitioner to discern among a host of operating conditions relevant to emerging learning methods. Specifically, we study the evolution of the optimization landscape of $α$-loss with respect to $α$ using tools drawn from the study of strictly-locally-quasi-convex functions in addition to geometric techniques. We interpret these results in terms of optimization complexity via normalized gradient descent.
Fully-parallel Convolutional Neural Network Hardware
Venue: arXiv cs.NE 2020
Year: 2020

A new trans-disciplinary knowledge area, Edge Artificial Intelligence or Edge Intelligence, is beginning to receive a tremendous amount of interest from the machine learning community due to the ever increasing popularization of the Internet of Things (IoT). Unfortunately, the incorporation of AI characteristics to edge computing devices presents the drawbacks of being power and area hungry for typical machine learning techniques such as Convolutional Neural Networks (CNN). In this work, we propose a new power-and-area-efficient architecture for implementing Articial Neural Networks (ANNs) in hardware, based on the exploitation of correlation phenomenon in Stochastic Computing (SC) systems. The architecture purposed can solve the difficult implementation challenges that SC presents for CNN applications, such as the high resources used in binary-tostochastic conversion, the inaccuracy produced by undesired correlation between signals, and the stochastic maximum function implementation. Compared with traditional binary logic implementations, experimental results showed an improvement of 19.6x and 6.3x in terms of speed performance and energy efficiency, for the FPGA implementation. We have also realized a full VLSI implementation of the proposed SC-CNN architecture demonstrating that our optimization achieve a 18x area reduction over previous SC-DNN architecture VLSI implementation in a comparable technological node. For the first time, a fully-parallel CNN as LENET-5 is embedded and tested in a single FPGA, showing the benefits of using stochastic computing for embedded applications, in contrast to traditional binary logic implementations.
Ecological Reinforcement Learning
Venue: arXiv cs.LG 2020
Year: 2020

Much of the current work on reinforcement learning studies episodic settings, where the agent is reset between trials to an initial state distribution, often with well-shaped reward functions. Non-episodic settings, where the agent must learn through continuous interaction with the world without resets, and where the agent receives only delayed and sparse reward signals, is substantially more difficult, but arguably more realistic considering real-world environments do not present the learner with a convenient "reset mechanism" and easy reward shaping. In this paper, instead of studying algorithmic improvements that can address such non-episodic and sparse reward settings, we instead study the kinds of environment properties that can make learning under such conditions easier. Understanding how properties of the environment impact the performance of reinforcement learning agents can help us to structure our tasks in ways that make learning tractable. We first discuss what we term "environment shaping" -- modifications to the environment that provide an alternative to reward shaping, and may be easier to implement. We then discuss an even simpler property that we refer to as "dynamism," which describes the degree to which the environment changes independent of the agent's actions and can be measured by environment transition entropy. Surprisingly, we find that even this property can substantially alleviate the challenges associated with non-episodic RL in sparse reward settings. We provide an empirical evaluation on a set of new tasks focused on non-episodic learning with sparse rewards. Through this study, we hope to shift the focus of the community towards analyzing how properties of the environment can affect learning and the ultimate type of behavior that is learned via RL.
Locally Masked Convolution for Autoregressive Models
Venue: arXiv cs.LG 2020
Year: 2020

High-dimensional generative models have many applications including image compression, multimedia generation, anomaly detection and data completion. State-of-the-art estimators for natural images are autoregressive, decomposing the joint distribution over pixels into a product of conditionals parameterized by a deep neural network, e.g. a convolutional neural network such as the PixelCNN. However, PixelCNNs only model a single decomposition of the joint, and only a single generation order is efficient. For tasks such as image completion, these models are unable to use much of the observed context. To generate data in arbitrary orders, we introduce LMConv: a simple modification to the standard 2D convolution that allows arbitrary masks to be applied to the weights at each location in the image. Using LMConv, we learn an ensemble of distribution estimators that share parameters but differ in generation order, achieving improved performance on whole-image density estimation (2.89 bpd on unconditional CIFAR10), as well as globally coherent image completions. Our code is available at https://ajayjain.github.io/lmconv.
Optimizing Downlink Resource Allocation in Multiuser MIMO Networks via Fractional Programming and the Hungarian Algorithm
Venue: arXiv cs.IT 2020
Year: 2020

Optimizing the sum-log-utility for the downlink of multi-frequency band, multiuser, multiantenna networks requires joint solutions to the associated beamforming and user scheduling problems through the use of cloud radio access network (CRAN) architecture; optimizing such a network is, however, non-convex and NP-hard. In this paper, we present a novel iterative beamforming and scheduling strategy based on fractional programming and the Hungarian algorithm. The beamforming strategy allows us to iteratively maximize the chosen objective function in a fashion similar to block coordinate ascent. Furthermore, based on the crucial insight that, in the downlink, the interference pattern remains fixed for a given set of beamforming weights, we use the Hungarian algorithm as an efficient approach to optimally schedule users for the given set of beamforming weights. Specifically, this approach allows us to select the best subset of users (amongst the larger set of all available users). Our simulation results show that, in terms of average sum-log-utility, as well as sum-rate, the proposed scheme substantially outperforms both the state-of-the-art multicell weighted minimum mean-squared error (WMMSE) and greedy proportionally fair WMMSE schemes, as well as standard interior-point and sequential quadratic solvers. Importantly, our proposed scheme is also far more computationally efficient than the multicell WMMSE scheme.
Multiverse: Dynamic VM Provisioning for Virtualized High Performance Computing Clusters
Venue: arXiv cs.DC 2020
Year: 2020

Traditionally, HPC workloads have been deployed in bare-metal clusters; but the advances in virtualization have led the pathway for these workloads to be deployed in virtualized clusters. However, HPC cluster administrators/providers still face challenges in terms of resource elasticity and virtual machine (VM) provisioning at large-scale, due to the lack of coordination between a traditional HPC scheduler and the VM hypervisor (resource management layer). This lack of interaction leads to low cluster utilization and job completion throughput. Furthermore, the VM provisioning delays directly impact the overall performance of jobs in the cluster. Hence, there is a need for effectively provisioning virtualized HPC clusters, which can best-utilize the physical hardware with minimal provisioning overheads. Towards this, we propose Multiverse, a VM provisioning framework, which can dynamically spawn VMs for incoming jobs in a virtualized HPC cluster, by integrating the HPC scheduler along with VM resource manager. We have implemented this framework on the Slurm} scheduler along with the vSphere VM resource manager. In order to reduce the VM provisioning overheads, we use instant cloning which shares both the disk and memory with the parent VM, when compared to full VM cloning which has to boot-up a new VM from scratch. Measurements with real-world HPC workloads demonstrate that, instant cloning is 2.5x faster than full cloning in terms of VM provisioning time. Further, it improves resource utilization by up to 40%, and cluster throughput by up to 1.5x, when compared to full clone for bursty job arrival scenarios.
Opinion Diffusion Software with Strategic Opinion Revelation and Unfriending
Venue: arXiv cs.SI 2020
Year: 2020

We present a novel software suite for social network modeling and opinion diffusion processes. Much research on social network science has assumed networks with static topologies. More recently, attention has been turned to networks that evolve. Although software for modeling both the topological evolution of networks and diffusion processes are constantly improving, very little attention has been paid to agent modeling. Our software is designed to be robust, modular, and extensible, providing the ability to model dynamic social network topologies and multidimensional diffusion processes, different styles of agent including non-homophilic paradigms, as well as a testing environment for multi-agent reinforcement learning (MARL) experiments with diverse sets of agent types. We also illustrate the value of diverse agent modeling, and environments that allow for strategic unfriending. Our work shows that polarization and consensus dynamics, as well as topological clustering effects, may rely more than previously known on individuals' goals for the composition of their neighborhood's opinions.
Graph Neural Networks and Reinforcement Learning for Behavior Generation in Semantic Environments
Venue: arXiv cs.LG 2020
Year: 2020

Most reinforcement learning approaches used in behavior generation utilize vectorial information as input. However, this requires the network to have a pre-defined input-size -- in semantic environments this means assuming the maximum number of vehicles. Additionally, this vectorial representation is not invariant to the order and number of vehicles. To mitigate the above-stated disadvantages, we propose combining graph neural networks with actor-critic reinforcement learning. As graph neural networks apply the same network to every vehicle and aggregate incoming edge information, they are invariant to the number and order of vehicles. This makes them ideal candidates to be used as networks in semantic environments -- environments consisting of objects lists. Graph neural networks exhibit some other advantages that make them favorable to be used in semantic environments. The relational information is explicitly given and does not have to be inferred. Moreover, graph neural networks propagate information through the network and can gather higher-degree information. We demonstrate our approach using a highway lane-change scenario and compare the performance of graph neural networks to conventional ones. We show that graph neural networks are capable of handling scenarios with a varying number and order of vehicles during training and application.
Exact Support Recovery in Federated Regression with One-shot Communication
Venue: arXiv cs.LG 2020
Year: 2020

Federated learning provides a framework to address the challenges of distributed computing, data ownership and privacy over a large number of distributed clients with low computational and communication capabilities. In this paper, we study the problem of learning the exact support of sparse linear regression in the federated learning setup. We provide a simple communication efficient algorithm which only needs one-shot communication with the centralized server to compute the exact support. Our method does not require the clients to solve any optimization problem and thus, can be run on devices with low computational capabilities. Our method is naturally robust to the problems of client failure, model poisoning and straggling clients. We formally prove that our method requires a number of samples per client that is polynomial with respect to the support size, but independent of the dimension of the problem. We require the number of distributed clients to be logarithmic in the dimension of the problem. If the predictor variables are mutually independent then the overall sample complexity matches the optimal sample complexity of the non-federated centralized setting. Furthermore, our method is easy to implement and has an overall polynomial time complexity.
Similarity Search with Tensor Core Units
Venue: arXiv cs.DS 2020
Year: 2020

Tensor Core Units (TCUs) are hardware accelerators developed for deep neural networks, which efficiently support the multiplication of two dense $\sqrt{m}\times \sqrt{m}$ matrices, where $m$ is a given hardware parameter. In this paper, we show that TCUs can speed up similarity search problems as well. We propose algorithms for the Johnson-Lindenstrauss dimensionality reduction and for similarity join that, by leveraging TCUs, achieve a $\sqrt{m}$ speedup up with respect to traditional approaches.
Paratransit Agency Responses to the Adoption of Sub-contracted Services Using Secure Technologies
Venue: arXiv cs.CY 2020
Year: 2020

Transportation agencies across the United States have the responsibility of providing transportation services for all travelers. Paratransit services which are designed to meet the needs of disabled travelers have been available to a certain extent for decades, but under the Americans with Disabilities Act mandate of 1990, uniform requirements were adopted across U.S. agencies. Most of these paratransit operators offer services which must be scheduled at least a day in advance. And, provision of these services by accessible busses is generally very expensive. Therefore, many agencies are considering sub-contracting some services to approved ride-hailing or taxi services. The purpose of this work is to examine the opinions of various public agencies with respect to the adoption of sub-contracted services through the use of secure technologies. Our research provides insight into the future of these partnerships. Agencies expressed interest in the use of privacy preserving secure technologies as well as a strong desire for better software solutions for paratransit passengers and operators. The on-line survey received thirty responses for a completion rate of 19.1%. Our primary findings are that a major concern of agencies for this sort of arrangement is the lack of Wheelchair Accessible Vehicles offered by taxis and TNCs and about 36% of the surveyed agencies have not considered such partnerships.
Automatic Kernel Generation for Volta Tensor Cores
Venue: arXiv cs.PL 2020
Year: 2020

A commonly occurring computation idiom in neural networks is to perform some pointwise operations on the result of a matrix multiplication. Such a sequence of operations is typically represented as a computation graph in deep learning compilers. When compiling to a GPU target, these computations can be individually mapped to manually tuned implementations provided by libraries such as cuBLAS and cuDNN. These libraries also provide off-the-shelf support for targeting tensor cores in NVIDIA GPUs, which can lead to huge performance boosts through their specialized support for mixed-precision matrix math. Alternatively, tensor cores can be programmed directly using CUDA APIs or inline assembly instructions, which opens up the possibility of generating efficient CUDA kernels automatically for such computations. Automatic kernel generation is particularly crucial when it is beneficial to generate efficient code for an entire computation graph by fusing several operations into a single device function instead of invoking a separate kernel for each of them. Polyhedral compilation techniques provide a systematic approach for the analysis and transformation of a sequence of affine loop-nests. In this paper, we describe a polyhedral approach to generate efficient CUDA kernels for matrix multiplication using inline assembly instructions for programming tensor cores on NVIDIA Volta GPUs. Furthermore, we build on this approach to generate fused kernels for computation sequences involving matrix multiplication and pointwise operations such as bias addition, ReLU activation etc. Experimental evaluation of these techniques show that automatically generated kernels can provide significantly better performance than manually tuned library implementations, with speedups ranging up to 2.55X.
Generalized Grasping for Mechanical Grippers for Unknown Objects with Partial Point Cloud Representations
Venue: arXiv cs.RO 2020
Year: 2020

We present a generalized grasping algorithm that uses point clouds (i.e. a group of points and their respective surface normals) to discover grasp pose solutions for multiple grasp types, executed by a mechanical gripper, in near real-time. The algorithm introduces two ideas: 1) a histogram of finger contact normals is used to represent a grasp 'shape' to guide a gripper orientation search in a histogram of object(s) surface normals, and 2) voxel grid representations of gripper and object(s) are cross-correlated to match finger contact points, i.e. grasp 'size', to discover a grasp pose. Constraints, such as collisions with neighbouring objects, are optionally incorporated in the cross-correlation computation. We show via simulations and experiments that 1) grasp poses for three grasp types can be found in near real-time, 2) grasp pose solutions are consistent with respect to voxel resolution changes for both partial and complete point cloud scans, and 3) a planned grasp is executed with a mechanical gripper.