OpenRaiser/Intern-Atlas huggingface dataset:
Mobile-Network Connected Drones: Field Trials, Simulations, and Design Insights
Venue: arXiv cs.NI 2019
Year: 2019

Drones are becoming increasingly used in a wide variety of industries and services and are delivering profound socioeconomic benefits. Technology needs to be in place to ensure safe operation and management of the growing fleet of drones. Mobile networks have connected tens of billions of devices on the ground in the past decades and are now ready to connect the drones flying in the sky. In this article, we share some of our findings in cellular connectivity for low altitude drones. We first present and analyze field measurement data collected during drone flights in a commercial Long-Term Evolution (LTE) network. We then present simulation results to shed light on the performance of a network when it is serving many drones simultaneously over a wide area. The results, analysis, and design insights presented in this article help enhance the understanding of the applicability and performance of providing mobile connectivity to low altitude drones.
Semantic White Balance: Semantic Color Constancy Using Convolutional Neural Network
Venue: arXiv cs.CV 2019
Year: 2019

The goal of computational color constancy is to preserve the perceptive colors of objects under different lighting conditions by removing the effect of color casts caused by the scene's illumination. With the rapid development of deep learning based techniques, significant progress has been made in image semantic segmentation. In this work, we exploit the semantic information together with the color and spatial information of the input image in order to remove color casts. We train a convolutional neural network (CNN) model that learns to estimate the illuminant color and gamma correction parameters based on the semantic information of the given image. Experimental results show that feeding the CNN with the semantic information leads to a significant improvement in the results by reducing the error by more than 40%.
Slate: extending Firedrake's domain-specific abstraction to hybridized solvers for geoscience and beyond
Venue: arXiv cs.MS 2019
Year: 2019

Within the finite element community, discontinuous Galerkin (DG) and mixed finite element methods have become increasingly popular in simulating geophysical flows. However, robust and efficient solvers for the resulting saddle-point and elliptic systems arising from these discretizations continue to be an on-going challenge. One possible approach for addressing this issue is to employ a method known as hybridization, where the discrete equations are transformed such that classic static condensation and local post-processing methods can be employed. However, it is challenging to implement hybridization as performant parallel code within complex models, whilst maintaining separation of concerns between applications scientists and software experts. In this paper, we introduce a domain-specific abstraction within the Firedrake finite element library that permits the rapid execution of these hybridization techniques within a code-generating framework. The resulting framework composes naturally with Firedrake's solver environment, allowing for the implementation of hybridization and static condensation as runtime-configurable preconditioners via the Python interface to PETSc, petsc4py. We provide examples derived from second order elliptic problems and geophysical fluid dynamics. In addition, we demonstrate that hybridization shows great promise for improving the performance of solvers for mixed finite element discretizations of equations related to large-scale geophysical flows.
Object Sorting Using a Global Texture-Shape 3D Feature Descriptor
Venue: arXiv cs.CV 2019
Year: 2019

Object recognition and grasping plays a key role in robotic systems, especially for the autonomous robots to implement object sorting tasks in a warehouse. In this paper, we present a global texture-shape 3D feature descriptor which can be utilized in a system of object recognition and grasping, and can perform object sorting tasks well. Our proposed descriptor stems from the clustered viewpoint feature histogram (CVFH), which relies on the geometrical information of the whole 3D object surface only, and can not perform well in recognizing the objects with similar geometrical information. Therefore, we extend the CVFH descriptor with texture and color information to generate a new global 3D feature descriptor. The proposed descriptor is evaluated in tasks of recognizing and classifying 3D objects by applying multi-class support vector machines (SVM) in both public 3D image dataset and real scenes. The results of evaluation show that the proposed descriptor achieves a significant better performance for object recognition compared with the original CVFH. Then, the proposed descriptor is applied in our object recognition and grasping system, showing that the proposed descriptor helps the system implement the object recognition, object grasping and object sorting tasks well.
Lie Transform--based Neural Networks for Dynamics Simulation and Learning
Venue: arXiv cs.NE 2019
Year: 2019

In the article, we discuss the architecture of the polynomial neural network that corresponds to the matrix representation of Lie transform. The matrix form of Lie transform is an approximation of the general solution of the nonlinear system of ordinary differential equations. The proposed architecture can be trained with small data sets, extrapolate predictions outside the training data, and provide a possibility for interpretation. We provide a theoretical explanation of the proposed architecture, as well as demonstrate it in several applications. We present the results of modeling and identification for both simple and well-known dynamical systems, and more complicated examples from price dynamics, chemistry, and accelerator physics. From a practical point of view, we describe the training of a Lie transform--based neural network with a small data set containing only 10 data points. We also demonstrate an interpretation of the fitted neural network by converting it to a system of differential equations.
The expressiveness of quasiperiodic and minimal shifts of finite type
Venue: arXiv cs.DM 2019
Year: 2019

We study multidimensional minimal and quasiperiodic shifts of finite type. We prove for these classes several results that were previously known for the shifts of finite type in general, without restriction. We show that some quasiperiodic shifts of finite type admit only non-computable configurations; we characterize the classes of Turing degrees that can be represented by quasiperiodic shifts of finite type. We also transpose to the classes of minimal/quasiperiodic shifts of finite type some results on subdynamics previously known for the effective shifts without restrictions: every effective minimal (quasiperiodic) shift of dimension $d$ can be represented as a projection of a subdynamics of a minimal (respectively, quasiperiodic) shift of finite type of dimension $d+1$.
Decomposition Methods with Deep Corrections for Reinforcement Learning
Venue: arXiv cs.LG 2019
Year: 2019

Decomposition methods have been proposed to approximate solutions to large sequential decision making problems. In contexts where an agent interacts with multiple entities, utility decomposition can be used to separate the global objective into local tasks considering each individual entity independently. An arbitrator is then responsible for combining the individual utilities and selecting an action in real time to solve the global problem. Although these techniques can perform well empirically, they rely on strong assumptions of independence between the local tasks and sacrifice the optimality of the global solution. This paper proposes an approach that improves upon such approximate solutions by learning a correction term represented by a neural network. We demonstrate this approach on a fisheries management problem where multiple boats must coordinate to maximize their catch over time as well as on a pedestrian avoidance problem for autonomous driving. In each problem, decomposition methods can scale to multiple boats or pedestrians by using strategies involving one entity. We verify empirically that the proposed correction method significantly improves the decomposition method and outperforms a policy trained on the full scale problem without utility decomposition.
Hierarchical Erasure Correction of Linear Codes
Venue: arXiv cs.IT 2019
Year: 2019

Linear codes over finite extension fields have widespread applications in theory and practice. In some scenarios, the decoder has a sequential access to the codeword symbols, giving rise to a hierarchical erasure structure. In this paper we develop a mathematical framework for hierarchical erasures over extension fields, provide several bounds and constructions, and discuss potential applications in distributed storage and flash memories. Our results show intimate connection to Universally Decodable Matrices, as well as to Reed-Solomon and Gabidulin codes.
An Empirical Evaluation of Deep Learning for ICD-9 Code Assignment using MIMIC-III Clinical Notes
Venue: arXiv cs.CL 2019
Year: 2019

Background and Objective: Code assignment is of paramount importance in many levels in modern hospitals, from ensuring accurate billing process to creating a valid record of patient care history. However, the coding process is tedious and subjective, and it requires medical coders with extensive training. This study aims to evaluate the performance of deep-learning-based systems to automatically map clinical notes to ICD-9 medical codes. Methods: The evaluations of this research are focused on end-to-end learning methods without manually defined rules. Traditional machine learning algorithms, as well as state-of-the-art deep learning methods such as Recurrent Neural Networks and Convolution Neural Networks, were applied to the Medical Information Mart for Intensive Care (MIMIC-III) dataset. An extensive number of experiments was applied to different settings of the tested algorithm. Results: Findings showed that the deep learning-based methods outperformed other conventional machine learning methods. From our assessment, the best models could predict the top 10 ICD-9 codes with 0.6957 F1 and 0.8967 accuracy and could estimate the top 10 ICD-9 categories with 0.7233 F1 and 0.8588 accuracy. Our implementation also outperformed existing work under certain evaluation metrics. Conclusion: A set of standard metrics was utilized in assessing the performance of ICD-9 code assignment on MIMIC-III dataset. All the developed evaluation tools and resources are available online, which can be used as a baseline for further research.
Going Deeper in Spiking Neural Networks: VGG and Residual Architectures
Venue: arXiv cs.CV 2019
Year: 2019

Over the past few years, Spiking Neural Networks (SNNs) have become popular as a possible pathway to enable low-power event-driven neuromorphic hardware. However, their application in machine learning have largely been limited to very shallow neural network architectures for simple problems. In this paper, we propose a novel algorithmic technique for generating an SNN with a deep architecture, and demonstrate its effectiveness on complex visual recognition problems such as CIFAR-10 and ImageNet. Our technique applies to both VGG and Residual network architectures, with significantly better accuracy than the state-of-the-art. Finally, we present analysis of the sparse event-driven computations to demonstrate reduced hardware overhead when operating in the spiking domain.
From Selective Deep Convolutional Features to Compact Binary Representations for Image Retrieval
Venue: arXiv cs.CV 2019
Year: 2019

In the large-scale image retrieval task, the two most important requirements are the discriminability of image representations and the efficiency in computation and storage of representations. Regarding the former requirement, Convolutional Neural Network (CNN) is proven to be a very powerful tool to extract highly discriminative local descriptors for effective image search. Additionally, in order to further improve the discriminative power of the descriptors, recent works adopt fine-tuned strategies. In this paper, taking a different approach, we propose a novel, computationally efficient, and competitive framework. Specifically, we firstly propose various strategies to compute masks, namely SIFT-mask, SUM-mask, and MAX-mask, to select a representative subset of local convolutional features and eliminate redundant features. Our in-depth analyses demonstrate that proposed masking schemes are effective to address the burstiness drawback and improve retrieval accuracy. Secondly, we propose to employ recent embedding and aggregating methods which can significantly boost the feature discriminability. Regarding the computation and storage efficiency, we include a hashing module to produce very compact binary image representations. Extensive experiments on six image retrieval benchmarks demonstrate that our proposed framework achieves the state-of-the-art retrieval performances.
Uncertainty Quantification for Maxwell's Eigenproblem based on Isogeometric Analysis and Mode Tracking
Venue: arXiv cs.CE 2019
Year: 2019

The electromagnetic field distribution as well as the resonating frequency of various modes in superconducting cavities used in particle accelerators for example are sensitive to small geometry deformations. The occurring variations are motivated by measurements of an available set of resonators from which we propose to extract a small number of relevant and independent deformations by using a truncated Karhunen-Loève expansion. The random deformations are used in an expressive uncertainty quantification workflow to determine the sensitivity of the eigenmodes. For the propagation of uncertainty, a stochastic collocation method based on sparse grids is employed. It requires the repeated solution of Maxwell's eigenvalue problem at predefined collocation points, i.e., for cavities with perturbed geometry. The main contribution of the paper is ensuring the consistency of the solution, i.e., matching the eigenpairs, among the various eigenvalue problems at the stochastic collocation points. To this end, a classical eigenvalue tracking technique is proposed that is based on homotopies between collocation points and a Newton-based eigenvalue solver. The approach can be efficiently parallelized while tracking the eigenpairs. In this paper, we propose the application of isogeometric analysis since it allows for the exact description of the geometrical domains with respect to common computer-aided design kernels, for a straightforward and convenient way of handling geometrical variations and smooth solutions.
Astor: Exploring the Design Space of Generate-and-Validate Program Repair beyond GenProg
Venue: arXiv cs.SE 2019
Year: 2019

During last years, researches have proposed novel repair approaches that automatically generate patches for repairing software bugs. Repair approaches can be loosely characterized along the main design philosophy such generate- and-validate or synthesis-based. Each of those repair approaches is a point in the design space of program repair. Our goal is to facilitate the design, development and evaluation of repair approaches by providing a framework that: a) contains components commonly present in approaches implementations thus new approaches can be built over them, b) provides built-in implementations of existing repair approach. This paper presents a framework named Astor that encores the design space of generate-and-validate repair approaches. Astor provides extension points that form the explicit decision space of program repair. Over those extension points, researchers can reuse existing components or implements new ones. Astor includes 6 Java implementation of repair approaches, including one of the pioneer: GenProg. Researcher have been already defining new approaches over Astor, proposing improvements of those built-in approaches by using the extension points, and executing approaches implementations from Astor in their evaluations. The implementations of the repair approaches built over Astor are capable of repair, in total, 98 real bugs from 5 large Java programs.
AMC: AutoML for Model Compression and Acceleration on Mobile Devices
Venue: arXiv cs.CV 2019
Year: 2019

Model compression is a critical technique to efficiently deploy neural network models on mobile devices which have limited computation resources and tight power budgets. Conventional model compression techniques rely on hand-crafted heuristics and rule-based policies that require domain experts to explore the large design space trading off among model size, speed, and accuracy, which is usually sub-optimal and time-consuming. In this paper, we propose AutoML for Model Compression (AMC) which leverage reinforcement learning to provide the model compression policy. This learning-based compression policy outperforms conventional rule-based compression policy by having higher compression ratio, better preserving the accuracy and freeing human labor. Under 4x FLOPs reduction, we achieved 2.7% better accuracy than the handcrafted model compression policy for VGG-16 on ImageNet. We applied this automated, push-the-button compression pipeline to MobileNet and achieved 1.81x speedup of measured inference latency on an Android phone and 1.43x speedup on the Titan XP GPU, with only 0.1% loss of ImageNet Top-1 accuracy.
Consistent Individualized Feature Attribution for Tree Ensembles
Venue: arXiv cs.LG 2019
Year: 2019

Interpreting predictions from tree ensemble methods such as gradient boosting machines and random forests is important, yet feature attribution for trees is often heuristic and not individualized for each prediction. Here we show that popular feature attribution methods are inconsistent, meaning they can lower a feature's assigned importance when the true impact of that feature actually increases. This is a fundamental problem that casts doubt on any comparison between features. To address it we turn to recent applications of game theory and develop fast exact tree solutions for SHAP (SHapley Additive exPlanation) values, which are the unique consistent and locally accurate attribution values. We then extend SHAP values to interaction effects and define SHAP interaction values. We propose a rich visualization of individualized feature attributions that improves over classic attribution summaries and partial dependence plots, and a unique "supervised" clustering (clustering based on feature attributions). We demonstrate better agreement with human intuition through a user study, exponential improvements in run time, improved clustering performance, and better identification of influential features. An implementation of our algorithm has also been merged into XGBoost and LightGBM, see http://github.com/slundberg/shap for details.
Towards Understanding the Generalization Bias of Two Layer Convolutional Linear Classifiers with Gradient Descent
Venue: arXiv cs.LG 2019
Year: 2019

A major challenge in understanding the generalization of deep learning is to explain why (stochastic) gradient descent can exploit the network architecture to find solutions that have good generalization performance when using high capacity models. We find simple but realistic examples showing that this phenomenon exists even when learning linear classifiers --- between two linear networks with the same capacity, the one with a convolutional layer can generalize better than the other when the data distribution has some underlying spatial structure. We argue that this difference results from a combination of the convolution architecture, data distribution and gradient descent, all of which are necessary to be included in a meaningful analysis. We provide a general analysis of the generalization performance as a function of data distribution and convolutional filter size, given gradient descent as the optimization algorithm, then interpret the results using concrete examples. Experimental results show that our analysis is able to explain what happens in our introduced examples.
Power Control via Stackelberg Game for Small-Cell Networks
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, power control for two-tier small-cell networks in the uplink is investigated. We formulate the power control problem as a Stackelberg game, where the macrocell user equipment (MUE) acts as the leader and the small-cell user equipments (SUEs) as the followers. To reduce the cross-tier and co-tier interference and also the power consumption of both the MUE and SUEs, we propose to impose a set of costs on their transmit powers and optimize not only the transmit rate but also the transmit power. The corresponding optimization problems are solved by two-layer iterations. In the inner iteration, the SUEs compete with each other and their optimal transmit powers are obtained through iterative computations. In the outer iteration, the MUE's optimal transmit power is obtained in a closed form based on the transmit powers of the SUEs through proper mathematical manipulations. We prove the convergence of the proposed power control scheme, and also theoretically show the existence and uniqueness of the Stackelberg equilibrium (SE) in the formulated Stackelberg game. Simulation results show great improvement of the proposed power control scheme especially for the MUE.
Sparse Matrix Multiplication and Triangle Listing in the Congested Clique Model
Venue: arXiv cs.DS 2019
Year: 2019

We multiply two $n \times n$ matrices $S,T$ over semirings in the Congested Clique model, where $n$ fully connected nodes communicate synchronously using $O(\log n)$-bit messages, within $O(nz(S)^{1/3} nz(T)^{1/3}/n + 1)$ rounds of communication, where $nz(A)$ denotes the number of non-zero elements in a matrix $A$. By leveraging the sparsity of the input matrices, our algorithm greatly reduces communication compared with general algorithms [Censor-Hillel et al., PODC 2015], improving upon the state-of-the-art for matrices with $o(n^2)$ non-zero elements. Our algorithm exhibits the additional strength of surpassing previous solutions also when only one matrix is sparse. This allows efficiently raising a sparse matrix to a power greater than 2. As applications, we speed up 4-cycle counting and APSP in sparse graphs. Our algorithmic contribution is a new \emph{deterministic} method of restructuring the input matrices in a sparsity-aware manner, which assigns each node with element-wise multiplication tasks that are not necessarily consecutive but are balanced, yielding communication-efficient multiplication. Moreover, this new deterministic method for restructuring matrices may be used to restructure the adjacency matrix of input graphs, enabling faster solutions for graph related problems. As an example, we present a new deterministic algorithm which solves the triangle listing problem in $O(m/n^{5/3} + 1)$ rounds, a complexity that was previously obtained by a \emph{randomized} algorithm [Pandurangan et al., SPAA 2018] and matches the lower bound of $\tildeΩ(n^{1/3})$ when $m=n^2$ of [Izumi and Le Gall, PODC 2017, Pandurangan et al., SPAA 2018]. Our triangle listing algorithm implies triangle counting with the same complexity of $O(m/n^{5/3} + 1)$ rounds, which is a \emph{cubic} improvement over the previous $O(m^2/n^3)$-round algorithm [Dolev et al., DISC 2012].
Quantum versus Classical Online Streaming Algorithms with Advice
Venue: arXiv cs.DS 2019
Year: 2019

We consider online algorithms with respect to the competitive ratio. Here, we investigate quantum and classical one-way automata with non-constant size of memory (streaming algorithms) as a model for online algorithms. We construct problems that can be solved by quantum online streaming algorithms better than by classical ones in a case of logarithmic or sublogarithmic size of memory, even if classical online algorithms get advice bits. Furthermore, we show that a quantum online algorithm with a constant number of qubits can be better than any deterministic online algorithm with a constant number of advice bits and unlimited computational power.