OpenRaiser/Intern-Atlas huggingface dataset:
Arbitrarily large iterative tomographic reconstruction on multiple GPUs using the TIGRE toolbox
Venue: arXiv cs.DC 2019
Year: 2019

Tomographic image sizes keep increasing over time and while the GPUs that compute the tomographic reconstruction are also increasing in memory size, they are not doing so fast enough to reconstruct the largest datasets. This problem is often solved by reconstructing data in large clusters of GPUs with enough devices to fit the measured X-ray projections and reconstructed volume. Often this requires tens of GPUs, which is a very economically expensive solution. Access to single-node machines designed to reconstruct using just one or a few GPUs is more common in the field, but current software does not allow iterative reconstruction of volumes that do not fit in those GPUs. In this work, we propose a strategy to execute efficiently the required operations for iterative reconstruction for arbitrarily large images with any number of GPUs with arbitrarily small memories in a single node. Strategies for both the forward and backprojection operators are presented, along with two regularization approaches that are easily generalized to other projection types or regularizers. The proposed improvement also accelerates reconstruction of smaller images on single or multiple GPUs, providing faster code for time-critical medical applications. The resulting algorithm has been added to the TIGRE toolbox, a repository for iterative reconstruction algorithms for general CT, but this memory-saving and problem-splitting strategy can be easily adapted for any other GPU-based CT code.
Learning Interpretable Features via Adversarially Robust Optimization
Venue: arXiv cs.CV 2019
Year: 2019

Neural networks are proven to be remarkably successful for classification and diagnosis in medical applications. However, the ambiguity in the decision-making process and the interpretability of the learned features is a matter of concern. In this work, we propose a method for improving the feature interpretability of neural network classifiers. Initially, we propose a baseline convolutional neural network with state of the art performance in terms of accuracy and weakly supervised localization. Subsequently, the loss is modified to integrate robustness to adversarial examples into the training process. In this work, feature interpretability is quantified via evaluating the weakly supervised localization using the ground truth bounding boxes. Interpretability is also visually assessed using class activation maps and saliency maps. The method is applied to NIH ChestX-ray14, the largest publicly available chest x-rays dataset. We demonstrate that the adversarially robust optimization paradigm improves feature interpretability both quantitatively and visually.
Super-resolution limit of the ESPRIT algorithm
Venue: arXiv cs.IT 2019
Year: 2019

The problem of imaging point objects can be formulated as estimation of an unknown atomic measure from its $M+1$ consecutive noisy Fourier coefficients. The standard resolution of this inverse problem is $1/M$ and super-resolution refers to the capability of resolving atoms at a higher resolution. When any two atoms are less than $1/M$ apart, this recovery problem is highly challenging and many existing algorithms either cannot deal with this situation or require restrictive assumptions on the sign of the measure. ESPRIT is an efficient method that does not depend on the sign of the measure. This paper provides an explicit error bound on the support matching distance of ESPRIT in terms of the minimum singular value of Vandermonde matrices. When the support consists of multiple well-separated clumps and noise is sufficiently small, the support error by ESPRIT scales like ${\rm SRF}^{2λ+2} \times {\rm Noise}$, where the Super-Resolution Factor (${\rm SRF}$) governs the difficulty of the problem and $λ$ is the cardinality of the largest clump. {If the support contains one clump of closely spaced atoms, the min-max error is ${\rm SRF}^{2λ+2} \times {\rm Noise}/M$. Our error bound matches the min-max rate up to a factor of $M$ in the small noise regime. Our results therefore establishes the near-optimality of ESPRIT,} and our theory is validated by numerical experiments.
A Taxonomy and Dataset for 360° Videos
Venue: arXiv cs.MM 2019
Year: 2019

In this paper, we propose a taxonomy for 360° videos that categorizes videos based on moving objects and camera motion. We gathered and produced 28 videos based on the taxonomy, and recorded viewport traces from 60 participants watching the videos. In addition to the viewport traces, we provide the viewers' feedback on their experience watching the videos, and we also analyze viewport patterns on each category.
DSAF: Dynamic Slice Allocation Framework for 5G Core Network
Venue: arXiv cs.NI 2019
Year: 2019

Network slicing is a key to supporting different quality-of-service requirements for users and application in the 5G network. However, allocating network slices efficiently while providing a minimum guaranteed level of service in a mobile core is challenging. To address this question, in our previous work we proposed an optimization model to allocate slices. It provided a static and manual allocation of slices. In this paper, we extend our work to dynamically allocated slices. We propose a dynamic slice allocation framework for the 5G core network. The proposed framework provides user-interaction to request slices and any required services that need to run on a slice(s). It can accept a single or multiple allocation requests, and it dynamically allocates them. Additionally, the framework allocates slices in a balanced fashion across available resources. We compare our framework with the First Come First Serve and First Available allocation scheme.
Charlotte: Composable Authenticated Distributed Data Structures, Technical Report
Venue: arXiv cs.DC 2019
Year: 2019

We present Charlotte, a framework for composable, authenticated distributed data structures. Charlotte data is stored in blocks that reference each other by hash. Together, all Charlotte blocks form a directed acyclic graph, the blockweb; all observers and applications use subgraphs of the blockweb for their own data structures. Unlike prior systems, Charlotte data structures are composable: applications and data structures can operate fully independently when possible, and share blocks when desired. To support this composability, we define a language-independent format for Charlotte blocks and a network API for Charlotte servers. An authenticated distributed data structure guarantees that data is immutable and self-authenticating: data referenced will be unchanged when it is retrieved. Charlotte extends these guarantees by allowing applications to plug in their own mechanisms for ensuring availability and integrity of data structures. Unlike most traditional distributed systems, including distributed databases, blockchains, and distributed hash tables, Charlotte supports heterogeneous trust: different observers may have their own beliefs about who might fail, and how. Despite heterogeneity of trust, Charlotte presents each observer with a consistent, available view of data. We demonstrate the flexibility of Charlotte by implementing a variety of integrity mechanisms, including consensus and proof of work. We study the power of disentangling availability and integrity mechanisms by building a variety of applications. The results from these examples suggest that developers can use Charlotte to build flexible, fast, composable applications with strong guarantees.
Illumination Normalization via Merging Locally Enhanced Textures for Robust Face Recognition
Venue: arXiv cs.CV 2019
Year: 2019

In order to improve the accuracy of face recognition under varying illumination conditions, a local texture enhanced illumination normalization method based on fusion of differential filtering images (FDFI-LTEIN) is proposed to weaken the influence caused by illumination changes. Firstly, the dynamic range of the face image in dark or shadowed regions is expanded by logarithmic transformation. Then, the global contrast enhanced face image is convolved with difference of Gaussian filters and difference of bilateral filters, and the filtered images are weighted and merged using a coefficient selection rule based on the standard deviation (SD) of image, which can enhance image texture information while filtering out most noise. Finally, the local contrast equalization (LCE) is performed on the fused face image to reduce the influence caused by over or under saturated pixel values in highlight or dark regions. Experimental results on the Extended Yale B face database and CMU PIE face database demonstrate that the proposed method is more robust to illumination changes and achieve higher recognition accuracy when compared with other illumination normalization methods and a deep CNNs based illumination invariant face recognition method
Analysis of Approximate Message Passing with Non-Separable Denoisers and Markov Random Field Priors
Venue: arXiv cs.IT 2019
Year: 2019

Approximate message passing (AMP) is a class of low-complexity, scalable algorithms for solving high-dimensional linear regression tasks where one wishes to recover an unknown signal from noisy, linear measurements. AMP is an iterative algorithm that performs estimation by updating an estimate of the unknown signal at each iteration and the performance of AMP (quantified, for example, by the mean squared error of its estimates) depends on the choice of a "denoiser" function that is used to produce these signal estimates at each iteration. An attractive feature of AMP is that its performance can be tracked by a scalar recursion referred to as state evolution. Previous theoretical analysis of the accuracy of the state evolution predictions has been limited to the use of only separable denoisers or block-separable denoisers, a class of denoisers that underperform when sophisticated dependencies exist between signal entries. Since signals with entrywise dependencies are common in image/video-processing applications, in this work we study the high-dimensional linear regression task when the dependence structure of the input signal is modeled by a Markov random field prior distribution. We provide a rigorous analysis of the performance of AMP, demonstrating the accuracy of the state evolution predictions, when a class of non-separable sliding-window denoisers is applied. Moreover, we provide numerical examples where AMP with sliding-window denoisers can successfully capture local dependencies in images.
Credit Scoring for Micro-Loans
Venue: arXiv cs.LG 2019
Year: 2019

Credit Scores are ubiquitous and instrumental for loan providers and regulators. In this paper we showcase how micro-loan credit system can be developed in real setting. We show what challenges arise and discuss solutions. Particularly, we are concerned about model interpretability and data quality. In the final section, we introduce semi-supervised algorithm that aids model development and evaluate its performance
Non-Conforming Mesh Refinement for High-Order Finite Elements
Venue: arXiv cs.NA 2019
Year: 2019

We propose a general algorithm for non-conforming adaptive mesh refinement (AMR) of unstructured meshes in high-order finite element codes. Our focus is on h-refinement with a fixed polynomial order. The algorithm handles triangular, quadrilateral, hexahedral and prismatic meshes of arbitrarily high order curvature, for any order finite element space in the de Rham sequence. We present a flexible data structure for meshes with hanging nodes and a general procedure to construct the conforming interpolation operator, both in serial and in parallel. The algorithm and data structure allow anisotropic refinement of tensor product elements in 2D and 3D, and support unlimited refinement ratios of adjacent elements. We report numerical experiments verifying the correctness of the algorithms, and perform a parallel scaling study to show that we can adapt meshes containing billions of elements and run efficiently on 393,000 parallel tasks. Finally, we illustrate the integration of dynamic AMR into a high-order Lagrangian hydrodynamics solver.
P4BFT: Hardware-Accelerated Byzantine-Resilient Network Control Plane
Venue: arXiv cs.NI 2019
Year: 2019

Byzantine Fault Tolerance (BFT) enables correct operation of distributed, i.e., replicated applications in the face of malicious take-over and faulty/buggy individual instances. Recently, BFT designs have gained traction in the context of Software Defined Networking (SDN). In SDN, controller replicas are distributed and their state replicated for high availability purposes. Malicious controller replicas, however, may destabilize the control plane and manipulate the data plane, thus motivating the BFT requirement. Nonetheless, deploying BFT in practice comes at a disadvantage of increased traffic load stemming from replicated controllers, as well as a requirement for proprietary switch functionalities, thus putting strain on switches' control plane where particular BFT actions must be executed in software. P4BFT leverages an optimal strategy to decrease the total amount of messages transmitted to switches that are the configuration targets of SDN controllers. It does so by means of message comparison and deduction of correct messages in the determined optimal locations in the data plane. In terms of the incurred control plane load, our P4-based data plane extensions outperform the existing solutions by ~33.2% and ~40.2% on average, in random 128-switch and Fat-Tree/Internet2 topologies, respectively. To validate the correctness and performance gains of P4BFT, we deploy bmv2 and Netronome Agilio SmartNIC-based topologies. The advantages of P4BFT can thus be reproduced both with software switches and "commodity" P4-enabled hardware. A hardware-accelerated controller packet comparison procedure results in an average 96.4% decrease in processing delay per request compared to existing software approaches.
Support Relation Analysis for Objects in Multiple View RGB-D Images
Venue: arXiv cs.CV 2019
Year: 2019

Understanding physical relations between objects, especially their support relations, is crucial for robotic manipulation. There has been work on reasoning about support relations and structural stability of simple configurations in RGB-D images. In this paper, we propose a method for extracting more detailed physical knowledge from a set of RGB-D images taken from the same scene but from different views using qualitative reasoning and intuitive physical models. Rather than providing a simple contact relation graph and approximating stability over convex shapes, our method is able to provide a detailed supporting relation analysis based on a volumetric representation. Specifically, true supporting relations between objects (e.g., if an object supports another object by touching it on the side or if the object above contributes to the stability of the object below) are identified. We apply our method to real-world structures captured in warehouse scenarios and show our method works as desired.
Single-Path NAS: Device-Aware Efficient ConvNet Design
Venue: arXiv cs.LG 2019
Year: 2019

Can we automatically design a Convolutional Network (ConvNet) with the highest image classification accuracy under the latency constraint of a mobile device? Neural Architecture Search (NAS) for ConvNet design is a challenging problem due to the combinatorially large design space and search time (at least 200 GPU-hours). To alleviate this complexity, we propose Single-Path NAS, a novel differentiable NAS method for designing device-efficient ConvNets in less than 4 hours. 1. Novel NAS formulation: our method introduces a single-path, over-parameterized ConvNet to encode all architectural decisions with shared convolutional kernel parameters. 2. NAS efficiency: Our method decreases the NAS search cost down to 8 epochs (30 TPU-hours), i.e., up to 5,000x faster compared to prior work. 3. On-device image classification: Single-Path NAS achieves 74.96% top-1 accuracy on ImageNet with 79ms inference latency on a Pixel 1 phone, which is state-of-the-art accuracy compared to NAS methods with similar latency (<80ms).
Low-Complexity Tilings of the Plane
Venue: arXiv cs.DM 2019
Year: 2019

A two-dimensional configuration is a coloring of the infinite grid Z^2 with finitely many colors. For a finite subset D of Z^2, the D-patterns of a configuration are the colored patterns of shape D that appear in the configuration. The number of distinct D-patterns of a configuration is a natural measure of its complexity. A configuration is considered having low complexity with respect to shape D if the number of distinct D-patterns is at most |D|, the size of the shape. This extended abstract is a short review of an algebraic method to study periodicity of such low complexity configurations.
A Scheme for Continuous Input to the Tsetlin Machine with Applications to Forecasting Disease Outbreaks
Venue: arXiv cs.LG 2019
Year: 2019

In this paper, we apply a new promising tool for pattern classification, namely, the Tsetlin Machine (TM), to the field of disease forecasting. The TM is interpretable because it is based on manipulating expressions in propositional logic, leveraging a large team of Tsetlin Automata (TA). Apart from being interpretable, this approach is attractive due to its low computational cost and its capacity to handle noise. To attack the problem of forecasting, we introduce a preprocessing method that extends the TM so that it can handle continuous input. Briefly stated, we convert continuous input into a binary representation based on thresholding. The resulting extended TM is evaluated and analyzed using an artificial dataset. The TM is further applied to forecast dengue outbreaks of all the seventeen regions in the Philippines using the spatio-temporal properties of the data. Experimental results show that dengue outbreak forecasts made by the TM are more accurate than those obtained by a Support Vector Machine (SVM), Decision Trees (DTs), and several multi-layered Artificial Neural Networks (ANNs), both in terms of forecasting precision and F1-score.
Virtual Mixup Training for Unsupervised Domain Adaptation
Venue: arXiv cs.CV 2019
Year: 2019

We study the problem of unsupervised domain adaptation which aims to adapt models trained on a labeled source domain to a completely unlabeled target domain. Recently, the cluster assumption has been applied to unsupervised domain adaptation and achieved strong performance. One critical factor in successful training of the cluster assumption is to impose the locally-Lipschitz constraint to the model. Existing methods only impose the locally-Lipschitz constraint around the training points while miss the other areas, such as the points in-between training data. In this paper, we address this issue by encouraging the model to behave linearly in-between training points. We propose a new regularization method called Virtual Mixup Training (VMT), which is able to incorporate the locally-Lipschitz constraint to the areas in-between training data. Unlike the traditional mixup model, our method constructs the combination samples without using the label information, allowing it to apply to unsupervised domain adaptation. The proposed method is generic and can be combined with most existing models such as the recent state-of-the-art model called VADA. Extensive experiments demonstrate that VMT significantly improves the performance of VADA on six domain adaptation benchmark datasets. For the challenging task of adapting MNIST to SVHN, VMT can improve the accuracy of VADA by over 30\%. Code is available at \url{https://github.com/xudonmao/VMT}.
Good Things Come to Those Who Swap Objects on Paths
Venue: arXiv cs.DS 2019
Year: 2019

We study a simple exchange market, introduced by Gourvés, Lesca and Wilczynski (IJCAI-17), where every agent initially holds a single object. The agents have preferences over the objects, and two agents may swap their objects if they both prefer the object of the other agent. The agents live in an underlying social network that governs the structure of the swaps: Two agents can only swap their objects if they are adjacent. We investigate the REACHABLE OBJECT problem, which asks whether a given starting situation can ever lead, by means of a sequence of swaps, to a situation where a given agent obtains a given object. Our results answer several central open questions on the complexity of REACHABLE OBJECT. First, the problem is polynomial-time solvable if the social network is a path. Second, the problem is NP-hard on cliques and generalized caterpillars. Finally, we establish a three-versus-four dichotomy result for preference lists of bounded length: The problem is easy if all preference lists have length at most three, and the problem becomes NP-hard even if all agents have preference lists of length at most four.
Hybrid Predictive Model: When an Interpretable Model Collaborates with a Black-box Model
Venue: arXiv cs.LG 2019
Year: 2019

Interpretable machine learning has become a strong competitor for traditional black-box models. However, the possible loss of the predictive performance for gaining interpretability is often inevitable, putting practitioners in a dilemma of choosing between high accuracy (black-box models) and interpretability (interpretable models). In this work, we propose a novel framework for building a Hybrid Predictive Model (HPM) that integrates an interpretable model with any black-box model to combine their strengths. The interpretable model substitutes the black-box model on a subset of data where the black-box is overkill or nearly overkill, gaining transparency at no or low cost of the predictive accuracy. We design a principled objective function that considers predictive accuracy, model interpretability, and model transparency (defined as the percentage of data processed by the interpretable substitute.) Under this framework, we propose two hybrid models, one substituting with association rules and the other with linear models, and we design customized training algorithms for both models. We test the hybrid models on structured data and text data where interpretable models collaborate with various state-of-the-art black-box models. Results show that hybrid models obtain an efficient trade-off between transparency and predictive performance, characterized by our proposed efficient frontiers.
Deep Unsupervised Cardinality Estimation
Venue: arXiv cs.DB 2019
Year: 2019

Cardinality estimation has long been grounded in statistical tools for density estimation. To capture the rich multivariate distributions of relational tables, we propose the use of a new type of high-capacity statistical model: deep autoregressive models. However, direct application of these models leads to a limited estimator that is prohibitively expensive to evaluate for range or wildcard predicates. To produce a truly usable estimator, we develop a Monte Carlo integration scheme on top of autoregressive models that can efficiently handle range queries with dozens of dimensions or more. Like classical synopses, our estimator summarizes the data without supervision. Unlike previous solutions, we approximate the joint data distribution without any independence assumptions. Evaluated on real-world datasets and compared against real systems and dominant families of techniques, our estimator achieves single-digit multiplicative error at tail, an up to 90$\times$ accuracy improvement over the second best method, and is space- and runtime-efficient.