OpenRaiser/Intern-Atlas huggingface dataset:
Automatic L3 slice detection in 3D CT images using fully-convolutional networks
Venue: arXiv cs.CV 2018
Year: 2018

The analysis of single CT slices extracted at the third lumbar vertebra (L3) has garnered significant clinical interest in the past few years, in particular in regards to quantifying sarcopenia (muscle loss). In this paper, we propose an efficient method to automatically detect the L3 slice in 3D CT images. Our method works with images with a variety of fields of view, occlusions, and slice thicknesses. 3D CT images are first converted into 2D via Maximal Intensity Projection (MIP), reducing the dimensionality of the problem. The MIP images are then used as input to a 2D fully-convolutional network to predict the L3 slice locations in the form of 2D confidence maps. In addition we propose a variant architecture with less parameters allowing 1D confidence map prediction and slightly faster prediction time without loss of accuracy. Quantitative evaluation of our method on a dataset of 1006 3D CT images yields a median error of 1mm, similar to the inter-rater median error of 1mm obtained from two annotators, demonstrating the effectiveness of our method in efficiently and accurately detecting the L3 slice.
Recognizing Graph Search Trees
Venue: arXiv cs.DM 2018
Year: 2018

Graph searches and the corresponding search trees can exhibit important structural properties and are used in various graph algorithms. The problem of deciding whether a given spanning tree of a graph is a search tree of a particular search on this graph was introduced by Hagerup and Nowak in 1985, and independently by Korach and Ostfeld in 1989 where the authors showed that this problem is efficiently solvable for DFS trees. A linear time algorithm for BFS trees was obtained by Manber in 1990. In this paper we prove that the search tree problem is also in P for LDFS, in contrast to LBFS, MCS, and MNS, where we show NP-completeness. We complement our results by providing linear time algorithms for these searches on split graphs.
Strength in Numbers: Trading-off Robustness and Computation via Adversarially-Trained Ensembles
Venue: arXiv cs.NE 2018
Year: 2018

While deep learning has led to remarkable results on a number of challenging problems, researchers have discovered a vulnerability of neural networks in adversarial settings, where small but carefully chosen perturbations to the input can make the models produce extremely inaccurate outputs. This makes these models particularly unsuitable for safety-critical application domains (e.g. self-driving cars) where robustness is extremely important. Recent work has shown that augmenting training with adversarially generated data provides some degree of robustness against test-time attacks. In this paper we investigate how this approach scales as we increase the computational budget given to the defender. We show that increasing the number of parameters in adversarially-trained models increases their robustness, and in particular that ensembling smaller models while adversarially training the entire ensemble as a single model is a more efficient way of spending said budget than simply using a larger single model. Crucially, we show that it is the adversarial training of the ensemble, rather than the ensembling of adversarially trained models, which provides robustness.
Second-Order Agents on Ring Digraphs
Venue: arXiv cs.MA 2018
Year: 2018

The paper addresses the problem of consensus seeking among second-order linear agents interconnected in a specific ring topology. Unlike the existing results in the field dealing with one-directional digraphs arising in various cyclic pursuit algorithms or two-directional graphs, we focus on the case where some arcs in a two-directional ring graph are dropped in a regular fashion. The derived condition for achieving consensus turns out to be independent of the number of agents in a network.
Solving Chance Constrained Optimization under Non-Parametric Uncertainty Through Hilbert Space Embedding
Venue: arXiv cs.RO 2018
Year: 2018

In this paper, we present an efficient algorithm for solving a class of chance constrained optimization under non-parametric uncertainty. Our algorithm is built on the possibility of representing arbitrary distributions as functions in Reproducing Kernel Hilbert Space (RKHS). We use this foundation to formulate chance constrained optimization as one of minimizing the distance between a desired distribution and the distribution of the constraint functions in the RKHS. We provide a systematic way of constructing the desired distribution based on a notion of scenario approximation. Furthermore, we use the kernel trick to show that the computational complexity of our reformulated optimization problem is comparable to solving a deterministic variant of the chance-constrained optimization. We validate our formulation on two important robotic/control applications: (i) reactive collision avoidance of mobile robots in uncertain dynamic environments and (ii) inverse dynamics based path tracking of manipulators under perception uncertainty. In both these applications, the underlying chance constraints are defined over highly non-linear and non-convex functions of the uncertain parameters and possibly also decision variables. We also benchmark our formulation with the existing approaches in terms of sample complexity and the achieved optimal cost highlighting significant improvements in both these metrics.
High-Dimensional Robust Mean Estimation in Nearly-Linear Time
Venue: arXiv cs.LG 2018
Year: 2018

We study the fundamental problem of high-dimensional mean estimation in a robust model where a constant fraction of the samples are adversarially corrupted. Recent work gave the first polynomial time algorithms for this problem with dimension-independent error guarantees for several families of structured distributions. In this work, we give the first nearly-linear time algorithms for high-dimensional robust mean estimation. Specifically, we focus on distributions with (i) known covariance and sub-gaussian tails, and (ii) unknown bounded covariance. Given $N$ samples on $\mathbb{R}^d$, an $ε$-fraction of which may be arbitrarily corrupted, our algorithms run in time $\tilde{O}(Nd) / \mathrm{poly}(ε)$ and approximate the true mean within the information-theoretically optimal error, up to constant factors. Previous robust algorithms with comparable error guarantees have running times $\tildeΩ(N d^2)$, for $ε= Ω(1)$. Our algorithms rely on a natural family of SDPs parameterized by our current guess $ν$ for the unknown mean $μ^\star$. We give a win-win analysis establishing the following: either a near-optimal solution to the primal SDP yields a good candidate for $μ^\star$ -- independent of our current guess $ν$ -- or the dual SDP yields a new guess $ν'$ whose distance from $μ^\star$ is smaller by a constant factor. We exploit the special structure of the corresponding SDPs to show that they are approximately solvable in nearly-linear time. Our approach is quite general, and we believe it can also be applied to obtain nearly-linear time algorithms for other high-dimensional robust learning problems.
Optimal Scheduling of Multi-Energy Systems with Flexible Electrical and Thermal Loads
Venue: arXiv cs.SY 2018
Year: 2018

This paper proposes a detailed optimal scheduling model of an exemplar multi-energy system comprising combined cycle power plants (CCPPs), battery energy storage systems, renewable energy sources, boilers, thermal energy storage systems,electric loads and thermal loads. The proposed model considers the detailed start-up and shutdown power trajectories of the gas turbines, steam turbines and boilers. Furthermore, a practical,multi-energy load management scheme is proposed within the framework of the optimal scheduling problem. The proposed load management scheme utilizes the flexibility offered by system components such as flexible electrical pump loads, electrical interruptible loads and a flexible thermal load to reduce the overall energy cost of the system. The efficacy of the proposed model in reducing the energy cost of the system is demonstrated in the context of a day-ahead scheduling problem using four illustrative scenarios.
MURAUER: Mapping Unlabeled Real Data for Label AUstERity
Venue: arXiv cs.CV 2018
Year: 2018

Data labeling for learning 3D hand pose estimation models is a huge effort. Readily available, accurately labeled synthetic data has the potential to reduce the effort. However, to successfully exploit synthetic data, current state-of-the-art methods still require a large amount of labeled real data. In this work, we remove this requirement by learning to map from the features of real data to the features of synthetic data mainly using a large amount of synthetic and unlabeled real data. We exploit unlabeled data using two auxiliary objectives, which enforce that (i) the mapped representation is pose specific and (ii) at the same time, the distributions of real and synthetic data are aligned. While pose specifity is enforced by a self-supervisory signal requiring that the representation is predictive for the appearance from different views, distributions are aligned by an adversarial term. In this way, we can significantly improve the results of the baseline system, which does not use unlabeled data and outperform many recent approaches already with about 1% of the labeled real data. This presents a step towards faster deployment of learning based hand pose estimation, making it accessible for a larger range of applications.
A lower bound for online rectangle packing
Venue: arXiv cs.DS 2018
Year: 2018

We slightly improve the known lower bound on the asymptotic competitive ratio for online bin packing of rectangles. We present a complete proof for the new lower bound, whose value is above 1.91.
A Hierarchical Neural Network for Sequence-to-Sequences Learning
Venue: arXiv cs.CL 2018
Year: 2018

In recent years, the sequence-to-sequence learning neural networks with attention mechanism have achieved great progress. However, there are still challenges, especially for Neural Machine Translation (NMT), such as lower translation quality on long sentences. In this paper, we present a hierarchical deep neural network architecture to improve the quality of long sentences translation. The proposed network embeds sequence-to-sequence neural networks into a two-level category hierarchy by following the coarse-to-fine paradigm. Long sentences are input by splitting them into shorter sequences, which can be well processed by the coarse category network as the long distance dependencies for short sentences is able to be handled by network based on sequence-to-sequence neural network. Then they are concatenated and corrected by the fine category network. The experiments shows that our method can achieve superior results with higher BLEU(Bilingual Evaluation Understudy) scores, lower perplexity and better performance in imitating expression style and words usage than the traditional networks.
Generalised Entropies and Metric-Invariant Optimal Countermeasures for Information Leakage under Symmetric Constraints
Venue: arXiv cs.IT 2018
Year: 2018

We introduce a novel generalization of entropy and conditional entropy from which most definitions from the literature can be derived as particular cases. Within this general framework, we investigate the problem of designing countermeasures for information leakage. In particular, we seek metric-invariant solutions, i.e., they are robust against the choice of entropy for quantifying the leakage. The problem can be modelled as an information channel from the system to an adversary, and the countermeasures can be seen as modifying this channel in order to minimise the amount of information that the outputs reveal about the inputs. Our main result is to fully solve the problem under the highly symmetrical design constraint that the number of inputs that can produce the same output is capped. Our proof is constructive and the optimal channels and the minimum leakage are derived in closed form.
IntegrationDistiller: Automating Integration Analysis and Testing of Object-Oriented Applications
Venue: arXiv cs.SE 2018
Year: 2018

Software systems typically consist of various interacting components and units. While these components can be tested and shown to work correctly in isolation, when integrated and start interacting with each other, they may fail to produce the desired behaviors and results. Integration testing plays an important role in revealing issues in interactions among cooperating components. Identifying different interaction scenarios, however, is not a trivial task when performing integration testing. On the other hand, most of the integration testing solutions proposed in the literature are manual which hinders their scalability and applicability when it comes to large industrial systems. In this paper we introduce IntegrationDistiller as an automated solution and tool to identify integration scenarios and generate test cases (in the form of method call sequences) for .NET applications. It works by analyzing the code and automatically identifying class couplings, interacting methods, as well as invocation points. Moreover, the tool also helps and supports testers in identifying timing issues at integration level by automatic code instrumentation at invocation points. The code analysis engine of IntegrationDistiller is built and automated using .NET compiler platform, known as Roslyn. Hence, this work is the first in utilizing Roslyn features for automatic integration analysis and integration test case generation. This work has been done as part of our collaboration with ABB Industrial Automation Control Technologies (IACT) in Västerås-Sweden to address the integration testing challenges of the software part of the ABB Ability 800xA distributed control systems.
Learning to attend in a brain-inspired deep neural network
Venue: arXiv cs.CV 2018
Year: 2018

Recent machine learning models have shown that including attention as a component results in improved model accuracy and interpretability, despite the concept of attention in these approaches only loosely approximating the brain's attention mechanism. Here we extend this work by building a more brain-inspired deep network model of the primate ATTention Network (ATTNet) that learns to shift its attention so as to maximize the reward. Using deep reinforcement learning, ATTNet learned to shift its attention to the visual features of a target category in the context of a search task. ATTNet's dorsal layers also learned to prioritize these shifts of attention so as to maximize success of the ventral pathway classification and receive greater reward. Model behavior was tested against the fixations made by subjects searching images for the same cued category. Both subjects and ATTNet showed evidence for attention being preferentially directed to target goals, behaviorally measured as oculomotor guidance to targets. More fundamentally, ATTNet learned to shift its attention to target like objects and spatially route its visual inputs to accomplish the task. This work makes a step toward a better understanding of the role of attention in the brain and other computational systems.
Data Driven Linearized AC Power Flow Model with Regression Analysis
Venue: arXiv cs.SY 2018
Year: 2018

Full AC power flow model is an accurate mathematical model for representing the physical power systems. In practice, however, the utilization of this model is limited due to the computational complexity associated with its nonlinear and nonconvexity characteristics. An alternative linearized DC power flow model is widely used in the power system operation and planning tools. However, when reactive power and voltage magnitude are of concern, DC power flow model will be useless. Therefore, a linearized AC (LAC) power flow model is needed to address this issue. This paper first introduces a regular LAC model. Subsequently, with the advance in regression analysis technique, a data driven linearized AC (DLAC) model is proposed to improve the regular LAC model. Numerical simulations conducted on the Tennessee Valley Authority (TVA) system demonstrate the performance and effectiveness of the proposed DLAC model.
Mean Local Group Average Precision (mLGAP): A New Performance Metric for Hashing-based Retrieval
Venue: arXiv cs.CV 2018
Year: 2018

The research on hashing techniques for visual data is gaining increased attention in recent years due to the need for compact representations supporting efficient search/retrieval in large-scale databases such as online images. Among many possibilities, Mean Average Precision(mAP) has emerged as the dominant performance metric for hashing-based retrieval. One glaring shortcoming of mAP is its inability in balancing retrieval accuracy and utilization of hash codes: pushing a system to attain higher mAP will inevitably lead to poorer utilization of the hash codes. Poor utilization of the hash codes hinders good retrieval because of increased collision of samples in the hash space. This means that a model giving a higher mAP values does not necessarily do a better job in retrieval. In this paper, we introduce a new metric named Mean Local Group Average Precision (mLGAP) for better evaluation of the performance of hashing-based retrieval. The new metric provides a retrieval performance measure that also reconciles the utilization of hash codes, leading to a more practically meaningful performance metric than conventional ones like mAP. To this end, we start by mathematical analysis of the deficiencies of mAP for hashing-based retrieval. We then propose mLGAP and show why it is more appropriate for hashing-based retrieval. Experiments on image retrieval are used to demonstrate the effectiveness of the proposed metric.
Design and analysis adaptivity in multi-resolution topology optimization
Venue: arXiv cs.CE 2018
Year: 2018

Multiresolution topology optimization (MTO) methods involve decoupling of the design and analysis discretizations, such that a high-resolution design can be obtained at relatively low analysis costs. Recent studies have shown that the MTO method can be approximately 3 and 30 times faster than the traditional topology optimization method for 2D and 3D problems, respectively. To further exploit the potential of decoupling analysis and design, we propose a dp-adaptive MTO method, which involves locally increasing/decreasing the shape function orders (p) and design resolution (d). The adaptive refinement/coarsening is performed using a composite refinement indicator which includes criteria based on analysis error, presence of intermediate densities as well as the occurrence of design artefacts referred to as QR-patterns. While standard MTO must rely on filtering to suppress QR-patterns, the proposed adaptive method ensures efficiently that these artefacts are suppressed in the final design, without sacrificing the design resolution. The applicability of the dp-adaptive MTO method is demonstrated on several 2D mechanical design problems. For all the cases, significant speed-ups in computational time are obtained. In particular for design problems involving low material volume fractions, speed-ups of up to a factor of 10 can be obtained over the conventional MTO method.
On Periodic Functions as Regularizers for Quantization of Neural Networks
Venue: arXiv cs.LG 2018
Year: 2018

Deep learning models have been successfully used in computer vision and many other fields. We propose an unorthodox algorithm for performing quantization of the model parameters. In contrast with popular quantization schemes based on thresholds, we use a novel technique based on periodic functions, such as continuous trigonometric sine or cosine as well as non-continuous hat functions. We apply these functions component-wise and add the sum over the model parameters as a regularizer to the model loss during training. The frequency and amplitude hyper-parameters of these functions can be adjusted during training. The regularization pushes the weights into discrete points that can be encoded as integers. We show that using this technique the resulting quantized models exhibit the same accuracy as the original ones on CIFAR-10 and ImageNet datasets.
Complement-Free Couples Must Communicate: A Hardness Result for Two-Player Combinatorial Auctions
Venue: arXiv cs.GT 2018
Year: 2018

We study the communication complexity of welfare maximization in combinatorial auctions with $m$ items and two subadditive bidders. A $\frac{1}{2}$-approximation can be guaranteed by a trivial randomized protocol with zero communication, or a trivial deterministic protocol with $O(1)$ communication. We show that outperforming these trivial protocols requires exponential communication, settling an open question of [DobzinskiNS10, Feige09]. Specifically, we show that any (randomized) protocol guaranteeing a $(\frac{1}{2}+\frac{6}{\log_2 m})$-approximation requires communication exponential in $m$. This is tight even up to lower-order terms: we further present a $(\frac{1}{2}+\frac{1}{O(\log m)})$-approximation in poly($m$) communication. To derive our results, we introduce a new class of subadditive functions that are "far from" fractionally subadditive functions, and may be of independent interest for future works. Beyond our main result, we consider the spectrum of valuations between fractionally-subadditive and subadditive via the MPH hierarchy. Finally, we discuss the implications of our results towards combinatorial auctions with strategic bidders.
Hydra: A Peer to Peer Distributed Training & Data Collection Framework
Venue: arXiv cs.DC 2018
Year: 2018

The world needs diverse and unbiased data to train deep learning models. Currently data comes from a variety of sources that are unmoderated to a large extent. The outcomes of training neural networks with unverified data yields biased models with various strains of homophobia, sexism and racism. Another trend observed in the world of deep learning is the rise of distributed training. Although cloud companies provide high performance compute for training models in the form of GPU's connected with a low latency network, using these services comes at a high cost. We propose Hydra, a system that seeks to solve both of these problems in a novel manner by proposing a decentralized distributed framework which utilizes the substantial amount of idle compute of everyday electronic devices like smartphones and desktop computers for training and data collection purposes. Hydra couples a specialized distributed training framework on a network of these low powered devices with a reward scheme that incentivizes users to provide high quality data to unleash the compute capability on this training framework. Such a system has the ability to capture data from a wide variety of diverse sources which has been an issue in the current scenario of deep learning. Hydra brings in several new innovations in training on low powered devices including a fault tolerant version of the All Reduce algorithm. Furthermore we introduce a reinforcement learning policy to decide the size of training jobs on different machines on a heterogeneous cluster of devices with varying network latencies for Synchronous SGD. The novel thing about such a network is the ability of each machine to shut down and resume training capabilities at any point of time without restarting the overall training. To enable such an asynchronous behaviour we propose a communication framework inspired by the Bittorrent protocol and the Kademlia DHT.