OpenRaiser/Intern-Atlas huggingface dataset:
Spatially-weighted Anomaly Detection
Venue: arXiv cs.CV 2018
Year: 2018

Many types of anomaly detection methods have been proposed recently, and applied to a wide variety of fields including medical screening and production quality checking. Some methods have utilized images, and, in some cases, a part of the anomaly images is known beforehand. However, this kind of information is dismissed by previous methods, because the methods can only utilize a normal pattern. Moreover, the previous methods suffer a decrease in accuracy due to negative effects from surrounding noises. In this study, we propose a spatially-weighted anomaly detection method (SPADE) that utilizes all of the known patterns and lessens the vulnerability to ambient noises by applying Grad-CAM, which is the visualization method of a CNN. We evaluated our method quantitatively using two datasets, the MNIST dataset with noise and a dataset based on a brief screening test for dementia.
Stability and performance guarantees in networks with cyclic dependencies
Venue: arXiv cs.NI 2018
Year: 2018

With the development of real-time networks such as reactive embedded systems, there is a need to compute deterministic performance bounds. This paper focuses on the performance guarantees and stability conditions in networks with cyclic dependencies in the network calculus framework. We first propose an algorithm that computes tight backlog bounds in tree networks for any set of flows crossing a server. Then, we show how this algorithm can be applied to improve bounds from the literature fir any topology, including cyclic networks. In particular, we show that the ring is stable in the network calculus framework.
Hybrid Active Inference
Venue: arXiv cs.AI 2018
Year: 2018

We describe a framework of hybrid cognition by formulating a hybrid cognitive agent that performs hierarchical active inference across a human and a machine part. We suggest that, in addition to enhancing human cognitive functions with an intelligent and adaptive interface, integrated cognitive processing could accelerate emergent properties within artificial intelligence. To establish this, a machine learning part learns to integrate into human cognition by explaining away multi-modal sensory measurements from the environment and physiology simultaneously with the brain signal. With ongoing training, the amount of predictable brain signal increases. This lends the agent the ability to self-supervise on increasingly high levels of cognitive processing in order to further minimize surprise in predicting the brain signal. Furthermore, with increasing level of integration, the access to sensory information about environment and physiology is substituted with access to their representation in the brain. While integrating into a joint embodiment of human and machine, human action and perception are treated as the machine's own. The framework can be implemented with invasive as well as non-invasive sensors for environment, body and brain interfacing. Online and offline training with different machine learning approaches are thinkable. Building on previous research on shared representation learning, we suggest a first implementation leading towards hybrid active inference with non-invasive brain interfacing and state of the art probabilistic deep learning methods. We further discuss how implementation might have effect on the meta-cognitive abilities of the described agent and suggest that with adequate implementation the machine part can continue to execute and build upon the learned cognitive processes autonomously.
Convex Clustering: Model, Theoretical Guarantee and Efficient Algorithm
Venue: arXiv cs.LG 2018
Year: 2018

Clustering is a fundamental problem in unsupervised learning. Popular methods like K-means, may suffer from poor performance as they are prone to get stuck in its local minima. Recently, the sum-of-norms (SON) model (also known as the clustering path) has been proposed in Pelckmans et al. (2005), Lindsten et al. (2011) and Hocking et al. (2011). The perfect recovery properties of the convex clustering model with uniformly weighted all pairwise-differences regularization have been proved by Zhu et al. (2014) and Panahi et al. (2017). However, no theoretical guarantee has been established for the general weighted convex clustering model, where better empirical results have been observed. In the numerical optimization aspect, although algorithms like the alternating direction method of multipliers (ADMM) and the alternating minimization algorithm (AMA) have been proposed to solve the convex clustering model (Chi and Lange, 2015), it still remains very challenging to solve large-scale problems. In this paper, we establish sufficient conditions for the perfect recovery guarantee of the general weighted convex clustering model, which include and improve existing theoretical results as special cases. In addition, we develop a semismooth Newton based augmented Lagrangian method for solving large-scale convex clustering problems. Extensive numerical experiments on both simulated and real data demonstrate that our algorithm is highly efficient and robust for solving large-scale problems. Moreover, the numerical results also show the superior performance and scalability of our algorithm comparing to the existing first-order methods. In particular, our algorithm is able to solve a convex clustering problem with 200,000 points in $\mathbb{R}^3$ in about 6 minutes.
Compiler Testing: A Systematic Literature Analysis
Venue: arXiv cs.SE 2018
Year: 2018

Compilers are widely-used infrastructures in accelerating the software development, and expected to be trustworthy. In the literature, various testing technologies have been proposed to guarantee the quality of compilers. However, there remains an obstacle to comprehensively characterize and understand compiler testing. To overcome this obstacle, we propose a literature analysis framework to gain insights into the compiler testing area. First, we perform an extensive search to construct a dataset related to compiler testing papers. Then, we conduct a bibliometric analysis to analyze the productive authors, the influential papers, and the frequently tested compilers based on our dataset. Finally, we utilize association rules and collaboration networks to mine the authorships and the communities of interests among researchers and keywords. Some valuable results are reported. We find that the USA is the leading country that contains the most influential researchers and institutions. The most active keyword is "random testing". We also find that most researchers have broad interests within small-scale collaborators in the compiler testing area.
Optimal Mechanism Design with Risk-loving Agents
Venue: arXiv cs.GT 2018
Year: 2018

One of the most celebrated results in mechanism design is Myerson's characterization of the revenue optimal auction for selling a single item. However, this result relies heavily on the assumption that buyers are indifferent to risk. In this paper we investigate the case where the buyers are risk-loving, i.e. they prefer gambling to being rewarded deterministically. We use the standard model for risk from expected utility theory, where risk-loving behavior is represented by a convex utility function. We focus our attention on the special case of exponential utility functions. We characterize the optimal auction and show that randomization can be used to extract more revenue than when buyers are risk-neutral. Most importantly, we show that the optimal auction is simple: the optimal revenue can be extracted using a randomized take-it-or-leave-it price for a single buyer and using a loser-pay auction, a variant of the all-pay auction, for multiple buyers. Finally, we show that these results no longer hold for convex utility functions beyond exponential.
Hierarchical Recurrent Filtering for Fully Convolutional DenseNets
Venue: arXiv cs.CV 2018
Year: 2018

Generating a robust representation of the environment is a crucial ability of learning agents. Deep learning based methods have greatly improved perception systems but still fail in challenging situations. These failures are often not solvable on the basis of a single image. In this work, we present a parameter-efficient temporal filtering concept which extends an existing single-frame segmentation model to work with multiple frames. The resulting recurrent architecture temporally filters representations on all abstraction levels in a hierarchical manner, while decoupling temporal dependencies from scene representation. Using a synthetic dataset, we show the ability of our model to cope with data perturbations and highlight the importance of recurrent and hierarchical filtering.
Linear Queries Estimation with Local Differential Privacy
Venue: arXiv cs.LG 2018
Year: 2018

We study the problem of estimating a set of $d$ linear queries with respect to some unknown distribution $\mathbf{p}$ over a domain $\mathcal{J}=[J]$ based on a sensitive data set of $n$ individuals under the constraint of local differential privacy. This problem subsumes a wide range of estimation tasks, e.g., distribution estimation and $d$-dimensional mean estimation. We provide new algorithms for both the offline (non-adaptive) and adaptive versions of this problem. In the offline setting, the set of queries are fixed before the algorithm starts. In the regime where $n\lesssim d^2/\log(J)$, our algorithms attain $L_2$ estimation error that is independent of $d$, and is tight up to a factor of $\tilde{O}\left(\log^{1/4}(J)\right)$. For the special case of distribution estimation, we show that projecting the output estimate of an algorithm due to [Acharya et al. 2018] on the probability simplex yields an $L_2$ error that depends only sub-logarithmically on $J$ in the regime where $n\lesssim J^2/\log(J)$. These results show the possibility of accurate estimation of linear queries in the high-dimensional settings under the $L_2$ error criterion. In the adaptive setting, the queries are generated over $d$ rounds; one query at a time. In each round, a query can be chosen adaptively based on all the history of previous queries and answers. We give an algorithm for this problem with optimal $L_{\infty}$ estimation error (worst error in the estimated values for the queries w.r.t. the data distribution). Our bound matches a lower bound on the $L_{\infty}$ error for the offline version of this problem [Duchi et al. 2013].
CanvasGAN: A simple baseline for text to image generation by incrementally patching a canvas
Venue: arXiv cs.CV 2018
Year: 2018

We propose a new recurrent generative model for generating images from text captions while attending on specific parts of text captions. Our model creates images by incrementally adding patches on a "canvas" while attending on words from text caption at each timestep. Finally, the canvas is passed through an upscaling network to generate images. We also introduce a new method for generating visual-semantic sentence embeddings based on self-attention over text. We compare our model's generated images with those generated Reed et. al.'s model and show that our model is a stronger baseline for text to image generation tasks.
Scaling Submodular Optimization Approaches for Control Applications in Networked Systems
Venue: arXiv cs.LG 2018
Year: 2018

Often times, in many design problems, there is a need to select a small set of informative or representative elements from a large ground set of entities in an optimal fashion. Submodular optimization that provides for a formal way to solve such problems, has recently received significant attention from the controls community where such subset selection problems are abound. However, scaling these approaches to large systems can be challenging because of the high computational complexity of the overall flow, in-part due to the high-complexity compute-oracles used to determine the objective function values. In this work, we explore a well-known paradigm, namely leader-selection in a multi-agent networked environment to illustrate strategies for scalable submodular optimization. We study the performance of the state-of-the-art stochastic and distributed greedy algorithms as well as explore techniques that accelerate the computation oracles within the optimization loop. We finally present results combining accelerated greedy algorithms with accelerated computation oracles and demonstrate significant speedups with little loss of optimality when compared to the baseline ordinary greedy algorithm.
Stability analysis of networked control systems with not necessarily UGES protocols
Venue: arXiv cs.SY 2018
Year: 2018

This note studies (practical) asymptotic stability of nonlinear networked control systems whose protocols are not necessarily uniformly globally exponentially stable. In particular, we propose a Lyapunov-based approach to establish (practical) asymptotic stability of the networked control systems. Considering so-called modified Round Robin and Try-Once-Discard protocols, which are only uniformly globally asymptotically stable, we explicitly construct Lyapunov functions for these two protocols, which fit our proposed setting. In order to optimize the usage of communication resource, we exploit the following transmission policy: wait for a certain minimum amount of time after the last sampling instant and then check a state-dependent criterion. When the latter condition is violated, a transmission occurs. In that way, the existence of the minimum amount of time between two consecutive transmission is established and so-called Zeno phenomenon, therefore, is avoided. Finally, illustrative examples are given to verify the effectiveness of our results.
Tuning for Tissue Image Segmentation Workflows for Accuracy and Performance
Venue: arXiv cs.DC 2018
Year: 2018

We propose a software platform that integrates methods and tools for multi-objective parameter auto- tuning in tissue image segmentation workflows. The goal of our work is to provide an approach for improving the accuracy of nucleus/cell segmentation pipelines by tuning their input parameters. The shape, size and texture features of nuclei in tissue are important biomarkers for disease prognosis, and accurate computation of these features depends on accurate delineation of boundaries of nuclei. Input parameters in many nucleus segmentation workflows affect segmentation accuracy and have to be tuned for optimal performance. This is a time-consuming and computationally expensive process; automating this step facilitates more robust image segmentation workflows and enables more efficient application of image analysis in large image datasets. Our software platform adjusts the parameters of a nuclear segmentation algorithm to maximize the quality of image segmentation results while minimizing the execution time. It implements several optimization methods to search the parameter space efficiently. In addition, the methodology is developed to execute on high performance computing systems to reduce the execution time of the parameter tuning phase. Our results using three real-world image segmentation workflows demonstrate that the proposed solution is able to (1) search a small fraction (about 100 points) of the parameter space, which contains billions to trillions of points, and improve the quality of segmentation output by 1.20x, 1.29x, and 1.29x, on average; (2) decrease the execution time of a segmentation workflow by up to 11.79x while improving output quality; and (3) effectively use parallel systems to accelerate parameter tuning and segmentation phases.
Anytime Stochastic Gradient Descent: A Time to Hear from all the Workers
Venue: arXiv cs.LG 2018
Year: 2018

In this paper, we focus on approaches to parallelizing stochastic gradient descent (SGD) wherein data is farmed out to a set of workers, the results of which, after a number of updates, are then combined at a central master node. Although such synchronized SGD approaches parallelize well in idealized computing environments, they often fail to realize their promised computational acceleration in practical settings. One cause is slow workers, termed stragglers, who can cause the fusion step at the master node to stall, which greatly slowing convergence. In many straggler mitigation approaches work completed by these nodes, while only partial, is discarded completely. In this paper, we propose an approach to parallelizing synchronous SGD that exploits the work completed by all workers. The central idea is to fix the computation time of each worker and then to combine distinct contributions of all workers. We provide a convergence analysis and optimize the combination function. Our numerical results demonstrate an improvement of several factors of magnitude in comparison to existing methods.
Characterizing Deep-Learning I/O Workloads in TensorFlow
Venue: arXiv cs.DC 2018
Year: 2018

The performance of Deep-Learning (DL) computing frameworks rely on the performance of data ingestion and checkpointing. In fact, during the training, a considerable high number of relatively small files are first loaded and pre-processed on CPUs and then moved to accelerator for computation. In addition, checkpointing and restart operations are carried out to allow DL computing frameworks to restart quickly from a checkpoint. Because of this, I/O affects the performance of DL applications. In this work, we characterize the I/O performance and scaling of TensorFlow, an open-source programming framework developed by Google and specifically designed for solving DL problems. To measure TensorFlow I/O performance, we first design a micro-benchmark to measure TensorFlow reads, and then use a TensorFlow mini-application based on AlexNet to measure the performance cost of I/O and checkpointing in TensorFlow. To improve the checkpointing performance, we design and implement a burst buffer. We find that increasing the number of threads increases TensorFlow bandwidth by a maximum of 2.3x and 7.8x on our benchmark environments. The use of the tensorFlow prefetcher results in a complete overlap of computation on accelerator and input pipeline on CPU eliminating the effective cost of I/O on the overall performance. The use of a burst buffer to checkpoint to a fast small capacity storage and copy asynchronously the checkpoints to a slower large capacity storage resulted in a performance improvement of 2.6x with respect to checkpointing directly to slower storage on our benchmark environment.
Eiffel: Efficient and Flexible Software Packet Scheduling
Venue: arXiv cs.NI 2018
Year: 2018

Packet scheduling determines the ordering of packets in a queuing data structure with respect to some ranking function that is mandated by a scheduling policy. It is the core component in many recent innovations to optimize network performance and utilization. Our focus in this paper is on the design and deployment of packet scheduling in software. Software schedulers have several advantages over hardware including shorter development cycle and flexibility in functionality and deployment location. We substantially improve current software packet scheduling performance, while maintaining flexibility, by exploiting underlying features of packet ranking; namely, packet ranks are integers and, at any point in time, fall within a limited range of values. We introduce Eiffel, a novel programmable packet scheduling system. At the core of Eiffel is an integer priority queue based on the Find First Set (FFS) instruction and designed to support a wide range of policies and ranking functions efficiently. As an even more efficient alternative, we also propose a new approximate priority queue that can outperform FFS-based queues for some scenarios. To support flexibility, Eiffel introduces novel programming abstractions to express scheduling policies that cannot be captured by current, state-of-the-art scheduler programming models. We evaluate Eiffel in a variety of settings and in both kernel and userspace deployments. We show that it outperforms state of the art systems by 3-40x in terms of either number of cores utilized for network processing or number of flows given fixed processing capacity.
A Survey of Neighbourhood Construction Models for Categorizing Data Points
Venue: arXiv cs.DS 2018
Year: 2018

Finding neighbourhood structures is very useful in extracting valuable relationships among data samples. This paper presents a survey of recent neighbourhood construction algorithms for pattern clustering and classifying data points. Extracting neighbourhoods and connections among the points is extremely useful for clustering and classifying the data. Many applications such as detecting social network communities, bundling related edges, and solving location and routing problems all indicate the usefulness of this problem. Finding data point neighbourhood in data mining and pattern recognition should generally improve knowledge extraction from databases. Several algorithms of data point neighbourhood construction have been proposed to analyse the data in this sense. They will be described and discussed from different aspects in this paper. Finally, the future challenges concerning the title of the present paper will be outlined.
NCARD: Improving Neighborhood Construction by Apollonius Region Algorithm based on Density
Venue: arXiv cs.CG 2018
Year: 2018

Due to the increased rate of information in the present era, local identification of similar and related data points by using neighborhood construction algorithms is highly significant for processing information in various sciences. Geometric methods are especially useful for their accuracy in locating highly similar neighborhood points using efficient geometric structures. Geometric methods should be examined for each individual point in neighborhood data set so that similar groups would be formed. Those algorithms are not highly accurate for high dimension of data. Due to the important challenges in data point analysis, we have used geometric method in which the Apollonius circle is used to achieve high local accuracy with high dimension data. In this paper, we propose a neighborhood construction algorithm, namely Neighborhood Construction by Apollonius Region Density (NCARD). In this study, the neighbors of data points are determined using not only the geometric structures, but also the density information. Apollonius circle, one of the state-of-the-art proximity geometry methods, Apollonius circle, is used for this purpose. For efficient clustering, our algorithm works better with high dimension of data than the previous methods; it is also able to identify the local outlier data. We have no prior information about the data in the proposed algorithm. Moreover, after locating similar data points with Apollonius circle, we will extract density and relationship among the points, and a unique and accurate neighborhood is created in this way. The proposed algorithm is more accurate than the state-of-the-art and well-known algorithms up to almost 8-13% in real and artificial data sets.
ASVRG: Accelerated Proximal SVRG
Venue: arXiv cs.LG 2018
Year: 2018

This paper proposes an accelerated proximal stochastic variance reduced gradient (ASVRG) method, in which we design a simple and effective momentum acceleration trick. Unlike most existing accelerated stochastic variance reduction methods such as Katyusha, ASVRG has only one additional variable and one momentum parameter. Thus, ASVRG is much simpler than those methods, and has much lower per-iteration complexity. We prove that ASVRG achieves the best known oracle complexities for both strongly convex and non-strongly convex objectives. In addition, we extend ASVRG to mini-batch and non-smooth settings. We also empirically verify our theoretical results and show that the performance of ASVRG is comparable with, and sometimes even better than that of the state-of-the-art stochastic methods.