OpenRaiser/Intern-Atlas huggingface dataset:
Towards Robust Deep Neural Networks with BANG
Venue: arXiv cs.CV 2018
Year: 2018

Machine learning models, including state-of-the-art deep neural networks, are vulnerable to small perturbations that cause unexpected classification errors. This unexpected lack of robustness raises fundamental questions about their generalization properties and poses a serious concern for practical deployments. As such perturbations can remain imperceptible - the formed adversarial examples demonstrate an inherent inconsistency between vulnerable machine learning models and human perception - some prior work casts this problem as a security issue. Despite the significance of the discovered instabilities and ensuing research, their cause is not well understood and no effective method has been developed to address the problem. In this paper, we present a novel theory to explain why this unpleasant phenomenon exists in deep neural networks. Based on that theory, we introduce a simple, efficient, and effective training approach, Batch Adjusted Network Gradients (BANG), which significantly improves the robustness of machine learning models. While the BANG technique does not rely on any form of data augmentation or the utilization of adversarial images for training, the resultant classifiers are more resistant to adversarial perturbations while maintaining or even enhancing the overall classification performance.
Global Minimum for a Finsler Elastica Minimal Path Approach
Venue: arXiv cs.CG 2018
Year: 2018

In this paper, we propose a novel curvature-penalized minimal path model via an orientation-lifted Finsler metric and the Euler elastica curve. The original minimal path model computes the globally minimal geodesic by solving an Eikonal partial differential equation (PDE). Essentially, this first-order model is unable to penalize curvature which is related to the path rigidity property in the classical active contour models. To solve this problem, we present an Eikonal PDE-based Finsler elastica minimal path approach to address the curvature-penalized geodesic energy minimization problem. We were successful at adding the curvature penalization to the classical geodesic energy. The basic idea of this work is to interpret the Euler elastica bending energy via a novel Finsler elastica metric that embeds a curvature penalty. This metric is non-Riemannian, anisotropic and asymmetric, and is defined over an orientation-lifted space by adding to the image domain the orientation as an extra space dimension. Based on this orientation lifting, the proposed minimal path model can benefit from both the curvature and orientation of the paths. Thanks to the fast marching method, the global minimum of the curvature-penalized geodesic energy can be computed efficiently. We introduce two anisotropic image data-driven speed functions that are computed by steerable filters. Based on these orientation-dependent speed functions, we can apply the proposed Finsler elastica minimal path model to the applications of closed contour detection, perceptual grouping and tubular structure extraction. Numerical experiments on both synthetic and real images show that these applications of the proposed model indeed obtain promising results.
Dynamic Structural Operational Semantics
Venue: arXiv cs.PL 2018
Year: 2018

We introduce Dynamic SOS as a framework for describing semantics of programming languages that include dynamic software upgrades, for upgrading software code during run-time. Dynamic SOS (DSOS) is built on top of the Modular SOS of P. Mosses, with an underlying category theory formalization. The idea of Dynamic SOS is to bring out the essential differences between dynamic upgrade constructs and program execution constructs. The important feature of Modular SOS (MSOS) that we exploit in DSOS is the sharp separation of the program execution code from the additional (data) structures needed at run-time. In DSOS we aim to achieve the same modularity and decoupling for dynamic software upgrades. This is partly motivated by the long term goal of having machine-checkable proofs for general results like type safety. We exemplify Dynamic SOS on two languages supporting dynamic software upgrades, namely the C-like Proteus, which supports updating of variables, functions, records, or types at specific program points, and Creol, which supports dynamic class upgrades in the setting of concurrent objects. Existing type analyses for software upgrades can be done on top of DSOS too, as we illustrate for Proteus. As a side result we define of a general encapsulating construction on Modular SOS useful in situations where a form of encapsulation of the execution is needed. We use encapsulation in the Creol setting of concurrent object-oriented programming with active objects and asynchronous method calls.
Learning to Detect Multiple Photographic Defects
Venue: arXiv cs.CV 2018
Year: 2018

In this paper, we introduce the problem of simultaneously detecting multiple photographic defects. We aim at detecting the existence, severity, and potential locations of common photographic defects related to color, noise, blur and composition. The automatic detection of such defects could be used to provide users with suggestions for how to improve photos without the need to laboriously try various correction methods. Defect detection could also help users select photos of higher quality while filtering out those with severe defects in photo curation and summarization. To investigate this problem, we collected a large-scale dataset of user annotations on seven common photographic defects, which allows us to evaluate algorithms by measuring their consistency with human judgments. Our new dataset enables us to formulate the problem as a multi-task learning problem and train a multi-column deep convolutional neural network (CNN) to simultaneously predict the severity of all the defects. Unlike some existing single-defect estimation methods that rely on low-level statistics and may fail in many cases on natural photographs, our model is able to understand image contents and quality at a higher level. As a result, in our experiments, we show that our model has predictions with much higher consistency with human judgments than low-level methods as well as several baseline CNN models. Our model also performs better than an average human from our user study.
Distributed Detection in Ad Hoc Networks Through Quantized Consensus
Venue: arXiv cs.IT 2018
Year: 2018

We study asymptotic performance of distributed detection in large scale connected sensor networks. Contrasting to the canonical parallel network where a single node has access to local decisions from all other nodes, each node can only exchange information with its direct neighbors in the present setting. We establish that, with each node employing an identical one-bit quantizer for local information exchange, a novel consensus reaching approach can achieve the optimal asymptotic performance of centralized detection as the network size scales. The statement is true under three different detection frameworks: the Bayesian criterion where the maximum a posteriori detector is optimal, the Neyman-Pearson criterion with a constant type-I error probability constraint, and the Neyman-Pearson criterion with an exponential type-I error probability constraint. Leveraging recent development in distributed consensus reaching using bounded quantizers with possibly unbounded data (which are log-likelihood ratios of local observations in the context of distributed detection), we design a one-bit deterministic quantizer with controllable threshold that leads to desirable consensus error bounds. The obtained bounds are key to establishing the optimal asymptotic detection performance. In addition, we examine non-asymptotic performance of the proposed approach, and show that the type-I and type-II error probabilities at each node can be made arbitrarily close to the centralized ones simultaneously when a continuity condition is satisfied.
Finding Better Active Learners for Faster Literature Reviews
Venue: arXiv cs.SE 2018
Year: 2018

Literature reviews can be time-consuming and tedious to complete. By cataloging and refactoring three state-of-the-art active learning techniques from evidence-based medicine and legal electronic discovery, this paper finds and implements FASTREAD, a faster technique for studying a large corpus of documents. This paper assesses FASTREAD using datasets generated from existing SE literature reviews (Hall, Wahono, Radjenović, Kitchenham et al.). Compared to manual methods, FASTREAD lets researchers find 95% relevant studies after reviewing an order of magnitude fewer papers. Compared to other state-of-the-art automatic methods, FASTREAD reviews 20-50% fewer studies while finding same number of relevant primary studies in a systematic literature review.
Impacts of Bad ESP (Early Size Predictions) on Software Effort Estimation
Venue: arXiv cs.SE 2018
Year: 2018

Context: Early size predictions (ESP) can lead to errors in effort predictions for software projects. This problem is particular acute in parametric effort models that give extra weight to size factors (for example, the COCOMO model assumes that effort is exponentially proportional to project size). Objective: To test if effort estimates are crippled by bad ESP. Method: Document inaccuracies in early size estimates. Use those error sizes to determine the implications of those inaccuracies via a Monte Carlo perturbation analysis of effort models and an analysis of the equations used in those effort models. Results: While many projects have errors in ESP of up to +/- 100%, those errors add very little to the overall effort estimate error. Specifically, we find no statistically significant difference in the estimation errors seen after increasing ESP errors from 0 to +/- 100%. An analysis of effort estimation models explains why this is so: the net additional impact of ESP error is relatively small compared to the other sources of error associated with in estimation models. Conclusion: ESP errors effect effort estimates by a relatively minor amount. As soon as a model uses a size estimate and other factors to predict project effort, then ESP errors are not crippling to the process of estimation
$k$-Distinct In- and Out-Branchings in Digraphs
Venue: arXiv cs.DS 2018
Year: 2018

An out-branching and an in-branching of a digraph $D$ are called $k$-distinct if each of them has $k$ arcs absent in the other. Bang-Jensen, Saurabh and Simonsen (2016) proved that the problem of deciding whether a strongly connected digraph $D$ has $k$-distinct out-branching and in-branching is fixed-parameter tractable (FPT) when parameterized by $k$. They asked whether the problem remains FPT when extended to arbitrary digraphs. Bang-Jensen and Yeo (2008) asked whether the same problem is FPT when the out-branching and in-branching have the same root. By linking the two problems with the problem of whether a digraph has an out-branching with at least $k$ leaves (a leaf is a vertex of out-degree zero), we first solve the problem of Bang-Jensen and Yeo (2008). We then develop a new digraph decomposition called the rooted cut decomposition and using it we prove that the problem of Bang-Jensen et al. (2016) is FPT for all digraphs. We believe that the \emph{rooted cut decomposition} will be useful for obtaining other results on digraphs.
Design of Data-Driven Mathematical Laws for Optimal Statistical Classification Systems
Venue: arXiv cs.LG 2018
Year: 2018

This article will devise data-driven, mathematical laws that generate optimal, statistical classification systems which achieve minimum error rates for data distributions with unchanging statistics. Thereby, I will design learning machines that minimize the expected risk or probability of misclassification. I will devise a system of fundamental equations of binary classification for a classification system in statistical equilibrium. I will use this system of equations to formulate the problem of learning unknown, linear and quadratic discriminant functions from data as a locus problem, thereby formulating geometric locus methods within a statistical framework. Solving locus problems involves finding equations of curves or surfaces defined by given properties and finding graphs or loci of given equations. I will devise three systems of data-driven, locus equations that generate optimal, statistical classification systems. Each class of learning machines satisfies fundamental statistical laws for a classification system in statistical equilibrium. Thereby, I will formulate three classes of learning machines that are scalable modules for optimal, statistical pattern recognition systems, all of which are capable of performing a wide variety of statistical pattern recognition tasks, where any given M-class statistical pattern recognition system exhibits optimal generalization performance for an M-class feature space.
Identification of release sources in advection-diffusion system by machine learning combined with Green function inverse method
Venue: arXiv cs.LG 2018
Year: 2018

The identification of sources of advection-diffusion transport is based usually on solving complex ill-posed inverse models against the available state- variable data records. However, if there are several sources with different locations and strengths, the data records represent mixtures rather than the separate influences of the original sources. Importantly, the number of these original release sources is typically unknown, which hinders reliability of the classical inverse-model analyses. To address this challenge, we present here a novel hybrid method for identification of the unknown number of release sources. Our hybrid method, called HNMF, couples unsupervised learning based on Nonnegative Matrix Factorization (NMF) and inverse-analysis Green functions method. HNMF synergistically performs decomposition of the recorded mixtures, finds the number of the unknown sources and uses the Green function of advection-diffusion equation to identify their characteristics. In the paper, we introduce the method and demonstrate that it is capable of identifying the advection velocity and dispersivity of the medium as well as the unknown number, locations, and properties of various sets of synthetic release sources with different space and time dependencies, based only on the recorded data. HNMF can be applied directly to any problem controlled by a partial-differential parabolic equation where mixtures of an unknown number of sources are measured at multiple locations.
ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
Venue: arXiv cs.CL 2018
Year: 2018

Machine learning about language can be improved by supplying it with specific knowledge and sources of external information. We present here a new version of the linked open data resource ConceptNet that is particularly well suited to be used with modern NLP techniques such as word embeddings. ConceptNet is a knowledge graph that connects words and phrases of natural language with labeled edges. Its knowledge is collected from many sources that include expert-created resources, crowd-sourcing, and games with a purpose. It is designed to represent the general knowledge involved in understanding language, improving natural language applications by allowing the application to better understand the meanings behind the words people use. When ConceptNet is combined with word embeddings acquired from distributional semantics (such as word2vec), it provides applications with understanding that they would not acquire from distributional semantics alone, nor from narrower resources such as WordNet or DBPedia. We demonstrate this with state-of-the-art results on intrinsic evaluations of word relatedness that translate into improvements on applications of word vectors, including solving SAT-style analogies.
Imposing higher-level Structure in Polyphonic Music Generation using Convolutional Restricted Boltzmann Machines and Constraints
Venue: arXiv cs.SD 2018
Year: 2018

We introduce a method for imposing higher-level structure on generated, polyphonic music. A Convolutional Restricted Boltzmann Machine (C-RBM) as a generative model is combined with gradient descent constraint optimisation to provide further control over the generation process. Among other things, this allows for the use of a "template" piece, from which some structural properties can be extracted, and transferred as constraints to the newly generated material. The sampling process is guided with Simulated Annealing to avoid local optima, and to find solutions that both satisfy the constraints, and are relatively stable with respect to the C-RBM. Results show that with this approach it is possible to control the higher-level self-similarity structure, the meter, and the tonal properties of the resulting musical piece, while preserving its local musical coherence.
Efficient Detection of Complex Event Patterns Using Lazy Chain Automata
Venue: arXiv cs.DB 2018
Year: 2018

Complex Event Processing (CEP) is an emerging field with important applications in many areas. CEP systems collect events arriving from input data streams and use them to infer more complex events according to predefined patterns. The Non-deterministic Finite Automaton (NFA) is one of the most popular mechanisms on which such systems are based. During the event detection process, NFAs incrementally extend previously observed partial matches until a full match for the query is found. As a result, each arriving event needs to be processed to determine whether a new partial match is to be initiated or an existing one extended. This method may be highly inefficient when many of the events do not result in output matches. We present a lazy evaluation mechanism that defers processing of frequent event types and stores them internally upon arrival. Events are then matched in ascending order of frequency, thus minimizing potentially redundant computations. We introduce a Lazy Chain NFA, which utilizes the above principle, and does not depend on the underlying pattern structure. An algorithm for constructing a Lazy Chain NFA for common pattern types is presented, including conjunction, negation and iteration. Finally, we experimentally evaluate our mechanism on real-world stock trading data. The results demonstrate a performance gain of two orders of magnitude over traditional NFA-based approaches, with significantly reduced memory resource requirements.
Lossy Coding of Correlated Sources over a Multiple Access Channel: Necessary Conditions and Separation Results
Venue: arXiv cs.IT 2018
Year: 2018

Lossy coding of correlated sources over a multiple access channel (MAC) is studied. First, a joint source-channel coding scheme is presented when the decoder has correlated side information. Next, the optimality of separate source and channel coding, that emerges from the availability of a common observation at the encoders, or side information at the encoders and the decoder, is investigated. It is shown that separation is optimal when the encoders have access to a common observation whose lossless recovery is required at the decoder, and the two sources are independent conditioned on this common observation. Optimality of separation is also proved when the encoder and the decoder have access to shared side information conditioned on which the two sources are independent. These separation results obtained in the presence of side information are then utilized to provide a set of necessary conditions for the transmission of correlated sources over a MAC without side information. Finally, by specializing the obtained necessary conditions to the transmission of binary and Gaussian sources over a MAC, it is shown that they can potentially be tighter than the existing results in the literature, providing a novel converse for this fundamental problem.
Dictionary Learning Based Sparse Channel Representation and Estimation for FDD Massive MIMO Systems
Venue: arXiv cs.IT 2018
Year: 2018

This paper addresses the problem of uplink and downlink channel estimation in FDD Massive MIMO systems. By utilizing sparse recovery and compressive sensing algorithms, we are able to improve the accuracy of the uplink/downlink channel estimation and reduce the number of uplink/downlink pilot symbols. Such successful channel estimation builds upon the assumption that the channel can be sparsely represented under some basis/dictionary. Previous works model the channel using some predefined basis/dictionary, while in this work, we present a dictionary learning based channel model such that a dictionary is learned from comprehensively collected channel measurements. The learned dictionary adapts specifically to the cell characteristics and promotes a more efficient and robust channel representation, which in turn improves the performance of the channel estimation. Furthermore, we extend the dictionary learning based channel model into a joint uplink/downlink learning framework by observing the reciprocity of the AOA/AOD between the uplink/downlink transmission, and propose a joint channel estimation algorithm that combines the uplink and downlink received training signals to obtain a more accurate channel estimate. In other words, the downlink training overhead, which is a bottleneck in FDD Massive MIMO system, can be reduced by utilizing the information from simpler uplink training.
Personalized Video Recommendation Using Rich Contents from Videos
Venue: arXiv cs.IR 2018
Year: 2018

Video recommendation has become an essential way of helping people explore the massive videos and discover the ones that may be of interest to them. In the existing video recommender systems, the models make the recommendations based on the user-video interactions and single specific content features. When the specific content features are unavailable, the performance of the existing models will seriously deteriorate. Inspired by the fact that rich contents (e.g., text, audio, motion, and so on) exist in videos, in this paper, we explore how to use these rich contents to overcome the limitations caused by the unavailability of the specific ones. Specifically, we propose a novel general framework that incorporates arbitrary single content feature with user-video interactions, named as collaborative embedding regression (CER) model, to make effective video recommendation in both in-matrix and out-of-matrix scenarios. Our extensive experiments on two real-world large-scale datasets show that CER beats the existing recommender models with any single content feature and is more time efficient. In addition, we propose a priority-based late fusion (PRI) method to gain the benefit brought by the integrating the multiple content features. The corresponding experiment shows that PRI brings real performance improvement to the baseline and outperforms the existing fusion methods.
MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving
Venue: arXiv cs.CV 2018
Year: 2018

While most approaches to semantic reasoning have focused on improving performance, in this paper we argue that computational times are very important in order to enable real time applications such as autonomous driving. Towards this goal, we present an approach to joint classification, detection and semantic segmentation via a unified architecture where the encoder is shared amongst the three tasks. Our approach is very simple, can be trained end-to-end and performs extremely well in the challenging KITTI dataset, outperforming the state-of-the-art in the road segmentation task. Our approach is also very efficient, taking less than 100 ms to perform all tasks.