OpenRaiser/Intern-Atlas huggingface dataset:
Weight Sharing is Crucial to Succesful Optimization
Venue: arXiv cs.LG 2017
Year: 2017

Exploiting the great expressive power of Deep Neural Network architectures, relies on the ability to train them. While current theoretical work provides, mostly, results showing the hardness of this task, empirical evidence usually differs from this line, with success stories in abundance. A strong position among empirically successful architectures is captured by networks where extensive weight sharing is used, either by Convolutional or Recurrent layers. Additionally, characterizing specific aspects of different tasks, making them "harder" or "easier", is an interesting direction explored both theoretically and empirically. We consider a family of ConvNet architectures, and prove that weight sharing can be crucial, from an optimization point of view. We explore different notions of the frequency, of the target function, proving necessity of the target function having some low frequency components. This necessity is not sufficient - only with weight sharing can it be exploited, thus theoretically separating architectures using it, from others which do not. Our theoretical results are aligned with empirical experiments in an even more general setting, suggesting viability of examination of the role played by interleaving those aspects in broader families of tasks.
Reservoir Computing for Detection of Steady State in Performance Tests of Compressors
Venue: arXiv cs.OH 2017
Year: 2017

Fabrication of devices in industrial plants often includes undergoing quality assurance tests or tests that seek to determine some attributes or capacities of the device. For instance, in testing refrigeration compressors, we want to find the true refrigeration capacity of the compressor being tested. Such test (also called an episode) may take up to four hours, being an actual hindrance to applying it to the total number of compressors produced. This work seeks to reduce the time spent on such industrial trials by employing Recurrent Neural Networks (RNNs) as dynamical models for detecting when a test is entering the so-called steady-state region. Specifically, we use Reservoir Computing (RC) networks which simplify the learning of RNNs by speeding up training time and showing convergence to a global optimum. Also, this work proposes a self-organized subspace projection method for RC networks which uses information from the beginning of the episode to define a cluster to which the episode belongs to. This assigned cluster defines a particular binary input that shifts the operating point of the reservoir to a subspace of trajectories for the duration of the episode. This new method is shown to turn the RC model robust in performance with respect to varying combination of reservoir parameters, such as spectral radius and leak rate, when compared to a standard RC network.
A watershed-based algorithm to segment and classify cells in fluorescence microscopy images
Venue: arXiv cs.CV 2017
Year: 2017

Imaging assays of cellular function, especially those using fluorescent stains, are ubiquitous in the biological and medical sciences. Despite advances in computer vision, such images are often analyzed using only manual or rudimentary automated processes. Watershed-based segmentation is an effective technique for identifying objects in images; it outperforms commonly used image analysis methods, but requires familiarity with computer-vision techniques to be applied successfully. In this report, we present and implement a watershed-based image analysis and classification algorithm in a GUI, enabling a broad set of users to easily understand the algorithm and adjust the parameters to their specific needs. As an example, we implement this algorithm to find and classify cells in a complex imaging assay for mitochondrial function. In a second example, we demonstrate a workflow using manual comparisons and receiver operator characteristics to optimize the algorithm parameters for finding live and dead cells in a standard viability assay. Overall, this watershed-based algorithm is more advanced than traditional thresholding and can produce optimized, automated results. By incorporating associated pre-processing steps in the GUI, the algorithm is also easily adjusted, rendering it user-friendly.
Efficient Textual Representation of Structure
Venue: arXiv cs.PL 2017
Year: 2017

This paper attempts a more formal approach to the legibility of text based programming languages, presenting, with proof, minimum possible ways of representing structure in text interleaved with information. This presumes that a minimalist approach is best for purposes of human readability, data storage and transmission, and machine evaluation. Several proposals are given for improving the expression of interleaved hierarchical structure. For instance, a single colon can replace a pair of brackets, and bracket types do not need to be repeated in both opening and closing symbols or words. Historic and customary uses of punctuation symbols guided the chosen form and nature of the improvements.
MobiRNN: Efficient Recurrent Neural Network Execution on Mobile GPU
Venue: arXiv cs.DC 2017
Year: 2017

In this paper, we explore optimizations to run Recurrent Neural Network (RNN) models locally on mobile devices. RNN models are widely used for Natural Language Processing, Machine Translation, and other tasks. However, existing mobile applications that use RNN models do so on the cloud. To address privacy and efficiency concerns, we show how RNN models can be run locally on mobile devices. Existing work on porting deep learning models to mobile devices focus on Convolution Neural Networks (CNNs) and cannot be applied directly to RNN models. In response, we present MobiRNN, a mobile-specific optimization framework that implements GPU offloading specifically for mobile GPUs. Evaluations using an RNN model for activity recognition shows that MobiRNN does significantly decrease the latency of running RNN models on phones.
Higher-order meshing of implicit geometries - part III: Conformal Decomposition FEM (CDFEM)
Venue: arXiv cs.NA 2017
Year: 2017

A higher-order accurate finite element method is proposed which uses automatically generated meshes based on implicit level-set data for the description of boundaries and interfaces in two and three dimensions. The method is an alternative for fictitious domain and extended finite element methods. The domain of interest is immersed in a background mesh composed by higher-order elements. The zero-level sets are identified and meshed followed by a decomposition of the cut background elements into conforming sub-elements. Adaptivity is a crucial ingredient of the method to guarantee the success of the mesh generation. It ensures the successful decomposition of cut elements and enables improved geometry descriptions and approximations. It is confirmed that higher-order accurate results with optimal convergence rates are achieved with the proposed conformal decomposition finite element method (CDFEM).
Graph-Cut RANSAC
Venue: arXiv cs.CV 2017
Year: 2017

A novel method for robust estimation, called Graph-Cut RANSAC, GC-RANSAC in short, is introduced. To separate inliers and outliers, it runs the graph-cut algorithm in the local optimization (LO) step which is applied when a so-far-the-best model is found. The proposed LO step is conceptually simple, easy to implement, globally optimal and efficient. GC-RANSAC is shown experimentally, both on synthesized tests and real image pairs, to be more geometrically accurate than state-of-the-art methods on a range of problems, e.g. line fitting, homography, affine transformation, fundamental and essential matrix estimation. It runs in real-time for many problems at a speed approximately equal to that of the less accurate alternatives (in milliseconds on standard CPU).
Personalized Age Progression with Bi-level Aging Dictionary Learning
Venue: arXiv cs.CV 2017
Year: 2017

Age progression is defined as aesthetically re-rendering the aging face at any future age for an individual face. In this work, we aim to automatically render aging faces in a personalized way. Basically, for each age group, we learn an aging dictionary to reveal its aging characteristics (e.g., wrinkles), where the dictionary bases corresponding to the same index yet from two neighboring aging dictionaries form a particular aging pattern cross these two age groups, and a linear combination of all these patterns expresses a particular personalized aging process. Moreover, two factors are taken into consideration in the dictionary learning process. First, beyond the aging dictionaries, each person may have extra personalized facial characteristics, e.g. mole, which are invariant in the aging process. Second, it is challenging or even impossible to collect faces of all age groups for a particular person, yet much easier and more practical to get face pairs from neighboring age groups. To this end, we propose a novel Bi-level Dictionary Learning based Personalized Age Progression (BDL-PAP) method. Here, bi-level dictionary learning is formulated to learn the aging dictionaries based on face pairs from neighboring age groups. Extensive experiments well demonstrate the advantages of the proposed BDL-PAP over other state-of-the-arts in term of personalized age progression, as well as the performance gain for cross-age face verification by synthesizing aging faces.
Joint Text Embedding for Personalized Content-based Recommendation
Venue: arXiv cs.IR 2017
Year: 2017

Learning a good representation of text is key to many recommendation applications. Examples include news recommendation where texts to be recommended are constantly published everyday. However, most existing recommendation techniques, such as matrix factorization based methods, mainly rely on interaction histories to learn representations of items. While latent factors of items can be learned effectively from user interaction data, in many cases, such data is not available, especially for newly emerged items. In this work, we aim to address the problem of personalized recommendation for completely new items with text information available. We cast the problem as a personalized text ranking problem and propose a general framework that combines text embedding with personalized recommendation. Users and textual content are embedded into latent feature space. The text embedding function can be learned end-to-end by predicting user interactions with items. To alleviate sparsity in interaction data, and leverage large amount of text data with little or no user interactions, we further propose a joint text embedding model that incorporates unsupervised text embedding with a combination module. Experimental results show that our model can significantly improve the effectiveness of recommendation systems on real-world datasets.
PMU-Based Estimation of Dynamic State Jacobian Matrix and Dynamic System State Matrix in Ambient Conditions
Venue: arXiv cs.SY 2017
Year: 2017

In this paper, a hybrid measurement- and model-based method is proposed which can estimate the dynamic state Jacobian matrix and the dynamic system state matrix in near real-time utilizing statistical properties extracted from PMU measurements. The proposed method can be used to detect and identify network topology changes that have not been reflected in an assumed network model. Additionally, an application of the estimated system state matrix in online dynamic stability monitoring is presented.
Automatically Discovering, Reporting and Reproducing Android Application Crashes
Venue: arXiv cs.SE 2017
Year: 2017

Mobile developers face unique challenges when detecting and reporting crashes in apps due to their prevailing GUI event-driven nature and additional sources of inputs (e.g., sensor readings). To support developers in these tasks, we introduce a novel, automated approach called CRASHSCOPE. This tool explores a given Android app using systematic input generation, according to several strategies informed by static and dynamic analyses, with the intrinsic goal of triggering crashes. When a crash is detected, CRASHSCOPE generates an augmented crash report containing screenshots, detailed crash reproduction steps, the captured exception stack trace, and a fully replayable script that automatically reproduces the crash on a target device(s). We evaluated CRASHSCOPE's effectiveness in discovering crashes as compared to five state-of-the-art Android input generation tools on 61 applications. The results demonstrate that CRASHSCOPE performs about as well as current tools for detecting crashes and provides more detailed fault information. Additionally, in a study analyzing eight real-world Android app crashes, we found that CRASHSCOPE's reports are easily readable and allow for reliable reproduction of crashes by presenting more explicit information than human written reports.
Graphons: A Nonparametric Method to Model, Estimate, and Design Algorithms for Massive Networks
Venue: arXiv cs.SI 2017
Year: 2017

Many social and economic systems are naturally represented as networks, from off-line and on-line social networks, to bipartite networks, like Netflix and Amazon, between consumers and products. Graphons, developed as limits of graphs, form a natural, nonparametric method to describe and estimate large networks like Facebook and LinkedIn. Here we describe the development of the theory of graphons, for both dense and sparse networks, over the last decade. We also review theorems showing that we can consistently estimate graphons from massive networks in a wide variety of models. Finally, we show how to use graphons to estimate missing links in a sparse network, which has applications from estimating social and information networks in development economics, to rigorously and efficiently doing collaborative filtering with applications to movie recommendations in Netflix and product suggestions in Amazon.
One look at the rating of scientific publications and corresponding toy-model
Venue: arXiv cs.DL 2017
Year: 2017

A toy-model of publications and citations processes is proposed. The model shows that the role of randomness in the processes is essential and cannot be ignored. Some other aspects of scientific publications rating are discussed.
Bio-inspired Evolutionary Game Dynamics on Complex Networks under Uncertain Cross-inhibitory Signals
Venue: arXiv cs.SY 2017
Year: 2017

Given a large population of players, each player has three possible choices between option 1 or 2 or no option. The two options are equally favorable and the population has to reach consensus on one of the two options quickly and in a distributed way. The more popular an option is, the more likely it is to be chosen by uncommitted players. Uncommitted players can be attracted by those committed to any of the other two options through a cross-inhibitory signal. This model originates in the context of honeybees swarms, and we generalize it to duopolistic competition and opinion dynamics. The contributions of this work include (1) the formulation of an evolutionary game model to explain the behavioral traits of the honeybees, (2) the study of the individuals and collective behavior including equilibrium points and stability, (3) the extension of the results to the case of structured environment via complex network theory, (4) the analysis of the impact of the connectivity on consensus, and (5) the study of absolute stability for the collective system under time-varying and uncertain cross-inhibitory parameter.
The Classical Complexity of Boson Sampling
Venue: arXiv cs.DS 2017
Year: 2017

We study the classical complexity of the exact Boson Sampling problem where the objective is to produce provably correct random samples from a particular quantum mechanical distribution. The computational framework was proposed by Aaronson and Arkhipov in 2011 as an attainable demonstration of `quantum supremacy', that is a practical quantum computing experiment able to produce output at a speed beyond the reach of classical (that is non-quantum) computer hardware. Since its introduction Boson Sampling has been the subject of intense international research in the world of quantum computing. On the face of it, the problem is challenging for classical computation. Aaronson and Arkhipov show that exact Boson Sampling is not efficiently solvable by a classical computer unless $P^{\#P} = BPP^{NP}$ and the polynomial hierarchy collapses to the third level. The fastest known exact classical algorithm for the standard Boson Sampling problem takes $O({m + n -1 \choose n} n 2^n )$ time to produce samples for a system with input size $n$ and $m$ output modes, making it infeasible for anything but the smallest values of $n$ and $m$. We give an algorithm that is much faster, running in $O(n 2^n + \operatorname{poly}(m,n))$ time and $O(m)$ additional space. The algorithm is simple to implement and has low constant factor overheads. As a consequence our classical algorithm is able to solve the exact Boson Sampling problem for system sizes far beyond current photonic quantum computing experimentation, thereby significantly reducing the likelihood of achieving near-term quantum supremacy in the context of Boson Sampling.
Application Layer Coding for IoT: Benefits, Limitations, and Implementation Aspects
Venue: arXiv cs.NI 2017
Year: 2017

One of the key technologies for future IoT/M2M systems are low power wide area networks, which are designed to support a massive number of low-end devices often in the unlicensed shared spectrum using random access protocols. However these usually operate without centralised control and since Automatic Repeat request and acknowledgement mechanisms are not very effective due to the strict duty cycles limits and high interference in the shared bands, many packets are lost from collisions. In this paper we analyse a recently proposed application layer coding scheme, which aims to recover lost packets by introducing redundancy in the form of a fountain code. We show how latency and decoding complexity is affected by the packet loss rate but also prove that there is a limit to what can be achieved by introducing more redundancy. The analysis is backed up by simulation results.
Hybrid Beamforming with Reduced Number of Phase Shifters for Massive MIMO Systems
Venue: arXiv cs.IT 2017
Year: 2017

In this paper, two novel hybrid beamforming methods are proposed to reduce the cost and power consumption of hybrid beamformers with subconnected phase shifter network structure in massive multiple-input multiple-output (MIMO) systems. This is achieved by replacing some of the phase shifters with switches which, in general, are cheaper and have lower power consumption compared to phase shifters. The proposed methods and the closed-form expressions of their performance are derived according to the properties of the elements of the singular vectors of the channel matrix. In the first approach, it is shown that by combining the subconnected phase shifter network with a fully-connected switch architecture, the number of the phase shifters can be reduced up to 50\% while the spectral efficiency is preserved. Then, in order to simplify the structure of the switch network, the fully-connected switches is replaced by subconnected switch network, e.g. binary switches. The analytical and simulation results indicate that just by using 25% of phase shifters 90% spectral efficiency can be achieved. Finally, simulation results indicate that similar behavior is observed when the wireless channel is considered to be sparse or correlated.
Exponential Stability Analysis via Integral Quadratic Constraints
Venue: arXiv cs.SY 2017
Year: 2017

The theory of integral quadratic constraints (IQCs) allows verification of stability and gain-bound properties of systems containing nonlinear or uncertain elements. Gain bounds often imply exponential stability, but it can be challenging to compute useful numerical bounds on the exponential decay rate. This work presents a generalization of the classical IQC results of Megretski and Rantzer that leads to a tractable computational procedure for finding exponential rate certificates that are far less conservative than ones computed from $L_2$ gain bounds alone. An expanded library of IQCs for certifying exponential stability is also provided and the effectiveness of the technique is demonstrated via numerical examples.
Time-Varying Formation Controllers for Unmanned Aerial Vehicles Using Deep Reinforcement Learning
Venue: arXiv cs.RO 2017
Year: 2017

We consider the problem of designing scalable and portable controllers for unmanned aerial vehicles (UAVs) to reach time-varying formations as quickly as possible. This brief confirms that deep reinforcement learning can be used in a multi-agent fashion to drive UAVs to reach any formation while taking into account optimality and portability. We use a deep neural network to estimate how good a state is, so the agent can choose actions accordingly. The system is tested with different non-high-dimensional sensory inputs without any change in the neural network architecture, algorithm or hyperparameters, just with additional training.
Types of Cognition and its Implications for future High-Level Cognitive Machines
Venue: arXiv cs.AI 2017
Year: 2017

This work summarizes part of current knowledge on High-level Cognitive process and its relation with biological hardware. Thus, it is possible to identify some paradoxes which could impact the development of future technologies and artificial intelligence: we may make a High-level Cognitive Machine, sacrificing the principal attribute of a machine, its accuracy.
Implementation, demonstration and validation of a user-defined wall-function for direct precipitation fouling in ANSYS Fluent
Venue: arXiv cs.CE 2017
Year: 2017

In a previous paper (Johnsen et al., 2015) and presentation (Johnsen et al., 2016), we developed and demonstrated a generic modelling framework for the modelling of direct precipitation fouling from multi-component fluid mixtures that become super-saturated at the wall. The modelling concept involves the 1-dimensional transport of the fluid species through the turbulent boundary layer close to the wall. The governing equations include the Reynolds-averaged (RANS) advection-diffusion equations for each fluid species, and the axial momentum and energy equations for the fluid mixture. The driving force for the diffusive transport is the local gradient in the species' chemical potential. Adsorption mechanisms are not modelled per se, but the time-scale of adsorption is reflected in the choice of Dirichlet boundary conditions for the depositing species, at the fluid-solid interface. In this paper, the modelling framework is implemented as a user-defined function (UDF) for the CFD software ANSYS Fluent, to act as a wall boundary condition for mass-transfer to the wall. The subgrid, 1-dimensional formulation of the model reduces the computational cost associated with resolving the fine length-scales at which the boundary-layer mass transfer is determined, and allows for efficient modelling of industry-scale heat exchangers suffering from fouling. The current paper describes the modelling framework, and demonstrates and validates its applicability in a simplified 2D heat exchanger geometry (experimental and detailed CFD modelling data by Pääkkönen et al. (2012, 2016)). By tuning the diffusivity, only, good agreement with the experimental data and the detailed CFD model was obtained, in terms of area-averaged deposition rates.