OpenRaiser/Intern-Atlas huggingface dataset:
Limited Feedback Channel Estimation in Massive MIMO with Non-uniform Directional Dictionaries
Venue: arXiv cs.IT 2018
Year: 2018

Channel state information (CSI) at the base station (BS) is crucial to achieve beamforming and multiplexing gains in multiple-input multiple-output (MIMO) systems. State-of-the-art limited feedback schemes require feedback overhead that scales linearly with the number of BS antennas, which is prohibitive for $5$G massive MIMO. This work proposes novel limited feedback algorithms that lift this burden by exploiting the inherent sparsity in double directional (DD) MIMO channel representation using overcomplete dictionaries. These dictionaries are associated with angle of arrival (AoA) and angle of departure (AoD) that specifically account for antenna directivity patterns at both ends of the link. The proposed algorithms achieve satisfactory channel estimation accuracy using a small number of feedback bits, even when the number of transmit antennas at the BS is large -- making them ideal for $5$G massive MIMO. Judicious simulations reveal that they outperform a number of popular feedback schemes, and underscore the importance of using angle dictionaries matching the given antenna directivity patterns, as opposed to uniform dictionaries. The proposed algorithms are lightweight in terms of computation, especially on the user equipment side, making them ideal for actual deployment in $5$G systems.
Blind Identification of Fully Observed Linear Time-Varying Systems via Sparse Recovery
Venue: arXiv cs.SY 2018
Year: 2018

Discrete-time linear time-varying (LTV) systems form a powerful class of models to approximate complex dynamical systems with nonlinear dynamics for the purpose of analysis, design and control. Motivated by inference of spatio-temporal dynamics in breast cancer research, we propose a method to efficiently solve an identification problem for a specific class of discrete-time LTV systems, in which the states are fully observed and there is no access to system inputs. In addition, it is assumed that we do not know on which states the inputs act, which can change between time steps, and that the total number of inputs is sparse over all states and over time. The problem is formulated as a compressive sensing problem, which incorporates the effect of measurement noise and which has a solution with a partially sparse support. We derive sufficient conditions for the unique recovery of the system model and input values, which lead to practical conditions on the number of experiments and rank conditions on system outputs. Synthetic experiments analyze the method's sensitivity to noise for randomly generated models.
Scalable Multi-Domain Dialogue State Tracking
Venue: arXiv cs.CL 2018
Year: 2018

Dialogue state tracking (DST) is a key component of task-oriented dialogue systems. DST estimates the user's goal at each user turn given the interaction until then. State of the art approaches for state tracking rely on deep learning methods, and represent dialogue state as a distribution over all possible slot values for each slot present in the ontology. Such a representation is not scalable when the set of possible values are unbounded (e.g., date, time or location) or dynamic (e.g., movies or usernames). Furthermore, training of such models requires labeled data, where each user turn is annotated with the dialogue state, which makes building models for new domains challenging. In this paper, we present a scalable multi-domain deep learning based approach for DST. We introduce a novel framework for state tracking which is independent of the slot value set, and represent the dialogue state as a distribution over a set of values of interest (candidate set) derived from the dialogue history or knowledge. Restricting these candidate sets to be bounded in size addresses the problem of slot-scalability. Furthermore, by leveraging the slot-independent architecture and transfer learning, we show that our proposed approach facilitates quick adaptation to new domains.
Theory of Deep Learning III: explaining the non-overfitting puzzle
Venue: arXiv cs.LG 2018
Year: 2018

A main puzzle of deep networks revolves around the absence of overfitting despite large overparametrization and despite the large capacity demonstrated by zero training error on randomly labeled data. In this note, we show that the dynamics associated to gradient descent minimization of nonlinear networks is topologically equivalent, near the asymptotically stable minima of the empirical error, to linear gradient system in a quadratic potential with a degenerate (for square loss) or almost degenerate (for logistic or crossentropy loss) Hessian. The proposition depends on the qualitative theory of dynamical systems and is supported by numerical results. Our main propositions extend to deep nonlinear networks two properties of gradient descent for linear networks, that have been recently established (1) to be key to their generalization properties: 1. Gradient descent enforces a form of implicit regularization controlled by the number of iterations, and asymptotically converges to the minimum norm solution for appropriate initial conditions of gradient descent. This implies that there is usually an optimum early stopping that avoids overfitting of the loss. This property, valid for the square loss and many other loss functions, is relevant especially for regression. 2. For classification, the asymptotic convergence to the minimum norm solution implies convergence to the maximum margin solution which guarantees good classification error for "low noise" datasets. This property holds for loss functions such as the logistic and cross-entropy loss independently of the initial conditions. The robustness to overparametrization has suggestive implications for the robustness of the architecture of deep convolutional networks with respect to the curse of dimensionality.
Using Deep Neural Network Approximate Bayesian Network
Venue: arXiv cs.LG 2018
Year: 2018

We present a new method to approximate posterior probabilities of Bayesian Network using Deep Neural Network. Experiment results on several public Bayesian Network datasets shows that Deep Neural Network is capable of learning joint probability distri- bution of Bayesian Network by learning from a few observation and posterior probability distribution pairs with high accuracy. Compared with traditional approximate method likelihood weighting sampling algorithm, our method is much faster and gains higher accuracy in medium sized Bayesian Network. Another advantage of our method is that our method can be parallelled much easier in GPU without extra effort. We also ex- plored the connection between the accuracy of our model and the number of training examples. The result shows that our model saturate as the number of training examples grow and we don't need many training examples to get reasonably good result. Another contribution of our work is that we have shown discriminative model like Deep Neural Network can approximate generative model like Bayesian Network.
Users Constraints in Itemset Mining
Venue: arXiv cs.AI 2018
Year: 2018

Discovering significant itemsets is one of the fundamental problems in data mining. It has recently been shown that constraint programming is a flexible way to tackle data mining tasks. With a constraint programming approach, we can easily express and efficiently answer queries with users constraints on items. However, in many practical cases it is possible that queries also express users constraints on the dataset itself. For instance, asking for a particular itemset in a particular part of the dataset. This paper presents a general constraint programming model able to handle any kind of query on the items or the dataset for itemset mining.
Time Aware Least Recent Used (TLRU) Cache Management Policy in ICN
Venue: arXiv cs.NI 2018
Year: 2018

The information centric networks (ICN) can be viewed as a network of caches. Conversely, ICN type of cache networks has distinctive features e.g, contents popularity, usability time of content and other factors inflicts some diverse requirements for cache eviction policies. In this paper we defined four important characteristics of a suitable eviction policy for ICN. We analysed well known eviction policies in view of defined characteristics. Based upon analysis we propose a new eviction scheme which is well suitable for ICN type of cache networks.
Analysis and Code Design for the Binary CEO Problem under Logarithmic Loss
Venue: arXiv cs.IT 2018
Year: 2018

In this paper, we propose an efficient coding scheme for the binary Chief Executive Officer (CEO) problem under logarithmic loss criterion. Courtade and Weissman obtained the exact rate-distortion bound for a two-link binary CEO problem under this criterion. We find the optimal test-channel model and its parameters for the encoder of each link by using the given bound. Furthermore, an efficient encoding scheme based on compound LDGM-LDPC codes is presented to achieve the theoretical rates. In the proposed encoding scheme, a binary quantizer using LDGM codes and a syndrome-decoding employing LDPC codes are applied. An iterative decoding is also presented as a fusion center to reconstruct the observation bits. The proposed decoder consists of a sum-product algorithm with a side information from other decoder and a soft estimator. The output of the CEO decoder is the probability of source bits conditional to the received sequences of both links. This method outperforms the majority-based estimation of the source bits utilized in the prior studies of the binary CEO problem. Our numerical examples verify a close performance of the proposed coding scheme to the theoretical bound in several cases.
Automated rating of recorded classroom presentations using speech analysis in kazakh
Venue: arXiv cs.CL 2018
Year: 2018

Effective presentation skills can help to succeed in business, career and academy. This paper presents the design of speech assessment during the oral presentation and the algorithm for speech evaluation based on criteria of optimal intonation. As the pace of the speech and its optimal intonation varies from language to language, developing an automatic identification of language during the presentation is required. Proposed algorithm was tested with presentations delivered in Kazakh language. For testing purposes the features of Kazakh phonemes were extracted using MFCC and PLP methods and created a Hidden Markov Model (HMM) [5], [5] of Kazakh phonemes. Kazakh vowel formants were defined and the correlation between the deviation rate in fundamental frequency and the liveliness of the speech to evaluate intonation of the presentation was analyzed. It was established that the threshold value between monotone and dynamic speech is 0.16 and the error for intonation evaluation is 19%.
Automated image segmentation for detecting cell spreading for metastasizing assessments of cancer development
Venue: arXiv cs.CV 2018
Year: 2018

The automated segmentation of cells in microscopic images is an open research problem that has important implications for studies of the developmental and cancer processes based on in vitro models. In this paper, we present the approach for segmentation of the DIC images of cultured cells using G-neighbor smoothing followed by Kauwahara filtering and local standard deviation approach for boundary detection. NIH FIJI/ImageJ tools are used to create the ground truth dataset. The results of this work indicate that detection of cell boundaries using segmentation approach even in the case of realistic measurement conditions is a challenging problem.
Performance Limits with Additive Error Metrics in Noisy Multi-Measurement Vector Problem
Venue: arXiv cs.IT 2018
Year: 2018

Real-world applications such as magnetic resonance imaging with multiple coils, multi-user communication, and diffuse optical tomography often assume a linear model where several sparse signals sharing common sparse supports are acquired by several measurement matrices and then contaminated by noise. Multi-measurement vector (MMV) problems consider the estimation or reconstruction of such signals. In different applications, the estimation error that we want to minimize could be the mean squared error or other metrics such as the mean absolute error and the support set error. Seeing that minimizing different error metrics is useful in MMV problems, we study information-theoretic performance limits for MMV signal estimation with arbitrary additive error metrics. We also propose a message passing algorithmic framework that achieves the optimal performance, and rigorously prove the optimality of our algorithm for a special case. We further conjecture the optimality of our algorithm for some general cases, and back it up through numerical examples. As an application of our MMV algorithm, we propose a novel setup for active user detection in multi-user communication and demonstrate the promise of our proposed setup.
Accurate reconstruction of image stimuli from human fMRI based on the decoding model with capsule network architecture
Venue: arXiv cs.CV 2018
Year: 2018

In neuroscience, all kinds of computation models were designed to answer the open question of how sensory stimuli are encoded by neurons and conversely, how sensory stimuli can be decoded from neuronal activities. Especially, functional Magnetic Resonance Imaging (fMRI) studies have made many great achievements with the rapid development of the deep network computation. However, comparing with the goal of decoding orientation, position and object category from activities in visual cortex, accurate reconstruction of image stimuli from human fMRI is a still challenging work. In this paper, the capsule network (CapsNet) architecture based visual reconstruction (CNAVR) method is developed to reconstruct image stimuli. The capsule means containing a group of neurons to perform the better organization of feature structure and representation, inspired by the structure of cortical mini column including several hundred neurons in primates. The high-level capsule features in the CapsNet includes diverse features of image stimuli such as semantic class, orientation, location and so on. We used these features to bridge between human fMRI and image stimuli. We firstly employed the CapsNet to train the nonlinear mapping from image stimuli to high-level capsule features, and from high-level capsule features to image stimuli again in an end-to-end manner. After estimating the serviceability of each voxel by encoding performance to accomplish the selecting of voxels, we secondly trained the nonlinear mapping from dimension-decreasing fMRI data to high-level capsule features. Finally, we can predict the high-level capsule features with fMRI data, and reconstruct image stimuli with the CapsNet. We evaluated the proposed CNAVR method on the dataset of handwritten digital images, and exceeded about 10% than the accuracy of all existing state-of-the-art methods on the structural similarity index (SSIM).
Scene-Adapted Plug-and-Play Algorithm with Guaranteed Convergence: Applications to Data Fusion in Imaging
Venue: arXiv cs.CV 2018
Year: 2018

The recently proposed plug-and-play (PnP) framework allows leveraging recent developments in image denoising to tackle other, more involved, imaging inverse problems. In a PnP method, a black-box denoiser is plugged into an iterative algorithm, taking the place of a formal denoising step that corresponds to the proximity operator of some convex regularizer. While this approach offers flexibility and excellent performance, convergence of the resulting algorithm may be hard to analyze, as most state-of-the-art denoisers lack an explicit underlying objective function. In this paper, we propose a PnP approach where a scene-adapted prior (i.e., where the denoiser is targeted to the specific scene being imaged) is plugged into ADMM (alternating direction method of multipliers), and prove convergence of the resulting algorithm. Finally, we apply the proposed framework in two different imaging inverse problems: hyperspectral sharpening/fusion and image deblurring from blurred/noisy image pairs.
Interactive Decomposition Multi-Objective Optimization via Progressively Learned Value Functions
Venue: arXiv cs.NE 2018
Year: 2018

Decomposition has become an increasingly popular technique for evolutionary multi-objective optimization (EMO). A decomposition-based EMO algorithm is usually designed to approximate a whole Pareto-optimal front (PF). However, in practice, the decision maker (DM) might only be interested in her/his region of interest (ROI), i.e., a part of the PF. Solutions outside that might be useless or even noisy to the decision-making procedure. Furthermore, there is no guarantee to find the preferred solutions when tackling many-objective problems. This paper develops an interactive framework for the decomposition-based EMO algorithm to lead a DM to the preferred solutions of her/his choice. It consists of three modules, i.e., consultation, preference elicitation and optimization. Specifically, after every several generations, the DM is asked to score a few candidate solutions in a consultation session. Thereafter, an approximated value function, which models the DM's preference information, is progressively learned from the DM's behavior. In the preference elicitation session, the preference information learned in the consultation module is translated into the form that can be used in a decomposition-based EMO algorithm, i.e., a set of reference points that are biased toward to the ROI. The optimization module, which can be any decomposition-based EMO algorithm in principle, utilizes the biased reference points to direct its search process. Extensive experiments on benchmark problems with three to ten objectives fully demonstrate the effectiveness of our proposed method for finding the DM's preferred solutions.
High Dimensional Spaces, Deep Learning and Adversarial Examples
Venue: arXiv cs.CV 2018
Year: 2018

In this paper, we analyze deep learning from a mathematical point of view and derive several novel results. The results are based on intriguing mathematical properties of high dimensional spaces. We first look at perturbation based adversarial examples and show how they can be understood using topological and geometrical arguments in high dimensions. We point out mistake in an argument presented in prior published literature, and we present a more rigorous, general and correct mathematical result to explain adversarial examples in terms of topology of image manifolds. Second, we look at optimization landscapes of deep neural networks and examine the number of saddle points relative to that of local minima. Third, we show how multiresolution nature of images explains perturbation based adversarial examples in form of a stronger result. Our results state that expectation of $L_2$-norm of adversarial perturbations is $O\left(\frac{1}{\sqrt{n}}\right)$ and therefore shrinks to 0 as image resolution $n$ becomes arbitrarily large. Finally, by incorporating the parts-whole manifold learning hypothesis for natural images, we investigate the working of deep neural networks and root causes of adversarial examples and discuss how future improvements can be made and how adversarial examples can be eliminated.
The New Threats of Information Hiding: the Road Ahead
Venue: arXiv cs.CR 2018
Year: 2018

Compared to cryptography, steganography is a less discussed domain. However, there is a recent trend of exploiting various information hiding techniques to empower malware, for instance to bypass security frameworks of mobile devices or to exfiltrate sensitive data. This is mostly due to the need to counteract increasingly sophisticated security mechanisms, such as code analysis, runtime countermeasures, or real-time traffic inspection tools. In this perspective, this paper presents malware exploiting information hiding in a broad sense, i.e., it does not focus on classical covert channels, but also discusses other camouflage techniques. Differently from other works, this paper solely focuses on real-world threats observed in the 2011 - 2017 timeframe. The observation indicates a growing number of malware equipped with some form of data hiding capabilities and a lack of effective and universal countermeasures.
Deep Learning for Identifying Potential Conceptual Shifts for Co-creative Drawing
Venue: arXiv cs.LG 2018
Year: 2018

We present a system for identifying conceptual shifts between visual categories, which will form the basis for a co-creative drawing system to help users draw more creative sketches. The system recognizes human sketches and matches them to structurally similar sketches from categories to which they do not belong. This would allow a co-creative drawing system to produce an ambiguous sketch that blends features from both categories.
Bridging the Gap Between Neural Networks and Neuromorphic Hardware with A Neural Network Compiler
Venue: arXiv cs.NE 2018
Year: 2018

Different from developing neural networks (NNs) for general-purpose processors, the development for NN chips usually faces with some hardware-specific restrictions, such as limited precision of network signals and parameters, constrained computation scale, and limited types of non-linear functions. This paper proposes a general methodology to address the challenges. We decouple the NN applications from the target hardware by introducing a compiler that can transform an existing trained, unrestricted NN into an equivalent network that meets the given hardware's constraints. We propose multiple techniques to make the transformation adaptable to different kinds of NN chips, and reliable for restrict hardware constraints. We have built such a software tool that supports both spiking neural networks (SNNs) and traditional artificial neural networks (ANNs). We have demonstrated its effectiveness with a fabricated neuromorphic chip and a processing-in-memory (PIM) design. Tests show that the inference error caused by this solution is insignificant and the transformation time is much shorter than the retraining time. Also, we have studied the parameter-sensitivity evaluations to explore the tradeoffs between network error and resource utilization for different transformation strategies, which could provide insights for co-design optimization of neuromorphic hardware and software.
Secretary problem: graphs, matroids and greedoids
Venue: arXiv cs.DM 2018
Year: 2018

In the paper the generalisation of the well known "secretary problem" is considered. The aim of the paper is to give a generalised model in such a way that the chosen set of the possible best $k$ elements have to be independent of all rejected elements. This condition is formulated using the theory of greedoids and in their special cases -- matroids and antimatroids. Examples of some special cases of greedoids (uniform, graphical matroids and binary trees) are considered.
Circuit Complexity of Bounded Planar Cutwidth Graph Matching
Venue: arXiv cs.CC 2018
Year: 2018

Recently, perfect matching in bounded planar cutwidth bipartite graphs (\BGGM) was shown to be in ACC$^0$ by Hansen et al.. They also conjectured that the problem is in AC$^0$. In this paper, we disprove their conjecture by showing that the problem is not in AC$^0[p^α]$ for every prime $p$. Our results show that the previous upper bound is almost tight. Our techniques involve giving a reduction from Parity to BGGM. A further improvement in lower bounds is difficult since we do not have an algebraic characterization for AC$^0[m]$ where $m$ is not a prime power. Moreover, this will also imply a separation of AC$^0[m]$ from P. Our results also imply a better lower bound for perfect matching in general bounded planar cutwidth graphs.