OpenRaiser/Intern-Atlas huggingface dataset:
Deep Multi-instance Networks with Sparse Label Assignment for Whole Mammogram Classification
Venue: arXiv cs.CV 2016
Year: 2016

Mammogram classification is directly related to computer-aided diagnosis of breast cancer. Traditional methods requires great effort to annotate the training data by costly manual labeling and specialized computational models to detect these annotations during test. Inspired by the success of using deep convolutional features for natural image analysis and multi-instance learning for labeling a set of instances/patches, we propose end-to-end trained deep multi-instance networks for mass classification based on whole mammogram without the aforementioned costly need to annotate the training data. We explore three different schemes to construct deep multi-instance networks for whole mammogram classification. Experimental results on the INbreast dataset demonstrate the robustness of proposed deep networks compared to previous work using segmentation and detection annotations in the training.
Sample-efficient Deep Reinforcement Learning for Dialog Control
Venue: arXiv cs.AI 2016
Year: 2016

Representing a dialog policy as a recurrent neural network (RNN) is attractive because it handles partial observability, infers a latent representation of state, and can be optimized with supervised learning (SL) or reinforcement learning (RL). For RL, a policy gradient approach is natural, but is sample inefficient. In this paper, we present 3 methods for reducing the number of dialogs required to optimize an RNN-based dialog policy with RL. The key idea is to maintain a second RNN which predicts the value of the current policy, and to apply experience replay to both networks. On two tasks, these methods reduce the number of dialogs/episodes required by about a third, vs. standard policy gradient methods.
Context and Interference Effects in the Combinations of Natural Concepts
Venue: arXiv cs.AI 2016
Year: 2016

The mathematical formalism of quantum theory exhibits significant effectiveness when applied to cognitive phenomena that have resisted traditional (set theoretical) modeling. Relying on a decade of research on the operational foundations of micro-physical and conceptual entities, we present a theoretical framework for the representation of concepts and their conjunctions and disjunctions that uses the quantum formalism. This framework provides a unified solution to the 'conceptual combinations problem' of cognitive psychology, explaining the observed deviations from classical (Boolean, fuzzy set and Kolmogorovian) structures in terms of genuine quantum effects. In particular, natural concepts 'interfere' when they combine to form more complex conceptual entities, and they also exhibit a 'quantum-type context-dependence', which are responsible of the 'over- and under-extension' that are systematically observed in experiments on membership judgments.
Similarity preserving compressions of high dimensional sparse data
Venue: arXiv cs.DS 2016
Year: 2016

The rise of internet has resulted in an explosion of data consisting of millions of articles, images, songs, and videos. Most of this data is high dimensional and sparse. The need to perform an efficient search for similar objects in such high dimensional big datasets is becoming increasingly common. Even with the rapid growth in computing power, the brute-force search for such a task is impractical and at times impossible. Therefore it is quite natural to investigate the techniques that compress the dimension of the data-set while preserving the similarity between data objects. In this work, we propose an efficient compression scheme mapping binary vectors into binary vectors and simultaneously preserving Hamming distance and Inner Product. The length of our compression depends only on the sparsity and is independent of the dimension of the data. Moreover our schemes provide one-shot solution for Hamming distance and Inner Product, and work in the streaming setting as well. In contrast with the "local projection" strategies used by most of the previous schemes, our scheme combines (using sparsity) the following two strategies: $1.$ Partitioning the dimensions into several buckets, $2.$ Then obtaining "global linear summaries" in each of these buckets. We generalize our scheme for real-valued data and obtain compressions for Euclidean distance, Inner Product, and $k$-way Inner Product.
On Random Weights for Texture Generation in One Layer Neural Networks
Venue: arXiv cs.CV 2016
Year: 2016

Recent work in the literature has shown experimentally that one can use the lower layers of a trained convolutional neural network (CNN) to model natural textures. More interestingly, it has also been experimentally shown that only one layer with random filters can also model textures although with less variability. In this paper we ask the question as to why one layer CNNs with random filters are so effective in generating textures? We theoretically show that one layer convolutional architectures (without a non-linearity) paired with the an energy function used in previous literature, can in fact preserve and modulate frequency coefficients in a manner so that random weights and pretrained weights will generate the same type of images. Based on the results of this analysis we question whether similar properties hold in the case where one uses one convolution layer with a non-linearity. We show that in the case of ReLu non-linearity there are situations where only one input will give the minimum possible energy whereas in the case of no nonlinearity, there are always infinite solutions that will give the minimum possible energy. Thus we can show that in certain situations adding a ReLu non-linearity generates less variable images.
Neural Machine Translation from Simplified Translations
Venue: arXiv cs.CL 2016
Year: 2016

Text simplification aims at reducing the lexical, grammatical and structural complexity of a text while keeping the same meaning. In the context of machine translation, we introduce the idea of simplified translations in order to boost the learning ability of deep neural translation models. We conduct preliminary experiments showing that translation complexity is actually reduced in a translation of a source bi-text compared to the target reference of the bi-text while using a neural machine translation (NMT) system learned on the exact same bi-text. Based on knowledge distillation idea, we then train an NMT system using the simplified bi-text, and show that it outperforms the initial system that was built over the reference data set. Performance is further boosted when both reference and automatic translations are used to learn the network. We perform an elementary analysis of the translated corpus and report accuracy results of the proposed approach on English-to-French and English-to-German translation tasks.
Few-Shot Object Recognition from Machine-Labeled Web Images
Venue: arXiv cs.CV 2016
Year: 2016

With the tremendous advances of Convolutional Neural Networks (ConvNets) on object recognition, we can now obtain reliable enough machine-labeled annotations easily by predictions from off-the-shelf ConvNets. In this work, we present an abstraction memory based framework for few-shot learning, building upon machine-labeled image annotations. Our method takes some large-scale machine-annotated datasets (e.g., OpenImages) as an external memory bank. In the external memory bank, the information is stored in the memory slots with the form of key-value, where image feature is regarded as key and label embedding serves as value. When queried by the few-shot examples, our model selects visually similar data from the external memory bank, and writes the useful information obtained from related external data into another memory bank, i.e., abstraction memory. Long Short-Term Memory (LSTM) controllers and attention mechanisms are utilized to guarantee the data written to the abstraction memory is correlated to the query example. The abstraction memory concentrates information from the external memory bank, so that it makes the few-shot recognition effective. In the experiments, we firstly confirm that our model can learn to conduct few-shot object recognition on clean human-labeled data from ImageNet dataset. Then, we demonstrate that with our model, machine-labeled image annotations are very effective and abundant resources to perform object recognition on novel categories. Experimental results show that our proposed model with machine-labeled annotations achieves great performance, only with a gap of 1% between of the one with human-labeled annotations.
Accurate Reconstruction of Finite Rate of Innovation Signals on the Sphere
Venue: arXiv cs.IT 2016
Year: 2016

We develop a method for the accurate reconstruction of non-bandlimited finite rate of innovation signals on the sphere. For signals consisting of a finite number of Dirac functions on the sphere, we develop an annihilating filter based method for the accurate recovery of parameters of the Dirac functions using a finite number of observations of the bandlimited signal. In comparison to existing techniques, the proposed method enables more accurate reconstruction primarily due to better conditioning of systems involved in the recovery of parameters. For the recovery of $K$ Diracs on the sphere, the proposed method requires samples of the signal bandlimited in the spherical harmonic~(SH) domain at SH degree equal or greater than $ K + \sqrt{K + \frac{1}{4}} - \frac{1}{2}$. In comparison to the existing state-of-the art technique, the required bandlimit, and consequently the number of samples, of the proposed method is the same or less. We also conduct numerical experiments to demonstrate that the proposed technique is more accurate than the existing methods by a factor of $10^{7}$ or more for $2 \le K\le 20$.
The iCrawl Wizard -- Supporting Interactive Focused Crawl Specification
Venue: arXiv cs.DL 2016
Year: 2016

Collections of Web documents about specific topics are needed for many areas of current research. Focused crawling enables the creation of such collections on demand. Current focused crawlers require the user to manually specify starting points for the crawl (seed URLs). These are also used to describe the expected topic of the collection. The choice of seed URLs influences the quality of the resulting collection and requires a lot of expertise. In this demonstration we present the iCrawl Wizard, a tool that assists users in defining focused crawls efficiently and semi-automatically. Our tool uses major search engines and Social Media APIs as well as information extraction techniques to find seed URLs and a semantic description of the crawl intent. Using the iCrawl Wizard even non-expert users can create semantic specifications for focused crawlers interactively and efficiently.
A Latent-class Model for Estimating Product-choice Probabilities from Clickstream Data
Venue: arXiv cs.AI 2016
Year: 2016

This paper analyzes customer product-choice behavior based on the recency and frequency of each customer's page views on e-commerce sites. Recently, we devised an optimization model for estimating product-choice probabilities that satisfy monotonicity, convexity, and concavity constraints with respect to recency and frequency. This shape-restricted model delivered high predictive performance even when there were few training samples. However, typical e-commerce sites deal in many different varieties of products, so the predictive performance of the model can be further improved by integration of such product heterogeneity. For this purpose, we develop a novel latent-class shape-restricted model for estimating product-choice probabilities for each latent class of products. We also give a tailored expectation-maximization algorithm for parameter estimation. Computational results demonstrate that higher predictive performance is achieved with our latent-class model than with the previous shape-restricted model and common latent-class logistic regression.
Supervised Learning for Optimal Power Flow as a Real-Time Proxy
Venue: arXiv cs.LG 2016
Year: 2016

In this work we design and compare different supervised learning algorithms to compute the cost of Alternating Current Optimal Power Flow (ACOPF). The motivation for quick calculation of OPF cost outcomes stems from the growing need of algorithmic-based long-term and medium-term planning methodologies in power networks. Integrated in a multiple time-horizon coordination framework, we refer to this approximation module as a proxy for predicting short-term decision outcomes without the need of actual simulation and optimization of them. Our method enables fast approximate calculation of OPF cost with less than 1% error on average, achieved in run-times that are several orders of magnitude lower than of exact computation. Several test-cases such as IEEE-RTS96 are used to demonstrate the efficiency of our approach.
Finite-Length Analysis of Frameless ALOHA with Multi-User Detection
Venue: arXiv cs.IT 2016
Year: 2016

In this paper we present a finite-length analysis of frameless ALOHA for a k multi-user detection scenario, i.e., assuming the receiver can resolve collisions of size k or smaller. The analysis is obtained via a dynamical programming approach, and employed to optimize the scheme's performance. We also assess the optimized performance as function of k. Finally, we verify the presented results through Monte Carlo simulations.
Artificial-Noise-Aided Secure Transmission with Directional Modulation based on Random Frequency Diverse Arrays
Venue: arXiv cs.IT 2016
Year: 2016

In this paper, we propose a novel directional modulation (DM) scheme based on random frequency diverse arrays with artificial noise (RFDA-DM-AN) to enhance physical layer security of wireless communications. Specifically, we first design the RFDA-DM-AN scheme by randomly allocating frequencies to transmit antennas, thereby achieving two-dimensionally (i.e., angle and range) secure transmissions, and outperforming the state-of-the-art one-dimensional (i.e., angle) phase array (PA) based DM scheme. Then we develop the closed-form expression of a lower bound on the ergodic secrecy capacity (ESC) of our RFDA-DM-AN scheme. Based on the theoretical lower bound derived, we further optimize the transmission power allocation between the useful signal and artificial noise (AN) in order to enhance the ESC. Simulation results show that 1) our RFDA-DM-AN scheme achieves a higher secrecy capacity than that of the PA based DM scheme, 2) the lower bound derived is shown to approach the ESC as the number of transmit antennas N increases and precisely matches the ESC when N is sufficiently large, and 3) the proposed optimum power allocation achieves the highest ESC compared with other power allocations in the RFDA-DM-AN.
Inferring the location of authors from words in their texts
Venue: arXiv cs.CL 2016
Year: 2016

For the purposes of computational dialectology or other geographically bound text analysis tasks, texts must be annotated with their or their authors' location. Many texts are locatable through explicit labels but most have no explicit annotation of place. This paper describes a series of experiments to determine how positionally annotated microblog posts can be used to learn location-indicating words which then can be used to locate blog texts and their authors. A Gaussian distribution is used to model the locational qualities of words. We introduce the notion of placeness to describe how locational words are. We find that modelling word distributions to account for several locations and thus several Gaussian distributions per word, defining a filter which picks out words with high placeness based on their local distributional context, and aggregating locational information in a centroid for each text gives the most useful results. The results are applied to data in the Swedish language.
Dynamic Action Recognition: A convolutional neural network model for temporally organized joint location data
Venue: arXiv cs.CV 2016
Year: 2016

Motivation: Recognizing human actions in a video is a challenging task which has applications in various fields. Previous works in this area have either used images from a 2D or 3D camera. Few have used the idea that human actions can be easily identified by the movement of the joints in the 3D space and instead used a Recurrent Neural Network (RNN) for modeling. Convolutional neural networks (CNN) have the ability to recognise even the complex patterns in data which makes it suitable for detecting human actions. Thus, we modeled a CNN which can predict the human activity using the joint data. Furthermore, using the joint data representation has the benefit of lower dimensionality than image or video representations. This makes our model simpler and faster than the RNN models. In this study, we have developed a six layer convolutional network, which reduces each input feature vector of the form 15x1961x4 to an one dimensional binary vector which gives us the predicted activity. Results: Our model is able to recognise an activity correctly upto 87% accuracy. Joint data is taken from the Cornell Activity Datasets which have day to day activities like talking, relaxing, eating, cooking etc.
Local Sparse Approximation for Image Restoration with Adaptive Block Size Selection
Venue: arXiv cs.CV 2016
Year: 2016

In this paper the problem of image restoration (denoising and inpainting) is approached using sparse approximation of local image blocks. The local image blocks are extracted by sliding square windows over the image. An adaptive block size selection procedure for local sparse approximation is proposed, which affects the global recovery of underlying image. Ideally the adaptive local block selection yields the minimum mean square error (MMSE) in recovered image. This framework gives us a clustered image based on the selected block size, then each cluster is restored separately using sparse approximation. The results obtained using the proposed framework are very much comparable with the recently proposed image restoration techniques.
NOP - A Simple Experimental Processor for Parallel Deployment
Venue: arXiv cs.DC 2016
Year: 2016

The design of a parallel computing system using several thousands or even up to a million processors asks for processing units that are simple and thus small in space, to make as many processing units as possible fit on a single die. The design presented herewith is far from being optimised, it is not meant to compete with industry performance devices. Its main purpose is to allow for a prototypical implementation of a dynamic software system as a proof of concept.
The Matrix Exponential Distribution - A Tool for Wireless System Performance Analysis
Venue: arXiv cs.IT 2016
Year: 2016

In [1], we introduced a new, matrix algebraic, performance analysis framework for wireless systems with fading channels based on the matrix exponential distribution. The main idea was to use the compact, powerful, and easy-to-use, matrix exponential (ME)-distribution for i) modeling the unprocessed channel signal to noise ratio (SNR), ii) exploiting the closure property of the ME-distribution for SNR processing operations to give the effective channel random variable (r.v.) on ME-distribution form, and then to iii) express the performance measure in a closed-form based on ME-distribution matrix/vector parameters only. In this work, we aim to more clearly present, formalize, refine and develop this unified bottom-up analysis framework, show its versatility to handle important communication cases, performance evaluation levels, and performance metrics. The bivariate ME-distribution is introduced here as yet another useful ME-tool, e.g. to account for dependency among two r.v.s. We propose that the ME-distribution may, in addition to fading, also characterize the pdf of discrete-time signal r.v.s, thus extending the ME-distribution matrix form to new generalized 1D/2D-Gaussian-, and Rayleigh-, distribution-like matrix forms. Our findings here, strengthen the observation from [1], [2], and indicates that the ME-distribution can be a promising tool for wireless system modeling and performance analysis.
CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
Venue: arXiv cs.CV 2016
Year: 2016

When building artificial intelligence systems that can reason and answer questions about visual data, we need diagnostic tests to analyze our progress and discover shortcomings. Existing benchmarks for visual question answering can help, but have strong biases that models can exploit to correctly answer questions without reasoning. They also conflate multiple sources of error, making it hard to pinpoint model weaknesses. We present a diagnostic dataset that tests a range of visual reasoning abilities. It contains minimal biases and has detailed annotations describing the kind of reasoning each question requires. We use this dataset to analyze a variety of modern visual reasoning systems, providing novel insights into their abilities and limitations.
Joint Transceiver and Offset Design for Visible Light Communications with Input-dependent Shot Noise
Venue: arXiv cs.IT 2016
Year: 2016

In this paper, we investigate the problem of the joint transceiver and offset design (JTOD) for point-to-point multiple-input-multiple-output (MIMO) and multiple user multiple-input-single-output (MU-MISO) visible light communication (VLC) systems. Both uplink and downlink multi-user scenarios are considered. The shot noise induced by the incoming signals is considered, leading to a more realistic MIMO VLC channel model. Under key lighting constraints, we formulate non-convex optimization problems aiming at minimizing the sum mean squared error. To optimize the transceiver and the offset jointly, a gradient projection based procedure is resorted to. When only imperfect channel state information is available, a semidefinite programming (SDP) based scheme is proposed to obtain robust transceiver and offset. The proposed method is shown to non-trivially outperform the conventional scaled zero forcing (ZF) and singular value decomposition (SVD) based equalization methods. The robust scheme works particularly well when the signal is much stronger than the noise.
ARES: Adaptive Receding-Horizon Synthesis of Optimal Plans
Venue: arXiv cs.AI 2016
Year: 2016

We introduce ARES, an efficient approximation algorithm for generating optimal plans (action sequences) that take an initial state of a Markov Decision Process (MDP) to a state whose cost is below a specified (convergence) threshold. ARES uses Particle Swarm Optimization, with adaptive sizing for both the receding horizon and the particle swarm. Inspired by Importance Splitting, the length of the horizon and the number of particles are chosen such that at least one particle reaches a next-level state, that is, a state where the cost decreases by a required delta from the previous-level state. The level relation on states and the plans constructed by ARES implicitly define a Lyapunov function and an optimal policy, respectively, both of which could be explicitly generated by applying ARES to all states of the MDP, up to some topological equivalence relation. We also assess the effectiveness of ARES by statistically evaluating its rate of success in generating optimal plans. The ARES algorithm resulted from our desire to clarify if flying in V-formation is a flocking policy that optimizes energy conservation, clear view, and velocity alignment. That is, we were interested to see if one could find optimal plans that bring a flock from an arbitrary initial state to a state exhibiting a single connected V-formation. For flocks with 7 birds, ARES is able to generate a plan that leads to a V-formation in 95% of the 8,000 random initial configurations within 63 seconds, on average. ARES can also be easily customized into a model-predictive controller (MPC) with an adaptive receding horizon and statistical guarantees of convergence. To the best of our knowledge, our adaptive-sizing approach is the first to provide convergence guarantees in receding-horizon techniques.