niryuu/sni-each-converted huggingface dataset:
Interlingua based Machine Translation (MT) aims to encode multiple languages into a common linguistic representation and then decode sentences in multiple target languages from this representation. In this work we explore this idea in the context of neural encoder decoder architectures, albeit on a smaller scale and without MT as the end goal. Specifically, we consider the case of three languages or modalities X , Z and Y wherein we are interested in generating sequences in Y starting from information available in X . However, there is no parallel training data available between X and Y but, training data is available between X & Z and Z & Y (as is often the case in many real world applications). Z thus acts as a pivot/bridge. An obvious solution, which is perhaps less elegant but works very well in practice is to train a two stage model which first converts from X to Z and then from Z to Y . Instead we explore an interlingua inspired solution which jointly learns to do the following (i) encodeX and Z to a common representation and (ii) decode Y from this common representation. We evaluate our model on two tasks: (i) bridge transliteration and (ii) bridge captioning. We report promising results in both these applications and believe that this is a right step towards truly interlingua inspired encoder decoder architectures.
Output: A Correlational Encoder Decoder Architecture for Pivot Based Sequence Generation
Argument Component Boundary Detection (ACBD) is an important sub-task in argumentation mining; it aims at identifying the word sequences that constitute argument components, and is usually considered as the first sub-task in the argumentation mining pipeline. Existing ACBD methods heavily depend on task-specific knowledge, and require considerable human efforts on feature-engineering. To tackle these problems, in this work, we formulate ACBD as a sequence labeling problem and propose a variety of Recurrent Neural Network (RNN) based methods, which do not use domain specific or handcrafted features beyond the relative position of the sentence in the document. In particular, we propose a novel joint RNN model that can predict whether sentences are argumentative or not, and use the predicted results to more precisely detect the argument component boundaries. We evaluate our techniques on two corpora from two different genres; results suggest that our joint RNN model obtain the state-of-the-art performance on both datasets.
Output: Joint RNN Model for Argument Component Boundary Detection
We introduce kLog, a novel approach to statistical relational learning. Unlike standard approaches, kLog does not represent a probability distribution directly. It is rather a language to perform kernel-based learning on expressive logical and relational representations. kLog allows users to specify learning problems declaratively. It builds on simple but powerful concepts: learning from interpretations, entity/relationship data modeling, logic programming, and deductive databases. Access by the kernel to the rich representation is mediated by a technique we call graphicalization: the relational representation is first transformed into a graph — in particular, a grounded entity/relationship diagram. Subsequently, a choice of graph kernel defines the feature space. kLog supports mixed numerical and symbolic data, as well as background knowledge in the form of Prolog or Datalog programs as in inductive logic programming systems. The kLog framework can be applied to tackle the same range of tasks that has made statistical relational learning so popular, including classification, regression, multitask learning, and collective classification. We also report about empirical comparisons, showing PF was a visiting professor at KU Leuven and FC a postdoctoral fellow at KU Leuven while this work was initiated ∗Corresponding author Email addresses: p-f@dsi.unifi.it (Paolo Frasconi), costa@informatik.uni-freiburg.de (Fabrizio Costa), Luc.DeRaedt@cs.kuleuven.be (Luc De Raedt), Kurt.DeGrave@cs.kuleuven.be (Kurt De Grave) Preprint submitted to Artificial Intelligence July 29, 2014 ar X iv :1 20 5. 39 81 v5 [ cs .A I] 2 8 Ju l 2 01 4 that kLog can be either more accurate, or much faster at the same level of accuracy, than Tilde and Alchemy. kLog is GPLv3 licensed and is available at http://klog.dinfo.unifi.it along with tutorials.
Output: kLog: A Language for Logical and Relational Learning with Kernels
In this work, we propose CLass-Enhanced Attentive Response (CLEAR): an approach to visualize and understand the decisions made by deep neural networks (DNNs) given a specific input. CLEAR facilitates the visualization of attentive regions and levels of interest of DNNs during the decision-making process. It also enables the visualization of the most dominant classes associated with these attentive regions of interest. As such, CLEAR can mitigate some of the shortcomings of heatmap-based methods associated with decision ambiguity, and allows for better insights into the decision-making process of DNNs. Quantitative and qualitative experiments across three different datasets demonstrate the efficacy of CLEAR for gaining a better understanding of the inner workings of DNNs during the decision-making process.
Output: Explaining the Unexplained: A CLass-Enhanced Attentive Response (CLEAR) Approach to Understanding Deep Neural Networks
Lexical features are a major source of information in state-of-the-art coreference resolvers. Lexical features implicitly model some of the linguistic phenomena at a fine granularity level. They are especially useful for representing the context of mentions. In this paper we investigate a drawback of using many lexical features in state-of-the-art coreference resolvers. We show that if coreference resolvers mainly rely on lexical features, they can hardly generalize to unseen domains. Furthermore, we show that the current coreference resolution evaluation is clearly flawed by only evaluating on a specific split of a specific dataset in which there is a notable overlap between the training, development and test sets.
Output: Lexical Features in Coreference Resolution: To be Used With Caution
This paper proposes a deep denoising auto-encoder technique to extract better acoustic features for speech synthesis. The technique allows us to automatically extract low-dimensional features from high dimensional spectral features in a non-linear, data-driven, unsupervised way. We compared the new stochastic feature extractor with conventional mel-cepstral analysis in analysis-by-synthesis and text-to-speech experiments. Our results confirm that the proposed method increases the quality of synthetic speech in both experiments.
Output: DEEP DENOISING AUTO-ENCODER FOR STATISTICAL SPEECH SYNTHESIS
This paper proposes a novel approach for relation extraction from free text which is trained to jointly use information from the text and from existing knowledge. Our model is based on two scoring functions that operate by learning low-dimensional embeddings of words and of entities and relationships from a knowledge base. We empirically show on New York Times articles aligned with Freebase relations that our approach is able to efficiently use the extra information provided by a large subset of Freebase data (4M entities, 23k relationships) to improve over existing methods that rely on text features alone.
Output: Connecting Language and Knowledge Bases with Embedding Models for Relation Extraction
The aim of this paper is to show the interest in fitting features with an α-stable distribution to classify imperfect data. The supervised pattern recognition is thus based on the theory of continuous belief functions, which is a way to consider imprecision and uncertainty of data. The distributions of features are supposed to be unimodal and estimated by a single Gaussian and α-stable model. Experimental results are first obtained from synthetic data by combining two features of one dimension and by considering a vector of two features. Mass functions are calculated from plausibility functions by using the generalized Bayes theorem. The same study is applied to the automatic classification of three types of sea floor (rock, silt and sand) with features acquired by a mono-beam echo-sounder. We evaluate the quality of the α-stable model and the Gaussian model by analyzing qualitative results, using a Kolmogorov-Smirnov test (K-S test), and quantitative results with classification rates. The performances of the belief classifier are compared with a Bayesian approach.
Output: Features modeling with an α-stable distribution: application to pattern recognition based on continuous belief functions
Nonlinear independent component analysis (ICA) provides an appealing framework<lb>for unsupervised feature learning, but the models proposed so far are not identifiable.<lb>Here, we first propose a new intuitive principle of unsupervised deep learning<lb>from time series which uses the nonstationary structure of the data. Our learning<lb>principle, time-contrastive learning (TCL), finds a representation which allows<lb>optimal discrimination of time segments (windows). Surprisingly, we show how<lb>TCL can be related to a nonlinear ICA model, when ICA is redefined to include<lb>temporal nonstationarities. In particular, we show that TCL combined with linear<lb>ICA estimates the nonlinear ICA model up to point-wise transformations of the<lb>sources, and this solution is unique — thus providing the first identifiability result<lb>for nonlinear ICA which is rigorous, constructive, as well as very general.
Output: Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA
In this paper, we improve the attention or alignment accuracy of neural machine translation by utilizing the alignments of training sentence pairs. We simply compute the distance between the machine attentions and the “true” alignments, and minimize this cost in the training procedure. Our experiments on large-scale Chinese-to-English task show that our model improves both translation and alignment qualities significantly over the large-vocabulary neural machine translation system, and even beats a state-of-the-art traditional syntax-based system.
Output: Supervised Attentions for Neural Machine Translation
In current perception systems applied to the rebuilding of the environment for intelligent vehicles, the part reserved to object association for the tracking is increasingly significant. This allows firstly to follow the objects temporal evolution and secondly to increase the reliability of environment perception. We propose in this communication the development of a multi­ objects association algorithm with ambiguity removal entering into the design of such a dynamic perception system for intelligent vehicles. This algorithm uses the belief theory and data modelling with fuzzy mathematics in order to be able to handle inaccurate as well as uncertain information due to imperfect sensors. These theories also allow the fusion of numerical as well as symbolic data. We develop in this article the problem of matching between known and perceived objects. This makes it possible to update a dynamic environment map for a vehicle. The belief theory will enable us to quantify the belief in the association of each perceived object with each known object. Conflicts can appear in the case of object appearance or disappearance, or in the case of a confused situation or bad perception. These conflicts are removed or solved using an assignment algorithm, giving a solution called the « best » and so ensuring the tracking of some objects present in our environment.
Output: Multi-objects association in perception of dynamical situation
In this article we present two ways of struc­ turing bodies of evidence, which allow us to reduce the complexity of the operations usu­ ally performed in the framework of evidence theory. The first structure just partitions the focal elements in a body of evidence by their cardinality. With this structure we are able to reduce the complexity on the calculation of the belief functions Bel, PI, and Q. The other structure proposed here, the Hierarchi­ cal Trees, permits us to reduce the complex­ ity of the calculation of Bel, PI, and Q, as well as of the Dempster's rule of combination in relation to the brute-force algorithm. Both these structures do not require the generation of all the subsets of the reference domain.
Output: Structuring Bodies of Evidence
The goal of constraint-based sequence mining is to find sequences of symbols that are included in a large number of input sequences and that satisfy some constraints specified by the user. Many constraints have been proposed in the literature, but a general framework is still missing. We investigate the use of constraint programming as general framework for this task. We first identify four categories of constraints that are applicable to sequence mining. We then propose two constraint programming formulations. The first formulation introduces a new global constraint called exists-embedding. This formulation is the most efficient but does not support one type of constraint. To support such constraints, we develop a second formulation that is more general but incurs more overhead. Both formulations can use the projected database technique used in specialised algorithms. Experiments demonstrate the flexibility towards constraint-based settings and compare the approach to existing methods.
Output: Constraint-based sequence mining using constraint programming
<lb>This work considers the problem of learning the structure of multivariate linear tree models, which<lb>include a variety of directed tree graphical models with continuous, discrete, and mixed latent variables<lb>such as linear-Gaussian models, hidden Markov models, Gaussian mixture models, and Markov evolu-<lb>tionary trees. The setting is one where we only have samples from certain observed variables in the tree,<lb>and our goal is to estimate the tree structure (i.e., the graph of how the underlying hidden variables are<lb>connected to each other and to the observed variables). We propose the Spectral Recursive Grouping al-<lb>gorithm, an efficient and simple bottom-up procedure for recovering the tree structure from independent<lb>samples of the observed variables. Our finite sample size bounds for exact recovery of the tree structure<lb>reveal certain natural dependencies on underlying statistical and structural properties of the underlying<lb>joint distribution. Furthermore, our sample complexity guarantees have no explicit dependence on the<lb>dimensionality of the observed variables, making the algorithm applicable to many high-dimensional set-<lb>tings. At the heart of our algorithm is a spectral quartet test for determining the relative topology of a<lb>quartet of variables from second-order statistics.
Output: Spectral Methods for Learning Multivariate Latent Tree Structure
We propose a new framework for abstractive text summarization based on a sequence-to-sequence oriented encoderdecoder model equipped with a deep recurrent generative decoder (DRGN). Latent structure information implied in the target summaries is learned based on a recurrent latent random model for improving the summarization quality. Neural variational inference is employed to address the intractable posterior inference for the recurrent latent variables. Abstractive summaries are generated based on both the generative latent variables and the discriminative deterministic states. Extensive experiments on some benchmark datasets in different languages show that DRGN achieves improvements over the state-ofthe-art methods.
Output: Deep Recurrent Generative Decoder for Abstractive Text Summarization∗
Joint matching over a collection of objects aims at aggregating information from a large collection of similar instances (e.g. images, graphs, shapes) to improve maps between pairs of them. Given multiple objects and matches computed between a few object pairs in isolation, the goal is to recover an entire collection of maps that are (1) globally consistent, and (2) close to the provided maps — and under certain conditions provably the ground-truth maps. Despite recent advances on this problem, the best-known recovery guarantees are limited to a small constant barrier — none of the existing methods find theoretical support when more than 50% of input correspondences are corrupted. Moreover, prior approaches focus mostly on fully similar objects, while it is practically more demanding to match instances that are only partially similar to each other (e.g., different views of a single physical object). In this paper, we propose an algorithm to jointly match multiple objects that exhibit only partial similarities, given a few (possibly highly incomplete) pairwise matches that are densely corrupted. By encoding a consistent partial map collection into a 0-1 semidefinite matrix, we propose to recover the ground-truth maps via a parameter-free convex program called MatchLift, following a spectral method that pre-estimates the total number of distinct elements to be matched. Numerically, this program can be efficiently solved via alternating direction methods of multipliers (ADMM) along with a greedy rounding strategy. Theoretically, MatchLift exhibits near-optimal error-correction ability, i.e. in the asymptotic regime it is guaranteed to work even when a dominant fraction 1 − Θ ( log n √ n ) of the input maps behave like random outliers. Furthermore, MatchLift succeeds with minimal input complexity, namely, perfect matching can be achieved as soon as the provided maps form a connected map graph. We evaluate the proposed algorithm on various benchmark data sets including synthetic examples and real-world examples, all of which confirm the practical applicability and usefulness of MatchLift.
Output: Near-Optimal Joint Object Matching via Convex Relaxation
Model-free deep reinforcement learning (RL) methods have been successful in a wide variety of simulated domains. However, a major obstacle facing deep RL in the real world is the high sample complexity of such methods. Unbiased batch policy-gradient methods offer stable learning, but at the cost of high variance, which often requires large batches, while TD-style methods, such as off-policy actor-critic and Q-learning, are more sample-efficient but biased, and often require costly hyperparameter sweeps to stabilize. In this work, we aim to develop methods that combine the stability of unbiased policy gradients with the efficiency of off-policy RL. We present Q-Prop, a policy gradient method that uses a Taylor expansion of the off-policy critic as a control variate. Q-Prop is both sample efficient and stable, and effectively combines the benefits of on-policy and offpolicy methods. We analyze the connection between Q-Prop and existing modelfree algorithms, and use control variate theory to derive two variants of Q-Prop with conservative and aggressive adaptation. We show that conservative Q-Prop provides substantial gains in sample efficiency over trust region policy optimization (TRPO) with generalized advantage estimation (GAE), and improves stability over deep deterministic policy gradient (DDPG), the state-of-the-art on-policy and off-policy methods, on OpenAI Gym’s MuJoCo continuous control environments.
Output: Q-PROP: SAMPLE-EFFICIENT POLICY GRADIENT WITH AN OFF-POLICY CRITIC
Linear-time computational techniques have been developed for combining evidence which is available on a number of contending hypotheses. They offer a means of making the computation-intensive calculations involved more efficient in certain circumstances. Unfortunately, they restrict the orthogonal sum of evidential functions to the dichotomous structure − applies only to elements and their complements. In this paper, we present a novel evidence structure in terms of a triplet and a set of algorithms for evidential reasoning. The merit of this structure is that it divides a set of evidence into three subsets, distinguishing trivial evidential elements from important ones − focusing some particular elements. It avoids the deficits of the dichotomous structure in representing the preference of evidence and estimating the basic probability assignment of evidence. We have established a formalism for this structure and the general formulae for combining pieces of evidence in the form of the triplet, which have been theoretically and empirically justified.
Output: An Efficient Triplet-based Algorithm for Evidential Reasoning
We present the first parser for UCCA, a cross-linguistically applicable framework for semantic representation, which builds on extensive typological work and supports rapid annotation. UCCA poses a challenge for existing parsing techniques, as it exhibits reentrancy (resulting in DAG structures), discontinuous structures and non-terminal nodes corresponding to complex semantic units. To our knowledge, the conjunction of these formal properties is not supported by any existing parser. Our transition-based parser, which uses a novel transition set and features based on bidirectional LSTMs, has value not just for UCCA parsing: its ability to handle more general graph structures can inform the development of parsers for other semantic DAG structures, and in languages that frequently use discontinuous structures.
Output: A Transition-Based Directed Acyclic Graph Parser for UCCA
Weather affects our mood and behaviors, and many aspects of our life. When it is sunny, most people become happier; but when it rains, some people get depressed. Despite this evidence and the abundance of data, weather has mostly been overlooked in the machine learning and data science research. This work presents a causal analysis of how weather affects TV watching patterns. We show that some weather attributes, such as pressure and precipitation, cause major changes in TV watching patterns. To the best of our knowledge, this is the first large-scale causal study of the impact of weather on TV watching patterns.
Output: Does Weather Matter? Causal Analysis of TV Logs
Modern malware is designed with mutation characteristics, namely polymorphism and metamorphism, which causes an enormous growth in the number of variants of malware samples. Categorization of malware samples on the basis of their behaviors is essential for the computer security community in order to group samples belonging to same family. Microsoft released a malware classification challenge in 2015 with a huge dataset of near 0.5 terabytes of data, containing more than 20K malware samples. The analysis of this dataset inspired the development of a novel paradigm that is effective in categorizing malware variants into their actual family groups. This paradigm is presented and discussed in the present paper, where emphasis has been given to the phases related to the extraction, and selection of a set of novel features for the effective representation of malware samples. Features can be grouped according to different characteristics of malware behavior, and their fusion is performed according to a per-class weighting paradigm. The proposed method achieved a very high accuracy (≈ 0.998) on the Microsoft Malware Challenge dataset.
Output: Novel feature extraction, selection and fusion for effective malware family classification
Explaining adaptive behavior is a central problem in artificial intelligence research. Here we formalize adaptive agents as mixture distributions over sequences of inputs and outputs (I/O). Each distribution of the mixture constitutes a ‘possible world’, but the agent does not know which of the possible worlds it is actually facing. The problem is to adapt the I/O stream in a way that is compatible with the true world. A natural measure of adaptation can be obtained by the KullbackLeibler (KL) divergence between the I/O distribution of the true world and the I/O distribution expected by the agent that is uncertain about possible worlds. In the case of pure input streams, the Bayesian mixture provides a well-known solution for this problem. We show, however, that in the case of I/O streams this solution breaks down, because outputs are issued by the agent itself and require a different probabilistic syntax as provided by intervention calculus. Based on this calculus, we obtain a Bayesian control rule that allows modeling adaptive behavior with mixture distributions over I/O streams. This rule might allow for a novel approach to adaptive control based on a minimum KLprinciple.
Output: A Bayesian Rule for Adaptive Control based on Causal Interventions
Integrating vision and language has long been a dream in work on artificial intelligence (AI). In the past two years, we have witnessed an explosion of work that brings together vision and language from images to videos and beyond. The available corpora have played a crucial role in advancing this area of research. In this paper, we propose a set of quality metrics for evaluating and analyzing the vision & language datasets and classify them accordingly. Our analyses show that the most recent datasets have been using more complex language and more abstract concepts, however, there are different strengths and weaknesses in each.
Output: On Available Corpora for Empirical Methods in Vision & Language
The Minimum Weight Dominating Set (MWDS) problem is an important generalization of the Minimum Dominating Set (MDS) problem with extensive applications. This paper proposes a new local search algorithm for the MWDS problem, which is based on two new ideas. The first idea is a heuristic called two-level configuration checking (CC), which is a new variant of a recent powerful configuration checking strategy (CC) for effectively avoiding the recent search paths. The second idea is a novel scoring function based on the frequency of being uncovered of vertices. Our algorithm is called CCFS, according to the names of the two ideas. The experimental results show that, CCFS performs much better than some state-of-the-art algorithms in terms of solution quality on a broad range of MWDS benchmarks.
Output: Local Search for Minimum Weight Dominating Set with Two-Level Configuration Checking and Frequency Based Scoring Function
In the mixture models problem it is assumed that there are K distributions θ1, . . . , θK and one gets to observe a sample from a mixture of these distributions with unknown coefficients. The goal is to associate instances with their generating distributions, or to identify the parameters of the hidden distributions. In this work we make the assumption that we have access to several samples drawn from the same K underlying distributions, but with different mixing weights. As with topic modeling, having multiple samples is often a reasonable assumption. Instead of pooling the data into one sample, we prove that it is possible to use the differences between the samples to better recover the underlying structure. We present algorithms that recover the underlying structure under milder assumptions than the current state of art when either the dimensionality or the separation is high. The methods, when applied to topic modeling, allow generalization to words not present in the training data.
Output: Using multiple samples to learn mixture models
We introduce and analyze a rigorous formulation of the dynamics of a signal processing scheme that aims at dense scanning of large input signals. Recently proposed methodologies lack a satisfactory discussion of whether they actually produce the correct results according to their definition, especially in the context of Convolutional Neural Networks. We improve on this through an exact characterization of the requirements for a sound sliding window approach. The tools developed in this paper are especially beneficial if Convolutional Neural Networks are employed, but can also be used as a more general framework to validate related approaches to signal scanning. The contributed theory helps to eliminate redundant computations and renders special case treatment unnecessary, resulting in a dramatic boost in efficiency particularly on massively parallel processors.
Output: A Theory for Rapid Exact Signal Scanning with Deep Multi-Scale Convolutional Neural Networks
Annotating semantic data with metadata is becoming more and more important to provide information about the statements being asserted. While initial solutions proposed a data model to represent a specific dimension of metainformation (such as time or provenance), the need for a general annotation framework which allows representing different context dimensions is needed. In this paper, we extend the 4dFluents ontology by Welty and Fikes—on associating temporal validity to statements—to any dimension of context, and discuss possible issues that multidimensional context representations have to face and how we address them.
Output: NdFluents: A Multi-dimensional Contexts Ontology
In this article we introduce how to put vague hyperprior on Dirichlet distribution, and we update the parameter of it by adaptive rejection sampling (ARS). Finally we analyze this hyperprior in an over-fitted mixture model by some synthetic experiments.
Output: Hyperprior on symmetric Dirichlet distribution
We propose Batch-Expansion Training (BET), a framework for running a batch optimizer on a gradually expanding dataset. As opposed to stochastic approaches, batches do not need to be resampled i.i.d. at every iteration, thus making BET more resource efficient in a distributed setting, and when disk-access is constrained. Moreover, BET can be easily paired with most batch optimizers, does not require any parameter-tuning, and compares favorably to existing stochastic and batch methods. We show that when the batch size grows exponentially with the number of outer iterations, BET achieves optimal Õ(1/ǫ) data-access convergence rate for strongly convex objectives.
Output: Batch-Expansion Training: An Efficient Optimization Paradigm for Machine Learning
Mediation is a process, in which both parties agree to resolve their dispute by negotiating over alternative solutions presented by a mediator. In order to construct such solutions, mediation brings more information and knowledge, and, if possible, resources to the negotiation table. The contribution of this paper is the automated mediation machinery which does that. It presents an argumentation-based mediation approach that extends the logic-based approach to argumentation-based negotiation involving BDI agents. The paper describes the mediation algorithm. For comparison it illustrates the method with a case study used in an earlier work. It demonstrates how the computational mediator can deal with realistic situations in which the negotiating agents would otherwise fail due to lack of knowledge and/or resources.
Output: Dispute Resolution Using Argumentation-Based Mediation
Epistemic logic with non-standard knowledge operators, especially the “knowing-value” operator, has recently gathered much attention. With the “knowing-value” operator, we can express knowledge of individual variables, but not of the relations between them in general. In this paper, we propose a new operator Kf to express knowledge of the functional dependencies between variables. The semantics of this Kf operator uses a function domain which imposes a constraint on what counts as a functional dependency relation. By adjusting this function domain, different interesting logics arise, and in this paper we axiomatize three such logics in a single agent setting. Then we show how these three logics can be unified by allowing the function domain to vary relative to different agents and possible worlds. A multiagent axiomatization is given in this case.
Output: Epistemic Logic with Functional Dependency Operator
We present a deep layered architecture that generalizes classical convolutional neural networks (ConvNets). The architecture, called SimNets, is driven by two operators, one being a similarity function whose family contains the convolution operator used in ConvNets, and the other is a new soft max-min-mean operator called MEX that realizes classical operators like ReLU and max pooling, but has additional capabilities that make SimNets a powerful generalization of ConvNets. Three interesting properties emerge from the architecture: (i) the basic input to hidden layer to output machinery contains as special cases kernel machines with the Exponential and Generalized Gaussian kernels, the output units being ”neurons in feature space” (ii) in its general form, the basic machinery has a higher abstraction level than kernel machines, and (iii) initializing networks using unsupervised learning is natural. Experiments demonstrate the capability of achieving state of the art accuracy with networks that are an order of magnitude smaller than comparable ConvNets.
Output: SimNets: A Generalization of Convolutional Networks
We introduce the Dynamic Capacity Network (DCN), a neural network that can adaptively assign its capacity across different portions of the input data. This is achieved by combining modules of two types: low-capacity subnetworks and high-capacity sub-networks. The low-capacity sub-networks are applied across most of the input, but also provide a guide to select a few portions of the input on which to apply the high-capacity sub-networks. The selection is made using a novel gradient-based attention mechanism, that efficiently identifies input regions for which the DCN’s output is most sensitive and to which we should devote more capacity. We focus our empirical evaluation on the Cluttered MNIST and SVHN image datasets. Our findings indicate that DCNs are able to drastically reduce the number of computations, compared to traditional convolutional neural networks, while maintaining similar or even better performance.
Output: Dynamic Capacity Networks
<lb>An associative memory is a framework of content-addressable memory that stores a collection of<lb>message vectors (or a dataset) over a neural network while enabling a neurally feasible mechanism to<lb>recover any message in the dataset from its noisy version. Designing an associative memory requires<lb>addressing two main tasks: 1) learning phase: given a dataset, learn a concise representation of the<lb>dataset in the form of a graphical model (or a neural network), 2) recall phase: given a noisy version of<lb>a message vector from the dataset, output the correct message vector via a neurally feasible algorithm<lb>over the network learnt during the learning phase. This paper studies the problem of designing a class<lb>of neural associative memories which learns a network representation for a large dataset that ensures<lb>correction against a large number of adversarial errors during the recall phase. Specifically, the associa-<lb>tive memories designed in this paper can store dataset containing exp(n) n-length message vectors over<lb>a network with O(n) nodes and can tolerate Ω( n<lb>polylogn ) adversarial errors. This paper carries out this<lb>memory design by mapping the learning phase and recall phase to the tasks of dictionary learning with<lb>a square dictionary and iterative error correction in an expander code, respectively.
Output: Associative Memory using Dictionary Learning and Expander Decoding
Davoud Mougouei School of Computer Science, Engineering, and Mathematics Flinders University Adelaide, Australia davoud.mougouei@inders.edu.au David M. W. Powers School of Computer Science, Engineering, and Mathematics Flinders University Adelaide, Australia david.powers@inders.edu.au Asghar Moeini School of Computer Science, Engineering, and Mathematics Flinders University Adelaide, Australia asghar.moeini@inders.edu.au
Output: An Integer Programming Model for Binary Knapsack Problem with Value-Related Dependencies among Elements
In this paper, we propose a novel unsupervised deep learning model, called PCA-based Convolutional Network (PCN). The architecture of PCN is composed of several feature extraction stages and a nonlinear output stage. Particularly, each feature extraction stage includes two layers: a convolutional layer and a feature pooling layer. In the convolutional layer, the filter banks are simply learned by PCA. In the nonlinear output stage, binary hashing is applied. For the higher convolutional layers, the filter banks are learned from the feature maps that were obtained in the previous stage. To test PCN, we conducted extensive experiments on some challenging tasks, including handwritten digits recognition, face recognition and texture classification. The results show that PCN performs competitive with or even better than state-of-theart deep learning models. More importantly, since there is no back propagation for supervised finetuning, PCN is much more efficient than existing deep networks.
Output: A PCA-Based Convolutional Network
String getQueueName(); abstract Serializable getCommandObject(Long workflowInstanceId,Serializable getCommandObject(Long workflowInstanceId, String taskName, String messageType, ExecutionContext context);
Output: Automatic Structure Discovery for Large Source Code
Wide-angle sonar mapping of the environ­ ment by mobile robot is nontrivial due to sev­ eral sources of uncertainty: dropouts due to "specular" reflections, obstacle location un­ certainty due to the wide beam, and distance measurement error. Earlier papers address the latter problems, but dropouts remain a problem in many environments. We present an approach that lifts the overoptimistic in­ dependence assumption used in earlier work, and use Bayes nets to represent the depen­ dencies between objects of the model. Ob­ jects of the model consist of readings, and of regions in which "quasi location invari­ ance" of the (possible) obstacles exists, with respect to the readings. Simulation supports the method's feasibility. The model is readily extensible to allow for prior distributions, as well as other types of sensing operations.
Output: Bayes Networks for Sonar Sensor Fusion
Multi-agent path planning is a challenging problem with numerous real-life applications. Running a centralized search such as A* in the combined state space of all units is complete and cost-optimal, but scales poorly, as the state space size is exponential in the number of mobile units. Traditional decentralized approaches, such as FAR and WHCA*, are faster and more scalable, being based on problem decomposition. However, such methods are incomplete and provide no guarantees with respect to the running time or the solution quality. They are not necessarily able to tell in a reasonable time whether they would succeed in finding a solution to a given instance. We introduce MAPP, a tractable algorithm for multi-agent path planning on undirected graphs. We present a basic version and several extensions. They have low-polynomial worst-case upper bounds for the running time, the memory requirements, and the length of solutions. Even though all algorithmic versions are incomplete in the general case, each provides formal guarantees on problems it can solve. For each version, we discuss the algorithm’s completeness with respect to clearly defined subclasses of instances. Experiments were run on realistic game grid maps. MAPP solved 99.86% of all mobile units, which is 18–22% better than the percentage of FAR and WHCA*. MAPP marked 98.82% of all units as provably solvable during the first stage of plan computation. Parts of MAPP’s computation can be re-used across instances on the same map. Speed-wise, MAPP is competitive or significantly faster than WHCA*, depending on whether MAPP performs all computations from scratch. When data that MAPP can re-use are preprocessed offline and readily available, MAPP is slower than the very fast FAR algorithm by a factor of 2.18 on average. MAPP’s solutions are on average 20% longer than FAR’s solutions and 7–31% longer than WHCA*’s solutions.
Output: MAPP: a Scalable Multi-Agent Path Planning Algorithm with Tractability and Completeness Guarantees
Message-passing algorithms have emerged as powerful techniques for approximate inference in graphical models. When these algorithms converge, they can be shown to find local (or sometimes even global) optima of variational formulations to the inference problem. But many of the most popular algorithms are not guaranteed to converge. This has lead to recent interest in convergent message-passing algorithms. In this paper, we present a unified view of convergent message-passing algorithms. We present a simple derivation of an abstract algorithm, tree-consistency bound optimization (TCBO) that is provably convergent in both its sum and max product forms. We then show that many of the existing convergent algorithms are instances of our TCBO algorithm, and obtain novel convergent algorithms “for free” by exchanging maximizations and summations in existing algorithms. In particular, we show that Wainwright’s non-convergent sum-product algorithm for tree based variational bounds, is actually convergent with the right update order for the case where trees are monotonic chains.
Output: Convergent message passing algorithms - a unifying view
Electric water heaters have the ability to store energy in their water buffer without impacting the comfort of the end user. This feature makes them a prime candidate for residential demand response. However, the stochastic and nonlinear dynamics of electric water heaters, makes it challenging to harness their flexibility. Driven by this challenge, this paper formulates the underlying sequential decision-making problem as a Markov decision process and uses techniques from reinforcement learning. Specifically, we apply an auto-encoder network to find a compact feature representation of the sensor measurements, which helps to mitigate the curse of dimensionality. A wellknown batch reinforcement learning technique, fitted Q-iteration, is used to find a control policy, given this feature representation. In a simulation-based experiment using an electric water heater with 50 temperature sensors, the proposed method was able to achieve good policies much faster than when using the full state information. In a lab experiment, we apply fitted Q-iteration to an electric water heater with eight temperature sensors. Further reducing the state vector did not improve the results of fitted Q-iteration. The results of the lab experiment, spanning 40 days, indicate that compared to a thermostat controller, the presented approach was able to reduce the total cost of energy consumption of the electric water heater by 15%.
Output: Reinforcement Learning Applied to an Electric Water Heater: From Theory to Practice
The online learning of deep neural networks is an interesting problem of machine learning because, for example, major IT companies want to manage the information of the massive data uploaded on the web daily, and this technology can contribute to the next generation of lifelong learning. We aim to train deep models from new data that consists of new classes, distributions, and tasks at minimal computational cost, which we call online deep learning. Unfortunately, deep neural network learning through classical online and incremental methods does not work well in both theory and practice. In this paper, we introduce dual memory architectures for online incremental deep learning. The proposed architecture consists of deep representation learners and fast learnable shallow kernel networks, both of which synergize to track the information of new data. During the training phase, we use various online, incremental ensemble, and transfer learning techniques in order to achieve lower error of the architecture. On the MNIST, CIFAR-10, and ImageNet image recognition tasks, the proposed dual memory architectures performs much better than the classical online and incremental ensemble algorithm, and their accuracies are similar to that of the batch learner. ICML workshop on Deep Learning 2015, Lille, France, 2015. Copyright 2015 by the author(s).
Output: Dual Memory Architectures for Fast Deep Learning of Stream Data via an Online-Incremental-Transfer Strategy
Humans are generally good at learning abstract concepts about objects and scenes (e.g. spatial orientation, relative sizes, etc.). Over the last years convolutional neural networks have achieved almost human performance in recognizing concrete classes (i.e. specific object categories). This paper tests the performance of a current CNN (GoogLeNet) on the task of differentiating between abstract classes which are trivially differentiable for humans. We trained and tested the CNN on the two abstract classes of horizontal and vertical orientation and determined how well the network is able to transfer the learned classes to other, previously unseen objects.
Output: Learning Abstract Classes using Deep Learning
Recent years have seen the development of a number<lb>of methods for multiagent planning under uncertainty<lb>that scale to tens or even hundreds of agents. However,<lb>most of these methods either make restrictive assump-<lb>tions on the problem domain, or provide approximate<lb>solutions without any guarantees on quality. To allow<lb>for meaningful benchmarking through measurable qual-<lb>ity guarantees on a very general class of problems, this<lb>paper introduces a family of influence-optimistic upper<lb>bounds for factored Dec-POMDPs. Intuitively, we de-<lb>rive bounds on very large multiagent planning problems<lb>by subdividing them in sub-problems, and at each of<lb>these sub-problems making optimistic assumptions with<lb>respect to the influence that will be exerted by the rest<lb>of the system. We numerically compare the different<lb>upper bounds and demonstrate how, for the first time<lb>ever, we can achieve a non-trivial guarantee that the<lb>heuristic solution of problems with hundreds of agents<lb>is close to optimal. Furthermore, we provide evidence<lb>that the upper bounds may improve the effectiveness of<lb>heuristic influence search, and discuss further potential<lb>applications to multiagent planning.
Output: Influence-Optimistic Local Values for Multiagent Planning — Extended Version
Graph-based representations of images have recently acquired an important role for classification purposes within the context of machine learning approaches. The underlying idea is to consider that relevant information of an image is implicitly encoded into the relationships between more basic entities that compose by themselves the whole image. The classification problem is then reformulated in terms of an optimization problem usually solved by a gradient-based search procedure. Vario-eta through structure is an approximate second order stochastic optimization technique that achieves a good trade-off between speed of convergence and the computational effort required. However, the robustness of this technique for large scale problems has not been yet assessed. In this paper we firstly provide a theoretical justification of the assumptions made by this optimization procedure. Secondly, a complexity analysis of the algorithm is performed to prove its suitability for large scale learning problems.
Output: Complexity Analysis of Vario-eta through Structure
An algorithm for pose and motion estimation using corresponding features in omnidirectional images and a digital terrain map is proposed. In previous paper, such algorithm for regular camera was considered. Using a Digital Terrain (or Digital Elevation) Map (DTM/DEM) as a global reference enables recovering the absolute position and orientation of the camera. In order to do this, the DTM is used to formulate a constraint between corresponding features in two consecutive frames. In this paper, these constraints are extended to handle non-central projection, as is the case with many omnidirectional systems. The utilization of omnidirectional data is shown to improve the robustness and accuracy of the navigation algorithm. The feasibility of this algorithm is established through lab experimentation with two kinds of omnidirectional acquisition systems. The first one is polydioptric cameras while the second is catadioptric camera.
Output: Pose and Motion from Omnidirectional Optical Flow and a Digital Terrain Map€
We develop a new semantics for defeasible infer­ ence based on extended probability measures al­ lowed to take infinitesimal values, on the inter­ pretation of defaults as generalized conditional probability constraints and on a preferred-model implementation of entropy-maximization.
Output: Defaults and lnfinitesimals Defeasible Inference by Nonarchimedean Entropy Maximization