niryuu/sni-each-converted huggingface dataset:
We introduce the first global recursive neural parsing model with optimality guarantees during decoding. To support global features, we give up dynamic programs and instead search directly in the space of all possible subtrees. Although this space is exponentially large in the sentence length, we show it is possible to learn an efficient A* parser. We augment existing parsing models, which have informative bounds on the outside score, with a global model that has loose bounds but only needs to model non-local phenomena. The global model is trained with a novel objective that encourages the parser to search both efficiently and accurately. The approach is applied to CCG parsing, improving state-of-the-art accuracy by 0.4 F1. The parser finds the optimal parse for 99.9% of held-out sentences, exploring on average only 190 subtrees.
Output: Global Neural CCG Parsing with Optimality Guarantees
We propose a model to learn visually grounded word embeddings (vis-w2v) to capture visual notions of semantic relatedness. While word embeddings trained using text have been extremely successful, they cannot uncover notions of semantic relatedness implicit in our visual world. For instance, visual grounding can help us realize that concepts like eating and staring at are related, since when people are eating something, they also tend to stare at the food. Grounding a rich variety of relations like eating and stare at in vision is a challenging task, despite recent progress in vision. We realize the visual grounding for words depends on the semantics of our visual world, and not the literal pixels. We thus use abstract scenes created from clipart to provide the visual grounding. We find that the embeddings we learn capture fine-grained visually grounded notions of semantic relatedness. We show improvements over text only word embeddings (word2vec) on three tasks: common-sense assertion classification, visual paraphrasing and text-based image retrieval. Our code and datasets will be available online.
Output: Visual Word2Vec (vis-w2v): Learning Visually Grounded Word Embeddings Using Abstract Scenes
Dialog act identification plays an important role in understanding conversations. It has been widely applied in many fields such as dialogue systems, automatic machine translation, automatic speech recognition, and especially useful in systems with human-computer natural language dialogue interfaces such as virtual assistants and chatbots. The first step of identifying dialog act is identifying the boundary of the dialog act in utterances. In this paper, we focus on segmenting the utterance according to the dialog act boundaries, i.e. functional segments identification, for Vietnamese utterances. We investigate carefully functional segment identification in two approaches: (1) machine learning approach using maximum entropy (ME) and conditional random fields (CRFs); (2) deep learning approach using bidirectional Long Short-Term Memory (LSTM) with a CRF layer (Bi-LSTM-CRF) on two different conversational datasets: (1) Facebook messages (Message data); (2) transcription from phone conversations (Phone data). To the best of our knowledge, this is the first work that applies deep learning based approach to dialog act segmentation. As the results show, deep learning approach performs appreciably better as to compare with traditional machine learning approaches. Moreover, it is also the first study that tackles dialog act and functional segment identification for Vietnamese. Keywords—Dialog act segmentation, functional segment, Vietnamese conversation.
Output: Dialogue Act Segmentation for Vietnamese Human-Human Conversational Texts
People exhibit a tendency to generalize a novel noun to the basic-level in a hierarchical taxonomy – a cognitively salient category such as “dog” – with the degree of generalization depending on the number and type of exemplars. Recently, a change in the presentation timing of exemplars has also been shown to have an effect, surprisingly reversing the prior observed pattern of basic-level generalization. We explore the precise mechanisms that could lead to such behavior by extending a computational model of word learning and word generalization to integrate cognitive processes of memory and attention. Our results show that the interaction of forgetting and attention to novelty, as well as sensitivity to both type and token frequencies of exemplars, enables the model to replicate the empirical results from different presentation timings. Our results reinforce the need to incorporate general cognitive processes within word learning models to better understand the range of observed behaviors in vocabulary acquisition.
Output: The Interaction of Memory and Attention in Novel Word Generalization: A Computational Investigation
Current Deep Learning approaches have been very successful using convolutional neural networks (CNN) trained on large graphical processing units (GPU)-based computers. Three limitations of this approach are: 1) they are based on a simple layered network topology, i.e., highly connected layers, without intra-layer connections; 2) the networks are manually configured to achieve optimal results, and 3) the implementation of neuron model is expensive in both cost and power. In this paper, we evaluate deep learning models using three different computing architectures to address these problems: quantum computing to train complex topologies, high performance computing (HPC) to automatically determine network topology, and neuromorphic computing for a low-power hardware implementation. We use the MNIST dataset for our experiment, due to input size limitations of current quantum computers. Our results show the feasibility of using the three architectures in tandem to address the above deep learning limitations. We show a quantum computer can find high quality values of intra-layer connections weights, in a tractable time as the complexity of the network increases; a high performance computer can find optimal layer-based topologies; and a neuromorphic computer can represent the complex topology and weights derived from the other architectures in low power memristive hardware. Notice: This manuscript has been authored by UT-Battelle, LLC under Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan). 1 ar X iv :1 70 3. 05 36 4v 1 [ cs .N E ] 1 5 M ar 2 01 7
Output: A Study of Complex Deep Learning Networks on High Performance, Neuromorphic, and Quantum Computers
Machine transliteration is the process of automatically transforming the script of a word from a source language to a target language, while preserving pronunciation. Sequence to sequence learning has recently emerged as a new paradigm in supervised learning. In this paper a character-based encoder-decoder model has been proposed that consists of two Recurrent Neural Networks. The encoder is a Bidirectional recurrent neural network that encodes a sequence of symbols into a fixed-length vector representation, and the decoder generates the target sequence using an attention-based recurrent neural network. The encoder, the decoder and the attention mechanism are jointly trained to maximize the conditional probability of a target sequence given a source sequence. Our experiments on different datasets show that the proposed encoderdecoder model is able to achieve significantly higher transliteration quality over traditional statistical models.
Output: Neural Machine Transliteration: Preliminary Results
Kernel approximation using randomized feature maps has recently gained a lot of interest. In this work, we identify that previous approaches for polynomial kernel approximation create maps that are rank deficient, and therefore do not utilize the capacity of the projected feature space effectively. To address this challenge, we propose compact random feature maps (CRAFTMaps) to approximate polynomial kernels more concisely and accurately. We prove the error bounds of CRAFTMaps demonstrating their superior kernel reconstruction performance compared to the previous approximation schemes. We show how structured random matrices can be used to efficiently generate CRAFTMaps, and present a single-pass algorithm using CRAFTMaps to learn non-linear multi-class classifiers. We present experiments on multiple standard data-sets with performance competitive with state-of-the-art results.
Output: Compact Random Feature Maps
The multi-agent path-finding (MAPF) problem has recently received a lot of attention. However, it does not capture important characteristics of many real-world domains, such as automated warehouses, where agents are constantly engaged with new tasks. In this paper, we therefore study a lifelong version of the MAPF problem, called the multiagent pickup and delivery (MAPD) problem. In the MAPD problem, agents have to attend to a stream of delivery tasks in an online setting. One agent has to be assigned to each delivery task. This agent has to first move to a given pickup location and then to a given delivery location while avoiding collisions with other agents. We present two decoupled MAPD algorithms, Token Passing (TP) and Token Passing with Task Swaps (TPTS). Theoretically, we show that they solve all well-formed MAPD instances, a realistic subclass of MAPD instances. Experimentally, we compare them against a centralized strawman MAPD algorithm without this guarantee in a simulated warehouse system. TP can easily be extended to a fully distributed MAPD algorithm and is the best choice when real-time computation is of primary concern since it remains efficient for MAPD instances with hundreds of agents and tasks. TPTS requires limited communication among agents and balances well between TP and the centralized MAPD algorithm.
Output: Lifelong Multi-Agent Path Finding for Online Pickup and Delivery Tasks∗
Recommendation systems are emerging as an important business application with significant economic impact. Currently popular systems include Amazon’s book recommendations, Netflix’s movie recommendations, and Pandora’s music recommendations. In this paper we address the problem of estimating probabilities associated with recommendation system data using non-parametric kernel smoothing. In our estimation we interpret missing items as randomly censored observations and obtain efficient computation schemes using combinatorial properties of generating functions. We demonstrate our approach with several case studies involving real world movie recommendation data. The results are comparable with state-of-the-art techniques while also providing probabilistic preference estimates outside the scope of traditional recommender systems.
Output: Estimating Probabilities in Recommendation Systems
Studies of the overall structure of vocabulary and its dynamics became possible due to creation of diachronic text corpora, especially Google Books Ngram. This article discusses the question of core change rate and the degree to which the core words cover the texts. Different periods of the last three centuries and six main European languages presented in Google Books Ngram are compared. The main result is high stability of core change rate, which is analogous to stability of the Swadesh list.
Output: Dynamics of core of language vocabulary
Stochastic gradient-boosted decision trees are widely employed for multivariate classification and regression tasks. This paper presents a speed-optimized and cache-friendly implementation for multivariate classification called FastBDT. FastBDT is one order of magnitude faster during the fitting-phase and application-phase, in comparison with popular implementations in software frameworks like TMVA, scikit-learn and XGBoost. The concepts used to optimize the execution time and performance studies are discussed in detail in this paper. The key ideas include: An equal-frequency binning on the input data, which allows replacing expensive floating-point with integer operations, while at the same time increasing the quality of the classification; a cache-friendly linear access pattern to the input data, in contrast to usual implementations, which exhibit a random access pattern. FastBDT provides interfaces to C/C++, Python and TMVA. It is extensively used in the field of high energy physics by the Belle II experiment.
Output: A speed-optimized and cache-friendly implementation of stochastic gradient-boosted decision trees for multivariate classification
We present LTLS, a technique for multiclass and multilabel prediction that can perform training and inference in logarithmic time and space. LTLS embeds large classification problems into simple structured prediction problems and relies on efficient dynamic programming algorithms for inference. We train LTLS with stochastic gradient descent on a number of multiclass and multilabel datasets and show that despite its small memory footprint it is often competitive with existing approaches.
Output: Log-time and Log-space Extreme Classification
In this paper we investigate the problem of localizing a mobile device based on readings from its embedded sensors utilizing machine learning methodologies. We consider a realworld environment, collect a large dataset of 3110 datapoints, and examine the performance of a substantial number of machine learning algorithms in localizing a mobile device. We have found algorithms that give a mean error as accurate as 0.76 meters, outperforming other indoor localization systems reported in the literature. We also propose a hybrid instance-based approach that results in a speed increase by a factor of ten with no loss of accuracy in a live deployment over standard instance-based methods, allowing for fast and accurate localization. Further, we determine how smaller datasets collected with less density affect accuracy of localization, important for use in real-world environments. Finally, we demonstrate that these approaches are appropriate for real-world deployment by evaluating their performance in an online, in-motion experiment.
Output: Machine Learning for Indoor Localization Using Mobile Phone-Based Sensors
The choice of approximate posterior distribution is one of the core problems in variational inference. Most applications of variational inference employ simple families of posterior approximations in order to allow for efficient inference, focusing on mean-field or other simple structured approximations. This restriction has a significant impact on the quality of inferences made using variational methods. We introduce a new approach for specifying flexible, arbitrarily complex and scalable approximate posterior distributions. Our approximations are distributions constructed through a normalizing flow, whereby a simple initial density is transformed into a more complex one by applying a sequence of invertible transformations until a desired level of complexity is attained. We use this view of normalizing flows to develop categories of finite and infinitesimal flows and provide a unified view of approaches for constructing rich posterior approximations. We demonstrate that the theoretical advantages of having posteriors that better match the true posterior, combined with the scalability of amortized variational approaches, provides a clear improvement in performance and applicability of variational inference.
Output: Variational Inference with Normalizing Flows
Sentences are important semantic units of natural language. A generic, distributional representation of sentences that can capture the latent semantics is beneficial to multiple downstream applications. We observe a simple geometry of sentences – the word representations of a given sentence (on average 10.23 words in all SemEval datasets with a standard deviation 4.84) roughly lie in a low-rank subspace (roughly, rank 4). Motivated by this observation, we represent a sentence by the low-rank subspace spanned by its word vectors. Such an unsupervised representation is empirically validated via semantic textual similarity tasks on 19 different datasets, where it outperforms the sophisticated neural network models, including skip-thought vectors, by 15% on average.
Output: Representing Sentences as Low-Rank Subspaces
Private Set Intersection (PSI) is usually implemented as a sequence of encryption rounds between pairs of users, whereas the present work implements PSI in a simpler fashion: each set only needs to be encrypted once, after which each pair of users need only one ordinary set comparison. This is typically orders of magnitude faster than ordinary PSI at the cost of some “fuzziness” in the matching, which may nonetheless be tolerable or even desirable. This is demonstrated in the case where the sets consist of English words processed with WordNet. Email: 1054h34@gmail.com
Output: Fast and Fuzzy Private Set Intersection
Submodular functions describe a variety of discrete problems in machine learn-<lb>ing, signal processing, and computer vision. However, minimizing submodular<lb>functions poses a number of algorithmic challenges. Recent work introduced an<lb>easy-to-use, parallelizable algorithm for minimizing submodular functions that<lb>decompose as the sum of “simple” submodular functions. Empirically, this al-<lb>gorithm performs extremely well, but no theoretical analysis was given. In this<lb>paper, we show that the algorithm converges linearly, and we provide upper and<lb>lower bounds on the rate of convergence. Our proof relies on the geometry of<lb>submodular polyhedra and draws on results from spectral graph theory.
Output: On the Convergence Rate of Decomposable Submodular Function Minimization
The holy Quran is the holy book of the Muslims. It contains information about many domains. Often people search for particular concepts of holy Quran based on the relations among concepts. An ontological modeling of holy Quran can be useful in such a scenario. In this paper, we have modeled nature related concepts of holy Quran using OWL (Web Ontology Language) / RDF (Resource Description Framework). Our methodology involves identifying nature related concepts mentioned in holy Quran and identifying relations among those concepts. These concepts and relations are represented as classes/instances and properties of an OWL ontology. Later, in the result section it is shown that, using the Ontological model, SPARQL queries can retrieve verses and concepts of interest. Thus, this modeling helps semantic search and query on the holy Quran. In this work, we have used English translation of the holy Quran by Sahih International, Protege OWL Editor and for querying we have used SPARQL. Keywords— Quranic Ontology; Semantic Quran; Quranic Knowledge Representation.
Output: Applying Ontological Modeling on Quranic “Nature” Domain
The game of Go is more challenging than other board games, due to the difficulty of constructing a position or move evaluation function. In this paper we investigate whether deep convolutional networks can be used to directly represent and learn this knowledge. We train a large 12-layer convolutional neural network by supervised learning from a database of human professional games. The network correctly predicts the expert move in 55% of positions, equalling the accuracy of a 6 dan human player. When the trained convolutional network was used directly to play games of Go, without any search, it beat the traditional-search program GnuGo in 97% of games, and matched the performance of a state-of-the-art Monte-Carlo tree search that simulates two million positions per move.
Output: MOVE EVALUATION IN GO USING DEEP CONVOLUTIONAL NEURAL NETWORKS
Cluster analysis plays an important role in decision making process for many knowledge-based systems. There exist a wide variety of different approaches for clustering applications including the heuristic techniques, probabilistic models, and traditional hierarchical algorithms. In this paper, a novel heuristic approach based on big bang-big crunch algorithm is proposed for clustering problems. The proposed method not only takes advantage of heuristic nature to alleviate typical clustering algorithms such as k-means, but it also benefits from the memory based scheme as compared to its similar heuristic techniques. Furthermore, the performance of the proposed algorithm is investigated based on several benchmark test functions as well as on the well-known datasets. The experimental results shows the significant superiority of the proposed method over the similar algorithms.
Output: Memory Enriched Big Bang Big Crunch Optimization Algorithm for Data Clustering
Performing sensitivity analysis for influence diagrams using the decision circuit framework is particularly convenient, since the partial derivatives with respect to every parameter are readily available [Bhattacharjya and Shachter, 2007; 2008]. In this paper we present three non-linear sensitivity analysis methods that utilize this partial derivative information and therefore do not require re-evaluating the decision situation multiple times. Specifically, we show how to efficiently compare strategies in decision situations, perform sensitivity to risk aversion and compute the value of perfect hedging [Seyller, 2008].
Output: Three new sensitivity analysis methods for influence diagrams
We present two related methods for creating MasterPrints, synthetic fingerprints that a fingerprint verification system identifies as many different people. Both methods start with training a Generative Adversarial Network (GAN) on a set of real fingerprint images. The generator network is then used to search for images that can be recognized as multiple individuals. The first method uses evolutionary optimization in the space of latent variables, and the second uses gradient-based search. Our method is able to design a MasterPrint that a commercial fingerprint system matches to 22% of all users in a strict security setting, and 75% of all users at a looser security setting.
Output: DeepMasterPrint: Generating Fingerprints for Presentation Attacks
This paper develops the idea of membership function assignment for OWL (Web Ontology Language) ontology elements in order to subsequently generate fuzzy rules from this ontology. The task of membership function assignment for OWL ontology elements had already been partially described, but this concerned the case , when several OWL ontologies of the same domain were available, and they were merged into a single ontology. The purpose of this paper is to present the way of membership function assignment for OWL ontology elements in the case, when there is the only one available ontology. Fuzzy rules, generated from the OWL ontology, are necessary for supplement of the SWES (Semantic Web Expert System) knowledge base. SWES is an expert system, which will be able to extract knowledge from OWL ontologies , found in the Web, and will serve as a universal expert for the user.
Output: Membership Function Assignment for Elements of Single OWL Ontology
Authors are encouraged to submit new papers to INFORMS journals by means of a style file template, which includes the journal title. However, use of a template does not certify that the paper has been accepted for publication in the named journal. INFORMS journal templates are for the exclusive purpose of submitting to an INFORMS journal and should not be used to distribute the papers in print or online or to submit the papers to another publication.
Output: A Dynamic Near-Optimal Algorithm for Online Linear Programming
For agents and robots to become more useful, they must be able to quickly learn from non-technical users. This paper investigates the problem of interactively learning behaviors communicated by a human teacher using positive and negative feedback. Much previous work on this problem has made the assumption that people provide feedback for decisions that is dependent on the behavior they are teaching and is independent from the learner’s current policy. We present empirical results that show this assumption to be false—whether human trainers give a positive or negative feedback for a decision is influenced by the learner’s current policy. We argue that policy-dependent feedback, in addition to being commonplace, enables useful training strategies from which agents should benefit. Based on this insight, we introduce Convergent Actor-Critic by Humans (COACH), an algorithm for learning from policy-dependent feedback that converges to a local optimum. Finally, we demonstrate that COACH can successfully learn multiple behaviors on a physical robot, even with noisy image features.
Output: Interactive Learning from Policy-Dependent Human Feedback
This paper considers a general data-fitting problem over a networked system, in which many computing nodes are connected by an undirected graph. This kind of problem can find many real-world applications and has been studied extensively in the literature. However, existing solutions either need a central controller for information sharing or requires slot synchronization among different nodes, which increases the difficulty of practical implementations, especially for a very large and heterogeneous system. As a contrast, in this paper, we treat the data-fitting problem over the network as a stochastic programming problem with many constraints. By adapting the results in a recent paper [18], we design a fully distributed and asynchronized stochastic gradient descent (SGD) algorithm. We show that our algorithm can achieve global optimality and consensus asymptotically by only local computations and communications. Additionally, we provide a sharp lower bound for the convergence speed in the regular graph case. This result fits the intuition and provides guidance to design a ‘good’ network topology to speed up the convergence. Also, the merit of our design is validated by experiments on both synthetic and real-world datasets.
Output: Fully Distributed and Asynchronized Stochastic Gradient Descent for Networked Systems
Data-to-text systems are powerful in generating reports from data automatically and thus they simplify the presentation of complex data. Rather than presenting data using visualisation techniques, datato-text systems use natural (human) language, which is the most common way for human-human communication. In addition, data-to-text systems can adapt their output content to users’ preferences, background or interests and therefore they can be pleasant for users to interact with. Content selection is an important part of every data-to-text system, because it is the module that determines which from the available information should be conveyed to the user. This survey initially introduces the field of data-to-text generation, describes the general data-to-text system architecture and then it reviews the state-ofthe-art content selection methods. Finally, it provides recommendations for choosing an approach and discusses opportunities for future research.
Output: Content Selection in Data-to-Text Systems: A Survey
Semi-supervised learning based on the low-density separation principle such as<lb>the cluster and manifold assumptions has been extensively studied in the last<lb>decades. However, such semi-supervised learning methods do not always per-<lb>form well due to violation of the cluster and manifold assumptions. In this paper,<lb>we propose a novel approach to semi-supervised learning that does not require<lb>such restrictive assumptions. Our key idea is to combine learning from positive<lb>and negative data (standard supervised learning) and learning from positive and<lb>unlabeled data (PU learning), the latter is guaranteed to be able to utilize unla-<lb>beled data without the cluster and manifold assumptions. We theoretically and<lb>experimentally show the usefulness of our approach.
Output: Beyond the Low-density Separation Principle: A Novel Approach to Semi-supervised Learning
In this paper we propose a neural network model with a novel Sequential Attention layer that extends soft attention by assigning weights to words in an input sequence in a way that takes into account not just how well that word matches a query, but how well surrounding words match. We evaluate this approach on the task of reading comprehension (Who did What and CNN) and show that it dramatically improves a strong baseline like the Stanford Reader. The resulting model is competitive with the state of the art.
Output: Sequential Attention
We propose two novel techniques — stacking bottleneck features and minimum generation error training criterion — to improve the performance of deep neural network (DNN)based speech synthesis. The techniques address the related issues of frame-by-frame independence and ignorance of the relationship between static and dynamic features, within current typical DNNbased synthesis frameworks. Stacking bottleneck features, which are an acoustically–informed linguistic representation, provides an efficient way to include more detailed linguistic context at the input. The minimum generation error training criterion minimises overall output trajectory error across an utterance, rather than minimising the error per frame independently, and thus takes into account the interaction between static and dynamic features. The two techniques can be easily combined to further improve performance. We present both objective and subjective results that demonstrate the effectiveness of the proposed techniques. The subjective results show that combining the two techniques leads to significantly more natural synthetic speech than from conventional DNN or long short-term memory (LSTM) recurrent neural network (RNN) systems.
Output: Improving Trajectory Modelling for DNN-based Speech Synthesis by using Stacked Bottleneck Features and Minimum Generation Error Training
When training deep neural networks, it is typically assumed that the training examples are uniformly difficult to learn. Or, to restate, it is assumed that the training error will be uniformly distributed across the training examples. Based on these assumptions, each training example is used an equal number of times. However, this assumption may not be valid in many cases. “Oddball SGD” (novelty-driven stochastic gradient descent) was recently introduced to drive training probabilistically according to the error distribution – training frequency is proportional to training error magnitude. In this article, using a deep neural network to encode a video, we show that oddball SGD can be used to enforce uniform error across the training set.
Output: Uniform Learning in a Deep Neural Network via "Oddball" Stochastic Gradient Descent
In this project, a rather complete proof-theoretical formalization of Lambek Calculus (non-associative with arbitrary extensions) has been ported from Coq proof assistent to HOL4 theorem prover, with some improvements and new theorems. Three deduction systems (Syntactic Calculus, Natural Deduction and Sequent Calculus) of Lambek Calculus are defined with many related theorems proved. The equivalance between these systems are formally proved. Finally, a formalization of Sequent Calculus proofs (where Coq has built-in supports) has been designed and implemented in HOL4. Some basic results including the subformula properties of the so-called “cut-free” proofs are formally proved. This work can be considered as the preliminary work towards a language parser based on category grammars which is not multimodal but still has ability to support context-sensitive languages through customized extensions.
Output: Formalized Lambek Calculus in Higher Order Logic (HOL4)
This paper addresses the problem of predicting the k events that are most likely to occur next, over historical real-time event streams. Existing approaches to causal prediction queries have a number of limitations. First, they exhaustively search over an acyclic causal network to find the most likely k effect events; however, data from real event streams frequently reflect cyclic causality. Second, they contain conservative assumptions intended to exclude all possible non-causal links in the causal network; it leads to the omission of many less-frequent but important causal links. We overcome these limitations by proposing a novel event precedence model and a runtime causal inference mechanism. The event precedence model constructs a first order absorbing Markov chain incrementally over event streams, where an edge between two events signifies a temporal precedence relationship between them, which is a necessary condition for causality. Then, the run-time causal inference mechanism learns causal relationships dynamically during query processing. This is done by removing some of the temporal precedence relationships that do not exhibit causality in the presence of other events in the event precedence model. This paper presents two query processing algorithms – one performs exhaustive search on the model and the other performs a more efficient reduced search with early termination. Experiments using two real datasets (cascading blackouts in power systems and web page views) verify the effectiveness of the probabilistic top-k prediction queries and the efficiency of the algorithms. Specifically, the reduced search algorithm reduced runtime, relative to exhaustive search, by 25− 80% (depending on the application) with only a small reduction in accuracy.
Output: Real-time Top-K Predictive Query Processing over Event Streams
We develop a probabilistic latent-variable model to discover semantic frames—types of events and their participants—from corpora. We present a Dirichlet-multinomial model in which frames are latent categories that explain the linking of verb-subject-object triples, given document-level sparsity. We analyze what the model learns, and compare it to FrameNet, noting it learns some novel and interesting frames. This document also contains a discussion of inference issues, including concentration parameter learning; and a small-scale error analysis of syntactic parsing accuracy. Note: this work was originally posted online October 2012 as part of CMU MLD’s Data Analysis Project requirement. This version has no new experiments or results, but has added some discussion of new related work.
Output: Learning Frames from Text with an Unsupervised Latent Variable Model
Drugs are frequently prescribed to patients with the aim of improving each patient’s medical state, but an unfortunate consequence of most prescription drugs is the occurrence of undesirable side effects. Side effects that occur in more than one in a thousand patients are likely to be signalled efficiently by current drug surveillance methods, however, these same methods may take decades before generating signals for rarer side effects, risking medical morbidity or mortality in patients prescribed the drug while the rare side effect is undiscovered. In this paper we propose a novel computational meta-analysis framework for signalling rare side effects that integrates existing methods, knowledge from the web, metric learning and semi-supervised clustering. The novel framework was able to signal many known rare and serious side effects for the selection of drugs investigated, such as tendon rupture when prescribed Ciprofloxacin or Levofloxacin, renal failure with Naproxen and depression associated with Rimonabant. Furthermore, for the majority of the drug investigated it generated signals for rare side effects at a more stringent signalling threshold than existing methods and shows the potential to become a fundamental part of post marketing surveillance to detect rare side effects.
Output: A Novel Semi-Supervised Algorithm for Rare Prescription Side Effect Discovery
We exhibit a strong link between frequentist PAC-Bayesian bounds and the Bayesian marginal likelihood. That is, for the negative log-likelihood loss function, we show that the minimization of PAC-Bayesian generalization bounds maximizes the Bayesian marginal likelihood. This provides an alternative explanation to the Bayesian Occam’s razor criteria, under the assumption that the data is generated by a i.i.d. distribution. Moreover, as the negative log-likelihood is an unbounded loss function, we motivate and propose a PAC-Bayesian theorem tailored for the sub-Gamma loss family, and we show that our approach is sound on classical Bayesian linear regression tasks.
Output: PAC-Bayesian Theory Meets Bayesian Inference
In educational technology and learning sciences, there are multiple uses for a predictive model of whether a student will perform a task correctly or not. For example, an intelligent tutoring system may use such a model to estimate whether or not a student has mastered a skill. We analyze the significance of data recency in making such predictions, i.e., asking whether relatively more recent observations of a student’s performance matter more than relatively older observations. We develop a new Recent-Performance Factors Analysis model that takes data recency into account. The new model significantly improves predictive accuracy over both existing logistic-regression performance models and over novel baseline models in evaluations on real-world and synthetic datasets. As a secondary contribution, we demonstrate how the widely used cross-validation with 0-1 loss is inferior to AIC and to cross-validation with L1 prediction error loss as a measure of model performance.
Output: Predicting Performance During Tutoring with Models of Recent Performance
We propose a supervised machine learning approach for boosting existing signal and image recovery methods and demonstrate its efficacy on example of image reconstruction in computed tomography. Our technique is based on a local nonlinear fusion of several image estimates, all obtained by applying a chosen reconstruction algorithm with different values of its control parameters. Usually such output images have different bias/variance trade-off. The fusion of the images is performed by feed-forward neural network trained on a set of known examples. Numerical experiments show an improvement in reconstruction quality relatively to existing direct and iterative reconstruction methods.
Output: Spatially-Adaptive Reconstruction in Computed Tomography using Neural Networks
We introduce a new interpretation of two re­ lated notions conditional utility and utility independence. Unlike the traditional inter­ pretation, the new interpretation render the notions the direct analogues of their prob­ abilistic counterparts. To capture these no­ tions formally, we appeal to the notion of util­ ity distribution, introduced in previous paper. We show that utility distributions, which have a structure that is identical to that of probability distributions, can be viewed as a special case of an additive multiattribute utility functions, and show how this special case permits us to capture the novel senses of conditional utility and utility independence. Finally, we present the notion of utility net­ works, which do for utilities what Bayesian networks do for probabilities. Specifically, utility networks exploit the new interpreta­ tion of conditional utility and utility indepen­ dence to compactly represent a utility distri­ bution.
Output: Conditional Utility, Utility Independence, and Utility Networks
We present the first differentially private algorithms for reinforcement learning, which apply to the task of evaluating a fixed policy. We establish two approaches for achieving differential privacy, provide a theoretical analysis of the privacy and utility of the two algorithms, and show promising results on simple empirical examples.
Output: Differentially Private Policy Evaluation∗
The high probability of hardware failures prevents many advanced robots (e.g. legged robots) to be confidently deployed in real-world situations (e.g post-disaster rescue). Instead of attempting to diagnose the failure(s), robots could adapt by trial-and-error in order to be able to complete their tasks. However, the best trial-and-error algorithms for robotics are all episodic: between each trial, the robot needs to be put back in the same state, that is, the robot is not learning autonomously. In this paper, we introduce a novel learning algorithm called “Reset-free Trial-and-Error” (RTE) that allows robots to recover from damage while completing their tasks. We evaluate it on a hexapod robot that is damaged in several ways (e.g. a missing leg, a shortened leg, etc.) and whose objective is to reach a sequence of targets in an arena. Our experiments show that the robot can recover most of its locomotion abilities in a few minutes, in an environment with obstacles, and without any human intervention. Overall, this new algorithm makes it possible to contemplate sending robots to places that are truly too dangerous for humans and in which robots cannot be rescued.
Output: Reset-free Trial-and-Error Learning for Data-Efficient Robot Damage Recovery
Coalition formation is a key topic in multi-agent systems. Coalitions enable agents to achieve goals that they may not have been able to achieve on their own. Previous work has shown problems in coalition games to be computationally hard. Wooldridge and Dunne (Artificial Intelligence 2006) studied the classical computational complexity of several natural decision problems in Coalitional Resource Games (CRG) games in which each agent is endowed with a set of resources and coalitions can bring about a set of goals if they are collectively endowed with the necessary amount of resources. The input of coalitional resource games bundles together several elements, e.g., the agent set Ag, the goal set G, the resource set R, etc. Shrot, Aumann and Kraus (AAMAS 2009) examine coalition formation problems in the CRG model using the theory of Parameterized Complexity. Their refined analysis shows that not all parts of input act equal some instances of the problem are indeed tractable while others still remain intractable. We answer an important question left open by Shrot, Aumann and Kraus by showing that the SC Problem (checking whether a Coalition is Successful) is W[1]-hard when parameterized by the size of the coalition. Then via a single theme of reduction from SC, we are able to show that various problems related to resources, resource bounds and resource conflicts introduced by Wooldridge et al are 1. W[1]-hard or co-W[1]-hard when parameterized by the size of the coalition. 2. para-NP-hard or co-para-NP-hard when parameterized by |R|. 3. FPT when parameterized by either |G| or |Ag|+ |R|. ∗Supported in part by Google Faculty Research Award, ONR Young Investigator Award and NSF CAREER award. †Department of Computer Science , University of Maryland at College Park, USA, email: rchitnis@cs.umd.edu ‡Department of Computer Science , University of Maryland at College Park, USA. email: hajiagha@cs.umd.edu §Department of Computer Science , University of Maryland at College Park, USA. email: vliaghat@cs.umd.edu
Output: Parameterized Complexity of Problems in Coalitional Resource Games
A Verbal Autopsy is the record of an interview about the circumstances of an uncertified death. In developing countries, if a death occurs away from health facilities, a field-worker interviews a relative of the deceased about the circumstances of the death; this Verbal Autopsy can be reviewed offsite. We report on a comparative study of the processes involved in Text Classification applied to classifying Cause of Death: feature value representation; machine learning classification algorithms; and feature reduction strategies in order to identify the suitable approaches applicable to the classification of Verbal Autopsy text. We demonstrate that normalised term frequency and the standard TFiDF achieve comparable performance across a number of classifiers. The results also show Support Vector Machine is superior to other classification algorithms employed in this research. Finally, we demonstrate the effectiveness of employing a ’locally-semisupervised’ feature reduction strategy in order to increase performance accuracy.
Output: A Comparative Study of Machine Learning Methods for Verbal Autopsy Text Classification
We consider principal component analysis for contaminated data-set in the high dimensional regime, where the dimensionality of each observation is comparable or even more than the number of observations. We propose a deterministic high-dimensional robust PCA algorithm which inherits all theoretical properties of its randomized counterpart, i.e., it is tractable, robust to contaminated points, easily kernelizable, asymptotic consistent and achieves maximal robustness – a breakdown point of 50%. More importantly, the proposed method exhibits significantly better computational efficiency, which makes it suitable for large-scale real applications.
Output: Robust PCA in High-dimension: A Deterministic Approach
Background: Lung cancer was known as primary cancers and the survival rate of cancer is about 15%. Early detection of lung cancer is the leading factor in survival rate. All symptoms (features) of lung cancer do not appear until the cancer spreads to other areas. It needs an accurate early detection of lung cancer, for increasing the survival rate. For accurate detection, it need characterizes efficient features and delete redundancy features among all features.Feature selection is the problem of selecting informative features among all features. Materialsand Methods: Lung cancer database consist of 32 patient records with 57 features. This database collected by Hong and Youngand indexed in the University of California Irvine repository. Experimental contents include the extracted from the clinical data and X-ray data, etc. The data described 3 types of pathological lung cancers and all features are taking an integer value 0-3. In our study, new method is proposed for identify efficient features of lung cancer. It is based on Hyper-Heuristic. Results:We obtained an accuracy of 80.63% using reduced 11 feature set.The proposed method compare to the accuracy of 5 machine learning feature selections.The accuracy of these 5 methods are 60.94, 57.81, 68.75, 60.94 and 68.75. Conclusions: The proposed method has better performance with the highest level of accuracy. Therefore, the proposed model is recommended for identifying an efficient symptom of Disease. These finding are very important in health research, particularly in allocation of medical resources for patients who predicted as high-risks
Output: Hyper-Heuristic Algorithm for Finding Efficient Features in Diagnose of Lung Cancer Disease
We consider the question of the stability of evolutionary algorithms to gradual changes,<lb>or drift, in the target concept. We define an algorithm to be resistant to drift if, for<lb>some inverse polynomial drift rate in the target function, it converges to accuracy 1 − ǫ<lb>with polynomial resources, and then stays within that accuracy indefinitely, except with<lb>probability ǫ at any one time. We show that every evolution algorithm, in the sense of<lb>Valiant [19], can be converted using the Correlational Query technique of Feldman [9], into<lb>such a drift resistant algorithm. For certain evolutionary algorithms, such as for Boolean<lb>conjunctions, we give bounds on the rates of drift that they can resist. We develop some<lb>new evolution algorithms that are resistant to significant drift. In particular, we give an<lb>algorithm for evolving linear separators over the spherically symmetric distribution that is<lb>resistant to a drift rate of O(ǫ/n), and another algorithm over the more general product<lb>normal distributions that resists a smaller drift rate. The above translation result can be also interpreted as one on the robustness of the notion of<lb>evolvability itself under changes of definition. As a second result in that direction we show<lb>that every evolution algorithm can be converted to a quasi-monotonic one that can evolve<lb>from any starting point without the performance ever dipping significantly below that of<lb>the starting point. This permits the somewhat unnatural feature of arbitrary performance<lb>degradations to be removed from several known robustness translations.
Output: Evolution with Drifting Targets