niryuu/sni-each-converted huggingface dataset:
We propose a novel module, the reviewer module, to improve the encoder-decoder learning framework. The reviewer module is generic, and can be plugged into an existing encoder-decoder model. The reviewer module performs a number of review steps with attention mechanism on the encoder hidden states, and outputs a fact vector after each review step; the fact vectors are used as the input of the attention mechanism in the decoder. We show that the conventional encoderdecoders are a special case of our framework. Empirically, we show that our framework can improve over state-of-the-art encoder-decoder systems on the tasks of image captioning and source code captioning.
Output: Encode, Review, and Decode: Reviewer Module for Caption Generation
Sentence similarity is considered the basis of many natural language tasks such as information retrieval, question answering and text summarization. The semantic meaning between compared text fragments is based on the words’ semantic features and their relationships. This article reviews a set of word and sentence similarity measures and compares them on benchmark datasets. On the studied datasets, results showed that hybrid semantic measures perform better than both knowledge and corpus based measures. General Terms Semantic Similarity, Natural Language Processing, Computational Linguistics, Text Similarity
Output: A Comprehensive Comparative Study of Word and Sentence Similarity Measures
This work introduces a probabilistic-based model for binary CSP that provides a fine grained analysis of its internal structure. Assuming that a domain modification could occur in the CSP, it shows how to express, in a predictive way, the probability that a domain value becomes inconsistent, then it express the expectation of the number of arc-inconsistent values in each domain of the constraint network. Thus, it express the expectation of the number of arc-inconsistent values for the whole constraint network. Next, it provides bounds for each of these three probabilistic indicators. Finally, a polytime algorithm, which propagates the probabilistic information, is presented.
Output: A Probabilistic-Based Model for Binary CSP
For any deep computational processing of language we need evidences, and one such set of evidences is corpus. This paper describes the development of a textbased corpus for the Bishnupriya Manipuri language. A Corpus is considered as a building block for any language processing tasks. Due to the lack of awareness like other Indian languages, it is also studied less frequently. As a result the language still lacks a good corpus and basic language processing tools. As per our knowledge this is the first effort to develop a corpus for Bishnupriya Manipuri language. KeywordsCorpus, language, Bishnupriya Manipuri, word analysis, word frequency
Output: Towards The Development of a Bishnupriya Manipuri Corpus
Collaborative security initiatives are increasingly often ad-<lb>vocated to improve timeliness and effectiveness of threat mit-<lb>igation. Among these, collaborative predictive blacklisting<lb>(CPB) aims to forecast attack sources based on alerts con-<lb>tributed by multiple organizations that might be targeted in<lb>similar ways. Alas, CPB proposals thus far have only fo-<lb>cused on improving hit counts, but overlooked the impact of<lb>collaboration on false positives and false negatives. More-<lb>over, sharing threat intelligence often prompts important pri-<lb>vacy, confidentiality, and liability issues. In this paper, we<lb>first provide a comprehensive measurement analysis of two<lb>state-of-the-art CPB systems: one that uses a trusted cen-<lb>tral party to collect alerts [Soldo et al., Infocom’10] and a<lb>peer-to-peer one relying on controlled data sharing [Freudi-<lb>ger et al., DIMVA’15], studying the impact of collaboration<lb>on both correct and incorrect predictions. Then, we present<lb>a novel privacy-friendly approach that significantly improves<lb>over previous work, achieving a better balance of true and<lb>false positive rates, while minimizing information disclosure.<lb>Finally, we present an extension that allows our system to<lb>scale to very large numbers of organizations.
Output: Building and Measuring Privacy-Preserving Predictive Blacklists
This essay investigates the question of how the naive Bayes classifier and the support vector machine compare in their ability to forecast the Stock Exchange of Thailand. The theory behind the SVM and the naive Bayes classifier is explored. The algorithms are trained using data from the month of January 2010, extracted from the MarketWatch.com website. Input features are selected based on previous studies of the SET100 Index. The Weka 3 software is used to create models from the labeled training data. Mean squared error and proportion of correctly classified instances, and a number of other error measurements are the used to compare the two algorithms. This essay shows that these two algorithms are currently not advanced enough to accurately model the stock exchange. Nevertheless, the naive Bayes is better than the support vector machine at predicting the Stock Exchange of Thailand.
Output: How do the naive Bayes classifier and the Support Vector Machine compare in their ability to forecast the Stock Exchange of Thailand?
We develop a framework for post model selection inference, via marginal screening, in linear regression. At the core of this framework is a result that characterizes the exact distribution of linear functions of the response y, conditional on the model being selected (“condition on selection” framework). This allows us to construct valid confidence intervals and hypothesis tests for regression coefficients that account for the selection procedure. In contrast to recent work in highdimensional statistics, our results are exact (non-asymptotic) and require no eigenvalue-like assumptions on the design matrix X. Furthermore, the computational cost of marginal regression, constructing confidence intervals and hypothesis testing is negligible compared to the cost of linear regression, thus making our methods particularly suitable for extremely large datasets. Although we focus on marginal screening to illustrate the applicability of the condition on selection framework, this framework is much more broadly applicable. We show how to apply the proposed framework to several other selection procedures including orthogonal matching pursuit, non-negative least squares, and marginal screening+Lasso.
Output: Exact Post Model Selection Inference for Marginal Screening
In this paper we describe a deep network architecture that maps visual input to control actions for a robotic planar reaching task with 100% reliability in real-world trials. Our network is trained in simulation and fine-tuned with a limited number of real-world images. The policy search is guided by a kinematics-based controller (K-GPS), which works more effectively and efficiently than ε-Greedy. A critical insight in our system is the need to introduce a bottleneck in the network between the perception and control networks, and to initially train these networks independently.
Output: Vision-Based Reaching Using Modular Deep Networks: from Simulation to the Real World
Automatic image annotation is one of the most challenging problems in machine vision areas. The goal of this task is to predict number of keywords automatically for images captured in real data. Many methods are based on visual features in order to calculate similarities between image samples. But the computation cost of these approaches is very high. These methods require many training samples to be stored in memory. To lessen this burden, a number of techniques have been developed to reduce the number of features in a dataset. Manifold learning is a popular approach to nonlinear dimensionality reduction. In this paper, we investigate Diffusion maps manifold learning method for web image auto-annotation task. Diffusion maps manifold learning method is used to reduce the dimension of some visual features. Extensive experiments and analysis on NUS-WIDE-LITE web image dataset with different visual features show how this manifold learning dimensionality reduction method can be applied effectively to image annotation.
Output: WEB IMAGE ANNOTATION BY DIFFUSION MAPS MANIFOLD LEARNING ALGORITHM
e success of deep learning depends on nding an architecture to t the task. As deep learning has scaled up to more challenging tasks, the architectures have become dicult to design by hand. is paper proposes an automated method, CoDeepNEAT, for optimizing deep learning architectures through evolution. By extending existing neuroevolution methods to topology, components, and hyperparameters, this method achieves results comparable to best human designs in standard benchmarks in object recognition and language modeling. It also supports building a real-world application of automated image captioning on a magazine website. Given the anticipated increases in available computing power, evolution of deep networks is promising approach to constructing deep learning applications in the future.
Output: Evolving Deep Neural Networks
In this paper, we consider the patient similarity matching problem over a cancer cohort of more than 220,000 patients. Our approach first leverages on Word2Vec framework to embed ICD codes into vector-valued representation. We then propose a sequential algorithm for case-control matching on this representation space of diagnosis codes. The novel practice of applying the sequential matching on the vector representation lifted the matching accuracy measured through multiple clinical outcomes. We reported the results on a large-scale dataset to demonstrate the effectiveness of our method. For such a large dataset where most clinical information has been codified, the new method is particularly relevant.
Output: Control Matching via Discharge Code Sequences
Joint extraction of entities and relations is an important task in information extraction. To tackle this problem, we firstly propose a novel tagging scheme that can convert the joint extraction task to a tagging problem. Then, based on our tagging scheme, we study different end-toend models to extract entities and their relations directly, without identifying entities and relations separately. We conduct experiments on a public dataset produced by distant supervision method and the experimental results show that the tagging based methods are better than most of the existing pipelined and joint learning methods. What’s more, the end-to-end model proposed in this paper, achieves the best results on the public dataset.
Output: Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme
We introduce a powerful recurrent neural network based method for novelty detection to the application of detecting radio anomalies. This approach holds promise in significantly increasing the ability of naive anomaly detection to detect small anomalies in highly complex complexity multi-user radio bands. We demonstrate the efficacy of this approach on a number of common real over the air radio communications bands of interest and quantify detection performance in terms of probability of detection an false alarm rates across a range of interference to band power ratios and compare to baseline methods.
Output: Recurrent Neural Radio Anomaly Detection
The activation function of Deep Neural Networks (DNNs) has undergone many changes during the last decades. Since the advent of the well-known non-saturated Rectified Linear Unit (ReLU), many have tried to further improve the performance of the networks with more elaborate functions. Examples are the Leaky ReLU (LReLU) to remove zero gradients and Exponential Linear Unit (ELU) to reduce bias shift. In this paper, we introduce the Parametric ELU (PELU), an adaptive activation function that allows the DNNs to adopt different non-linear behaviors throughout the training phase. We contribute in three ways: (1) we show that PELU increases the network flexibility to counter vanishing gradient, (2) we provide a gradient-based optimization framework to learn the parameters of the function, and (3) we conduct several experiments on MNIST, CIFAR-10/100 and ImageNet with different network architectures, such as NiN, Overfeat, All-CNN, ResNet and Vgg, to demonstrate the general applicability of the approach. Our proposed PELU has shown relative error improvements of 4.45% and 5.68% on CIFAR10 and 100, and as much as 7.28% with only 0.0003% parameter increase on ImageNet, along with faster convergence rate in almost all test scenarios. We also observed that Vgg using PELU tended to prefer activations saturating close to zero, as in ReLU, except at last layer, which saturated near -2. These results suggest that varying the shape of the activations during training along with the other parameters helps to control vanishing gradients and bias shift, thus facilitating learning.
Output: Parametric Exponential Linear Unit for Deep Convolutional Neural Networks
Successive elimination of candidates is often a route to making manipulation intractable to compute. We prove that eliminating candidates does not necessarily increase the computational complexity of manipulation. However, for many voting rules used in practice, the computational complexity increases. For example, it is already known that it is NP-hard to compute how a single voter can manipulate the result of single transferable voting(the elimination version of plurality voting). We show here that it is NP-hard to compute how a single voter can manipulate the result of the elimination version of veto voting, of the closely related Coombs’ rule, and of the elimination versions of a general class of scoring rules.
Output: Eliminating the Weakest Link: Making Manipulation Intractable?
Network embedding (NE) is playing a critical role in network analysis, due to its ability to represent vertices with efficient low-dimensional embedding vectors. However, existing NE models aim to learn a fixed context-free embedding for each vertex, and neglect the diverse roles when interacting with other vertices. In this paper, we assume that one vertex usually shows different aspects when interacting with different neighbor vertices, and should own different embeddings respectively. Therefore, we present ContextAware Network Embedding (CANE), a novel NE model to address this issue. CANE learns context-aware embeddings for vertices with mutual attention mechanism and is expected to model the semantic relationships between vertices more precisely. In experiments, we compare our model with existing NE models on three real-world datasets. Experimental results shows that CANE achieves significant improvement than state-of-the-art methods on link prediction, and comparable performance on vertex classification.
Output: CANE: Context-Aware Network Embedding for Relation Modeling
We describe a question answering model that applies to both images and structured knowledge bases.The model uses natural language strings to automatically assemble neural networks from a collection of composable modules. Parameters for these modules are learned jointly with network-assembly parameters via reinforcement learning, with only (world, question, answer) triples as supervision. Our approach, which we term a dynamic neural module network, achieves state-of-the-art results on benchmark datasets in both visual and structured domains.
Output: Learning to Compose Neural Networks for Question Answering
Recurrent neural network(RNN) has been broadly applied to natural language processing(NLP) problems. This kind of neural network is designed for modeling sequential data and has been testified to be quite efficient in sequential tagging tasks. In this paper, we propose to use bi-directional RNN with long short-term memory(LSTM) units for Chinese word segmentation, which is a crucial preprocess task for modeling Chinese sentences and articles. Classical methods focus on designing and combining hand-craft features from context, whereas bi-directional LSTM network(BLSTM) does not need any prior knowledge or pre-designing, and it is expert in keeping the contextual information in both directions. Experiment result shows that our approach gets stateof-the-art performance in word segmentation on both traditional Chinese datasets and simplified Chinese datasets.
Output: Bi-directional LSTM Recurrent Neural Network for Chinese Word Segmentation
Motivated by concerns for user privacy, we design a steganographic system (“stegosystem”) that enables two users to exchange encrypted messages without an adversary detecting that such an exchange is taking place. We propose a new linguistic stegosystem based on a Long ShortTerm Memory (LSTM) neural network. We demonstrate our approach on the Twitter and Enron email datasets and show that it yields high-quality steganographic text while significantly improving capacity (encrypted bits per word) relative to the state-of-the-art.
Output: Generating Steganographic Text with LSTMs
The Swiss avalanche bulletin is produced twice a day in four languages. Due to the lack of time available for manual translation, a fully automated translation system is employed, based on a catalogue of predefined phrases and predetermined rules of how these phrases can be combined to produce sentences. Because this catalogue of phrases is limited to a small sublanguage, the system is able to automatically translate such sentences from German into the target languages French, Italian and English without subsequent proofreading or correction. Having been operational for two winter seasons, we assess here the quality of the produced texts based on two different surveys where participants rated texts from real avalanche bulletins from both origins, the catalogue of phrases versus manually written and translated texts. With a mean recognition rate of 55%, users can hardly distinguish between the two types of texts, and give very similar ratings with respect to their language quality. Overall, the output from the catalogue system can be considered virtually equivalent to a text written by avalanche forecasters and then manually translated by professional translators. Furthermore, forecasters declared that all relevant situations were captured by the system with sufficient accuracy. Forecaster's working load did not change with the introduction of the catalogue: the extra time to find matching sentences is compensated by the fact that they no longer need to double-check manually translated texts. The reduction of daily translation costs is expected to offset the initial development costs within a few years.
Output: Fully automatic multi-language translation with a catalogue of phrases – successful employment for the Swiss avalanche bulletin
Programs to solve so-called constraint problems are complex pieces of software which require many design decisions to be made more or less arbitrarily by the implementer. These decisions affect the performance of the finished solver significantly [13]. Once a design decision has been made, it cannot easily be reversed, although a different decision may be more appropriate for a particular problem. We investigate using machine learning to make these decisions automatically depending on the problem to solve with the alldifferent constraint as an example. Our system is capable of making non-trivial, multi-level decisions that improve over always making a default choice.
Output: Using machine learning to make constraint solver implementation decisions
In this paper, we develop a framework for information theoretic learning based on infinitely divisible matrices. We formulate an entropy-like functional on positive definite matrices based on Renyi’s axiomatic definition of entropy and examine some key properties of this functional that lead to the concept of infinite divisibility. The proposed formulation avoids the plug in estimation of density and brings along the representation power of reproducing kernel Hilbert spaces. As an application example, we derive a supervised metric learning algorithm using a matrix based analogue to conditional entropy achieving results comparable with the state of the art.
Output: Information Theoretic Learning with Infinitely Divisible Kernels
Special-purpose constraint propagation algorithms frequently make implicit use of short supports — by examining a subset of the variables, they can infer support (a justification that a variable-value pair may still form part of an assignment that satisfies the constraint) for all other variables and values and save substantial work – but short supports have not been studied in their own right. The two main contributions of this paper are the identification of short supports as important for constraint propagation, and the introduction of HaggisGAC, an efficient and effective general purpose propagation algorithm for exploiting short supports. Given the complexity of HaggisGAC, we present it as an optimised version of a simpler algorithm ShortGAC. Although experiments demonstrate the efficiency of ShortGAC compared with other general-purpose propagation algorithms where a compact set of short supports is available, we show theoretically and experimentally that HaggisGAC is even better. We also find that HaggisGAC performs better than GAC-Schema on full-length supports. We also introduce a variant algorithm HaggisGACStable, which is adapted to avoid work on backtracking and in some cases can be faster and have significant reductions in memory use. All the proposed algorithms are excellent for propagating disjunctions of constraints. In all experiments with disjunctions we found our algorithms to be faster than Constructive Or and GAC-Schema by at least an order of magnitude, and up to three orders of magnitude.
Output: Short and Long Supports for Constraint Propagation
Functional neuroimaging can measure the brain’s response to an external stimulus. It is used to perform brain mapping: identifying from these observations the brain regions involved. This problem can be cast into a linear supervised learning task where the neuroimaging data are used as predictors for the stimulus. Brain mapping is then seen as a support recovery problem. On functional MRI (fMRI) data, this problem is particularly challenging as i) the number of samples is small due to limited acquisition time and ii) the variables are strongly correlated. We propose to overcome these difficulties using sparse regression models over new variables obtained by clustering of the original variables. The use of randomization techniques, e.g. bootstrap samples, and clustering of the variables improves the recovery properties of sparse methods. We demonstrate the benefit of our approach on an extensive simulation study as well as two fMRI datasets.
Output: Small-sample brain mapping: sparse recovery on spatially correlated designs with randomization and clustering
A dictionary defines words in terms of other words. Definitions can tell you the meanings of words you don’t know, but only if you know the meanings of the defining words. How many words do you need to know (and which ones) in order to be able to learn all the rest from definitions? We reduced dictionaries to their “grounding kernels” (GKs), about 10% of the dictionary, from which all the other words could be defined. The GK words turned out to have psycholinguistic correlates: they were learned at an earlier age and more concrete than the rest of the dictionary. But one can compress still more: the GK turns out to have internal structure, with a strongly connected “kernel core” (KC) and a surrounding layer, from which a hierarchy of definitional distances can be derived, all the way out to the periphery of the full dictionary. These definitional distances, too, are correlated with psycholinguistic variables (age of acquisition, concreteness, imageability, oral and written frequency) and hence perhaps with the “mental lexicon” in each of our heads.
Output: Hierarchies in Dictionary Definition Space
Nanson’s and Baldwin’s voting rules select a winner by successively eliminating candidates with low Borda scores. We show that these rules have a number of desirable computational properties. In particular, with unweighted votes, it is NP-hard to manipulate either rule with one manipulator, whilst with weighted votes, it is NP-hard to manipulate either rule with a small number of candidates and a coalition of manipulators. As only a couple of other voting rules are known to be NP-hard to manipulate with a single manipulator, Nanson’s and Baldwin’s rules appear to be particularly resistant to manipulation from a theoretical perspective. We also propose a number of approximation methods for manipulating these two rules. Experiments demonstrate that both rules are often difficult to manipulate in practice. These results suggest that elimination style voting rules deserve further study.
Output: Manipulation of Nanson’s and Baldwin’s Rules
<lb>We study the problem of stochastic optimization for deep learning in the paral-<lb>lel computing environment under communication constraints. A new algorithm<lb>is proposed in this setting where the communication and coordination of work<lb>among concurrent processes (local workers), is based on an elastic force which<lb>links the parameters they compute with a center variable stored by the parameter<lb>server (master). The algorithm enables the local workers to perform more explo-<lb>ration, i.e. the algorithm allows the local variables to fluctuate further from the<lb>center variable by reducing the amount of communication between local workers<lb>and the master. We empirically demonstrate that in the deep learning setting, due<lb>to the existence of many local optima, allowing more exploration can lead to the<lb>improved performance. We propose synchronous and asynchronous variants of<lb>the new algorithm. We provide the stability analysis of the asynchronous vari-<lb>ant in the round-robin scheme and compare it with the more common parallelized<lb>method ADMM. We show that the stability of EASGD is guaranteed when a simple<lb>stability condition is satisfied, which is not the case for ADMM. We additionally<lb>propose the momentum-based version of our algorithm that can be applied in both<lb>synchronous and asynchronous settings. Asynchronous variant of the algorithm<lb>is applied to train convolutional neural networks for image classification on the<lb>CIFAR and ImageNet datasets. Experiments demonstrate that the new algorithm<lb>accelerates the training of deep architectures compared to DOWNPOUR and other<lb>common baseline approaches and furthermore is very communication efficient.
Output: Deep learning with Elastic Averaging SGD
The follow the leader (FTL) algorithm, perhaps the simplest of all online learning algorithms, is known to perform well when the loss functions it is used on are convex and positively curved. In this paper we ask whether there are other “lucky” settings when FTL achieves sublinear, “small” regret. In particular, we study the fundamental problem of linear prediction over a non-empty convex, compact domain. Amongst other results, we prove that the curvature of the boundary of the domain can act as if the losses were curved: In this case, we prove that as long as the mean of the loss vectors have positive lengths bounded away from zero, FTL enjoys a logarithmic growth rate of regret, while, e.g., for polytope domains and stochastic data it enjoys finite expected regret. Building on a previously known meta-algorithm, we also get an algorithm that simultaneously enjoys the worst-case guarantees and the bound available for FTL.
Output: Following the Leader and Fast Rates in Linear Prediction: Curved Constraint Sets and Other Regularities∗
Supervised (linear) embedding models like Wsabie [5] and PSI [1] have proven successful at ranking, recommendation and annotation tasks. However, despite being scalable to large datasets they do not take full advantage of the extra data due to their linear nature, and typically underfit. We propose a new class of models which aim to provide improved performance while retaining many of the benefits of the existing class of embedding models. Our new approach works by iteratively learning a linear embedding model where the next iteration’s features and labels are reweighted as a function of the previous iteration. We describe several variants of the family, and give some initial results. 1 (Supervised) Linear Embedding Models Standard linear embedding models are of the form: f(x, y) = xUV y = ∑
Output: Affinity Weighted Embedding
We propose a method combining relational-logic representations with neural network learning. A general lifted architecture, possibly reflecting some background domain knowledge, is described through relational rules which may be handcrafted or learned. The relational rule-set serves as a template for unfolding possibly deep neural networks whose structures also reflect the structures of given training or testing relational examples. Different networks corresponding to different examples share their weights, which co-evolve during training by stochastic gradient descent algorithm. The framework allows for hierarchical relational modeling constructs and learning of latent relational concepts through shared hidden layers weights corresponding to the rules. Discovery of notable relational concepts and experiments on 78 relational learning benchmarks demonstrate favorable performance of the method.
Output: Lifted Relational Neural Networks
This paper addresses the question of how language use affects community reaction to comments in online discussion forums, and the relative importance of the message vs. the messenger. A new comment ranking task is proposed based on community annotated karma in Reddit discussions, which controls for topic and timing of comments. Experimental work with discussion threads from six subreddits shows that the importance of different types of language features varies with the community of interest.
Output: Talking to the crowd: What do people react to in online discussions?
Texts present coherent stories that have a particular theme or overall setting, for example science fiction or western. In this paper, we present a text generation method called rewriting that edits existing human-authored narratives to change their theme without changing the underlying story. We apply the approach to math word problems, where it might help students stay more engaged by quickly transforming all of their homework assignments to the theme of their favorite movie without changing the math concepts that are being taught. Our rewriting method uses a twostage decoding process, which proposes new words from the target theme and scores the resulting stories according to a number of factors defining aspects of syntactic, semantic, and thematic coherence. Experiments demonstrate that the final stories typically represent the new theme well while still testing the original math concepts, outperforming a number of baselines. We also release a new dataset of human-authored rewrites of math word problems in several themes.
Output: A Theme-Rewriting Approach for Generating Algebra Word Problems
Large-scale multi-relational embedding refers to the task of learning the latent representations for entities and relations in large knowledge graphs. An effective and scalable solution for this problem is crucial for the true success of knowledgebased inference in a broad range of applications. This paper proposes a novel framework for optimizing the latent representations with respect to the analogical properties of the embedded entities and relations. By formulating the learning objective in a differentiable fashion, our model enjoys both theoretical power and computational scalability, and significantly outperformed a large number of representative baseline methods on benchmark datasets. Furthermore, the model offers an elegant unification of several well-known methods in multi-relational embedding, which can be proven to be special instantiations of our framework.
Output: Analogical Inference for Multi-relational Embeddings
In many domains it is desirable to assess the pref­ erences of users in a qualitative rather than quan­ titative way. Such representations of qualitative preference orderings form an important compo­ nent of automated decision tools. We propose a graphical representation of preferences that re­ flects conditional dependence and independence of preference statements under a ceteris paribus (all else being equal) interpretation. Such a rep­ resentation is often compact and arguably natural. We describe several search algorithms for domi­ nance testing based on this representation; these algorithms are quite effective, especially in spe­ cific network topologies, such as chainand tree­ structured networks, as well as polytrees.
Output: Reasoning With Conditional Ceteris Paribus Preference Statements
Determining semantic textual similarity is a core research subject in natural language processing. Since vector-based models for sentence representation often use shallow information, capturing accurate semantics is difficult. By contrast, logical semantic representations capture deeper levels of sentence semantics, but their symbolic nature does not offer graded notions of textual similarity. We propose a method for determining semantic textual similarity by combining shallow features with features extracted from natural deduction proofs of bidirectional entailment relations between sentence pairs. For the natural deduction proofs, we use ccg2lambda, a higherorder automatic inference system, which converts Combinatory Categorial Grammar (CCG) derivation trees into semantic representations and conducts natural deduction proofs. Experiments show that our system was able to outperform other logicbased systems and that features derived from the proofs are effective for learning textual similarity.
Output: Determining Semantic Textual Similarity using Natural Deduction Proofs
Visual question answering (VQA) has witnessed great progress since May, 2015 as a classic problem unifying visual and textual data into a system. Many enlightening VQA works explore deep into the image and question encodings and fusing methods, of which attention is the most effective and infusive mechanism. Current attention based methods focus on adequate fusion of visual and textual features, but lack the attention to where people focus to ask questions about the image. Traditional attention based methods attach a single value to the feature at each spatial location, which losses many useful information. To remedy these problems, we propose a general method to perform saliency-like pre-selection on overlapped region features by the interrelation of bidirectional LSTM (BiLSTM), and use a novel element-wise multiplication based attention method to capture more competent correlation information between visual and textual features. We conduct experiments on the large-scale COCO-VQA dataset and analyze the effectiveness of our model demonstrated by strong empirical results.
Output: Task-driven Visual Saliency and Attention-based Visual Question Answering
To cope with the high level of ambiguity faced in domains such as Computer<lb>Vision or Natural Language processing, robust prediction methods often search<lb>for a diverse set of high-quality candidate solutions or proposals. In structured<lb>prediction problems, this becomes a daunting task, as the solution space (image<lb>labelings, sentence parses, etc.) is exponentially large. We study greedy algo-<lb>rithms for finding a diverse subset of solutions in structured-output spaces by<lb>drawing new connections between submodular functions over combinatorial item<lb>sets and High-Order Potentials (HOPs) studied for graphical models. Specifically,<lb>we show via examples that when marginal gains of submodular diversity functions<lb>allow structured representations, this enables efficient (sub-linear time) approxi-<lb>mate maximization by reducing the greedy augmentation step to inference in a<lb>factor graph with appropriately constructed HOPs. We discuss benefits, trade-<lb>offs, and show that our constructions lead to significantly better proposals.
Output: Submodular meets Structured: Finding Diverse Subsets in Exponentially-Large Structured Item Sets
Time series is attracting more attention across statistics, machine learning and pattern recognition as it appears widely in both industry and academia, but few advances has been achieved in effective time series visualization due to its temporal dimensionality and complex dynamics. Inspired by recent effort on using network metrics to characterize time series for classification, we present an approach to visualize time series as complex networks based on first order Markov process and temporal ordering. Different to classical bar charts, line plots and other statistics based graph, our approach delivers more intuitive visualization that better preserves both the temporal dependency and frequency structures. It provides a natural inverse operation to map the graph back to time series, making it possible to use graph statistics to characterize time series for better visual exploration and statistical analysis. Our experimental results suggest the effectiveness on various tasks such as system identification, classification and anomaly detection on both synthetic and the real time series data.
Output: Encoding Temporal Markov Dynamics in Graph for Time Series Visualization
The article deals with the issue of modification of metric classification algorithms. In particular, it studies the algorithm k-Nearest Neighbours for its application to sequential data. A method of generalization of metric classification algorithms is proposed. As a part of it, there has been developed an algorithm for solving the problem of classification and labelling of sequential data. The advantages of the developed algorithm of classification in comparison with the existing one are also discussed in the article. There is a comparison of the effectiveness of the proposed algorithm with the algorithm of CRF in the task of chunking in the open data set CoNLL2000.
Output: Generalization of metric classification algorithms for sequences classification and labelling
This is a machine learning application paper involving big data. We present high-accuracy prediction methods of rare events in semi-structured machine log files, which are produced at high velocity and high volume by NORC’s computer-assisted telephone interviewing (CATI) network for conducting surveys. We judiciously apply natural language processing (NLP) techniques and data-mining strategies to train effective learning and prediction models for classifying uncommon error messages in the log—without access to source code, updated documentation or dictionaries. In particular, our simple but effective approach of features preallocation for learning from imbalanced data coupled with naive Bayes classifiers can be conceivably generalized to supervised or semisupervised learning and prediction methods for other critical events such as cyberattack detection.
Output: Machine Learning for Machine Data from a CATI Network
Mental health forums are online communities where people express their issues and seek help from moderators and other users. In such forums, there are often posts with severe content indicating that the user is in acute distress and there is a risk of attempted self-harm. Moderators need to respond to these severe posts in a timely manner to prevent potential selfharm. However, the large volume of daily posted content makes it difficult for the moderators to locate and respond to these critical posts. We present a framework for triaging user content into four severity categories which are defined based on indications of self-harm ideation. Our models are based on a feature-rich classification framework which includes lexical, psycholinguistic, contextual and topic modeling features. Our approaches improve the state of the art in triaging the content severity in mental health forums by large margins (up to 17% improvement over the F-1 scores). Using the proposed model, we analyze the mental state of users and we show that overall, long-term users of the forum demonstrate a decreased severity of risk over time. Our analysis on the interaction of the moderators with the users further indicates that without an automatic way to identify critical content, it is indeed challenging for the moderators to provide timely response to the users in
Output: Triaging Content Severity in Online Mental Health Forums∗
In this paper, we present a multimodal Recurrent Neural Network (m-RNN) model for generating novel image captions. It directly models the probability distribution of generating a word given previous words and an image. Image captions are generated according to this distribution. The model consists of two sub-networks: a deep recurrent neural network for sentences and a deep convolutional network for images. These two sub-networks interact with each other in a multimodal layer to form the whole m-RNN model. The effectiveness of our model is validated on four benchmark datasets (IAPR TC-12, Flickr 8K, Flickr 30K and MS COCO). Our model outperforms the state-of-the-art methods. In addition, we apply the m-RNN model to retrieval tasks for retrieving images or sentences, and achieves significant performance improvement over the state-of-the-art methods which directly optimize the ranking objective function for retrieval. The project page of this work is: www.stat.ucla.edu/ ̃junhua.mao/m-RNN.html. 1
Output: DEEP CAPTIONING WITH MULTIMODAL RECURRENT NEURAL NETWORKS (M-RNN)
We present a self-training approach to unsupervised dependency parsing that reuses existing supervised and unsupervised parsing algorithms. Our approach, called ‘iterated reranking’ (IR), starts with dependency trees generated by an unsupervised parser, and iteratively improves these trees using the richer probability models used in supervised parsing that are in turn trained on these trees. Our system achieves 1.8% accuracy higher than the stateof-the-part parser of Spitkovsky et al. (2013) on the WSJ corpus.
Output: Unsupervised Dependency Parsing: Let’s Use Supervised Parsers
This paper presents an empirical study of two widely-used sequence prediction models, Conditional Random Fields (CRFs) and Long Short-Term Memory Networks (LSTMs), on two fundamental tasks for Vietnamese text processing, including part-of-speech tagging and named entity recognition. We show that a strong lower bound for labeling accuracy can be obtained by relying only on simple word-based features with minimal handcrafted feature engineering, of 90.65% and 86.03% performance scores on the standard test sets for the two tasks respectively. In particular, we demonstrate empirically the surprising efficiency of word embeddings in both of the two tasks, with both of the two models. We point out that the state-of-the-art LSTMs model does not always outperform significantly the traditional CRFs model, especially on moderate-sized data sets. Finally, we give some suggestions and discussions for efficient use of sequence labeling models in practical applications.
Output: An Empirical Study of Discriminative Sequence Labeling Models for Vietnamese Text Processing
Variational inference provides a powerful tool for approximate probabilistic inference on complex, structured models. Typical variational inference methods, however, require to use inference networks with computationally tractable probability density functions. This largely limits the design and implementation of variational inference methods. We consider wild variational inference methods that do not require tractable density functions on the inference networks, and hence can be applied in more challenging cases. As an example of application, we treat stochastic gradient Langevin dynamics (SGLD) as an inference network, and use our methods to automatically adjust the step sizes of SGLD, yielding significant improvement over the hand-designed step size schemes.
Output: TWO METHODS FOR WILD VARIATIONAL INFERENCE
Sleep stages pattern provides important clues in diagnosing the presence of sleep disorder. By analyzing sleep stages pattern and extracting its features from EEG, EOG, and EMG signals, we can classify sleep stages. This study presents a novel classification model for predicting sleep stages with a high accuracy. The main idea is to combine the generative capability of Deep Belief Network (DBN) with a discriminative ability and sequence pattern recognizing capability of Long Short-term Memory (LSTM). We use DBN that is treated as an automatic higher level features generator. The input to DBN is 28 ”handcrafted” features as used in previous sleep stages studies. We compared our method with other techniques which combined DBN with Hidden Markov Model (HMM).In this study, we exploit the sequence or time series characteristics of sleep dataset. To the best of our knowledge, most of the present sleep analysis from polysomnogram relies only on single instanced label (nonsequence) for classification. In this study, we used two datasets: an open data set that is treated as a benchmark; the other dataset is our sleep stages dataset (available for download) to verify the results further. Our experiments showed that the combination of DBN with LSTM gives better overall accuracy 98.75% (Fscore=0.9875) for benchmark dataset and 98.94% (Fscore=0.9894) for MKG dataset. This result is better than the state of the art of sleep stages classification that was 91.31%i. Keywords— sleep stages classification, long short term memory, deep belief network, deep learning
Output: Combining Generative and Discriminative Neural Networks for Sleep Stages Classification
In supervised binary hashing, one wants to learn a function that maps a high-dimensional feature vector to a vector of binary codes, for application to fast image retrieval. This typically results in a difficult optimization problem, nonconvex and nonsmooth, because of the discrete variables involved. Much work has simply relaxed the problem during training, solving a continuous optimization, and truncating the codes a posteriori. This gives reasonable results but is quite suboptimal. Recent work has tried to optimize the objective directly over the binary codes and achieved better results, but the hash function was still learned a posteriori, which remains suboptimal. We propose a general framework for learning hash functions using affinity-based loss functions that uses auxiliary coordinates. This closes the loop and optimizes jointly over the hash functions and the binary codes so that they gradually match each other. The resulting algorithm can be seen as a corrected, iterated version of the procedure of optimizing first over the codes and then learning the hash function. Compared to this, our optimization is guaranteed to obtain better hash functions while being not much slower, as demonstrated experimentally in various supervised datasets. In addition, our framework facilitates the design of optimization algorithms for arbitrary types of loss and hash functions.
Output: Optimizing affinity-based binary hashing using auxiliary coordinates
Abstract This paper reconsiders the problem of the absent-minded driver who must choose between alternatives with different payoff with imperfect recall and varying degrees of knowledge of the system. The classical absent-minded driver problem represents the case with limited information and it has bearing on the general area of communication and learning, social choice, mechanism design, auctions, theories of knowledge, belief, and rational agency. Within the framework of extensive games, this problem has applications to many artificial intelligence scenarios. It is obvious that the performance of the agent improves as information available increases. It is shown that a non-uniform assignment strategy for successive choices does better than a fixed probability strategy. We consider both classical and quantum approaches to the problem. We argue that the superior performance of quantum decisions with access to entanglement cannot be fairly compared to a classical algorithm. If the cognitive systems of agents are taken to have access to quantum resources, or have a quantum mechanical basis, then that can be leveraged into superior performance.
Output: The Absent-Minded Driver Problem Redux