niryuu/sni-each-converted huggingface dataset:
Modern conflict-driven clause-learning (CDCL) Boolean SAT solvers provide efficient automatic analysis of real-world feature models (FM) of systems ranging from cars to operating systems. It is well-known that solver-based analysis of real-world FMs scale very well even though SAT instances obtained from such FMs are large, and the corresponding analysis problems are known to be NP-complete. To better understand why SAT solvers are so effective, we systematically studied many syntactic and semantic characteristics of a representative set of large real-world FMs. We discovered that a key reason why large real-world FMs are easy-toanalyze is that the vast majority of the variables in these models are unrestricted, i.e., the models are satisfiable for both true and false assignments to such variables under the current partial assignment. Given this discovery and our understanding of CDCL SAT solvers, we show that solvers can easily find satisfying assignments for such models without too many backtracks relative to the model size, explaining why solvers scale so well. Further analysis showed that the presence of unrestricted variables in these real-world models can be attributed to their high-degree of variability. Additionally, we experimented with a series of well-known nonbacktracking simplifications that are particularly effective in solving FMs. The remaining variables/clauses after simplifications, called the core, are so few that they are easily solved even with backtracking, further strengthening our conclusions. We explain the connection between our findings and backdoors, an idea posited by theorists to explain the power of SAT solvers. This connection strengthens our hypothesis that SAT-based analysis of FMs is easy. In contrast to our findings, previous research characterizes the difficulty of analyzing randomly-generated FMs in terms of treewidth. Our experiments suggest that the difficulty of analyzing realworld FMs cannot be explained in terms of treewidth. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. SPLC 2015, July 20 24, 2015, Nashville, TN, USA c © 2015 ACM. ISBN 978-1-4503-3613-0/15/07. . . $15.00 DOI: http://dx.doi.org/10.1145/2791060.2791070 CCS Concepts •Software and its engineering → Software product lines;
Output: SAT-based Analysis of Large Real-world Feature Models is Easy
A framework is presented for unsupervised learning of representations based on infomax principle for large-scale neural populations. We use an asymptotic approximation to the Shannon’s mutual information for a large neural population to demonstrate that a good initial approximation to the global information-theoretic optimum can be obtained by a hierarchical infomax method. Starting from the initial solution, an efficient algorithm based on gradient descent of the final objective function is proposed to learn representations from the input datasets, and the method works for complete, overcomplete, and undercomplete bases. As confirmed by numerical experiments, our method is robust and highly efficient for extracting salient features from input datasets. Compared with the main existing methods, our algorithm has a distinct advantage in both the training speed and the robustness of unsupervised representation learning. Furthermore, the proposed method is easily extended to the supervised or unsupervised model for training deep structure networks.
Output: NEURAL POPULATION INFOMAX
We study the generalization properties of stochastic gradient methods for learning with convex loss functions and linearly parameterized functions. We show that, in the absence of penalizations or constraints, the stability and approximation properties of the algorithm can be controlled by tuning either the step-size or the number of passes over the data. In this view, these parameters can be seen to control a form of implicit regularization. Numerical results complement the theoretical findings.
Output: Generalization Properties and Implicit Regularization for Multiple Passes SGM
Offline handwriting recognition systems require cropped text line images for both<lb>training and recognition. On the one hand, the annotation of position and tran-<lb>script at line level is costly to obtain. On the other hand, automatic line seg-<lb>mentation algorithms are prone to errors, compromising the subsequent recogni-<lb>tion. In this paper, we propose a modification of the popular and efficient multi-<lb>dimensional long short-term memory recurrent neural networks (MDLSTM-<lb>RNNs) to enable end-to-end processing of handwritten paragraphs. More partic-<lb>ularly, we replace the collapse layer transforming the two-dimensional represen-<lb>tation into a sequence of predictions by a recurrent version which can recognize<lb>one line at a time. In the proposed model, a neural network performs a kind of<lb>implicit line segmentation by computing attention weights on the image represen-<lb>tation. The experiments on paragraphs of Rimes and IAM database yield results<lb>that are competitive with those of networks trained at line level, and constitute a<lb>significant step towards end-to-end transcription of full documents.
Output: Joint Line Segmentation and Transcription for End-to-End Handwritten Paragraph Recognition
The role of sentiment analysis is increasingly emerging to study software developers’ emotions by mining crowdgenerated content within social software engineering tools. However, off-the-shelf sentiment analysis tools have been trained on non-technical domains and general-purpose social media, thus resulting in misclassifications of technical jargon and problem reports. Here, we present Senti4SD, a classifier specifically trained to support sentiment analysis in developers’ communication channels. Senti4SD is trained and validated using a gold standard of Stack Overflow questions, answers, and comments manually annotated for sentiment polarity. It exploits a suite of both lexiconand keyword-based features, as well as semantic features based on word embedding. With respect to a mainstream off-the-shelf tool, which we use as a baseline, Senti4SD reduces the misclassifications of neutral and positive posts as emotionally negative. To encourage replications, we release a lab package including the classifier, the word embedding space, and the gold standard with annotation guidelines.
Output: Sentiment Polarity Detection for Software Development
The human perceptual system can make complex inferences on faces, ranging from the objective evaluations regarding gender, ethnicity, expression, age, identity, etc. to subjective judgments on facial attractiveness, trustworthiness, sociability, friendliness, etc. Whereas the objective aspects have been extensively studied, less attention has been paid to modeling the subjective perception of faces. Here, we adapt 6 state-of-the-art neural networks pretrained on various image tasks (object classification, face identification, face localization) to predict human ratings on 40 social judgments of faces in the 10k US Adult Face Database. Supervised ridge regression on PCA of the conv5 2 layer in VGG-16 network gives best predictions on the average human ratings. Human group agreement was evaluated by repeatedly randomly splitting the raters into two halves for each face, and calculating the Pearson correlation between the two sets of averaged ratings. Due to this methodology, the models correlations with the average human ratings can exceed this score. We find that 1) model performance grows as the consensus on a face trait increases, and 2) model correlations are always higher than human correlations with each other. These results illustrate the learnability of the subjective perception of faces, especially when there is consensus, and the striking versatility and transferability of representations learned for object recognition. This work has strong applications to social robotics, allowing robots to infer human judgments of each other.
Output: Learning to see people like people
This work aims to address the problem of imagebased question-answering (QA) with new models and datasets. In our work, we propose to use recurrent neural networks and visual semantic embeddings without intermediate stages such as object detection and image segmentation. Our model performs 1.8 times better than the recently published results on the same dataset. Another main contribution is an automatic question generation algorithm that converts the currently available image description dataset into QA form, resulting in a 10 times bigger dataset with more evenly distributed answers.
Output: Image Question Answering: A Visual Semantic Embedding Model and a New Dataset
ABSTRACT: The "easy" problem of cognitive science is explaining how and why we can do what we can do. The "hard" problem is explaining how and why we feel. Turing's methodology for cognitive science (the Turing Test) is based on doing: Design a model that can do anything a human can do, indistinguishably from a human, to a human, and you have explained cognition. Searle has shown that the successful model cannot be solely computational. Sensory-motor robotic capacities are necessary to ground some, at least, of the model's words, in what the robot can do with the things in the world that the words are about. But even grounding is not enough to guarantee that -nor to explain how and why -the model feels (if it does). That problem is much harder to solve (and perhaps insoluble).
Output: Alan Turing and the “Hard” and “Easy” Problem of Cognition: Doing and Feeling
Multi Expression Programming (MEP) is an evolutionary technique that may be used for solving computationally difficult problems. MEP uses a linear solution representation. Each MEP individual is a string encoding complex expressions (computer programs). A MEP individual may encode multiple solutions of the current problem. In this paper MEP is used for evolving a Traveling Salesman Problem (TSP) heuristic for graphs satisfying triangle inequality. Evolved MEP heuristic is compared with Nearest Neighbor Heuristic (NN) and Minimum Spanning Tree Heuristic (MST) on some difficult problems in TSPLIB. For most of the considered problems the evolved MEP heuristic outperforms NN and MST. The obtained algorithm was tested against some problems in TSPLIB. The results emphasizes that evolved MEP heuristic is a powerful tool for solving difficult TSP instances.
Output: Evolving TSP heuristics using Multi Expression Programming
The purpose of this paper is to serve as a reference guide for the development of chatterbots implemented with the AIML language. In order to achieve this, the main concepts in Pattern Recognition area are described because the AIML uses such theoretical framework in their syntactic and semantic structures. After that, AIML language is described and each AIML command/tag is followed by an application example. Also, the usage of AIML embedded tags for the handling of sequence dialogue limitations between humans and machines is shown. Finally, computer systems that assist in the design of chatterbots with the AIML language are classified and described.
Output: ARTIFICIAL INTELLIGENCE MARKUP LANGUAGE: A BRIEF TUTORIAL
This paper explores an incremental training strategy for the skip-gram model with negative sampling (SGNS) from both empirical and theoretical perspectives. Existing methods of neural word embeddings, including SNGS, are multi-pass algorithms and thus cannot perform incremental model update. To address this problem, we present a simple incremental extension of SNGS and provide a thorough theoretical analysis to demonstrate its validity. Empirical experiments demonstrated the correctness of the theoretical analysis as well as the practical usefulness of the incremental algorithm.
Output: Incremental Skip-gram Model with Negative Sampling
SentiWordNet is an important lexical resource supporting sentiment analysis in opinion mining applications. In this paper, we propose a novel approach to construct a Vietnamese SentiWordNet (VSWN). SentiWordNet is typically generated from WordNet in which each synset has numerical scores to indicate its opinion polarities. Many previous studies obtained these scores by applying a machine learning method to WordNet. However, Vietnamese WordNet is not available unfortunately by the time of this paper. Therefore, we propose a method to construct VSWN from a Vietnamese dictionary, not from WordNet. We show the effectiveness of the proposed method by generating a VSWN with 39,561 synsets automatically. The method is experimentally tested with 266 synsets with aspect of positivity and negativity. It attains a competitive result compared with English SentiWordNet that is 0.066 and 0.052 differences for positivity and negativity sets respectively.
Output: Construction of Vietnamese SentiWordNet by using Vietnamese Dictionary
Text categorization is the process of grouping documents into categories based on their contents. This process is important to make information retrieval easier, and it became more important due to the huge textual information available online. The main problem in text categorization is how to improve the classification accuracy. Although Arabic text categorization is a new promising field, there are a few researches in this field. This paper proposes a new method for Arabic text categorization using vector evaluation. The proposed method uses a categorized Arabic documents corpus, and then the weights of the tested document's words are calculated to determine the document keywords which will be compared with the keywords of the corpus categorizes to determine the tested document's best category.
Output: ARABIC TEXT CATEGORIZATION ALGORITHM USING VECTOR EVALUATION METHOD
<lb>Sequentially learning to place items in multi-position displays or lists is a task that can be cast into the<lb>multiple-play semi-bandit setting. However, a major concern in this context is when the system cannot decide<lb>whether the user feedback for each item is actually exploitable. Indeed, much of the content may have been<lb>simply ignored by the user. The present work proposes to exploit available information regarding the display<lb>position bias under the so-called Position-based click model (PBM). We first discuss how this model differs<lb>from the Cascade model and its variants considered in several recent works on multiple-play bandits. We<lb>then provide a novel regret lower bound for this model as well as computationally efficient algorithms that<lb>display good empirical and theoretical performance.
Output: Multiple-Play Bandits in the Position-Based Model
Knowledge representation is a key component to the success of all rule based systems including learning classifier systems (LCSs). This component brings insight into how to partition the problem space what in turn seeks prominent role in generalization capacity of the system as a whole. Recently, knowledge representation component has received great deal of attention within data mining communities due to its impacts on rule based systems in terms of efficiency and efficacy. The current work is an attempt to find a comprehensive and yet elaborate view into the existing knowledge representation techniques in LCS domain in general and XCS in specific. To achieve the objectives, knowledge representation techniques are grouped into different categories based on the classification approach in which they are incorporated. In each category, the underlying rule representation schema and the format of classifier condition to support the corresponding representation are presented. Furthermore, a precise explanation on the way that each technique partitions the problem space along with the extensive experimental results is provided. To have an elaborated view on the functionality of each technique, a comparative analysis of existing techniques on some conventional problems is provided. We expect F. Shoeleh Computer Science and Engineering Dept. Shiraz University, Shiraz, Iran. E-mail: shoeleh@cse.shirazu.ac.ir M. Majd E-mail: majd@cse.shirazu.ac.ir A. Hamzeh E-mail: ali@cse.shirazu.ac.ir S. Hashemi E-mail: s hashemi@shirazu.ac.ir this survey to be of interest to the LCS researchers and practitioners since it provides a guideline for choosing a proper knowledge representation technique for a given problem and also opens up new streams of research on this topic.
Output: Knowledge Representation in Learning Classifier Systems: A Survey
Stochastic local search (SLS) algorithms have exhibited great effectiveness in finding models of random instances of the Boolean satisfiability problem (SAT). As one of the most widely known and used SLS algorithm, WalkSAT plays a key role in the evolutions of SLS for SAT, and also hold stateof-the-art performance on random instances. This work proposes a novel implementation for WalkSAT which decreases the redundant calculations leading to a dramatically speeding up, thus dominates the latest version of WalkSAT including its advanced variants.
Output: An Efficient Implementation for WalkSAT
Artificial intelligence is commonly defined as the ability to achieve goals in the world. In the reinforcement learning framework, goals are encoded as reward functions that guide agent behaviour, and the sum of observed rewards provide a notion of progress. However, some domains have no such reward signal, or have a reward signal so sparse as to appear absent. Without reward feedback, agent behaviour is typically random, often dithering aimlessly and lacking intentionality. In this paper we present an algorithm capable of learning purposeful behaviour in the absence of rewards. The algorithm proceeds by constructing temporally extended actions (options), through the identification of purposes that are “just out of reach” of the agents current behaviour. These purposes establish intrinsic goals for the agent to learn, ultimately resulting in a suite of behaviours that encourage the agent to visit different parts of the state space. Moreover, the approach is particularly suited for settings where rewards are very sparse, and such behaviours can help in the exploration of the environment until reward is observed.
Output: Learning Purposeful Behaviour in the Absence of Rewards
<lb>We study a general version of the adversarial online learning problem. We are given a decision<lb>set X in a reflexive Banach space X and a sequence of reward vectors in the dual space of X . At<lb>each iteration, we choose an action from X , based on the observed sequence of previous rewards.<lb>Our goal is to minimize regret, defined as the gap between the realized reward and the reward<lb>of the best fixed action in hindsight. Using results from infinite dimensional convex analysis, we<lb>generalize the method of Dual Averaging (or Follow the Regularized Leader) to our setting and<lb>obtain general upper bounds on the worst-case regret that subsume a wide range of results from the<lb>literature. Under the assumption of uniformly continuous rewards, we obtain explicit anytime regret<lb>bounds in a setting where the decision set is the set of probability distributions on a compact metric<lb>space S whose Radon-Nikodym derivatives are elements of L(S) for some p > 1. Importantly,<lb>we make no convexity assumptions on either the set S or the reward functions. We also prove<lb>a general lower bound on the worst-case regret for any online algorithm. We then apply these<lb>results to the problem of learning in repeated continuous two-player zero-sum games, in which<lb>players’ strategy sets are compact metric spaces. In doing so, we first prove that if both players<lb>play a Hannan-consistent strategy, then with probability 1 the empirical distributions of play weakly<lb>converge to the set of Nash equilibria of the game. We then show that, under mild assumptions,<lb>Dual Averaging on the (infinite-dimensional) space of probability distributions indeed achieves<lb>Hannan-consistency. Finally, we illustrate our results through numerical examples.<lb>
Output: Minimizing Regret on Reflexive Banach Spaces and Learning Nash Equilibria in Continuous Zero-Sum Games
Nowadays, engineers have to develop software often without even knowing which hardware it will eventually run on in numerous mobile phones, tablets, desktops, laptops, data centers, supercomputers and cloud services. Unfortunately, optimizing compilers are not keeping pace with ever increasing complexity of ever changing computer systems anymore and may produce severely underperforming executable codes while wasting expensive resources and energy. We present the first to our knowledge practical, collaborative and publicly available solution to this problem. We help the software engineering community gradually implement and share light-weight wrappers around any software piece with more than one implementation or optimization choice available. These wrappers are connected with a public Collective Mind autotuning infrastructure and repository of knowledge to continuously monitor all important characteristics of these pieces (computational species) across numerous existing hardware configurations in realistic environments together with randomly selected optimizations. At the same time, Collective Mind Node) allows to easily crowdsource time-consuming autotuning across existing Android-based mobile device including commodity mobile phones and tables. Similar to natural sciences, we can now continuously track all winning solutions (optimizations for a given hardware such as compiler flags, OpenCL/CUDA/OpenMP/MPI/skeleton parameters, number of threads and any other exposed by users) that minimize all costs of a computation (execution time, energy spent, code size, failures, memory and storage footprint, optimization time, faults, contentions, inaccuracy and so on) of a given species on a Pareto frontier along with any unexpected behavior at c-mind.org/repo . Furthermore, the community can continuously classify solutions, prune redundant ones, and correlate them with various features of software, its inputs (data sets) and used hardware either manually (similar to Wikipedia) or using available big data analytics and machine learning techniques. Our approach can also help computer engineering community create the first public, realistic, large, diverse, distributed, representative, and continuously evolving benchmark with related optimization knowledge while gradually covering all possible software and hardware to be able to predict best optimizations and improve compilers depending on usage scenarios and requirements. Such continuously growing collective knowledge accessible via simple web service can become an integral part of the practical software and hardware co-design of self-tuning computer systems as we demonstrate in several real usage scenarios validated in industry..
Output: Collective Mind, Part II: Towards Performance- and Cost-Aware Software Engineering as a Natural Science
We study truthful mechanisms for matching and related problems in a partial information<lb>setting, where the agents’ true utilities are hidden, and the algorithm only has access to ordi-<lb>nal preference information. Our model is motivated by the fact that in many settings, agents<lb>cannot express the numerical values of their utility for different outcomes, but are still able<lb>to rank the outcomes in their order of preference. Specifically, we study problems where the<lb>ground truth exists in the form of a weighted graph of agent utilities, but the algorithm can<lb>only elicit the agents’ private information in the form of a preference ordering for each agent<lb>induced by the underlying weights. Against this backdrop, we design truthful algorithms to<lb>approximate the true optimum solution with respect to the hidden weights. Our techniques<lb>yield universally truthful algorithms for a number of graph problems: a 1.76-approximation<lb>algorithm for Max-Weight Matching, 2-approximation algorithm for Max k-matching, a 6-<lb>approximation algorithm for Densest k-subgraph, and a 2-approximation algorithm for Max<lb>Traveling Salesman as long as the hidden weights constitute a metric. We also provide im-<lb>proved approximation algorithms for such problems when the agents are not able to lie about<lb>their preferences. Our results are the first non-trivial truthful approximation algorithms for<lb>these problems, and indicate that in many situations, we can design robust algorithms even<lb>when the agents may lie and only provide ordinal information instead of precise utilities.
Output: Truthful Mechanisms for Matching and Clustering in an Ordinal World
The Cmabrigde Uinervtisy (Cambridge University) effect from the psycholinguistics literature has demonstrated a robust word processing mechanism in humans, where jumbled words (e.g. Cmabrigde / Cambridge) are recognized with little cost. Inspired by the findings from the Cmabrigde Uinervtisy effect, we propose a word recognition model based on a semi-character level recursive neural network (scRNN). In our experiments, we demonstrate that scRNN has significantly more robust performance in word spelling correction (i.e. word recognition) compared to existing spelling checkers. Furthermore, we demonstrate that the model is cognitively plausible by replicating a psycholinguistics experiment about human reading difficulty using our model.
Output: Robsut Wrod Reocginiton via semi-Character Recurrent Neural Network
Segmental conditional random fields (SCRFs) and connectionist temporal classification (CTC) are two sequence labeling objectives used for end-to-end training of speech recognition models. Both models define the transcription probability by marginalizing decisions about latent segmentation alternatives to derive a sequence probability: the former uses a globally normalized joint model of segment labels and durations, and the latter classifies each frame as either an output symbol or a “continuation” of the previous label. In this paper, we train a recognition model by optimizing an interpolation between the SCRF and CTC losses, where the same recurrent neural network (RNN) encoder used for feature extraction for both outputs. We find that this multi-task objective improves recognition accuracy when decoding with either the SCRF or CTC models. Additionally, we show that CTC can also be used to pretrain the RNN encoder, which improves the convergence rate when learning the joint model.
Output: Multi-task Learning with CTC and Segmental CRF for Speech Recognition
We present a novel approach to learning an HMM whose outputs are distributed according to a parametric family. This is done by decoupling the learning task into two steps: first estimating the output parameters, and then estimating the hidden states transition probabilities. The first step is accomplished by fitting a mixture model to the output stationary distribution. Given the parameters of this mixture model, the second step is formulated as the solution of an easily solvable convex quadratic program. We provide an error analysis for the estimated transition probabilities and show they are robust to small perturbations in the estimates of the mixture parameters. Finally, we support our analysis with some encouraging empirical results.
Output: On learning parametric-output HMMs
In many real tasks the features are evolving, with some features being vanished and some other features augmented. For example, in environment monitoring some sensors expired whereas some new ones deployed; in mobile game recommendation some games dropped whereas some new ones added. Learning with such incremental and decremental features is crucial but rarely studied, particularly when the data coming like a stream and thus it is infeasible to keep the whole data for optimization. In this paper, we study this challenging problem and present the OPID approach. Our approach attempts to compress important information of vanished features into functions of survived features, and then expand to include the augmented features. It is the one-pass learning approach, which only needs to scan each instance once and does not need to store the whole data, and thus satisfy the evolving streaming data nature. The effectiveness of our approach is validated theoretically and empirically.
Output: One-Pass Learning with Incremental and Decremental Features
RDF and Description Logics work in an open-world setting where absence of information is not information about absence. Nevertheless, Description Logic axioms can be interpreted in a closed-world setting and in this setting they can be used for both constraint checking and closed-world recognition against information sources. When the information sources are expressed in well-behaved RDF or RDFS (i.e., RDF graphs interpreted in the RDF or RDFS semantics) this constraint checking and closed-world recognition is simple to describe. Further this constraint checking can be implemented as SPARQL querying and thus effectively per-
Output: Using Description Logics for RDF Constraint Checking and Closed-World Recognition
This paper deals with the problem of esti­ mating the probability that one event was a cause of another in a given scenario. Us­ ing structural-semantical definitions of the probabilities of necessary or sufficient cau­ sation (or both), we show how to optimally bound these quantities from data obtained in experimental and observational studies, making minimal assumptions concerning the data-generating process. In particular, we strengthen the results of Pearl (1999) by weakening the data-generation assumptions and deriving theoretically sharp bounds on the probabilities of causation. These results delineate precisely how empirical data can be used both in settling questions of attribution and in solving attribution-related problems of decision making.
Output: Probabilities of Causation: Bounds and Identification
We study the use of greedy feature selection methods for morphosyntactic tagging under a number of different conditions. We compare a static ordering of features to a dynamic ordering based on mutual information statistics, and we apply the techniques to standalone taggers as well as joint systems for tagging and parsing. Experiments on five languages show that feature selection can result in more compact models as well as higher accuracy under all conditions, but also that a dynamic ordering works better than a static ordering and that joint systems benefit more than standalone taggers. We also show that the same techniques can be used to select which morphosyntactic categories to predict in order to maximize syntactic accuracy in a joint system. Our final results represent a substantial improvement of the state of the art for several languages, while at the same time reducing both the number of features and the running time by up to 80% in some cases.
Output: Static and Dynamic Feature Selection in Morphosyntactic Analyzers
We report on a series of experiments with convolutional neural networks (CNN) trained on top of pre-trained word vectors for sentence-level classification tasks. We first show that a simple CNN with little hyperparameter tuning and static vectors achieves excellent results on multiple benchmarks. Learning task-specific vectors through fine-tuning offers further gains in performance. We additionally propose a simple modification to the architecture to allow for the use of both taskspecific and static word vectors. The CNN models discussed herein improve upon the state-of-the-art on 4 out of 7 tasks, which include sentiment analysis and question classification.
Output: Convolutional Neural Networks for Sentence Classification
Chinese word segmentation is a fundamental task for Chinese language processing. The granularity mismatch problem is the main cause of the errors. This paper showed that the binary tree representation can store outputs with different granularity. A binary tree based framework is also designed to overcome the granularity mismatch problem. There are two steps in this framework, namely tree building and tree pruning. The tree pruning step is specially designed to focus on the granularity problem. Previous work for Chinese word segmentation such as the sequence tagging can be easily employed in this framework. This framework can also provide quantitative error analysis methods. The experiments showed that after using a more sophisticated tree pruning function for a state-of-the-art conditional random field based baseline, the error reduction can be up to 20%.
Output: Binary Tree based Chinese Word Segmentation
As statistical classifiers become integrated into realworld applications, it is important to consider not only their accuracy but also their robustness to changes in the data distribution. In this paper, we consider the case where there is an unobserved confounding variable z that influences both the features x and the class variable y. When the influence of z changes from training to testing data, we find that the classifier accuracy can degrade rapidly. In our approach, we assume that we can predict the value of z at training time with some error. The prediction for z is then fed to Pearl’s back-door adjustment to build our model. Because of the attenuation bias caused by measurement error in z, standard approaches to controlling for z are ineffective. In response, we propose a method to properly control for the influence of z by first estimating its relationship with the class variable y, then updating predictions for z to match that estimated relationship. By adjusting the influence of z, we show that we can build a model that exceeds competing baselines on accuracy as well as on robustness over a range of confounding relationships.
Output: Controlling for Unobserved Confounds in Classification Using Correlational Constraints
A rich coreset is a subset of the data which contains nearly all the essential information. We give deterministic, low order polynomial-time algorithms to construct rich coresets for simple and multiple response linear regression, together with lower bounds indicating that there is not much room for improvement upon our results.
Output: Rich Coresets For Constrained Linear Regression
This paper describes our participation in Task 5 track 2 of SemEval 2017 to predict the sentiment of financial news headlines for a specific company on a continuous scale between -1 and 1. We tackled the problem using a number of approaches, utilising a Support Vector Regression (SVR) and a Bidirectional Long Short-Term Memory (BLSTM). We found an improvement of 4-6% using the LSTM model over the SVR and came fourth in the track. We report a number of different evaluations using a finance specific word embedding model and reflect on the effects of using different evaluation metrics.
Output: Lancaster A at SemEval-2017 Task 5: Evaluation metrics matter: predicting sentiment from financial news headlines
The ability of robots to estimate their location is crucial for a wide variety of autonomous operations. In settings where GPS is unavailable, rangeor bearing-only observations relative to a set of fixed beacons provide an effective means of estimating a robot’s location as it navigates. The accuracy of such a beacon-based localization system depends both on how beacons are spatially distributed in the environment, and how the robot’s location is inferred based on noisy measurements of range or bearing. However, it is computationally challenging to search for a placement and an inference strategy that, together, are optimal. Existing methods decouple these decisions, forgoing optimality for tractability. We propose a new optimization approach to jointly determine the beacon placement and inference algorithm. We model inference as a neural network and incorporate beacon placement as a differentiable neural layer. This formulation allows us to optimize placement and inference by jointly training the inference network and beacon layer. We evaluate our method on different localization problems and demonstrate performance that exceeds hand-crafted baselines.
Output: Jointly Optimizing Placement and Inference for Beacon-based Localization
In this paper a new heuristic optimization algorithm has been introduced based on the performance of the major football leagues within each season in EU countries. The algorithm starts with an initial population including three different groups of teams: the wealthiest (strongest), the regular, the poorest (weakest). Each individual of population constitute a football team while each player is an indication of a player in a post. The optimization can hopefully occurs when the competition among the teams in all the leagues is imitated as the strongest teams usually purchase the best players of the regular teams and in turn, regular teams purchase the best players of the weakest who should always discover young players instead of buying professionals. It has been shown that the algorithm can hopefully converge to an acceptable solution solving various benchmarks.
Output: Soccer League Optimization: A heuristic Algorithm Inspired by the Football System in European Countries
Framing is a political strategy in which politicians carefully word their statements in order to control public perception of issues. Previous works exploring political framing typically analyze frame usage in longer texts, such as Congressional speeches. We present a collection of weakly supervised models which harness collective classification to predict the frames used in political discourse on the microblogging platform, Twitter. Our global probabilistic models show that by combining both lexical features of tweets and network-based behavioral features of Twitter, we are able to increase the average, unsupervised F1 score by 21.52 points over a lexical baseline alone.
Output: Leveraging Behavioral and Social Information for Weakly Supervised Collective Classification of Political Discourse on Twitter
For most reinforcement learning approaches, the learning is performed by maximizing an accumulative reward that is expectedly and manually defined for specific tasks. However, in real world, rewards are emergent phenomena from the complex interactions between agents and environments. In this paper, we propose an implicit generic reward model for reinforcement learning. Unlike those rewards that are manually defined for specific tasks, such implicit reward is task independent. It only comes from the deviation from the agents’ previous experiences.
Output: Experience enrichment based task independent reward model
We consider the problem of Bayesian parameter estimation for deep neural networks, which is important in problem settings where we may have little data, and/ or where we need accurate posterior predictive densities p(y|x,D), e.g., for applications involving bandits or active learning. One simple approach to this is to use online Monte Carlo methods, such as SGLD (stochastic gradient Langevin dynamics). Unfortunately, such a method needs to store many copies of the parameters (which wastes memory), and needs to make predictions using many versions of the model (which wastes time). We describe a method for “distilling” a Monte Carlo approximation to the posterior predictive density into a more compact form, namely a single deep neural network. We compare to two very recent approaches to Bayesian neural networks, namely an approach based on expectation propagation [HLA15] and an approach based on variational Bayes [BCKW15]. Our method performs better than both of these, is much simpler to implement, and uses less computation at test time.
Output: Bayesian Dark Knowledge
<lb>This work addresses the problem of regret minimization in non-stochastic multi-<lb>armed bandit problems, focusing on performance guarantees that hold with high<lb>probability. Such results are rather scarce in the literature since proving them re-<lb>quires a large deal of technical effort and significant modifications to the standard,<lb>more intuitive algorithms that come only with guarantees that hold on expectation.<lb>One of these modifications is forcing the learner to sample arms from the uniform<lb>distribution at least Ω(<lb>√<lb>T ) times over T rounds, which can adversely affect per-<lb>formance if many of the arms are suboptimal. While it is widely conjectured that<lb>this property is essential for proving high-probability regret bounds, we show in<lb>this paper that it is possible to achieve such strong results without this undesirable<lb>exploration component. Our result relies on a simple and intuitive loss-estimation<lb>strategy called Implicit eXploration (IX) that allows a remarkably clean analy-<lb>sis. To demonstrate the flexibility of our technique, we derive several improved<lb>high-probability bounds for various extensions of the standard multi-armed bandit<lb>framework. Finally, we conduct a simple experiment that illustrates the robustness<lb>of our implicit exploration technique.
Output: Explore no more: Improved high-probability regret bounds for non-stochastic bandits
Deep Neural Networks (DNNs) are analyzed via the theoretical framework of the information bottleneck (IB) principle. We first show that any DNN can be quantified by the mutual information between the layers and the input and output variables. Using this representation we can calculate the optimal information theoretic limits of the DNN and obtain finite sample generalization bounds. The advantage of getting closer to the theoretical limit is quantifiable both by the generalization bound and by the network’s simplicity. We argue that both the optimal architecture, number of layers and features/connections at each layer, are related to the bifurcation points of the information bottleneck tradeoff, namely, relevant compression of the input layer with respect to the output layer. The hierarchical representations at the layered network naturally correspond to the structural phase transitions along the information curve. We believe that this new insight can lead to new optimality bounds and deep learning algorithms.
Output: Deep Learning and the Information Bottleneck Principle
OmniGraph, a novel representation to support a range of NLP classification tasks, integrates lexical items, syntactic dependencies and frame semantic parses into graphs. Feature engineering is folded into the learning through convolution graph kernel learning to explore different extents of the graph. A high-dimensional space of features includes individual nodes to complex networks. In experiments on a text-forecasting problem that predicts stock price change from news for company mentions, OmniGraph beats several benchmarks based on bag-of-words, syntactic dependencies, and semantic trees. The highly expressive features OmniGraph discovers provide insights into the semantics across distinct market sectors. To demonstrate the method’s generality, we also report its high performance results on a fine-grained sentiment corpus.
Output: OmniGraph: Rich Representation and Graph Kernel Learning
Although end-to-end Neural Machine Translation (NMT) has achieved remarkable progress in the past two years, it suffers from a major drawback: translations generated by NMT systems often lack of adequacy. It has been widely observed that NMT tends to repeatedly translate some source words while mistakenly ignoring other words. To alleviate this problem, we propose a novel encoder-decoder-reconstructor framework for NMT. The reconstructor, incorporated into the NMT model, manages to reconstruct the input source sentence from the hidden layer of the output target sentence, to ensure that the information in the source side is transformed to the target side as much as possible. Experiments show that the proposed framework significantly improves the adequacy of NMT output and achieves superior translation result over state-of-theart NMT and statistical MT systems.
Output: Neural Machine Translation with Reconstruction
One of the most prominent challenges in clustering is “the user’s dilemma,” which is the problem of selecting an appropriate clustering algorithm for a specific task. A formal approach for addressing this problem relies on the identification of succinct, user-friendly properties that formally capture when certain clustering methods are preferred over others. Until now these properties focused on advantages of classical Linkage-Based algorithms, failing to identify when other clustering paradigms, such as popular center-based methods, are preferable. We present surprisingly simple new properties that delineate the differences between common clustering paradigms, which clearly and formally demonstrates advantages of centerbased approaches for some applications. These properties address how sensitive algorithms are to changes in element frequencies, which we capture in a generalized setting where every element is associated with a real-valued weight. ∗E-mail: mackerman@fsu.edu †E-mail: shai@cs.uwaterloo.ca ‡E-mail: simina.branzei@gmail.com §E-mail: dloker@cs.uwaterloo.ca 1 ar X iv :1 10 9. 18 44 v2 [ cs .L G ] 4 O ct 2 01 6
Output: Weighted Clustering
We introduce a new discrepancy score between two distributions that gives an indi-<lb>cation on their similarity. While much research has been done to determine if two<lb>samples come from exactly the same distribution, much less research considered<lb>the problem of determining if two finite samples come from similar distributions.<lb>The new score gives an intuitive interpretation of similarity; it optimally perturbs<lb>the distributions so that they best fit each other. The score is defined between<lb>distributions, and can be efficiently estimated from samples. We provide conver-<lb>gence bounds of the estimated score, and develop hypothesis testing procedures<lb>that test if two data sets come from similar distributions. The statistical power of<lb>this procedures is presented in simulations. We also compare the score’s capacity<lb>to detect similarity with that of other known measures on real data.
Output: The Perturbed Variation
We address the problem of localizing noncollaborative WiFi devices in a large region. Our main motive is to localize humans by localizing their WiFi devices, e.g. during search-and-rescue operations after a natural disaster. We use an active sensing approach that relies on Unmanned Aerial Vehicles (UAVs) to collect signal-strength measurements at informative locations. The problem is challenging since the measurement is received at arbitrary times and they are received only when the UAV is in close proximity to the device. For these reasons, it is extremely important to make prudent decision with very few measurements. We use the Bayesian optimization approach based on Gaussian process (GP) regression. This approach works well for our application since GPs give reliable predictions with very few measurements while Bayesian optimization makes a judicious trade-off between exploration and exploitation. In field experiments conducted over a region of 1000 × 1000 m, we show that our approach reduces the search area to less than 100 meters around the WiFi device within 5 minutes only. Overall, our approach localizes the device in less than 15 minutes with an error of less than 20 meters.
Output: UAVs using Bayesian Optimization to Locate WiFi Devices