niryuu/sni-each-converted huggingface dataset:
Objective: Allowing patients to access their own electronic health record (EHR) notes through online patient portals has the potential to improve patient-centered care. However, medical jargon, which abounds in EHR notes, has been shown to be a barrier for patient EHR comprehension. Existing knowledge bases that link medical jargon to lay terms or definitions play an important role in alleviating this problem but have low coverage of medical jargon in EHRs. We developed a data-driven approach that mines EHRs to identify and rank medical jargon based on its importance to patients, to support the building of EHR-centric lay language resources. Methods: We developed an innovative adapted distant supervision (ADS) model based on support vector machines to rank medical jargon from EHRs. For distant supervision, we utilized the open-access, collaborative consumer health vocabulary, a large, publicly available resource that links lay terms to medical jargon. We explored both knowledge-based features from the Unified Medical Language System and distributed word representations (word embeddings) learned from unlabeled large corpora. We evaluated the ADS model using physician-identified important medical terms. Results: Our ADS model significantly surpassed two state-of-the-art automatic term recognition methods, TF*IDF and C-Value, yielding 0.810 ROC-AUC versus 0.710 and 0.667, respectively. Our model identified over 10K important medical jargon terms after ranking over 100K candidate terms mined from over 7,500 EHR narratives. Conclusion: Our work is an important step towards enriching lexical resources that link medical jargon to lay terms/definitions to support patient EHR comprehension. The identified medical jargon terms and their rankings are available upon request.
Output: Ranking medical jargon in electronic health record notes by adapted distant supervision
An efficient speech to text converter for mobile application is presented in this work. The prime motive is to formulate a system which would give optimum performance in terms of complexity, accuracy, delay and memory requirements for mobile environment. The speech to text converter consists of two stages namely front-end analysis and pattern recognition. The front end analysis involves preprocessing and feature extraction. The traditional voice activity detection algorithms which track only energy cannot successfully identify potential speech from input because the unwanted part of the speech also has some energy and appears to be speech. In the proposed system , VAD that calculates energy of high frequency part separately as zero crossing rate to differentiate noise from speech is used. Mel Frequency Cepstral Coefficient (MFCC) is used as feature extraction method and Generalized Regression Neural Network is used as recognizer. MFCC provides low word error rate and better feature extraction. Neural Network improves the accuracy. Thus a small database containing all possible syllable pronunciation of the user is sufficient to give recognition accuracy closer to 100%. Thus the proposed technique entertains realization of real time speaker independent applications like mobile phones, PDAs etc.
Output: Speaker Independent Continuous Speech to Text Converter for Mobile Application
We propose an information theoretic framework for quantitative assessment of acoustic modeling for hidden Markov model (HMM) based automatic speech recognition (ASR). Acoustic modeling yields the probabilities of HMM sub-word states for a short temporal window of speech acoustic features. We cast ASR as a communication channel where the input sub-word probabilities convey the information about the output HMM state sequence. The quality of the acoustic model is thus quantified in terms of the information transmitted through this channel. The process of inferring the most likely HMM state sequence from the sub-word probabilities is known as decoding. HMM based decoding assumes that an acoustic model yields accurate state-level probabilities and the data distribution given the underlying hidden state is independent of any other state in the sequence. We quantify 1) the acoustic model accuracy and 2) its robustness to mismatch between data and the HMM conditional independence assumption in terms of some mutual information quantities. In this context, exploiting deep neural network (DNN) posterior probabilities leads to a simple and straightforward analysis framework to assess shortcomings of the acoustic model for HMM based decoding. This analysis enables us to evaluate the Gaussian mixture acoustic model (GMM) and the importance of many hidden layers in DNNs without any need of explicit speech recognition. In addition, it sheds light on the contribution of low-dimensional models to enhance acoustic modeling for better compliance with the HMM based decoding requirements.
Output: Information Theoretic Analysis of DNN-HMM Acoustic Modeling
Machine Learning (ML) has found it particularly useful in malware detection. However, as the malware evolves very fast, the stability of the feature extracted from malware serves as a critical issue in malware detection. The recent success of deep learning in image recognition, natural language processing, and machine translation indicates a potential solution for stabilizing the malware detection effectiveness. In this research, we haven’t extract selected any features (e.g., the control-flow of op-code, classes, methods of functions and the timing they are invoked etc.) from Android apps. We develop our own method for translating Android apps into rgb color code and transform them to a fixed-sized encoded image. After that, the encoded image is fed to convolutional neural network (CNN) for automatic feature extraction and learning, reducing the expert’s intervention. Deep learning usually involves a large number of parameters that cannot be learned from only a small dataset. In this way, we currently have collected 1500k Android apps samples, have run our system over these 800k malware samples (benign and malicious samples are roughly equal-sized), and also through our back-end (60 million monthly active users and 10k new malware samples per day), we can effectively detect the malware. We believe that our methodology and the corresponding use of deep learning malware classification can overcome the weakness, and computational cost of the common static/dynamic analysis process or machine learning-based of Android malware detection approach.
Output: R2-D2: ColoR-inspired Convolutional NeuRal Network (CNN)-based AndroiD Malware Detections
We consider online content recommendation with implicit feedback through pairwise comparisons, formalized as the so-called dueling bandit problem. We study the dueling bandit problem in the Condorcet winner setting, and consider two notions of regret: the more well-studied strong regret, which is 0 only when both arms pulled are the Condorcet winner; and the less well-studied weak regret, which is 0 if either arm pulled is the Condorcet winner. We propose a new algorithm for this problem, Winner Stays (WS), with variations for each kind of regret: WS for weak regret (WS-W) has expected cumulative weak regret that is O(N), and O(N log(N)) if arms have a total order; WS for strong regret (WS-S) has expected cumulative strong regret of O(N + N log(T )), and O(N log(N) +N log(T )) if arms have a total order. WS-W is the first dueling bandit algorithm with weak regret that is constant in time. WS is simple to compute, even for problems with many arms, and we demonstrate through numerical experiments on simulated and real data that WS has significantly smaller regret than existing algorithms in both the weakand strong-regret settings.
Output: Dueling Bandits with Weak Regret
In this paper we analyse network motifs in the co-occurrence directed networks constructed from five different texts (four books and one portal) in the Croatian language. After preparing the data and network construction, we perform the network motif analysis. We analyse the motif frequencies and Z-scores in the five networks. We present the triad significance profile for five datasets. Furthermore, we compare our results with the existing results for the linguistic networks. Firstly, we show that the triad significance profile for the Croatian language is very similar with the other languages and all the networks belong to the same family of networks. However, there are certain differences between the Croatian language and other analysed languages. We conclude that this is due to the free word-order of the Croatian language.
Output: Network Motifs Analysis of Croatian Literature
We introduce a batched lazy algorithm for supervised classification using decision trees. It avoids unnecessary visits to irrelevant nodes when it is used to make predictions with either eagerly or lazily trained decision trees. A set of experiments demonstrate that the proposed algorithm can outperform both the conventional and lazy decision tree algorithms in terms of computation time as well as memory consumption, without compromising accuracy.
Output: Batched Lazy Decision Trees
Hierarchical Reinforcement Learning has been previously shown to speed up the convergence rate of RL planning algorithms as well as mitigate feature-based model misspecification Mankowitz et al. (2016a,b); Bacon & Precup (2015). To do so, it utilizes hierarchical abstractions, also known as skills – a type of temporally extended action Sutton et al. (1999) to plan at a higher level, abstracting away from the lower-level details. We incorporate risk sensitivity, also referred to as Situational Awareness (SA) , into hierarchical RL for the first time by defining and learning risk aware skills in a Probabilistic Goal Semi-Markov Decision Process (PG-SMDP). This is achieved using our novel Situational Awareness by Risk-Conscious Skills (SARiCoS) algorithm which comes with a theoretical convergence guarantee. We show in a RoboCup soccer domain that the learned risk aware skills exhibit complex human behaviors such as ‘time-wasting’ in a soccer game. In addition, the learned risk aware skills are able to mitigate reward-based model misspecification.
Output: Situational Awareness by Risk-Conscious Skills
Reasoning does not work well when done in isolation from its significance, both to the needs and interests of an agent and with respect to the wider world. Moreover, those issues may best be handled with a new sort of data structure that goes beyond the knowledge base and incorporates aspects of perceptual knowledge and even more, in which a kind of anticipatory action may be key. Out of the Ivory Tower Reasoning is one of the oldest topics in artificial intelligence (AI). And it has made lots of progress, in the form of commonsense reasoning (CSR), planning, automated theorem-proving, and more. But I suspect it has hit a barrier that must be surmounted if we are to approach anything like human-level inference. Here I give evidence for such a barrier, and ideas about dealing with it, loosely based on evidence from human behavior. In rough synopsis, reasoning does not work well when done in isolation from its broader significance, both for the needs and interests of an agent and for the wider world. Moreover, those issues may best be handled with a new sort of data structure that goes beyond the knowledge base (KB) and incorporates aspects of perceptual knowledge and even more, in which a kind of anticipatory action many be key. I suspect this has ties with recent calls to “put the Science” back in AI (Levesque 2013, Langley 2012). For what I am arguing, in some sense, is that reasoning should be regarded as “in the wild” as events unfold rather than confined to management of an isolated KB; and that this speaks to an agent interacting with the world, rather than a puzzle in abstract inference (yet I will also argue that even “pure” reasoning as in mathematics hugely benefits from many connections with the world). And finally, we then will end up studying the nature of world-embedded cognitive agents, humans included. But this is very broadbrushed and general, whereas my main point is a technical suggestion about reasoning informed by meaning, especially meaning concerning experience and action. This is a slightly modified version of a paper that appeared in AAAI2016. One quick example at the outset: The Wason Selection Task (Wason 1968) shows that human inference is strongly aided when the details of the task at hand have real meaning that the subjects can relate to in terms of things that matter to them, helping keep attention on what is relevant; and this holds even when the task in the abstract is a matter of so-called pure logic. While this could be seen as a defect in human reasoning, something computers would never trip up on, I think it points in the opposite direction: inference without broader meaning is not worth much, and not worth being good at. I will illustrate my main points with a series of examples based on the activities of proving, planning, and understanding.
Output: Five Dimensions of Reasoning in the Wild
Relation extraction is a fundamental task in information extraction. Most existing methods have heavy reliance on annotations labeled by human experts, which are costly and time-consuming. To overcome this drawback, we propose a novel framework, REHESSION, to conduct relation extractor learning using annotations from heterogeneous information source, e.g., knowledge base and domain heuristics. These annotations, referred as heterogeneous supervision, often conflict with each other, which brings a new challenge to the original relation extraction task: how to infer the true label from noisy labels for a given instance. Identifying context information as the backbone of both relation extraction and true label discovery, we adopt embedding techniques to learn the distributed representations of context, which bridges all components with mutual enhancement in an iterative fashion. Extensive experimental results demonstrate the superiority of REHESSION over the state-of-the-art.
Output: Heterogeneous Supervision for Relation Extraction: A Representation Learning Approach
Conditional independence and Markov prop­ erties are powerful tools allowing expression of multidimensional probability distributions by means of low-dimensional ones. As mul­ tidimensional possibilistic models have been studied for several years, the demand for analogous tools in possibility theory seems to be quite natural. This paper is intended to be a promotion of de Cooman's measure­ theoretic approach to possibility theory, as this approach allows us to find analogies to many important results obtained in prob­ abilistic framework. First we recall semi­ graphoid properties of conditional possibilis­ tic independence, parameterized by a contin­ uous t-norm, and find sufficient conditions for a class of Archimedean t-norms to have the graphoid property. Then we introduce Markov properties and factorization of possi­ bility distributions (again parameterized by a continuous t-norm) and find the relation­ ships between them. These results are ac­ companied by a number of counterexamples, which show that the assumptions of specific theorems are substantial.
Output: Conditional Independence and Markov Properties in Possibility Theory
In this paper we compare different types of recurrent units in recurrent neural networks (RNNs). Especially, we focus on more sophisticated units that implement a gating mechanism, such as a long short-term memory (LSTM) unit and a recently proposed gated recurrent unit (GRU). We evaluate these recurrent units on the tasks of polyphonic music modeling and speech signal modeling. Our experiments revealed that these advanced recurrent units are indeed better than more traditional recurrent units such as tanh units. Also, we found GRU to be comparable to LSTM.
Output: Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
The preimage of the activities of all the nodes at a certain level of a deep network is the set of inputs that result in the same node activity. For fully connected multi layer rectifier networks we demonstrate how to compute the preimages of activities at arbitrary levels from knowledge of the parameters in a deep rectifying network by disregarding the effects of max-pooling. If the preimage set of a certain activity in the network contains elements from more than one class it means that these classes are irreversibly mixed. This implies that preimage sets which are piecewise linear manifolds are building blocks for describing the input manifolds specific classes, i.e. all preimages should ideally be from the same class. We believe that the knowledge of how to compute preimages will be valuable in understanding the efficiency displayed by deep learning networks and could potentially be used in designing more efficient training algorithms
Output: THE PREIMAGE OF RECTIFIER NETWORK ACTIVITIES
Driven by the multi-level structure of human intracranial electroencephalogram (iEEG) recordings of epileptic seizures, we introduce a new variant of a hierarchical Dirichlet Process—the multi-level clustering hierarchical Dirichlet Process (MLC-HDP)—that simultaneously clusters datasets on multiple levels. Our seizure dataset contains brain activity recorded in typically more than a hundred individual channels for each seizure of each patient. The MLC-HDP model clusters over channels-types, seizure-types, and patient-types simultaneously. We describe this model and its implementation in detail. We also present the results of a simulation study comparing the MLC-HDP to a similar model, the Nested Dirichlet Process and finally demonstrate the MLC-HDP’s use in modeling seizures across multiple patients. We find the MLC-HDP’s clustering to be comparable to independent human physician clusterings. To our knowledge, the MLCHDP model is the first in the epilepsy literature capable of clustering seizures within and between patients.
Output: A Hierarchical Dirichlet Process Model with Multiple Levels of Clustering for Human EEG Seizure Modeling
Local search methods can quickly find good quality solutions in cases where systematic search methods might take a large amount of time. Moreover, in the context of pattern set mining, exhaustive search methods are not applicable due to the large search space they have to explore. In this paper, we propose the application of stochastic local search to solve the pattern set mining. Specifically, to the task of concept learning. We applied a number of local search algorithms on a standard benchmark instances for pattern set mining and the results show the potentials for further exploration.
Output: Stochastic Local Search for Pattern Set Mining
We present a new system S for handling uncertainty in a quantified modal logic (first-order modal logic). The system is based on both probability theory and proof theory. The system is derived from Chisholm’s epistemology. We concretize Chisholm’s system by grounding his undefined and primitive (i.e. foundational) concept of reasonableness in probability and proof theory. S can be useful in systems that have to interact with humans and provide justifications for their uncertainty. As a demonstration of the system, we apply the system to provide a solution to the lottery paradox. Another advantage of the system is that it can be used to provide uncertainty values for counterfactual statements. Counterfactuals are statements that an agent knows for sure are false. Among other cases, counterfactuals are useful when systems have to explain their actions to users (If I had not done α, then φ would have happened). Uncertainties for counterfactuals fall out naturally from our system. Efficient reasoning in just simple first-order logic is a hard problem. Resolution-based first-order reasoning systems have made significant progress over the last several decades in building systems that have solved non-trivial tasks (even unsolved conjectures in mathematics). We present a sketch of a novel algorithm for reasoning that extends firstorder resolution. Finally, while there have been many systems of uncertainty for propositional logics, first-order logics and propositional modal logics, there has been very little work in building systems of uncertainty for first-order modal logics. The work described below is in progress; and once finished will address this lack.
Output: Strength Factors: An Uncertainty System for a Quantified Modal Logic
With the rise of Social Media, people obtain and share information almost instantly on a 24/7 basis. Many research areas have tried to gain valuable insights from these large volumes of freely available user generated content. The research areas of intelligent transportation systems and smart cities are no exception. However, extracting meaningful and actionable knowledge from user generated content is a complex endeavor. First, each social media service has its own data collection specificities and constraints, second the volume of messages/posts produced can be overwhelming for automatic processing and mining, and last but not the least, social media texts are usually short, informal, with a lot of abbreviations, jargon, slang and idioms. In this thesis, we try to tackle some of the aforementioned challenges with the goal of extracting knowledge from social media streams that might be useful in the context of intelligent transportation systems and smart cities. We designed and developed a framework for collection, processing and mining of geo-located Tweets. More specifically, it provides functionalities for parallel collection of geo-located tweets from multiple pre-defined bounding boxes (cities or regions), including filtering of non complying tweets, text pre-processing for Portuguese and English language, topic modeling, and transportation-specific text classifiers, as well as, aggregation and data visualization. We performed an extensive exploratory data analysis of geo-located tweets in 5 different cities: Rio de Janeiro, São Paulo, New York City, London and Melbourne, comprising a total of more than 43 millions tweets in a period of 3 months. Furthermore, we performed a large scale topic modelling comparison between Rio de Janeiro and São Paulo. As far as we know this is the largest scale content analysis of geo-located tweets from Brazil. Interestingly, most of the topics are shared between both cities which despite being in the same country are considered very different regarding population, economy and lifestyle. We take advantage of recent developments in word embeddings and train such representations from the collections of geo-located tweets. We then use a combination of bag-of-embeddings and traditional bag-of-words to train travel-related classifiers in both Portuguese and English to filter travel-related content from non-related. We created specific gold-standard data to perform empirical evaluation of the resulting classifiers. Results are in line with research work in other application areas by showing the robustness of using word embeddings to learn word similarities that bag-of-words is not able to capture. The source code and resources developed in this dissertation will be publicly available to foster further developments by the research community in smart cities and intelligent transportation systems.
Output: Social Media Text Processing and Semantic Analysis for Smart Cities
Named Entity Recognition (NER) is a key NLP task, which is all the more chal-<lb>lenging on Web and user-generated content with their diverse and continuously<lb>changing language. This paper aims to quantify how this diversity impacts<lb>state-of-the-art NER methods, by measuring named entity (NE) and context<lb>variability, feature sparsity, and their effects on precision and recall. In particu-<lb>lar, our findings indicate that NER approaches struggle to generalise in diverse<lb>genres with limited training data. Unseen NEs, in particular, play an impor-<lb>tant role, which have a higher incidence in diverse genres such as social media<lb>than in more regular genres such as newswire. Coupled with a higher incidence<lb>of unseen features more generally and the lack of large training corpora, this<lb>leads to significantly lower F1 scores for diverse genres as compared to more<lb>regular ones. We also find that leading systems rely heavily on surface forms<lb>found in training data, having problems generalising beyond these, and offer<lb>explanations for this observation.
Output: Generalisation in Named Entity Recognition: A Quantitative Analysis
Passengers’ experience is becoming a key metric to evaluate the air transportation system’s performance. Efficient and robust tools to handle airport operations are needed along with a better understanding of passengers’ interests and concerns. Among various airport operations, this paper studies airport gate scheduling for improved passengers’ experience. Three objectives accounting for passengers, aircraft, and operation are presented. Trade-offs between these objectives are analyzed, and a balancing objective function is proposed. The results show that the balanced objective can improve the efficiency of traffic flow in passenger terminals and on ramps, as well as the robustness of gate operations.
Output: Airport Gate Scheduling for Passengers, Aircraft, and Operation
For mobile telecom operators, it is critical to build preference profiles of their customers and connected users, which can help operators make better marketing strategies, and provide more personalized services. With the deployment of deep packet inspection (DPI) in telecom networks, it is possible for the telco operators to obtain user online preference. However, DPI has its limitations and user preference derived only from DPI faces sparsity and cold start problems. To better infer the user preference, social correlation in telco users network derived from Call Detailed Records (CDRs) with regard to online preference is investigated. Though widely verified in several online social networks, social correlation between online preference of users in mobile telco networks, where the CDRs derived relationship are of less social properties and user mobile internet surfing activities are not visible to neighbourhood, has not been explored at a large scale. Based on a real world telecom dataset including CDRs and preference of more than 550K users for several months, we verified that correlation does exist between online preference in such ambiguous social network. Furthermore, we found that the stronger ties that users build, the more similarity between their preference may have. After defining the preference inferring task as a Top-K recommendation problem, we incorporated Matrix Factorization Collaborative Filtering model with social correlation and tie strength based on call patterns to generate Top-K preferred categories for users. The proposed Tie Strength Augmented Social Recommendation (TSASoRec) model takes data sparsity and cold start user problems into Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. KDD ’16 August 13–17, 2016, San Francisco, California, USA c © 2016 ACM. ISBN XXX-X-XXXX-XXXX-X/XX/XX. . . $10.00 DOI: 10.1145/1235 account, considering both the recorded and missing recorded category entries. The experiment on real dataset shows the proposed model can better infer user preference, especially for cold start users.
Output: On Tie Strength Augmented Social Correlation for Inferring Preference of Mobile Telco Users
This paper demonstrates a data-driven control approach for demand response in real-life residential buildings. The objective is to optimally schedule the heating cycles of the Domestic Hot Water (DHW) buffer to maximize the selfconsumption of the local photovoltaic (PV) production. A modelbased reinforcement learning technique is used to tackle the underlying sequential decision-making problem. The proposed algorithm learns the stochastic occupant behavior, predicts the PV production and takes into account the dynamics of the system. A real-life experiment with six residential buildings is performed using this algorithm. The results show that the self-consumption of the PV production is significantly increased, compared to the default thermostat control. Reinforcement Learning, Demand Response, Domestic Hot Water, Field Experiment March 1, 2017
Output: Using Reinforcement Learning for Demand Response of Domestic Hot Water Buffers: a Real-Life Demonstration
We study the power of different types of adaptive (nonoblivious) adversaries in the setting of prediction with expert advice, under both full-information and bandit feedback. We measure the player’s performance using a new notion of regret, also known as policy regret, which better captures the adversary’s adaptiveness to the player’s behavior. In a setting where losses are allowed to drift, we characterize —in a nearly complete manner— the power of adaptive adversaries with bounded memories and switching costs. In particular, we show that with switching costs, the attainable rate with bandit feedback is Θ̃(T ). Interestingly, this rate is significantly worse than the Θ( √ T ) rate attainable with switching costs in the full-information case. Via a novel reduction from experts to bandits, we also show that a bounded memory adversary can force Θ̃(T ) regret even in the full information case, proving that switching costs are easier to control than bounded memory adversaries. Our lower bounds rely on a new stochastic adversary strategy that generates loss processes with strong dependencies.
Output: Online Learning with Switching Costs and Other Adaptive Adversaries
The practicality of a video surveillance system is adversely limited by the amount of queries that can be placed on human resources and their vigilance in response. To transcend this limitation, a major effort under way is to include software that (fully or at least semi) automatically mines video footage, reducing the burden imposed to the system. Herein, we propose a semi-supervised incremental learning framework for evolving visual streams in order to develop a robust and flexible track classification system. Our proposed method learns from consecutive batches by updating an ensemble in each time. It tries to strike a balance between performance of the system and amount of data which needs to be labelled. As no restriction is considered, the system can address many practical problems in an evolving multi-camera scenario, such as concept drift, class evolution and various length of video streams which have not been addressed before. Experiments were performed on synthetic as well as real-world visual data in non-stationary environments, showing high accuracy with fairly little human collaboration.
Output: Active Mining of Parallel Video Streams
This paper addresses the task of AMR-totext generation by leveraging synchronous node replacement grammar. During training, graph-to-string rules are learned using a heuristic extraction algorithm. At test time, a graph transducer is applied to collapse input AMRs and generate output sentences. Evaluated on SemEval-2016 Task 8, our method gives a BLEU score of 25.62, which is the best reported so far.
Output: AMR-to-text Generation with Synchronous Node Replacement Grammar
The relationship between belief networks and relational databases is examined. Based on this analysis, a method to construct belief networks automatically from statistical rela­ tional data is proposed. A comparison be­ tween our method and other methods shows that our method has several advantages when generalization or prediction is deeded.
Output: From Relational Databases to Belief Networks
Recently emerged intelligent assistants on smartphones and home electronics (e.g., Siri and Alexa) can be seen as novel hybrids of domain-specific taskoriented spoken dialogue systems and open-domain non-task-oriented ones. To realize such hybrid dialogue systems, this paper investigates determining whether or not a user is going to have a chat with the system. To address the lack of benchmark datasets for this task, we construct a new dataset consisting of 15, 160 utterances collected from the real log data of a commercial intelligent assistant (and will release the dataset to facilitate future research activity). In addition, we investigate using tweets and Web search queries for handling open-domain user utterances, which characterize the task of chat detection. Experiments demonstrated that, while simple supervised methods are effective, the use of the tweets and search queries further improves the F1-score from 86.21 to 87.53.
Output: Chat Detection in an Intelligent Assistant: Combining Task-oriented and Non-task-oriented Spoken Dialogue Systems
In this paper, we consider several types of in­ formation and methods of combination asso­ ciated with incomplete probabilistic systems. \Ve discriminate between 'a priori' and evi­ dential information. The former one is a de­ scription of the whole population, the latest is a restriction based on observations for a particular case. Then, we proposse different combination methods for each one of them. 'We also consider conditioning as the hetero­ geneous combination of 'a priori' and eviden­ tial information. The evidential information is represented as a convex set of likelihood func­ tions. These will have an associated possi­ bility distribution with behavior according to classical Possibility Theory.
Output: COMBINATION OF UPPER AND LOWER PROBABILITIES
We describe an adaptation and application of a search-based structured prediction algorithm “Searn” to unsupervised learning problems. We show that it is possible to reduce unsupervised learning to supervised learning and demonstrate a high-quality unsupervised shift-reduce parsing model. We additionally show a close connection between unsupervised Searn and expectation maximization. Finally, we demonstrate the efficacy of a semi-supervised extension. The key idea that enables this is an application of the predict-self idea for unsupervised learning.
Output: Unsupervised Search-based Structured Prediction
As an emerging research topic, online class imbalance learning often combines the challenges of both class imbalance and concept drift. It deals with data streams having very skewed class distributions, where concept drift may occur. It has recently received increased research attention; however, very little work addresses the combined problem where both class imbalance and concept drift coexist. As the first systematic study of handling concept drift in class-imbalanced data streams, this paper first provides a comprehensive review of current research progress in this field, including current research focuses and open challenges. Then, an in-depth experimental study is performed, with the goal of understanding how to best overcome concept drift in online learning with class imbalance. Based on the analysis, a general guideline is proposed for the development of an effective algorithm.
Output: A Systematic Study of Online Class Imbalance Learning with Concept Drift
Clustering large datasets is a fundamental problem with a number of applications in machine learning. Data is often collected on different sites and clustering needs to be performed in a distributed manner with low communication. We would like the quality of the clustering in the distributed setting to match that in the centralized setting for which all the data resides on a single site. In this work, we study both graph and geometric clustering problems in two distributed models: (1) a point-to-point model, and (2) a model with a broadcast channel. We give protocols in both models which we show are nearly optimal by proving almost matching communication lower bounds. Our work highlights the surprising power of a broadcast channel for clustering problems; roughly speaking, to spectrally cluster n points or n vertices in a graph distributed across s servers, for a worst-case partitioning the communication complexity in a point-to-point model is n · s, while in the broadcast model it is n + s. A similar phenomenon holds for the geometric setting as well. We implement our algorithms and demonstrate this phenomenon on real life datasets, showing that our algorithms are also very efficient in practice. ∗A preliminary version of this paper appears at the 30th Annual Conference on Neural Information Processing Systems (NIPS), 2016. †Department of Computer Science, Indiana University, Bloomington, USA. Work supported in part by NSF CCF1525024 and IIS-1633215. Email: jiecchen@indiana.edu ‡Department of Computer Science, University of Bristol, Bristol, UK. h.sun@bristol.ac.uk §IBM Research Almaden, San Jose, USA. dpwoodru@us.ibm.com ¶Department of Computer Science, Indiana University, Bloomington, USA. Work supported in part by NSF CCF1525024 and IIS-1633215. Email: qzhangcs@indiana.edu ar X iv :1 70 2. 00 19 6v 1 [ cs .D S] 1 F eb 2 01 7
Output: Communication-Optimal Distributed Clustering∗
Logical inference algorithms for conditional independence (CI) statements have important applications from testing consistency during knowledge elicitation to constraintbased structure learning of graphical models. We prove that the implication problem for CI statements is decidable, given that the size of the domains of the random variables is known and fixed. We will present an approximate logical inference algorithm which combines a falsification and a novel validation algorithm. The validation algorithm represents each set of CI statements as a sparse 0-1 matrixA and validates instances of the implication problem by solving specific linear programs with constraint matrix A. We will show experimentally that the algorithm is both effective and efficient in validating and falsifying instances of the probabilistic CI implication problem.
Output: Logical Inference Algorithms and Matrix Representations for Probabilistic Conditional Independence
In this paper, a framework for testing Deep Neural Network (DNN) design in Python is presented. First, big data, machine learning (ML), and Artificial Neural Networks (ANNs) are discussed to familiarize the reader with the importance of such a system. Next, the benefits and detriments of implementing such a system in Python are presented. Lastly, the specifics of the system are explained, and some experimental results are presented to prove the effectiveness of the system.
Output: A Framework for Distributed Deep Learning Layer Design in Python
Lipreading, i.e. speech recognition from visual-only recordings of a speaker’s face, can be achieved with a processing pipeline based solely on neural networks, yielding significantly better accuracy than conventional methods. Feedforward and recurrent neural network layers (namely Long Short-Term Memory; LSTM) are stacked to form a single structure which is trained by back-propagating error gradients through all the layers. The performance of such a stacked network was experimentally evaluated and compared to a standard Support Vector Machine classifier using conventional computer vision features (Eigenlips and Histograms of Oriented Gradients). The evaluation was performed on data from 19 speakers of the publicly available GRID corpus. With 51 different words to classify, we report a best word accuracy on held-out evaluation speakers of 79.6% using the end-toend neural network-based solution (11.6% improvement over the best feature-based solution evaluated).
Output: LIPREADING WITH LONG SHORT-TERM MEMORY
We consider the problem of multiple agents sensing and acting in environments<lb>with the goal of maximising their shared utility. In these environments, agents must<lb>learn communication protocols in order to share information that is needed to solve<lb>the tasks. By embracing deep neural networks, we are able to demonstrate end-<lb>to-end learning of protocols in complex environments inspired by communication<lb>riddles and multi-agent computer vision problems with partial observability. We<lb>propose two approaches for learning in these domains: Reinforced Inter-Agent<lb>Learning (RIAL) and Differentiable Inter-Agent Learning (DIAL). The former uses<lb>deep Q-learning, while the latter exploits the fact that, during learning, agents can<lb>backpropagate error derivatives through (noisy) communication channels. Hence,<lb>this approach uses centralised learning but decentralised execution. Our experi-<lb>ments introduce new environments for studying the learning of communication<lb>protocols and present a set of engineering innovations that are essential for success<lb>in these domains.
Output: Learning to Communicate with Deep Multi-Agent Reinforcement Learning
Given the limited performance of 2D cellular automata in terms of space when the number of documents increases and in terms of visualization clusters, our motivation was to experiment these cellular automata by increasing the size to view the impact of size on quality of results. The representation of textual data was carried out by a vector model whose components are derived from the overall balancing of the used corpus Term Frequency – Inverse Document Frequency (TF IDF).The WorldNet thesaurus has been used to address the problem of the lemmatization of the words because the representation used in this study is that of the bags of words. Another independent method of the language was used to represent textual records is that of the n-grams. Several measures of similarity have been tested. To validate the classification we have used two measures of assessment based on the recall and precision (f-measure and entropy). The results are promising and confirm the idea to increase the dimension to the problem of the spatiality of the classes. The results obtained in terms of purity class (ie the minimum value of entropy) shows that the number of documents over longer believes the results are better for 3D cellular automata, which was not obvious to 2D the dimension. In terms of spatial navigation, cellular automata provide very good 3D performance visualization than 2D cellular automata.
Output: Visualization and clustering by 3D cellular automata: Application to unstructured data
Natural language inference (NLI) is a fundamentally important task in natural language processing that has many applications. The recently released Stanford Natural Language Inference (SNLI) corpus has made it possible to develop and evaluate learning-centered methods such as deep neural networks for the NLI task. In this paper, we propose a special long short-term memory (LSTM) architecture for NLI. Our model builds on top of a recently proposed neutral attention model for NLI but is based on a significantly different idea. Instead of deriving sentence embeddings for the premise and the hypothesis to be used for classification, our solution uses a matching-LSTM that performs word-by-word matching of the hypothesis with the premise. This LSTM is able to place more emphasis on important word-level matching results. In particular, we observe that this LSTM remembers important mismatches that are critical for predicting the contradiction or the neutral relationship label. Our experiments on the SNLI corpus show that our model outperforms the state of the art, achieving an accuracy of 86.1% on the test data.
Output: Learning Natural Language Inference with LSTM
In this paper, we present a novel framework incorporating a combination of sparse models in different domains. We posit the observed data as generated from a linear combination of a sparse Gaussian Markov model (with a sparse precision matrix) and a sparse Gaussian independence model (with a sparse covariance matrix). We provide efficient methods for decomposition of the data into two domains, viz., Markov and independence domains. We characterize a set of sufficient conditions for identifiability and model consistency. Our decomposition method is based on a simple modification of the popular `1-penalized maximumlikelihood estimator (`1-MLE). We establish that our estimator is consistent in both the domains, i.e., it successfully recovers the supports of both Markov and independence models, when the number of samples n scales as n = Ω(d log p), where p is the number of variables and d is the maximum node degree in the Markov model. Our conditions for recovery are comparable to those of `1-MLE for consistent estimation of a sparse Markov model, and thus, we guarantee successful high-dimensional estimation of a richer class of models under comparable conditions. Our experiments validate these results and also demonstrate that our models have better inference accuracy under simple algorithms such as loopy belief propagation.
Output: High-Dimensional Covariance Decomposition into Sparse Markov and Independence Domains
As a well-known clustering algorithm, Fuzzy C-Means (FCM) allows each input sample to belong to more than one cluster, providing more flexibility than non-fuzzy clustering methods. However, the accuracy of FCM is subject to false detections caused by noisy records, weak feature selection and low certainty of the algorithm in some cases. The false detections are very important in some decision-making application domains like network security and medical diagnosis, where weak decisions based on such false detections may lead to catastrophic outcomes. They are mainly emerged from making decisions about a subset of records that do not provide enough evidence to make a good decision. In this paper, we propose a method for detecting such ambiguous records in FCM by introducing a certainty factor to decrease invalid detections. This approach enables us to send the detected ambiguous records to another discrimination method for a deeper investigation, thus increasing the accuracy by lowering the error rate. Most of the records are still processed quickly and with low error rate which prevents performance loss compared to similar hybrid methods. Experimental results of applying the proposed method on several datasets from different domains show a significant decrease in error rate as well as improved sensitivity of the algorithm.
Output: Ambiguity-Driven Fuzzy C-Means Clustering: How to Detect Uncertain Clustered Records
Given the incessant growth of documents describing the opinions of different people circulating on the web, including Web 2.0 has made it possible to give an opinion on any product in the net. In this paper, we examine the various opinions expressed in the tweets and classify them (positive, negative or neutral) by using the emoticons for the Bayesian method and adjectives and adverbs for the Turney’s method.
Output: Detecting Opinions in Tweets
In this work we focus on efficient heuristics for solving a class of stochastic planning problems that arise in a variety of business, investment, and industrial applications. The problem is best de­ scribed in terms of future buy and sell contracts. By buying less reliable, but less expensive, buy (supply) contracts, a company or a trader can cover a position of more reliable and more expen­ sive sell contracts. The goal is to maximize the expected net gain (profit) by constructing a close to optimum portfolio out of the available buy and sell contracts. This stochastic planning problem can be formulated as a two-stage stochastic linear programm ing problem with recourse. However, this formalization leads to solutions that are ex­ ponential in the number of possible failure com­ binations. Thus, this approach is not feasible for large scale problems. In this work we investi­ gate heuristic approximation techniques alleviat­ ing the efficiency problem. We primarily focus on the clustering approach and devise heuristics for finding clusterings leading to good approxi­ mations. We illustrate the quality and feasibility of the approach through experimental data.
Output: A Clustering Approach to Solving Large Stochastic Matching Problems