Query: A New View of Predictive State Methods for Dynamical System Learning

Relevant passages:
Hilbert Space Embeddings of Hidden Markov Models <=SEP=> Hidden Markov Models (HMMs) are important tools for modeling sequence data. However, they are restricted to discrete latent states, and are largely restricted to Gaussian and discrete observations. And, learning algorithms for HMMs have predominantly relied on local search heuristics, with the exception of spectral methods such as those described below. We propose a nonparametric HMM that extends traditional HMMs to structured and non-Gaussian continuous distributions. Furthermore, we derive a localminimum-free kernel spectral algorithm for learning these HMMs. We apply our method to robot vision data, slot car inertial sensor data and audio event classification data, and show that in these applications, embedded HMMs exceed the previous state-of-the-art performance.

Closing the learning-planning loop with predictive state representations <=SEP=> A central problem in artificial intelligence is that of planning to maximize future reward under uncertainty in a partially observable environment. In this paper we propose and demonstrate a novel algorithm which accurately learns a model of such an environment directly from sequences of action-observation pairs. We then close the loop from observations to actions by planning in the learned model and recovering a policy which is near-optimal in the original environment. Specifically, we present an efficient and statistically consistent spectral algorithm for learning the parameters of a Predictive State Representation (PSR). We demonstrate the algorithm by learning a model of a simulated high-dimensional, vision-based mobile robot planning task, and then perform approximate point-based planning in the learned PSR. Analysis of our results shows that the algorithm learns a state space which efficiently captures the essential features of the environment. This representation allows accurate prediction with a small number of parameters, and enables successful and efficient planning.

A Spectral Algorithm for Learning Hidden Markov Models <=SEP=> Hidden Markov Models (HMMs) are one of the most fundamental a nd widely used statistical tools for modeling discrete time series. Typically, they are learned using sea rch heuristics (such as the Baum-Welch / EM algorithm), which suffer from the usual local optima issues. While in gen eral these models are known to be hard to learn with samples from the underlying distribution, we provide t h first provably efficient algorithm (in terms of sample and computational complexity) for learning HMMs under a nat ur l separation condition. This condition is roughly analogous to the separation conditions considered for lear ning mixture distributions (where, similarly, these model s are hard to learn in general). Furthermore, our sample compl exity results do not explicitly depend on the number of distinct (discrete) observations — they implicitly depend on this number through spectral properties of the underlyin g HMM. This makes the algorithm particularly applicable to se ttings with a large number of observations, such as those in natural language processing where the space of observati on is sometimes the words in a language. Finally, the algorithm is particularly simple, relying only on a singula r value decomposition and matrix multiplications.

Hilbert space embeddings of conditional distributions with applications to dynamical systems <=SEP=> In this paper, we extend the Hilbert space embedding approach to handle conditional distributions. We derive a kernel estimate for the conditional embedding, and show its connection to ordinary embeddings. Conditional embeddings largely extend our ability to manipulate distributions in Hilbert spaces, and as an example, we derive a nonparametric method for modeling dynamical systems where the belief state of the system is maintained as a conditional embedding. Our method is very general in terms of both the domains and the types of distributions that it can handle, and we demonstrate the effectiveness of our method in various dynamical systems. We expect that conditional embeddings will have wider applications beyond modeling dynamical systems.

Knowledge tracing: Modeling the acquisition of procedural knowledge <=SEP=> This paper describes an effort to model students' changing knowledge state during skill acquisition. Students in this research are learning to write short programs with the ACT Programming Tutor (APT). APT is constructed around a production rule cognitive model of programming knowledge, called theideal student model. This model allows the tutor to solve exercises along with the student and provide assistance as necessary. As the student works, the tutor also maintains an estimate of the probability that the student has learned each of the rules in the ideal model, in a process calledknowledge tracing. The tutor presents an individualized sequence of exercises to the student based on these probability estimates until the student has ‘mastered’ each rule. The programming tutor, cognitive model and learning and performance assumptions are described. A series of studies is reviewed that examine the empirical validity of knowledge tracing and has led to modifications in the process. Currently the model is quite successful in predicting test performance. Further modifications in the modeling process are discussed that may improve performance levels.

Differential evolution algorithm with multiple mutation strategies based on roulette wheel selection <=SEP=> In this paper, we propose a differential evolution (DE) algorithm variant with a combination of multiple mutation strategies based on roulette wheel selection, which we call MMRDE. We first propose a new, reflection-based mutation operation inspired by the reflection operations in the Nelder–Mead method. We design an experiment to compare its performance with seven mutation strategies, and we prove its effectiveness at balancing exploration and exploitation of DE. Although our reflection-based mutation strategy can balance exploration and exploitation of DE, it is still prone to premature convergence or evolutionary stagnation when solving complex multimodal optimization problems. Therefore, we add two basic strategies to help maintain population diversity and increase the robustness. We use roulette wheel selection to arrange mutation strategies based on their success rates for each individual. MMRDE is tested with some improved DE variants based on 28 benchmark functions for real-parameter optimization that have been recommended by the Institute of Electrical and Electronics Engineers CEC2013 special session. Experimental results indicate that the proposed algorithm shows its effectiveness at cooperative work with multiple strategies. It can obtain a good balance between exploration and exploitation. The proposed algorithm can guide the search for a global optimal solution with quick convergence compared with other improved DE variants.

Managing the knowledge lifecycle: A integrated knowledge management process model <=SEP=> The main purpose of this study is to propose an integrated conceptual model for exploring knowledge management process. Reviewing literature on knowledge management and tracing its historical background, definitions and dominant paradigms and analysis 32 frameworks of KM, a relatively integrated model of KM was presented in this paper. The study found many similarities of models and its presents a model which combine all the best features of the models analyzed. A 10-stage integrated model was proposed. These stages include some activities such as knowledge goal setting, identification, acquisition, creation, organization, sharing, evaluation, preservation, retention and updating and KM effectiveness evaluation. The findings can be used by managers in all organizational environments to implementation and auditing of KM practices.

An Augmented Reality museum guide <=SEP=> Recent years have seen advances in many enabling augmented reality technologies. Furthermore, much research has been carried out on how augmented reality can be used to enhance existing applications. This paper describes our experiences with an AR-museum guide that combines some of the latest technologies. Amongst other technologies, markerless tracking, hybrid tracking, and an ultra-mobile-PC were used. Like existing audio guides, the AR-guide can be used by any museum visitor, during a six-month exhibition on Islamic art. We provide a detailed description of the museumpsilas motivation for using AR, of our experiences in developing the system, and the initial results of user surveys. Taking this information into account, we can derive possible system improvements.

Security in Cloud Computing Using Cryptographic Algorithms Miss . <=SEP=> Cloud computing is the concept implemented to decipher the Daily Computing Problems. Cloud computing is basically virtual pool of resources and it provides these resources to users via internet. Cloud computing is the internet based development and used in computer technology. The prevalent problem associated with cloud computing is data privacy, security, anonymity and reliability etc. But the most important between them is security and how cloud provider assures it. To secure the Cloud means secure the treatments (calculations) and storage (databases hosted by the Cloud provider). In this paper we analyses different security issues to cloud and different cryptographic algorithms adoptable to better security for the cloud. Keywords— Cloud Computing, Cryptographic Algorithm, Internet, Security Algorithms, Security Attacks, Security Issue

CEFAM: Comprehensive Evaluation Framework for Agile Methodologies <=SEP=> Agile software development is regarded as an effective and efficient approach, mainly due to its ability to accommodate rapidly changing requirements, and to cope with modern software development challenges. There is therefore a strong tendency to use agile software development methodologies where applicable; however, the sheer number of existing agile methodologies and their variants hinders the selection of an appropriate agile methodology or method chunk. Methodology evaluation tools address this problem through providing detailed evaluations, yet no comprehensive evaluation framework is available for agile methodologies. We introduce the comprehensive evaluation framework for agile methodologies (CEFAM) as an evaluation tool for project managers and method engineers. The hierarchical (and mostly quantitative) evaluation criterion set introduced in this evaluation framework enhances the usability of the framework and provides results that are precise enough to be useful for the selection, adaptation and construction of agile methodologies.

Design and operation of Wi-Fi agribot integrated system <=SEP=> Robotics in agriculture is not a new concept; in controlled environments (green houses), it has a history of over 20 years. Research has been performed to develop harvesters for cherry tomatoes, cucumbers, mushrooms, and other fruits. In horticulture, robots have been introduced to harvest citrus and apples. In this paper autonomous robot for agriculture (AgriBot) is a prototype and implemented for performing various agricultural activities like seeding, weeding, spraying of fertilizers, insecticides. AgriBot is controlled with a Arduino Mega board having At mega 2560 microcontroller. The powerful Raspberry Pi a mini computer is used to control and monitor the working of the robot. The Arduino Mega is mounted on a robot allowing for access to all of the pins for rapid prototyping. Its hexapod body can autonomously walk in any direction, avoiding objects with its ultrasonic proximity sensor. Its walking algorithms allow it to instantly change direction and walk in any new direction without turning its body. An underbody sensory array allows the robot to know if a seed has been planted in the area at the optimal spacing and depth. AgriBot can then dig a hole, plant a seed in the hole, cover the seed with soil, and apply any pre-emergence fertilizers and/or herbicides along with the marking agent. AgriBot can then signal to other robots in the immediate proximity that it needs help planting in that area or that this area has been planted and to move on by communicating through Wi-Fi.

A practical VEP-based brain-computer interface <=SEP=> This paper introduces the development of a practical brain-computer interface at Tsinghua University. The system uses frequency-coded steady-state visual evoked potentials to determine the gaze direction of the user. To ensure more universal applicability of the system, approaches for reducing user variation on system performance have been proposed. The information transfer rate (ITR) has been evaluated both in the laboratory and at the Rehabilitation Center of China, respectively. The system has been proved to be applicable to >90% of people with a high ITR in living environments.

Projector Calibration by "Inverse Camera Calibration" <=SEP=> The accuracy of 3-D reconstructions depends substantially on the accuracy of active vision system calibration. In this work, the problem of video projector calibration is solved by inverting the standard camera calibration work flow. The calibration procedure requires a single camera, which does not need to be calibrated and which is used as the sensor whether projected dots and calibration pattern landmarks, such as the checkerboard corners, coincide. The method iteratively adjusts the projected dots to coincide with the landmarks and the final coordinates are used as inputs to a camera calibration method. The otherwise slow iterative adjustment is accelerated by estimating a plane homography between the detected landmarks and the projected dots, which makes the calibration method fast.

FEEL: Featured Event Embedding Learning <=SEP=> Statistical script learning is an effective way to acquire world knowledge which can be used for commonsense reasoning. Statistical script learning induces this knowledge by observing event sequences generated from texts. The learned model thus can predict subsequent events, given earlier events. Recent approaches rely on learning event embeddings which capture script knowledge. In this work, we suggest a general learning model–Featured Event Embedding Learning (FEEL)–for injecting event embeddings with fine grained information. In addition to capturing the dependencies between subsequent events, our model can take into account higher level abstractions of the input event which help the model generalize better and account for the global context in which the event appears. We evaluated our model over three narrative cloze tasks, and showed that our model is competitive with the most recent state-of-the-art. We also show that our resulting embedding can be used as a strong representation for advanced semantic tasks such as discourse parsing and sentence semantic relatedness.

Attribute-based encryption with non-monotonic access structures <=SEP=> We construct an Attribute-Based Encryption (ABE) scheme that allows a user's private key to be expressed in terms of any access formula over attributes. Previous ABE schemes were limited to expressing only monotonic access structures. We provide a proof of security for our scheme based on the Decisional Bilinear Diffie-Hellman (BDH) assumption. Furthermore, the performance of our new scheme compares favorably with existing, less-expressive schemes.

Browser Fingerprinting as user tracking technology <=SEP=> The Web has become an indispensable part of our society and is currently most commonly used mode of information delivery. Millions of users access the free services provided by the websites on daily basis and while providing these free services websites track and profile their web users. In this environment, the ability to track users and their online habits can be very lucrative for advertising companies, yet very intrusive for the privacy of users. The objective of this paper is to study about the increasingly common yet hardly discussed technique of identifying individual Web users and tracking them across multiple websites known as “Browser Fingerprinting”. A unique browser fingerprint is derived by the unique pattern of information visible whenever a computer visits a website. The permutations thus collected are sufficiently distinct that they can be used as a tool for tracking. Unlike cookies, Fingerprints are generated on server side and are difficult for a user to influence. The main objective of this research is study about how the fingerprinting was evolved, its positives and negatives, what threat it poses to users' online privacy and what countermeasures could be used to prevent it. This paper will also analyse which different properties the browsers send to the server, allowing a unique fingerprint of those browsers to be created.

Zone Divisional Hierarchical Routing Protocol for Wireless Sensor Network <=SEP=> Clustering prolongs energy resources, improves scalability and preserves communication bandwidth. Clustering is either classified as static and dynamic or as equal and unequal. In cluster based routing protocols that employ multi-hop communication, imbalanced energy consumption among the nodes results in hot-spots. Unequal clustering overcomes hot-spots but requires a high overhead and is prone to connectivity issues. To offer guaranteed connectivity and alleviate the hot-spot problem, a zone divisional hierarchical routing protocol has been proposed in this paper. The network is divided into equal sized static rectangular clusters which are assigned to two zones namely near zone and far zone. The zone facing the base station is known as the near zone and rest of the network space makes up the far zone which is further divided into sub-zones. Dual cluster heads approach for sharing the reception, aggregation and forwarding tasks is proposed. The performance evaluation of the proposed protocol against the existing protocols reveals that the method offers energy-efficient multi-hop communication support, uses negligible overhead, prevents creation of hot-spots, avoids early death of nodes, uses balanced energy consumption across the network and maximizes the network lifetime.

MOBA: a New Arena for Game AI <=SEP=> Games have always been popular testbeds for Artificial Intelligence (AI). In the last decade, we have seen the rise of the Multiple Online Battle Arena (MOBA) games, which are the most played games nowadays. In spite of this, there are few works that explore MOBA as a testbed for AI Research. In this paper we present and discuss the main features and opportunities offered by MOBA games to Game AI Research. We describe the various challenges faced along the game and also propose a discrete model that can be used to better understand and explore the game. With this, we aim to encourage the use of MOBA as a novel research platform for Game AI.

A 20-GHz phase-locked loop for 40-gb/s serializing transmitter in 0.13-/spl mu/m CMOS <=SEP=> A 20-GHz phase-locked loop with 4.9 ps/sub pp//0.65 ps/sub rms/ jitter and -113.5 dBc/Hz phase noise at 10-MHz offset is presented. A half-duty sampled-feedforward loop filter that simply replaces the resistor with a switch and an inverter suppresses the reference spur down to -44.0 dBc. A design iteration procedure is outlined that minimizes the phase noise of a negative-g/sub m/ oscillator with a coupled microstrip resonator. Static frequency dividers made of pulsed latches operate faster than those made of flip-flops and achieve near 2:1 frequency range. The phase-locked loop fabricated in a 0.13-/spl mu/m CMOS operates from 17.6 to 19.4GHz and dissipates 480mW.

Color Balance and Fusion for Underwater Image Enhancement <=SEP=> We introduce an effective technique to enhance the images captured underwater and degraded due to the medium scattering and absorption. Our method is a single image approach that does not require specialized hardware or knowledge about the underwater conditions or scene structure. It builds on the blending of two images that are directly derived from a color-compensated and white-balanced version of the original degraded image. The two images to fusion, as well as their associated weight maps, are defined to promote the transfer of edges and color contrast to the output image. To avoid that the sharp weight map transitions create artifacts in the low frequency components of the reconstructed image, we also adapt a multiscale fusion strategy. Our extensive qualitative and quantitative evaluation reveals that our enhanced images and videos are characterized by better exposedness of the dark regions, improved global contrast, and edges sharpness. Our validation also proves that our algorithm is reasonably independent of the camera settings, and improves the accuracy of several image processing applications, such as image segmentation and keypoint matching.

Rationale-Augmented Convolutional Neural Networks for Text Classification <=SEP=> We present a new Convolutional Neural Network (CNN) model for text classification that jointly exploits labels on documents and their constituent sentences. Specifically, we consider scenarios in which annotators explicitly mark sentences (or snippets) that support their overall document categorization, i.e., they provide rationales. Our model exploits such supervision via a hierarchical approach in which each document is represented by a linear combination of the vector representations of its component sentences. We propose a sentence-level convolutional model that estimates the probability that a given sentence is a rationale, and we then scale the contribution of each sentence to the aggregate document representation in proportion to these estimates. Experiments on five classification datasets that have document labels and associated rationales demonstrate that our approach consistently outperforms strong baselines. Moreover, our model naturally provides explanations for its predictions.

Informing Pedagogical Action: Aligning Learning Analytics With Learning Design <=SEP=> This article considers the developing field of learning analytics and argues that to move from small-scale practice to broad scale applicability, there is a need to establish a contextual framework that helps teachers interpret the information that analytics provides. The article presents learning design as a form of documentation of pedagogical intent that can provide the context for making sense of diverse sets of analytic data. We investigate one example of learning design to explore how broad categories of analytics—which we call checkpoint and process analytics—can inform the interpretation of outcomes from a learning design and facilitate pedagogical action.

Maternal depression and early childhood growth in developing countries: systematic review and meta-analysis. <=SEP=> OBJECTIVE
To investigate the relationship between maternal depression and child growth in developing countries through a systematic literature review and meta-analysis.


METHODS
Six databases were searched for studies from developing countries on maternal depression and child growth published up until 2010. Standard meta-analytical methods were followed and pooled odds ratios (ORs) for underweight and stunting in the children of depressed mothers were calculated using random effects models for all studies and for subsets of studies that met strict criteria on study design, exposure to maternal depression and outcome variables. The population attributable risk (PAR) was estimated for selected studies.


FINDINGS
Seventeen studies including a total of 13,923 mother and child pairs from 11 countries met inclusion criteria. The children of mothers with depression or depressive symptoms were more likely to be underweight (OR: 1.5; 95% confidence interval, CI: 1.2-1.8) or stunted (OR: 1.4; 95% CI: 1.2-1.7). Subanalysis of three longitudinal studies showed a stronger effect: the OR for underweight was 2.2 (95% CI: 1.5-3.2) and for stunting, 2.0 (95% CI: 1.0-3.9). The PAR for selected studies indicated that if the infant population were entirely unexposed to maternal depressive symptoms 23% to 29% fewer children would be underweight or stunted.


CONCLUSION
Maternal depression was associated with early childhood underweight and stunting. Rigorous prospective studies are needed to identify mechanisms and causes. Early identification, treatment and prevention of maternal depression may help reduce child stunting and underweight in developing countries.

Application of Artificial Neural Networks to Multiple Criteria Inventory Classification <=SEP=> Original scientific paper Inventory classification is a very important part of inventory control which represents the technique of operational research discipline. A systematic approach to the inventory control and classification may have a significant influence on company competitiveness. The paper describes the results obtained by investigating the application of neural networks in multiple criteria inventory classification. Various structures of a back-propagation neural network have been analysed and the optimal one with the minimum Root Mean Square error selected. The predicted results are compared to those obtained by the multiple criteria classification using the analytical hierarchy process.

Methods for Pretreatment of Lignocellulosic Biomass for Efficient Hydrolysis and Biofuel Production <=SEP=> Industrial & Engineering Chemistry Research is published by the American Chemical Society. 1155 Sixteenth Street N.W., Washington, DC 20036 Review Methods for Pretreatment of Lignocellulosic Biomass for Efficient Hydrolysis and Biofuel Production Parveen Kumar, Diane M. Barrett, Michael J. Delwiche, and Pieter Stroeve Ind. Eng. Chem. Res., Article ASAP • DOI: 10.1021/ie801542g • Publication Date (Web): 20 March 2009 Downloaded from http://pubs.acs.org on March 26, 2009

Automatic Game Design via Mechanic Generation <=SEP=> Game designs often center on the game mechanics— rules governing the logical evolution of the game. We seek to develop an intelligent system that generates computer games. As first steps towards this goal we present a composable and cross-domain representation for game mechanics that draws from AI planning action representations. We use a constraint solver to generate mechanics subject to design requirements on the form of those mechanics—what they do in the game. A planner takes a set of generated mechanics and tests whether those mechanics meet playability requirements—controlling how mechanics function in a game to affect player behavior. We demonstrate our system by modeling and generating mechanics in a role-playing game, platformer game, and combined role-playing-platformer game.

Central Retinal Artery Occlusion and Partial Ophthalmoplegia Caused by Hyaluronic Acid Injection Filler Cosmetic <=SEP=> Central retinal artery occlusion is a rare ocular emergency condition that can lead to a total blindness. Hereby we report a 30-year-old woman with a sudden visual loss of the left eye after being injected by hyaluronic acid filler at the dorsum nasal area for cosmetic purpose. Visual acuity was light perception while anterior segment and intraocular pressure were within normal limit. Partial ophthalmoplegia include restriction to the nasal gaze were found. Funduscopic examination revealed total retinal edema with a cherry red spot at the fovea. Ocular massage was performed as initial management followed by humor aqueous paracentesis. All procedures were done within 90 minutes. The patients discharge with the visual improvement to hand motion and the Partial ophthalmophlegia has been improved after being evaluated less than a month.

Critiquing-based recommenders: survey and emerging trends <=SEP=> Critiquing-based recommender systems elicit users’ feedback, called critiques, which they made on the recommended items. This conversational style of interaction is in contract to the standard model where users receive recommendations in a single interaction. Through the use of the critiquing feedback, the recommender systems are able to more accurately learn the users’ profiles, and therefore suggest better recommendations in the subsequent rounds. Critiquing-based recommenders have been widely studied in knowledge-, content-, and preference-based recommenders and are beginning to be tried in several online websites, such as MovieLens. This article examines the motivation and development of the subject area, and offers a detailed survey of the state of the art concerning the design of critiquing interfaces and development of algorithms for critiquing generation. With the help of categorization analysis, the survey reveals three principal branches of critiquing based recommender systems, using respectively natural language based, system-suggested, and user-initiated critiques. Representative example systems will be presented and analyzed for each branch, and their respective pros and cons will be discussed. Subsequently, a hybrid framework is developed to unify the advantages of different methods and overcome their respective limitations. Empirical findings from user studies are further presented, indicating how hybrid critiquing supports could effectively enable end-users to achieve more confident decisions. Finally, the article will point out several future trends to boost the advance of critiquing-based recommenders.

Expert Level Control of Ramp Metering Based on Multi-Task Deep Reinforcement Learning <=SEP=> This paper shows how the recent breakthroughs in reinforcement learning (RL) that have enabled robots to learn to play arcade video games, walk, or assemble colored bricks, can be used to perform other tasks that are currently at the core of engineering cyberphysical systems. We present the first use of RL for the control of systems modeled by discretized non-linear partial differential equations (PDEs) and devise a novel algorithm to use non-parametric control techniques for large multi-agent systems. Cyberphysical systems (e.g., hydraulic channels, transportation systems, the energy grid, and electromagnetic systems) are commonly modeled by PDEs, which historically have been a reliable way to enable engineering applications in these domains. However, it is known that the control of these PDE models is notoriously difficult. We show how neural network-based RL enables the control of discretized PDEs whose parameters are unknown, random, and time-varying. We introduce an algorithm of mutual weight regularization (MWR), which alleviates the curse of dimensionality of multi-agent control schemes by sharing experience between agents while giving each agent the opportunity to specialize its action policy so as to tailor it to the local parameters of the part of the system it is located in. A discretized PDE, such as the scalar Lighthill–Whitham–Richards PDE can indeed be considered as a macroscopic freeway traffic simulator and which presents the most salient challenges for learning to control large cyberphysical system with multiple agents. We consider two different discretization procedures and show the opportunities offered by applying deep reinforcement for continuous control on both. Using a neural RL PDE controller on a traffic flow simulation based on a Godunov discretization of the San Francisco Bay Bridge, we are able to achieve precise adaptive metering without model calibration thereby beating the state of the art in traffic metering. Furthermore, with the more accurate BeATS simulator, we manage to achieve a control performance on par with ALINEA, a state-of-the-art parametric control scheme, and show how using MWR improves the learning procedure.

Design Issues and Challenges in Wireless Sensor Networks <=SEP=> Wireless Sensor Networks (WSNs) are composed self-organized wireless ad hoc networks which comprise of a large number of resource constrained sensor nodes. The major areas of research in WSN is going on hardware, and operating system of WSN, deployment, architecture, localization, synchronization, programming models, data aggregation and dissemination, database querying, architecture, middleware, quality of service and security. This paper study highlights ongoing research activities and issues that affect the design and performance of Wireless Sensor Network.