Query: Actor-critic algorithms

Relevant passages:
Average cost temporal-difference learning <=SEP=> We propose a variant of temporal-di!erence learning that approximates average and di!erential costs of an irreducible aperiodic Markov chain. Approximations are comprised of linear combinations of "xed basis functions whose weights are incrementally updated during a single endless trajectory of the Markov chain. We present a proof of convergence (with probability 1) and a characterization of the limit of convergence. We also provide a bound on the resulting approximation error that exhibits an interesting dependence on the `mixing timea of the Markov chain. The results parallel previous work by the authors, involving approximations of discounted cost-to-go. ! 1999 Elsevier Science Ltd. All rights reserved.

Simulation-based optimization of Markov reward processes <=SEP=> We propose a simulation based algorithm for optimizing the average reward in a Markov Reward Process that depends on a set of parameters As a special case the method applies to Markov Decision Processes where optimization takes place within a parametrized set of policies The algorithm involves the simulation of a single sample path and can be implemented on line A convergence result with probability is provided This research was supported by contracts with Siemens AG Munich Germany and Alcatel Bell Belgium and by contract DMI with the National Science Foundation Introduction Markov Decision Processes and the associated dynamic programming DP methodology Ber a Put provide a general framework for posing and analyzing problems of se quential decision making under uncertainty DP methods rely on a suitably de ned value function that has to be computed for every state in the state space However many inter esting problems involve very large state spaces curse of dimensionality In addition DP assumes the availability of an exact model in the form of transition probabilities In many practical situations such a model is not available and one must resort to simulation or experimentation with an actual system For all of these reasons dynamic programming in its pure form may be inapplicable The e orts to overcome the aforementioned di culties involve two main ideas The use of simulation to estimate quantities of interest thus avoiding model based computations The use of parametric representations to overcome the curse of dimensionality Parametric representations and the associated algorithms can be broadly classi ed into three main categories a Parametrized value functions Instead of associating a value V i with each state i one uses a parametric form V i r where r is a vector of tunable parameters weights and V is a so called approximation architecture For example V i r could be the output of a multilayer perceptron with weights r when the input is i Other representations are possible e g involving polynomials linear combina tions of feature vectors state aggregation etc When the main ideas from DP are combined with such parametric representations one obtains methods that go un der the names of reinforcement learning or neuro dynamic programming see BT SB for textbook expositions as well as the references therein A key char acteristic is that policy optimization is carried out in an indirect fashion one tries to obtain a good approximation of the optimal value function of dynamic programming and uses it to construct policies that are close to optimal Such methods are reason ably well though not fully understood and there have been some notable practical successes see BT SB for an overview including the world class backgammon player by Tesauro Tes b Parametrized policies In an alternative approach which is the one considered in this paper the tuning of a parametrized value function is bypassed Instead one considers a class of policies described in terms of a parameter vector Simulation is employed to estimate the gradient of the performance metric with respect to and the policy is improved by updating in a gradient direction In some cases the re quired gradient can be estimated using IPA in nitesimal perturbation analysis see e g HC Gla CR and the references therein For general Markov processes and in the absence of special structure IPA is inapplicable but gradient estimation is still possible using likelihood ratio methods Gly Gly GG LEc GI c Actor critic methods A third approach which is a combination of the rst two includes parametrizations of the policy actor and of the value function critic BSA While such methods seem particularly promising theoretical understand ing has been limited to the impractical case of lookup representations one parameter per state KB This paper concentrates on methods based on policy parametrization and approx imate gradient improvement in the spirit of item b above While we are primarily interested in the case of Markov Decision Processes almost everything applies to the more general case of Markov Reward Processes that depend on a parameter vector and we proceed within this broader context We start with a formula for the gradient of the performance metric that has been presented in di erent forms and for various contexts in Gly CC FH JSJ TH CW We then suggest a method for estimating the terms that appear in that formula This leads to a simulation based method that updates the parameter vector at every regeneration time in an approximate gradient direction Furthermore we show how to construct an on line method that updates the parameter vector at each time step The resulting method has some conceptual similarities with those described in CR that reference assumes however the availability of an IPA estimator with certain guaranteed properties that are absent in our context and in JSJ which however does not contain convergence results The method that we propose only keeps in memory and updates K numbers where K is the dimension of Other than itself this includes a vector similar to the eligibility trace in Sutton s temporal di erence methods and as in JSJ an estimate of the average reward under the current value of If that estimate was accurate our method would be a standard stochastic gradient algorithm However as keeps changing is generally a biased estimate of the true average reward and the mathematical structure of our method is more complex than that of stochastic gradient algorithms For reasons that will become clearer later standard approaches e g martingale arguments or the ODE approach do not seem to su ce for establishing convergence and a more elaborate proof is necessary Our gradient estimator can also be derived or interpreted in terms of likelihood ratios Gly GG It takes the same form as the one presented in p of Gly but it is used di erently The development in Gly leads to a consistent estimator of the gradient assuming that a very large number of regenerative cycles are estimated while keeping the policy parameter at a xed value Presumably would be then updated after such a long simulation In contrast our method updates much more frequently and retains the desired convergence properties despite the fact that any single cycle results in a biased gradient estimate An alternative simulation based stochastic gradient method again based on a likeli hood ratio formula has been provided in Gly and uses the simulation of two regen erative cycles to construct an unbiased estimate of the gradient We note some of the di erences with the latter work First the methods in Gly involve a larger number of auxiliary quantities that are propagated in the course of a regenerative cycle Second our method admits a modi cation see Sections that can make it applicable even if the time until the next regeneration is excessive in which case likelihood ratio based methods su er from excessive variance Third our estimate  of the average reward is obtained as a weighted average of all past rewards not just over the last regenerative cycle In contrast an approach such as the one in Gly would construct an independent estimate of  during each regenerative cycle which should result in higher variance Finally our method brings forth and makes crucial use of the value di erential reward function of dy namic programming This is important because it paves the way for actor critic methods in which the variance associated with the estimates of the di erential rewards is poten tially reduced by means of learning value function approximation Indeed subsequent to the rst writing of this paper this latter approach has been pursued in KT SMS In summary the main contributions of this paper are as follows We introduce a new algorithm for updating the parameters of a Markov Reward Process on the basis of a single sample path The parameter updates can take place either during visits to a certain recurrent state or at every time step We also specialize the method to Markov Decision Processes with parametrically represented policies In this case the method does not require the transition probabilities to be known We establish that the gradient with respect to the parameter vector of the perfor mance metric converges to zero with probability which is the strongest possible result for gradient related stochastic approximation algorithms The method admits approximate variants with reduced variance such as the one described in Section or various types of actor critic methods The remainder of this paper is organized as follows In Section we introduce our framework and assumptions and state some background results including a formula for the gradient of the performance metric In Section we present an algorithm that per forms updates during visits to a certain recurrent state present our main convergence result and provide a heuristic argument Sections and deal with variants of the algo rithm that perform updates at every time step In Section we specialize our methods to the case of Markov Decision Processes that are optimized within a possibly restricted set of parametrically represented randomized policies We present some numerical results in Section and conclude in Section The lengthy proof of our main results is developed in the appendices Markov Reward Processes Depending on a Parameter In this section we present our general framework make a few assumptions and state some basic results that will be needed later We consider a discrete time nite state Markov chain fing with state space S f Ng whose transition probabilities depend on a parameter vector K and are denoted by pij P in j j in i Whenever the state is equal to i we receive a one stage reward that also depends on and is denoted by gi For every K let P be the stochastic matrix with entries pij Let P fP j Kg be the set of all such matrices and let P be its closure Note that every element of P is also a stochastic matrix and therefore de nes a Markov chain on the same s

Neuro-dynamic programming <=SEP=> What should you think more? Time to get this [PDF? It is easy then. You can only sit and stay in your place to get this book. Why? It is on-line book store that provide so many collections of the referred books. So, just with internet connection, you can enjoy downloading this book and numbers of books that are searched for now. By visiting the link page download that we have provided, the book that you refer so much can be found. Just save the requested book downloaded and then you can enjoy the book to read every time and place you want.

Learning from delayed rewards <=SEP=> 

Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming <=SEP=> This paper extends previous work with Dyna a class of architectures for intelligent systems based on approximating dynamic program ming methods Dyna architectures integrate trial and error reinforcement learning and execution time planning into a single process operating alternately on the world and on a learned model of the world In this paper I present and show results for two Dyna archi tectures The Dyna PI architecture is based on dynamic programming s policy iteration method and can be related to existing AI ideas such as evaluation functions and uni versal plans reactive systems Using a nav igation task results are shown for a simple Dyna PI system that simultaneously learns by trial and error learns a world model and plans optimal routes using the evolving world model The Dyna Q architecture is based on Watkins s Q learning a new kind of rein forcement learning Dyna Q uses a less famil iar set of data structures than does Dyna PI but is arguably simpler to implement and use We show that Dyna Q architectures are easy to adapt for use in changing environments Introduction to Dyna How should a robot decide what to do The traditional answer in AI has been that it should deduce its best action in light of its current goals and world model i e that it should plan However it is now widely recognized that planning s usefulness is limited by its computational complexity and by its dependence on an accurate world model An alternative approach is to do the planning in advance and compile its result into a set of rapid reactions or situation action rules which are then used for real time decision making Yet a third approach is to learn a good set of reactions by trial and error this has the advantage of eliminating the dependence on a world model In this paper I brie y introduce Dyna a class of simple architectures integrating and permitting tradeo s among these three approaches Dyna architectures use machine learning algo rithms to approximate the conventional optimal con trol technique known as dynamic programming DP Bellman Ross DP itself is not a learn ing method but rather a computational method for determining optimal behavior given a complete model of the task to be solved It is very similar to state space search but di ers in that it is more incremental and never considers actual action sequences explicitly only single actions at a time This makes DP more amenable to incremental planning at execution time and also makes it more suitable for stochastic or in completely modeled environments as it need not con sider the extremely large number of sequences possi ble in an uncertain environment Learned world mod els are likely to be stochastic and uncertain making DP approaches particularly promising for learning sys tems Dyna architectures are those that learn a world model online while using approximations to DP to learn and plan optimal behavior Intuitively Dyna is based on the old idea that planning is like trial and error learning from hypothet ical experience Craik Dennett The theory of Dyna is based on the theory of DP e g Ross and on DP s relationship to reinforcement learning Watkins Barto Sutton Watkins to temporal di erence learning Sutton and to AI methods for planning and search Korf Werbos has previously argued for the general idea of building AI systems that approx imate dynamic programming and Whitehead and others Sutton Barto Sutton Pinette Rumelhart et al have presented results for the speci c idea of augmenting a reinforcement learning system with a world model used for planning Dyna PI Dyna by Approximating Policy Iteration I call the rst Dyna architecture Dyna PI because it is based on approximating a DP method known as pol icy iteration Howard The Dyna PI architec ture consists of four components interacting as shown in Figure The policy is simply the function formed by the current set of reactions it receives as input a description of the current state of the world and pro duces as output an action to be sent to the world The world represents the task to be solved prototypi cally it is the robot s external environment The world receives actions from the policy and produces a next state output and a reward output The overall task is de ned as maximizing the long term average reward per time step cf Russell The architecture also includes an explicit world model The world model is intended to mimic the one step input output behavior of the real world Finally the Dyna PI architecture in cludes an evaluation function that rapidly maps states to values much as the policy rapidly maps states to actions The evaluation function the policy and the world model are each updated by separate learning processes WORLD Action Reward (scalar) Heuristic Reward (scalar) State EVALUATION FUNCTION

Static hand gesture recognition using neural networks <=SEP=> This paper presents a novel technique for hand gesture recognition through human–computer interaction based on shape analysis. The main objective of this effort is to explore the utility of a neural network-based approach to the recognition of the hand gestures. A unique multi-layer perception of neural network is built for classification by using backpropagation learning algorithm. The goal of static hand gesture recognition is to classify the given hand gesture data represented by some features into some predefined finite number of gesture classes. The proposed system presents a recognition algorithm to recognize a set of six specific static hand gestures, namely: Open, Close, Cut, Paste, Maximize, and Minimize. The hand gesture image is passed through three stages, preprocessing, feature extraction, and classification. In preprocessing stage some operations are applied to extract the hand gesture from its background and prepare the hand gesture image for the feature extraction stage. In the first method, the hand contour is used as a feature which treats scaling and translation of problems (in some cases). The complex moment algorithm is, however, used to describe the hand gesture and treat the rotation problem in addition to the scaling and translation. The algorithm used in a multi-layer neural network classifier which uses back-propagation learning algorithm. The results show that the first method has a performance of 70.83% recognition, while the second method, proposed in this article, has a better performance of 86.38% recognition rate.

Intrusion Detection Model Based On Particle Swarm Optimization and Support Vector Machine <=SEP=> Advance in information and communication technologies, force us to keep most of the information electronically, consequently, the security of information has become a fundamental issue. The traditional intrusion detection systems look for unusual or suspicious activity, such as patterns of network traffic that are likely indicators of unauthorized activity. However, normal operation often produces traffic that matches likely "attack signature", resulting in false alarms. One main drawback is the inability of detecting new attacks which do not have known signatures. In this paper particle swarm optimization (PSO) is used to implement a feature selection, and support vector machine (SVMs) with the one-versus-rest method serve as a fitness function of PSO for classification problems from the literature. Experimental result shows that our method allows us to recognize not only known attacks but also to detect suspicious activity that may be the result of a new, unknown attack. Our method simplifies features effectively and obtains a higher classification accuracy compared to other methods

Wireless Relay Communications with Unmanned Aerial Vehicles: Performance and Optimization <=SEP=> In this paper, we investigate a communication system in which unmanned aerial vehicles (UAVs) are used as relays between ground-based terminals and a network base station. We develop an algorithm for optimizing the performance of the ground-to-relay links through control of the UAV heading angle. To quantify link performance, we define the ergodic normalized transmission rate (ENTR) for the links between the ground nodes and the relay, and derive a closed-form expression for it in terms of the eigenvalues of the channel correlation matrix. We show that the ENTR can be approximated as a sinusoid with an offset that depends on the heading of the UAV. Using this observation, we develop a closed-form expression for the UAV heading that maximizes the uplink network data rate while keeping the rate of each individual link above a certain threshold. When the current UAV relay assignments cannot meet the minimum link requirements, we investigate the deployment and heading control problem for new UAV relays as they are added to the network, and propose a smart handoff algorithm that updates node and relay assignments as the topology of the network evolves.

Modeling Social Perception of Faces [Social Sciences] <=SEP=> The face is our primary source of visual information for identifying people and reading their emotional and mental states. With the exception of prosopagnosics (who are unable to recognize faces) and those suffering from such disorders of social cognition as autism, people are extremely adept at these two tasks. However, our cognitive powers in this regard come at the price of reading too much into the human face. The face is often treated as a window into a person's true nature. Given the agreement in social perception of faces, this paper discusses that it should be possible to model this perception.

Supporting Situationally Aware Cybersecurity Systems 30 th September 2015 <=SEP=> In this report, we describe the Unified Cyber Security ontology (UCO) to support situational awareness in cyber security systems. The ontology is an effort to incorporate and integrate heterogeneous information available from different cyber security systems and most commonly used cyber security standards for information sharing and exchange. The ontology has also been mapped to a number of existing cyber security ontologies as well as concepts in the Linked Open Data cloud. Similar to DBpedia which serves as the core for Linked Open Data cloud, we envision UCO to serve as the core for the specialized cyber security Linked Open Data cloud which would evolve and grow with the passage of time with additional cybersecurity data sets as they become available. We also present a prototype system and concrete use-cases supported by the UCO ontology. To the best of our knowledge, this is the first cyber security ontology that has been mapped to general world ontologies to support broader and diverse security use-cases. We compare the resulting ontology with previous efforts, discuss its strengths and limitations, and describe potential future work directions.

Training Spiking ConvNets by STDP and Gradient Descent <=SEP=> This paper proposes a new method for training multi-layer spiking convolutional neural networks (CNNs). Training a multi-layer spiking network poses difficulties because the output spikes do not have derivatives and the commonly use backpropagation method for non-spiking networks is not easily applied. Our method uses a novel version of layered spike-timing- dependent plasticity (STDP) that incorporates supervised and unsupervised components. Our method starts with conventional learning methods and converts them to spatio-temporally local rules suited for spiking neural networks (SNNs). The training process uses two components for unsupervised feature extraction and supervised classification. The first component is a new STDP rule for spike-based representation learning which trains convolutional filters. The second introduces a new STDP-based supervised learning rule for spike pattern classification via an approximation to gradient descent. Stacking these components implements a novel spiking CNN of integrate-and-fire (IF) neurons with performances comparable with the state-of-the-art deep SNNs. The experimental results show the success of the proposed model for the MNIST handwritten digit classification. Our network architecture is the only high performance, spiking CNN which provides bio-inspired STDP rules in a hierarchy of feature extraction and classification in an entirely spike-based framework.

Practical Implementations of Arithmetic Coding 1 <=SEP=> We provide a tutorial on arithmetic coding, showing how it provides nearly optimal data compression and how it can be matched with almost any probabilistic model. We indicate the main disadvantage of arithmetic coding, its slowness, and give the basis of a fast, space-e cient, approximate arithmetic coder with only minimal loss of compression e ciency. Our coder is based on the replacement of arithmetic by table lookups coupled with a new deterministic probability estimation scheme. Index terms : Data compression, arithmetic coding, adaptive modeling, analysis of algorithms, data structures, low precision arithmetic. A similar version of this paper appears in Image and Text Compression, James A. Storer, ed., Kluwer Academic Publishers, Norwell, MA, 1992, 85{112. A shortened version of this paper appears in the proceedings of the International Conference on Advances in Communication and Control (COMCON 3), Victoria, British Columbia, Canada, October 16{18, 1991. Support was provided in part by NASA Graduate Student Researchers Program grant NGT{ 50420 and by a National Science Foundation Presidential Young Investigators Award grant with matching funds from IBM. Additional support was provided by a Universities Space Research Association/CESDIS associate membership. Support was provided in part by National Science Foundation Presidential Young Investigator Award CCR{9047466 with matching funds from IBM, by NSF research grant CCR{9007851, by Army Research O ce grant DAAL03{91{G{0035, and by the O ce of Naval Research and the Defense Advanced Research Projects Agency under contract N00014{91{J{4052ARPA Order No. 8225. Additional support was provided by a Universities Space Research Association/CESDIS associate membership.

Emotional Branding and the Strategic Value of the Doppelgänger Brand Image <=SEP=> Emotional branding is widely heralded as a key to marketing success. However, little attention has been given to the risks posed by this strategy. This article argues that emotional-branding strategies are conducive to the emergence of a doppelgänger brand image, which is defined as a family of disparaging images and meanings about a brand that circulate throughout popular culture. This article’s thesis is that a doppelgänger brand image can undermine the perceived authenticity of an emotional-branding story and, thus, the identity value that the brand provides to consumers. The authors discuss how the tenets of emotional branding paradoxically encourage the formation and propagation of doppelgänger brand imagery. This article develops the counterintuitive proposition that rather than merely being a threat to be managed, a doppelgänger brand image can actually benefit a brand by providing early warning signs that an emotional-branding story is beginning to lose its cultural resonance. This article demonstrates these ideas through an analysis of the doppelgänger brand image that is beginning to haunt a paragon of emotional branding—Starbucks. The authors conclude with a discussion of how marketing managers can proactively use the insights gained by analyzing a doppelgänger brand image.

Unsupervised Context-Sensitive Spelling Correction of English and Dutch Clinical Free-Text with Word and Character N-Gram Embeddings <=SEP=> We present an unsupervised context-sensitive spelling correction method for clinical free-text that uses word and character n-gram embeddings. Our method generates misspelling replacement candidates and ranks them according to their semantic fit, by calculating a weighted cosine similarity between the vectorized representation of a candidate and the misspelling context. To tune the parameters of this model, we generate self-induced spelling error corpora. We perform our experiments for two languages. For English, we greatly outperform off-the-shelf spelling correction tools on a manually annotated MIMIC-III test set, and counter the frequency bias of a noisy channel model, showing that neural embeddings can be successfully exploited to improve upon the state-of-the-art. For Dutch, we also outperform an off-the-shelf spelling correction tool on manually annotated clinical records from the Antwerp University Hospital, but can offer no empirical evidence that our method counters the frequency bias of a noisy channel model in this case as well. However, both our context-sensitive model and our implementation of the noisy channel model obtain high scores on the test set, establishing a state-of-the-art for Dutch clinical spelling correction with the noisy channel model.

A longitudinal study of the social and emotional predictors and consequences of cyber and traditional bullying victimisation <=SEP=> Few longitudinal studies have investigated how cyberbullying interacts with traditional bullying among young people, who are increasingly using online environments to seek information, entertainment and to socialise. This study aimed to identify the associations between the relative contribution of cyberbullying victimisation and traditional bullying victimisation on social and emotional antecedents and outcomes among adolescents. Participants were a cohort of 1,504 adolescents from 16 Australian schools followed from age 13 to 15 years. Adolescents experiencing social and emotional difficulties were more likely to be cyberbullied and traditionally bullied, than traditionally bullied only. Those targeted in both ways experienced more harm and stayed away from school more often than those traditionally bullied only. These findings suggest a high coexistence of cyber and traditional bullying behaviours and their antecedents, and higher levels of harm from a combination of these behaviours for adolescents over time. Future research should engage students as co-researchers to enhance school and parent strategies to support adolescents experiencing difficulties, and to reduce the likelihood of both cyber and traditional bullying.

Seeing Stars When There Aren’t Many Stars: Graph-Based Semi-Supervised Learning For Sentiment Categorization <=SEP=> We present a graph-based semi-supervised learning algorithm to address the sentiment analysis task of rating inference. Given a set of documents (e.g., movie reviews) and accompanying ratings (e.g., “4 stars”), the task calls for inferring numerical ratings for unlabeled documents based on the perceived sentiment expressed by their text. In particular, we are interested in the situation where labeled data is scarce. We place this task in the semi-supervised setting and demonstrate that considering unlabeled reviews in the learning process can improve ratinginference performance. We do so by creating a graph on both labeled and unlabeled data to encode certain assumptions for this task. We then solve an optimization problem to obtain a smooth rating function over the whole graph. When only limited labeled data is available, this method achieves significantly better predictive accuracy over other methods that ignore the unlabeled examples during training.

CNN-SLAM: Real-Time Dense Monocular SLAM with Learned Depth Prediction <=SEP=> Given the recent advances in depth prediction from Convolutional Neural Networks (CNNs), this paper investigates how predicted depth maps from a deep neural network can be deployed for the goal of accurate and dense monocular reconstruction. We propose a method where CNN-predicted dense depth maps are naturally fused together with depth measurements obtained from direct monocular SLAM, based on a scheme that privileges depth prediction in image locations where monocular SLAM approaches tend to fail, e.g. along low-textured regions, and vice-versa. We demonstrate the use of depth prediction to estimate the absolute scale of the reconstruction, hence overcoming one of the major limitations of monocular SLAM. Finally, we propose a framework to efficiently fuse semantic labels, obtained from a single frame, with dense SLAM, so to yield semantically coherent scene reconstruction from a single view. Evaluation results on two benchmark datasets show the robustness and accuracy of our approach.

Development of a one-axis actively regulated bearingless motor with a repulsive type passive magnetic bearing <=SEP=> This paper describes a novel one-axis actively regulated bearingless motor with a spinning top-shaped rotor, which is intended for a use as a cooling fan. This motor requires only one set of three-phase winding and thus one three-phase inverter to realize both rotation and magnetic suspension, and thus it is dubbed a single-drive bearingless motor. An axial position of the rotor is actively regulated and radial and tilting motions are passively stabilized by the passive magnetic bearing. Torque and axial suspension force are regulated by q-axis and d-axis currents, respectively. In this paper, the structure and principle, design, and test results of the proposed bearingless motor are presented.

Mutual information and minimum mean-square error in multiuser Gaussian channels <=SEP=> 1. Introduction Due to the lack of explicit closed form expressions of the mutual information for binary inputs, which were provided only for the BPSK and QPSK for the single input single output (SISO) case, [1], [2], [3], it is of particular importance to address connections between information theory and estimation theory for the multiuser case. Connections between information theory and estimation theory dates back to the work of Duncan, in [4] who showed that for the continuous-time additive white Gaussian noise (AWGN) channel, the filtering minimum mean squared error (causal estimation) is twice the input-output mutual information for any underlying signal distribution. Recently, Guo, Shamai, and Verdu have illuminated intimate connections between information theory and estimation theory in a seminal paper, [1]. In particular, Guo et al. have shown that in the classical problem of information transmission through the conventional AWGN channel, the derivative of the mutual information with respect to the SNR is equal to the smoothing minimum mean squared error (noncausal estimation); a relationship that holds for scalar, vector, discrete-time and continuous-time channels regardless of the input statistics. There have been extensions of these results to the case of mismatched input distributions in the scalar Gaussian channel in [5] and [6]. However, the fundamental relation between the derivative of the mutual information and the MMSE, known as I-MMSE identity, and defined for point to point channels with any noise or input distributions in [1] is not anymore suitable for the multiuser case. Therefore, in this paper, we revisit the connections between the mutual information and the MMSE for the multiuser setup. We generalize the I-MMSE relation to the multiuser case. In particular, we prove that the derivative of the mutual information with respect to the signal to noise ratio (SNR) is equal to the minimum mean squared error (MMSE) plus a covariance induced due to the interference, quantified by a term with respect to the cross correlation of the users inputs' estimates, their channels, and their precoding matrices. Further, we capitalize on this unveiled multiuser I-MMSE relation to derive the components of the multiuser mutual information. In particular, we derive the derivative of the conditinal and non-conditional mutual information with respect to the SNR. Further extensions of this result allows a generalization of the relations of linear vector Gaussian channels in [7] to multiuser channels. In particular, [8], [9] generalize the I-MMSE relation to the per-user gradient …

Mersenne Twister A Pseudo-Random Number Generator <=SEP=> Pseudo random number generators have been widely used in numbers of applications, particularly simulation and cryptography. Among them is a Mersenne Twister. Introduced in 1998 by Makoto Matsumoto and Takuji Nishimura, it has been a highly preferred generator since it provides long period, high order of dimensional equidistribution, speed and reliability. In this paper, we will focus mainly on describing the theories and algorithm of the Mersenne Twister. We also briefly introduce its variants and discuss its limitations and improvements.

Evaluating WAP News Sites: The Webqual/m Approach <=SEP=> This paper reports on the evaluation of wireless Internet news sites using the WebQual/m instrument. From initial application in the domain of traditional Internet Web sites, the instrument has been adapted for sites delivered using the wireless application protocol (WAP). The WebQual approach is to assess the Web-site quality from the perspective of the 'voice of the customer’, an approach adopted in quality function deployment. The WebQual/m instrument is used to assess customer perceptions of information, site and useroriented qualities. In particular, the qualities of three UK-based WAP news sites are assessed via an online questionnaire. The results are reported and analysed and demonstrate considerable variations in the offerings of the news sites. The findings and their implications for mobile commerce are discussed and some conclusions and directions for further research are provided.

Phialemoniopsis endophytica sp. nov., a new species of endophytic fungi from Luffa cylindrica in Henan, China <=SEP=> During a survey of endophytic fungi in the cucurbit plants collected from Henan, China, a new species, Phialemoniopsis endophytica was isolated from the lower stem of Luffa cylindrica. It differs from other Phialemoniopsis species by its cylindrical to flask-shaped phialides, falcate conidia with blunt ends, ostiolate pycnidium-like conidiomata without marginal setae and ellipsoidal chlamydospores. Multi-locus (ITS, LSU, ACT, and TUB) phylogenetic analysis confirmed that P. endophytica is distinct from other species. A synopsis of the morphological characters of the new species is provided.

Learning user interaction models for predicting web search result preferences <=SEP=> Evaluating user preferences of web search results is crucial for search engine development, deployment, and maintenance. We present a real-world study of modeling the behavior of web search users to predict web search result preferences. Accurate modeling and interpretation of user behavior has important applications to ranking, click spam detection, web search personalization, and other tasks. Our key insight to improving robustness of interpreting implicit feedback is to model query-dependent deviations from the expected "noisy" user behavior. We show that our model of clickthrough interpretation improves prediction accuracy over state-of-the-art clickthrough methods. We generalize our approach to model user behavior beyond clickthrough, which results in higher preference prediction accuracy than models based on clickthrough information alone. We report results of a large-scale experimental evaluation that show substantial improvements over published implicit feedback interpretation methods.

Detection and Localization of Texture-less Objects with Deep Neural Networks <=SEP=> This thesis studies Faster R-CNN, the state-of-art method for object detection in RGB images, and proposes its extension to RGB-D images. Solutions to the following problems are proposed and evaluated: filling missing values in depth images, depth encoding (raw depth vs. surface normals), extension of the CNN architecture to accept the extra depth information, and initialization of weights in the extended network. The overall best results were achieved with a network that accepts an extra depth channel, pre-processed by the iterative median filter to fill in the missing values, and has the depth weights in the first convolutional layer initialized with the mean of the color weights that were pretrained on ImageNet. However, the improvement over the original method using only RGB channels is not significant (mAP was increased by 1 − 2%), which suggests a need for different incorporation of the depth information.

Diffusion dynamics on multiplex networks <=SEP=> We study the time scales associated with diffusion processes that take place on multiplex networks, i.e., on a set of networks linked through interconnected layers. To this end, we propose the construction of a supra-laplacian matrix, which consists of a dimensional lifting of the laplacian matrix of each layer of the multiplex network. We use perturbative analysis to reveal analytically the structure of eigenvectors and eigenvalues of the complete network in terms of the spectral properties of the individual layers. The spectrum of the supra-laplacian allows us to understand the physics of diffusionlike processes on top of multiplex networks.

Dynamical Approximation by Hierarchical Tucker and Tensor-Train Tensors <=SEP=> We extend results on the dynamical low-rank approximation for the treatment of time-dependent matrices and tensors (Koch and Lubich; see [SIAM J. Matrix Anal. Appl., 29 (2007), pp. 434–454], [SIAM J. Matrix Anal. Appl., 31 (2010), pp. 2360–2375]) to the recently proposed hierarchical Tucker (HT) tensor format (Hackbusch and Kühn; see [J. Fourier Anal. Appl., 15 (2009), pp. 706–722]) and the tensor train (TT) format (Oseledets; see [SIAM J. Sci. Comput., 33 (2011), pp. 2295–2317]), which are closely related to tensor decomposition methods used in quantum physics and chemistry. In this dynamical approximation approach, the time derivative of the tensor to be approximated is projected onto the time-dependent tangent space of the approximation manifold along the solution trajectory. This approach can be used to approximate the solutions to tensor differential equations in the HT or TT format and to compute updates in optimization algorithms within these reduced tensor formats. By deriving and analyzing the tangent space projector for the manifold of HT/TT tensors of fixed rank, we obtain curvature estimates, which allow us to obtain quasi-best approximation properties for the dynamical approximation, showing that the prospects and limitations of the ansatz are similar to those of the dynamical low rank approximation for matrices. Our results are exemplified by numerical experiments.

Gender Classification of Blog Authors: With Feature Engineering and Deep Learning using LSTM Networks <=SEP=> In this paper, we present two approaches to automatically classify the gender of blog authors: the first is a manual feature extraction based system incorporating two novel feature classes: variable length character sequence patterns and thirteen new word classes, along with an added class of surface features while the second is a first-ever application of a memory variant of Recurrent Neural Networks, i.e. Bidirectional Long Short Term Memory Networks (BLSTMs) on this task. We use two blog data sets to report our results: the first is a well-explored one used by the previous state-of-the-art model while the other is a 20 times larger corpus. For the first system, we use a voting of machine learning classifiers to obtain an improved accuracy with respect to the previous feature mining systems on the former data set. Using our second approach, we show that the accuracy obtained using such deep LSTMs is comparable to the current state-of-the-art deep learning system for the task of gender classification. Finally, we carry out a comparative study of performance of both the systems on the two data sets.

Decision Making Under Uncertainty in Visualisation <=SEP=> Decision making under uncertainty can lead to irrational behaviour; such errors are often being referred to as cognitive biases. Related work in this area has tended to focus on the human’s analytic and sensemaking processes. This paper puts forward a novel perspective on this, proposing that some cognitive biases can also occur in the process of viewing visualisations. Consequently, this source of error may have a negative impact on decision making. This paper presents examples of situations where cognitive biases in visualisation can occur and outlines a future user study to investigate the anchoring and adjustment cognitive biases in visualisation.

Detection of SQL injection and XSS attacks in three tier web applications <=SEP=> Web applications are used on a large scale worldwide, which handles sensitive personal data of users. With web application that maintains data ranging from as simple as telephone number to as important as bank account information, security is a prime point of concern. With hackers aimed to breakthrough this security using various attacks, we are focusing on SQL injection attacks and XSS attacks. SQL injection attack is very common attack that manipulates the data passing through web application to the database servers through web servers in such a way that it alters or reveals database contents. While Cross Site Scripting (XSS) attacks focuses more on view of the web application and tries to trick users that leads to security breach. We are considering three tier web applications with static and dynamic behavior, for security. Static and dynamic mapping model is created to detect anomalies in the class of SQL Injection and XSS attacks.

The reviewing of object files: Object-specific integration of information <=SEP=> A series of experiments explored a form of object-specific priming. In all experiments a preview field containing two or more letters is followed by a target letter that is to be named. The displays are designed to produce a perceptual interpretation of the target as a new state of an object that previously contained one of the primes. The link is produced in different experiments by a shared location, by a shared relative position in a moving pattern, or by successive appearance in the same moving frame. An object-specific advantage is consistently observed: naming is facilitated by a preview of the target, if (and in some cases only if) the two appearances are linked to the same object. The amount and the object specificity of the preview benefit are not affected by extending the preview duration to 1 s, or by extending the temporal gap between fields to 590 ms. The results are interpreted in terms of a reviewing process, which is triggered by the appearance of the target and retrieves just one of the previewed items. In the absence of an object link, the reviewing item is selected at random. We develop the concept of an object file as a temporary episodic representation, within which successive states of an object are linked and integrated.