Above are papers related to the following paper: [Paper_ID] Paper #1152904. 	[Title] GIBBS-MARKOV MODELS [Abstract] In this paper we present a framework for building probabilistic automata parameterized by context-dependent probabilities. Gibbs distributions are used to model state transitions and output generation, and parameter estimation is carried out using an EM algorithm where the M-step uses a generalized iterative scaling procedure. We discuss relations with certain classes of stochastic feedforward neural networks, a geometric interpretation for parameter estimation, and a simple example of a statistical language model constructed using this methodology. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1135746. 	[Title] Learning Markov chains with variable memory length from noisy output [Abstract] The problem of modeling complicated data sequences, such as DNA or speech, often arises in practice. Most of the algorithms select a hypothesis from within a model class assuming that the observed sequence is the direct output of the underlying generation process. In this paper we consider the case when the output passes through a memoryless noisy channel before observation. In particular, we show that in the class of Markov chains with variable memory length, learning is affected by factors, which, despite being super-polynomial, are still small in some practical cases. Markov models with variable memory length, or probabilistic finite suffix automata, were introduced in learning theory by Ron, Singer and Tishby who also described a polynomial time learning algorithm [11, 12]. We present a modification of the algorithm which uses a noise-corrupted sample and has knowledge of the noise structure. The same algorithm is still viable if the noise is not known exactly but a good estimation is available. Finally, some experimental results are presented for removing noise from corrupted English text, and to measure how the performance of the learning algorithm is affected by the size of the noisy sample and the noise rate. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1129608. 	[Title] A New Look at Tree Models for Multiple Sequence Alignment [Abstract] Evolutionary trees are frequently used as the underlying model in the design of algorithms, optimization criteria and software packages for multiple sequence alignment (MSA). In this paper, we reexamine the suitability of trees as a universal model for MSA in light of the broad range of biological questions that MSA's are used to address. A tree model consists of a tree topology and a model of accepted mutations along the branches. After surveying the major applications of MSA, examples from the molecular biology literature are used to illustrate situations in which this tree model fails. This occurs when the relationship between residues in a column cannot be described by a tree; for example, in some structural and functional applications of MSA. It also occurs in situations, such as lateral gene transfer, where an entire gene cannot be modeled by a unique tree. In cases of nonparsimonous data or convergent evolution, it may be difficult to find a consistent mutational model. We hope that this survey will promote dialogue between biologists and computer scientists, leading to more biologically realistic research on MSA. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #390922. 	[Title] A MEAN FIELD LEARNING ALGORITHM FOR UNSUPERVISED NEURAL NETWORKS [Abstract] We introduce a learning algorithm for unsupervised neural networks based on ideas from statistical mechanics. The algorithm is derived from a mean field approximation for large, layered sigmoid belief networks. We show how to (approximately) infer the statistics of these networks without resort to sampling. This is done by solving the mean field equations, which relate the statistics of each unit to those of its Markov blanket. Using these statistics as target values, the weights in the network are adapted by a local delta rule. We evaluate the strengths and weaknesses of these networks for problems in statistical pattern recognition. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #910. 	[Title] Learning networks for face analysis and synthesis [Abstract] This paper presents an overview of the face-related projects in our group. The unifying theme underlying our work is the use of example-based learning methods for both analyzing and synthesizing face images. We label the example face images (and for the problem of face detection, "near miss" faces as well) with descriptive parameters for pose, expression, identity, and face vs. non-face. Then, by using example-based learning techniques, we develop networks for performing analysis tasks such as pose and expression estimation, face recognition, and face detection in cluttered scenes. In addition to these analysis applications, we show how the example-based technique can also be used as a novel method for image synthesis that is for computer graphics. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1152272. 	[Title] GIBBS-MARKOV MODELS [Abstract] In this paper we present a framework for building probabilistic automata parameterized by context-dependent probabilities. Gibbs distributions are used to model state transitions and output generation, and parameter estimation is carried out using an EM algorithm where the M-step uses a generalized iterative scaling procedure. We discuss relations with certain classes of stochastic feedforward neural networks, a geometric interpretation for parameter estimation, and a simple example of a statistical language model constructed using this methodology. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #29723. 	[Title] Boltzmann Chains and Hidden Markov Models [Abstract] We propose a statistical mechanical framework for the modeling of discrete time series. Maximum likelihood estimation is done via Boltzmann learning in one-dimensional networks with tied weights. We call these networks Boltzmann chains and show that they contain hidden Markov models (HMMs) as a special case. Our framework also motivates new architectures that address particular shortcomings of HMMs. We look at two such architectures: parallel chains that model feature sets with disparate time scales, and looped networks that model long-term dependencies between hidden states. For these networks, we show how to implement the Boltzmann learning rule exactly, in polynomial time, without resort to simulated or mean-field annealing. The necessary computations are done by exact decimation procedures from statistical mechanics. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #3229. 	[Title] Book Review  Introduction to the Theory of Neural Computation Reviewed by: 2 [Abstract] Neural computation, also called connectionism, parallel distributed processing, neural network modeling or brain-style computation, has grown rapidly in the last decade. Despite this explosion, and ultimately because of impressive applications, there has been a dire need for a concise introduction from a theoretical perspective, analyzing the strengths and weaknesses of connectionist approaches and establishing links to other disciplines, such as statistics or control theory. The Introduction to the Theory of Neural Computation by Hertz, Krogh and Palmer (subsequently referred to as HKP) is written from the perspective of physics, the home discipline of the authors. The book fulfills its mission as an introduction for neural network novices, provided that they have some background in calculus, linear algebra, and statistics. It covers a number of models that are often viewed as disjoint. Critical analyses and fruitful comparisons between these models Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #28957. 	[Title] Operations for Learning with Graphical Models decomposition techniques and the demonstration that graphical models provide [Abstract] This paper is a multidisciplinary review of empirical, statistical learning from a graphical model perspective. Well-known examples of graphical models include Bayesian networks, directed graphs representing a Markov chain, and undirected networks representing a Markov field. These graphical models are extended to model data analysis and empirical learning using the notation of plates. Graphical operations for simplifying and manipulating a problem are provided including decomposition, differentiation, and the manipulation of probability models from the exponential family. Two standard algorithm schemas for learning are reviewed in a graphical framework: Gibbs sampling and the expectation maximization algorithm. Using these operations and schemas, some popular algorithms can be synthesized from their graphical specification. This includes versions of linear regression, techniques for feed-forward networks, and learning Gaussian and discrete Bayesian networks from data. The paper concludes by sketching some implications for data analysis and summarizing how some popular algorithms fall within the framework presented. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #50336. 	[Title] Computing upper and lower bounds on likelihoods in intractable networks [Abstract] We present deterministic techniques for computing upper and lower bounds on marginal probabilities in sigmoid and noisy-OR networks. These techniques become useful when the size of the network (or clique size) precludes exact computations. We illustrate the tightness of the bounds by numerical experi ments. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #50337. 	[Title] Recursive algorithms for approximating probabilities in graphical models [Abstract] MIT Computational Cognitive Science Technical Report 9604 Abstract We develop a recursive node-elimination formalism for efficiently approximating large probabilistic networks. No constraints are set on the network topologies. Yet the formalism can be straightforwardly integrated with exact methods whenever they are/become applicable. The approximations we use are controlled: they maintain consistently upper and lower bounds on the desired quantities at all times. We show that Boltzmann machines, sigmoid belief networks, or any combination (i.e., chain graphs) can be handled within the same framework. The accuracy of the methods is verified exper imentally. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1152162. 	[Title] GIBBS-MARKOV MODELS [Abstract] In this paper we present a framework for building probabilistic automata parameterized by context-dependent probabilities. Gibbs distributions are used to model state transitions and output generation, and parameter estimation is carried out using an EM algorithm where the M-step uses a generalized iterative scaling procedure. We discuss relations with certain classes of stochastic feedforward neural networks, a geometric interpretation for parameter estimation, and a simple example of a statistical language model constructed using this methodology. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #10531. 	[Title] Using Dirichlet Mixture Priors to Derive Hidden Markov Models for Protein Families [Abstract] A Bayesian method for estimating the amino acid distributions in the states of a hidden Markov model (HMM) for a protein family or the columns of a multiple alignment of that family is introduced. This method uses Dirichlet mixture densities as priors over amino acid distributions. These mixture densities are determined from examination of previously constructed HMMs or multiple alignments. It is shown that this Bayesian method can improve the quality of HMMs produced from small training sets. Specific experiments on the EF-hand motif are reported, for which these priors are shown to produce HMMs with higher likelihood on unseen data, and fewer false positives and false negatives in a database search task. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #66982. 	[Title] Learning Generative Models with the Up-Propagation Algorithm [Abstract] Up-propagation is an algorithm for inverting and learning neural network generative models. Sensory input is processed by inverting a model that generates patterns from hidden variables using top-down connections. The inversion process is iterative, utilizing a negative feedback loop that depends on an error signal propagated by bottom-up connections. The error signal is also used to learn the generative model from examples. The algorithm is benchmarked against principal component analysis in In his doctrine of unconscious inference, Helmholtz argued that perceptions are formed by the interaction of bottom-up sensory data with top-down expectations. According to one interpretation of this doctrine, perception is a procedure of sequential hypothesis testing. We propose a new algorithm, called up-propagation, that realizes this interpretation in layered neural networks. It uses top-down connections to generate hypotheses, and bottom-up connections to revise them. It is important to understand the difference between up-propagation and its ancestor, the backpropagation algorithm[1]. Backpropagation is a learning algorithm for recognition models. As shown in Figure 1a, bottom-up connections recognize patterns, while top-down connections propagate an error signal that is used to learn the recognition model. In contrast, up-propagation is an algorithm for inverting and learning generative models, as shown in Figure 1b. Top-down connections generate patterns from a set of hidden variables. Sensory input is processed by inverting the generative model, recovering hidden variables that could have generated the sensory data. This operation is called either pattern recognition or pattern analysis, depending on the meaning of the hidden variables. Inversion of the generative model is done iteratively, through a negative feedback loop driven by an error signal from the bottom-up connections. The error signal is also used for learning the connections experiments on images of handwritten digits. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #194223. 	[Title] Bayesian Unsupervised Learning of Higher Order Structure [Abstract] Multilayer architectures such as those used in Bayesian belief networks and Helmholtz machines provide a powerful framework for representing and learning higher order statistical relations among inputs. Because exact probability calculations with these models are often intractable, there is much interest in finding approximate algorithms. We present an algorithm that efficiently discovers higher order structure using EM and Gibbs sampling. The model can be interpreted as a stochastic recurrent network in which ambiguity in lower-level states is resolved through feedback from higher levels. We demonstrate the performance of the algorithm on bench mark problems. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #43698. 	[Title] Dirichlet Mixtures: A Method for Improving Detection of Weak but Significant Protein Sequence Homology [Abstract] UCSC Technical Report UCSC-CRL-96-09 Abstract This paper presents the mathematical foundations of Dirichlet mixtures, which have been used to improve database search results for homologous sequences, when a variable number of sequences from a protein family or domain are known. We present a method for condensing the information in a protein database into a mixture of Dirichlet densities. These mixtures are designed to be combined with observed amino acid frequencies, to form estimates of expected amino acid probabilities at each position in a profile, hidden Markov model, or other statistical model. These estimates give a statistical model greater generalization capacity, such that remotely related family members can be more reliably recognized by the model. Dirichlet mixtures have been shown to outperform substitution matrices and other methods for computing these expected amino acid distributions in database search, resulting in fewer false positives and false negatives for the families tested. This paper corrects a previously published formula for estimating these expected probabilities, and contains complete derivations of the Dirichlet mixture formulas, methods for optimizing the mixtures to match particular databases, and suggestions for efficient implementation. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #31927. 	[Title] A Generalized Hidden Markov Model for the Recognition of Human Genes in DNA [Abstract] We present a statistical model of genes in DNA. A Generalized Hidden Markov Model (GHMM) provides the framework for describing the grammar of a legal parse of a DNA sequence (Stormo & Haussler 1994). Probabilities are assigned to transitions between states in the GHMM and to the generation of each nucleotide base given a particular state. Machine learning techniques are applied to optimize these probabilities using a standardized training set. Given a new candidate sequence, the best parse is deduced from the model using a dynamic programming algorithm to identify the path through the model with maximum probability. The GHMM is flexible and modular, so new sensors and additional states can be inserted easily. In addition, it provides simple solutions for integrating cardinality constraints, reading frame constraints, "indels", and homology searching. The description and results of an implementation of such a gene-finding model, called Genie, is presented. The exon sensor is a codon frequency model conditioned on windowed nucleotide frequency and the preceding codon. Two neural networks are used, as in (Brunak, Engelbrecht, & Knudsen 1991), for splice site prediction. We show that this simple model performs quite well. For a cross-validated standard test set of 304 genes [ftp://www-hgc.lbl.gov/pub/genesets] in human DNA, our gene-finding system identified up to 85% of protein-coding bases correctly with a specificity of 80%. 58% of exons were exactly identified with a specificity of 51%. Genie is shown to perform favorably compared with several other gene-finding systems. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #194617. 	[Title] Finding Genes in DNA with a Hidden Markov Model [Abstract] This study describes a new Hidden Markov Model (HMM) system for segmenting uncharacterized genomic DNA sequences into exons, introns, and intergenic regions. Separate HMM modules were designed and trained for specific regions of DNA: exons, introns, intergenic regions, and splice sites. The models were then tied together to form a biologically feasible topology. The integrated HMM was trained further on a set of eukaryotic DNA sequences, and tested by using it to segment a separate set of sequences. The resulting HMM system, which is called VEIL (Viterbi Exon-Intron Locator), obtains an overall accuracy on test data of 92% of total bases correctly labelled, with a correlation coefficient of 0.73. Using the more stringent test of exact exon prediction, VEIL correctly located both ends of 53% of the coding exons, and 49% of the exons it predicts are exactly correct. These results compare favorably to the best previous results for gene structure prediction, and demonstrate the benefits of using HMMs for this problem. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #35905. 	[Title] Equivalence of Linear Boltzmann Chains and Hidden Markov Models sequence L, is: where Z(; A; [Abstract] Several authors have made a link between hidden Markov models for time series and energy-based models (Luttrell 1989, Williams 1990, Saul and Jordan 1995). Saul and Jordan (1995) discuss a linear Boltzmann chain model with state-state transition energies A ii 0 (going from state i to state i 0 ) and symbol emission energies B ij , under which the probability of an entire state fi l ; j l g L Whilst any HMM can be written as a linear Boltzmann chain by setting exp(A ii 0 ) = a ii 0 , exp(B ij ) = b ij and exp( i ) = i , not all linear Boltzmann chains can be represented as HMMs (Saul and Jordan 1995). However, the difference between the two models is minimal. To be precise, if the final hidden Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1140547. 	[Title] Localist Attractor Networks [Abstract] Attractor networks, which map a continuous input space to a discrete output space, are useful for pattern completion, cleaning up noisy or missing features in an input. However, designing a net to have a given set of attractors is notoriously tricky; training procedures are CPU intensive and often produce spurious attractors and ill-conditioned attractor basins. These difficulties occur because each connection in the network participates in the encoding of multiple attractors. We describe an alternative formulation of attractor networks in which the encoding of knowledge is local, not distributed. Although localist attractor nets have similar dynamics to their distributed counterparts, they are much easier to work with and interpret. We propose a statistical formulation of localist attractor net dynamics, which yields a convergence proof and a mathematical interpretation of model parameters. We present simulation experiments that explore the behavior of lo-calist attractor nets, showing that they produce a gang effectthe presence of an attractor enhances the attractor basins of neighboring attractorsand that spurious attractors occur only at points of symmetry in state space. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #686532. 	[Title] Homology Detection via Family Pairwise Search a straightforward generalization of pairwise sequence comparison algorithms to [Abstract] The function of an unknown biological sequence can often be accurately inferred by identifying sequences homologous to the original sequence. Given a query set of known homologs, there exist at least three general classes of techniques for finding additional homologs: pairwise sequence comparisons, motif analysis, and hidden Markov modeling. Pairwise sequence comparisons are typically employed when only a single query sequence is known. Hidden Markov models (HMMs), on the other hand, are usually trained with sets of more than 100 sequences. Motif-based methods fall in between these two extremes. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #35922. 	[Title] Mean Field Theory for Sigmoid Belief Networks [Abstract] We develop a mean field theory for sigmoid belief networks based on ideas from statistical mechanics. Our mean field theory provides a tractable approximation to the true probability distribution in these networks; it also yields a lower bound on the likelihood of evidence. We demonstrate the utility of this framework on a benchmark problem in statistical pattern recognition|the classification of handwritten digits. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1105116. 	[Title] Large Deviation Methods for Approximate Probabilistic Inference, with Rates of Convergence a free parameter. The [Abstract] We study layered belief networks of binary random variables in which the conditional probabilities Pr[childjparents] depend monotonically on weighted sums of the parents. For these networks, we give efficient algorithms for computing rigorous bounds on the marginal probabilities of evidence at the output layer. Our methods apply generally to the computation of both upper and lower bounds, as well as to generic transfer function parameterizations of the conditional probability tables (such as sigmoid and noisy-OR). We also prove rates of convergence of the accuracy of our bounds as a function of network size. Our results are derived by applying the theory of large deviations to the weighted sums of parents at each node in the network. Bounds on the marginal probabilities are computed from two contributions: one assuming that these weighted sums fall near their mean values, and the other assuming that they do not. This gives rise to an interesting trade-off between probable explanations of the evidence and improbable deviations from the mean. In networks where each child has N parents, the gap between our upper and lower bounds behaves as a sum of two terms, one of order p In addition to providing such rates of convergence for large networks, our methods also yield efficient algorithms for approximate inference in fixed networks. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1063773. 	[Title] A Gentle Guide to Multiple Alignment Version Please send comments, critique, flames and praise Instructions [Abstract] Prerequisites. An understanding of the dynamic programming (edit distance) approach to pairwise sequence alignment is useful for parts 1.3, 1.4, and 2. Also, familiarity with the use of Internet resources would be helpful for part 3. For the former, see Chapters 1.1 - 1.3, and for the latter, see Chapter 2 of the Hypertext Book of the GNA-VSNS Biocomputing Course at http://www.techfak.uni-bielefeld.de/bcd/Curric/welcome.html. General Rationale. You will understand why Multiple Alignment is considered a challenging problem, you will study approaches that try to reduce the number of steps needed to calculate the optimal solution, and you will study fast heuristics. In a case study involving immunoglobulin sequences, you will study multiple alignments obtained from WWW servers, recapitulating results from an original paper. Revision History. Version 1.01 on 17 Sep 1995. Expanded Ex.9. Updated Ex.46. Revised Solution Sheet -re- Ex.3+12. Marked more Exercises by "A" (to be submitted to the Instructor). Various minor clarifications in content Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #686559. 	[Title] Parameterization studies for the SAM and HMMER methods of hidden Markov model generation [Abstract] Multiple sequence alignment of distantly related viral proteins remains a challenge to all currently available alignment methods. The hidden Markov model approach offers a new, flexible method for the generation of multiple sequence alignments. The results of studies attempting to infer appropriate parameter constraints for the generation of de novo HMMs for globin, kinase, aspartic acid protease, and ribonuclease H sequences by both the SAM and HMMER methods are described. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1129442. 	[Title] Minimum-Risk Profiles of Protein Families Based on Statistical Decision Theory [Abstract] Statistical decision theory provides a principled way to estimate amino acid frequencies in conserved positions of a protein family. The goal is to minimize the risk function, or the expected squared-error distance between the estimates and the true population frequencies. The minimum-risk estimates are obtained by adding an optimal number of pseudocounts to the observed data. Two formulas are presented, one for pseudocounts based on marginal amino acid frequencies and one for pseudocounts based on the observed data. Experimental results show that profiles constructed using minimal-risk estimates are more discriminating than those constructed using existing methods. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #286562. 	[Title] Hidden Markov Model Analysis of Motifs in Steroid Dehydrogenases and their Homologs [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #31336. 	[Title] The megaprior heuristic for discovering protein sequence patterns [Abstract] Several computer algorithms for discovering patterns in groups of protein sequences are in use that are based on fitting the parameters of a statistical model to a group of related sequences. These include hidden Markov model (HMM) algorithms for multiple sequence alignment, and the MEME and Gibbs sampler algorithms for discovering motifs. These algorithms are sometimes prone to producing models that are incorrect because two or more patterns have been combined. The statistical model produced in this situation is a convex combination (weighted average) of two or more different models. This paper presents a solution to the problem of convex combinations in the form of a heuristic based on using extremely low variance Dirichlet mixture priors as part of the statistical model. This heuristic, which we call the megaprior heuristic, increases the strength (i.e., decreases the variance) of the prior in proportion to the size of the sequence dataset. This causes each column in the final model to strongly resemble the mean of a single component of the prior, regardless of the size of the dataset. We describe the cause of the convex combination problem, analyze it mathematically, motivate and describe the implementation of the megaprior heuristic, and show how it can effectively eliminate the problem of convex combinations in protein sequence pattern discovery. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1124844. 	[Title] Stochastic Decomposition of DNA Sequences Using Hidden Markov Models [Abstract] This work presents an application of a machine learning for characterizing an important property of natural DNA sequences compositional inhomogeneity. Compositional segments often correspond to meaningful biological units. Taking into account such inhomogeneity is a prerequisite of successful recognition of functional features in DNA sequences, especially, protein-coding genes. Here we present a technique for DNA segmentation using hidden Markov models. A DNA sequence is represented by a chain of homogeneous segments, each described by one of a few statistically discriminated hidden states, whose contents form a first-order Markov chain. The technique is used to describe and compare chromosomes I and IV of the completely sequenced Saccharomyces cerevisiae (yeast) genome. Our results indicate the existence of a few well separated states, which gives support to the isochore theory. We also explore the model's likelihood landscape and analyze the dynamics of the optimization process, thus addressing the problem of reliability of the obtained optima and efficiency of the algorithms. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #390894. 	[Title] A variational approach to Bayesian logistic regression models and their extensions [Abstract] We consider a logistic regression model with a Gaussian prior distribution over the parameters. We show that accurate variational techniques can be used to obtain a closed form posterior distribution over the parameters given the data thereby yielding a posterior predictive model. The results are readily extended to (binary) belief networks. For belief networks we also derive closed form posteriors in the presence of missing values. Finally, we show that the dual of the regression problem gives a latent variable density model, the variational formulation of which leads to exactly solvable EM updates. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #4335. 	[Title] A VIEW OF THE EM ALGORITHM THAT JUSTIFIES INCREMENTAL, SPARSE, AND OTHER VARIANTS [Abstract] The EM algorithm performs maximum likelihood estimation for data in which some variables are unobserved. We present a function that resembles negative free energy and show that the M step maximizes this function with respect to the model parameters and the E step maximizes it with respect to the distribution over the unobserved variables. From this perspective, it is easy to justify an incremental variant of the EM algorithm in which the distribution for only one of the unobserved variables is recalculated in each E step. This variant is shown empirically to give faster convergence in a mixture estimation problem. A variant of the algorithm that exploits sparse conditional distributions is also described, and a wide range of other variant algorithms are also seen to be possible. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1107567. 	[Title] Learning Algorithms with Applications to Robot Navigation and Protein Folding [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #390896. 	[Title] IMPROVING THE MEAN FIELD APPROXIMATION VIA THE USE OF MIXTURE DISTRIBUTIONS [Abstract] Mean field methods provide computationally efficient approximations to posterior probability distributions for graphical models. Simple mean field methods make a completely factorized approximation to the posterior, which is unlikely to be accurate when the posterior is multimodal. Indeed, if the posterior is multi-modal, only one of the modes can be captured. To improve the mean field approximation in such cases, we employ mixture models as posterior approximations, where each mixture component is a factorized distribution. We describe efficient methods for optimizing the parameters in these models. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #31349. 	[Title] Meta-MEME: Motif-based Hidden Markov Models of Protein Families [Abstract]  Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #1123576. 	[Title] Density Networks and their Application to Protein Modelling [Abstract] I define a latent variable model in the form of a neural network for which only target outputs are specified; the inputs are unspecified. Although the inputs are missing, it is still possible to train this model by placing a simple probability distribution on the unknown inputs and maximizing the probability of the data given the parameters. The model can then discover for itself a description of the data in terms of an underlying latent variable space of lower dimensionality. I present preliminary results of the application of these models to protein data. Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #31353. 	[Title] Hidden Markov Models in Computational Biology: Applications to Protein Modeling UCSC-CRL-93-32 Keywords: Hidden Markov Models, [Abstract] Hidden Markov Models (HMMs) are applied to the problems of statistical modeling, database searching and multiple sequence alignment of protein families and protein domains. These methods are demonstrated on the globin family, the protein kinase catalytic domain, and the EF-hand calcium binding motif. In each case the parameters of an HMM are estimated from a training set of unaligned sequences. After the HMM is built, it is used to obtain a multiple alignment of all the training sequences. It is also used to search the SWISS-PROT 22 database for other sequences that are members of the given protein family, or contain the given domain. The HMM produces multiple alignments of good quality that agree closely with the alignments produced by programs that incorporate three-dimensional structural information. When employed in discrimination tests (by examining how closely the sequences in a database fit the globin, kinase and EF-hand HMMs), the HMM is able to distinguish members of these families from non-members with a high degree of accuracy. Both the HMM and PRO-FILESEARCH (a technique used to search for relationships between a protein sequence and multiply aligned sequences) perform better in these tests than PROSITE (a dictionary of sites and patterns in proteins). The HMM appears to have a slight advantage Please summarise all the given information.

Above are papers related to the following paper: [Paper_ID] Paper #396412. 	[Title] Boltzmann Machine learning using mean field theory and linear response correction [Abstract] We present a new approximate learning algorithm for Boltzmann Machines, using a systematic expansion of the Gibbs free energy to second order in the weights. The linear response correction to the correlations is given by the Hessian of the Gibbs free energy. The computational complexity of the algorithm is cubic in the number of neurons. We compare the performance of the exact BM learning algorithm with first order (Weiss) mean field theory and second order (TAP) mean field theory. The learning task consists of a fully connected Ising spin glass model on 10 neurons. We conclude that 1) the method works well for paramagnetic problems 2) the TAP correction gives a significant improvement over the Weiss mean field theory, both for paramagnetic and spin glass problems and 3) that the inclusion of diagonal weights improves the Weiss approximation for paramagnetic problems, but not for spin glass problems. Please summarise all the given information.

Paper #1152904 has the title `GIBBS-MARKOV MODELS  
` and the abstract `In this paper we present a framework for building probabilistic automata parameterized by context-dependent probabilities. Gibbs distributions are used to model state transitions and output generation, and parameter estimation is carried out using an EM algorithm where the M-step uses a generalized iterative scaling procedure. We discuss relations with certain classes of stochastic feedforward neural networks, a geometric interpretation for parameter estimation, and a simple example of a statistical language model constructed using this methodology. 
`.
Question: Which category should Paper #1152904 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].


Paper #1152904 has the title `GIBBS-MARKOV MODELS  
` and the abstract `In this paper we present a framework for building probabilistic automata parameterized by context-dependent probabilities. Gibbs distributions are used to model state transitions and output generation, and parameter estimation is carried out using an EM algorithm where the M-step uses a generalized iterative scaling procedure. We discuss relations with certain classes of stochastic feedforward neural networks, a geometric interpretation for parameter estimation, and a simple example of a statistical language model constructed using this methodology. 
`.
Question: Which category should Paper #1152904 be classified as ?
You can select one from ['Neural_Networks', 'Case_Based', 'Theory', 'Reinforcement_Learning', 'Probabilistic_Methods', 'Rule_Learning', 'Genetic_Algorithms'].

This paper should be classified into: Neural_Networks