conditional scores and optimal scores for generalized linear measurement error models abstract this paper studies estimation of the parameters of generalized linear models in canonical form when the explanatory vector is measured with independent normal error for the functional case i e when the explanatory vectors are fixed constants unbiased score functions are obtained by conditioning on certain sufficient statistics this work generalizes results obtained for logistic regression in the case that the explanatory vectors are independent and identically distributed with unknown distribtuion efficient score functions are obtained using the theory developed in begun et al 1983 keywords conditional score function efficient score function functional model generalized linear model measurement error structural model

gradient systems in view of information geometry abstract dualistic properties of a gradient flow on a manifold m associated with a dualistic structure g is studied from an information geometrical viewpoint some useful applications are also investigated

information geometry on hierarchy of probability distributions an exponential family or mixture family of probability distributions has a natural hierarchical structure this paper gives an orthogonal decomposition of such a system based on information geometry a typical example is the decomposition of stochastic dependency among a number of random variables in general they have a complex structure of dependencies pairwise dependency is easily represented by correlation but it is more difficult to measure effects of pure triplewise or higher order interactions dependencies among these variables stochastic dependency is decomposed quantitatively into an orthogonal sum of pairwise triplewise and further higher order dependencies this gives a new invariant decomposition of joint entropy this problem is important for extracting intrinsic interactions in firing patterns of an ensemble of neurons and for estimating its functional connections the orthogonal decomposition is given in a wide class of hierarchical structures including both exponential and mixture families as an example we decompose the dependency in a higher order markov chain into a sum of those in various lower order markov chains

regression analysis when covariates are regression parameters of a random effects model for observed longitudinal measurements summary we consider regression analysis when covariate variables are the underlying regression coefficients of another linear mixed model a naive approach is t o use each subject s repeated measurements which are assumed to follow a linear mixed model and obtain subject specific estimated coefficients to replace the covariate variables however directly replacing the unobserved covariates in the primary regression by these estimated coefficients may result in a significantly biased estimator the aforementioned problem can be evaluated as a generalization of the classical additive error model where repeated measures are considered as replicates to correct for these biases we investigate a pseudo expected estimating equation eee estimator a regression calibration rc estimator and a refined version of the rc estimator for linear regression the first two estimators are identical under certain conditions however when the primary regression model is a nonlinear model the rc estimator is usually biased we thus consider a refined regression calibration estimator whose performance is close to that of the pseudo eee estimator but does not require numerical integration the rc estimator is also extended to the proportional hazards regression model in addition to the distribution theory we evaluate the methods through simulation studies the methods are applied to analyze a real dataset from a child growth study

fast training of recurrent networks based on the em algorithm in this work a probabilistic model is established for recurrent networks the expectation maximization em algorithm is then applied to derive a new fast training algorithm for recurrent networks through mean field approximation this new algorithm converts training a complicated recurrent network into training an array of individual feedforward neurons these neurons are then trained via a linear weighted regression algorithm the training time has been improved by five to 15 times on benchmark problems

a multiclass classification method based on decoding of binary classifiers in this letter we present new methods of multiclass classification that combine multiple binary classifiers misclassification of each binary classifier is formulated as a bit inversion error with probabilistic models by making an analogy to the context of information transmission theory dependence between binary classifiers is incorporated into our model which makes a decoder a type of boltzmann machine we performed experimental studies using a synthetic data set data sets from the uci repository and bioinformatics data sets and the results show that the proposed methods are superior to the existing multiclass classification methods

an information geometric approach to ml estimation with incomplete data application to semiblind mimo channel identification in this paper we cast the stochastic maximum likelihood estimation of parameters with incomplete data in an information geometric framework in this vein we develop the information geometric identification igid algorithm the algorithm consists of iterative alternating projections on two sets of probability distributions pds i e likelihood pds and data empirical distributions a gaussian assumption on the source distribution permits a closed form low complexity solution for these projections the method is applicable to a wide range of problems however in this paper the emphasis is on semiblind identification of unknown parameters in a multiple input multiple output mimo communications system it is shown by simulations that the performance of the algorithm in terms of both estimation error and bit error rate ber is similar to that of the expectation maximization em based algorithm proposed previously by aldana et al but with a substantial improvement in computational speed especially for large constellations

four types of learning curves if machines are learning to make decisions given a number of examples the generalization error e t is defined as the average probability that an incorrect decision is made for a new example by a machine when trained with t examples the generalization error decreases as t increases and the curve e t is called a learning curve the present paper uses the bayesian approach to show that given the annealed approximation learning curves can be classified into four asymptotic types if the machine is deterministic with noiseless teacher signals then 1 e at 1 when the correct machine parameter is unique and 2 e at 2 when the set of the correct parameters has a finite measure if the teacher signals are noisy then 3 e at 1 2 for a deterministic machine and 4 e c at 1 for a stochastic machine

hebbian learning of recurrent connections a geometrical perspective we show how a hopfield network with modifiable recurrent connections undergoing slow hebbian learning can extract the underlying geometry of an input space first we use a slow and fast analysis to derive an averaged system whose dynamics derives from an energy function and therefore always converges to equilibrium points the equilibria reflect the correlation structure of the inputs a global object extracted through local recurrent interactions only second we use numerical methods to illustrate how learning extracts the hidden geometrical structure of the inputs indeed multidimensional scaling methods make it possible to project the final connectivity matrix onto a euclidean distance matrix in a high dimensional space with the neurons labeled by spatial position within this space the resulting network structure turns out to be roughly convolutional the residual of the projection defines the nonconvolutional part of the connectivity which is minimized in the process finally we show how restricting the dimension of the space where the neurons live gives rise to patterns similar to cortical maps we motivate this using an energy efficiency argument based on wire length minimization finally we show how this approach leads to the emergence of ocular dominance or orientation columns in primary visual cortex via the self organization of recurrent rather than feedforward connections in addition we establish that the nonconvolutional or long range connectivity is patchy and is co aligned in the case of orientation learning

a joint model for survival and longitudinal data measured with error the relationship between a longitudinal covariate and a failure time process can be assessed using the cox proportional hazards regression model we consider the problem of estimating the parameters in the cox model when the longitudinal covariate is measured infrequently and with measurement error we assume a repeated measures random effects model for the covariate process estimates of the parameters are obtained by maximizing the joint likelihood for the covariate process and the failure time process this approach uses the available information optimally because we use both the covariate and survival data simultaneously parameters are estimated using the expectation maximization algorithm we argue that such a method is superior to naive methods where one maximizes the partial likelihood of the cox model using the observed covariate values it also improves on two stage methods where in the first stage empirical bayes estimates of the covariate process are computed and then used as time dependent covariates in a second stage to find the parameters in the cox model that maximize the partial likelihood

how to center deep boltzmann machines this work analyzes centered restricted boltzmann machines rbms and centered deep boltzmann machines dbms where centering is done by subtracting offset values from visible and hidden variables we show analytically that i centered and normal boltzmann machines bms and thus rbms and dbms are different parameterizations of the same model class such that any normal bm rbm dbm can be transformed to an equivalent centered bm rbm dbm and vice versa and that this equivalence generalizes to artificial neural networks in general ii the expected performance of centered binary bms rbms dbms is invariant under simultaneous flip of data and offsets for any off set value in the range of zero to one iii centering can be reformulated as a different update rule for normal bms rbms dbms and iv using the enhanced gradient is equivalent to setting the offset values to the average over model and data mean furthermore we present numerical simulations suggesting that i optimal generative performance is achieved by subtracting mean values from visible as well as hidden variables ii centered binary rbms dbms reach significantly higher log likelihood values than normal binary rbms dbms iii centering variants whose offsets depend on the model mean like the enhanced gradient suffer from severe divergence problems iv learning is stabilized if an exponentially moving average over the batch means is used for the offset values instead of the current batch mean which also prevents the enhanced gradient from severe divergence v on a similar level of log likelihood values centered binary rbms dbms have smaller weights and bigger bias parameters than normal binary rbms dbms vi centering leads to an update direction that is closer to the natural gradient which is extremely efficient for training as we show for small binary rbms vii centering eliminates the need for greedy layer wise pre training of dbms which often even deteriorates the results independently of whether centering is used or not and ix centering is also beneficial for auto encoders

information geometry and plefka s mean field theory an alternate derivation of plefka s method is presented which is obtained from minimizing gibbs energy it is shown that this method can be rederived using a perturbation expansion of kullback leibler divergence the latter having a nice information geometric interpretation

conditional estimation for generalized linear models when covariates are subject specific parameters in a mixed model for longitudinal measurements the relationship between a primary endpoint and features of longitudinal profiles of a continuous response is often of interest and a relevant framework is that of a generalized linear model with covariates that are subject specific random effects in a linear mixed model for the longitudinal measurements naive implementation by imputing subject specific effects from individual regression fits yields biased inference and several methods for reducing this bias have been proposed these require a parametric normality assumption on the random effects which may be unrealistic adapting a strategy of stefanski and carroll 1987 biometrika74 703 716 we propose estimators for the generalized linear model parameters that require no assumptions on the random effects and yield consistent inference regardless of the true distribution the methods are illustrated via simulation and by application to a study of bone mineral density in women transitioning to menopause

statistical einstein manifolds of exponential families with group invariant potential functions abstract this paper mainly contributes to a classification of statistical einstein manifolds namely statistical manifolds at the same time are einstein manifolds a statistical manifold is a riemannian manifold each of whose points is a probability distribution with the fisher information metric as a riemannian metric information geometry was developed to understand the intrinsic properties of statistical models which play important roles in statistical inference etc among all these models exponential families is one of the most important kinds whose geometric structures are fully determined by their potential functions to classify statistical einstein manifolds we derive partial differential equations for potential functions of exponential families special solutions of these equations are obtained through the ansatz method as well as group invariant solutions via reductions using lie point symmetries

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

information geometry of boltzmann machines a boltzmann machine is a network of stochastic neurons the set of all the boltzmann machines with a fixed topology forms a geometric manifold of high dimension where modifiable synaptic weights of connections play the role of a coordinate system to specify networks a learning trajectory for example is a curve in this manifold it is important to study the geometry of the neural manifold rather than the behavior of a single network in order to know the capabilities and limitations of neural networks of a fixed topology using the new theory of information geometry a natural invariant riemannian metric and a dual pair of affine connections on the boltzmann neural network manifold are established the meaning of geometrical structures is elucidated from the stochastic and the statistical point of view this leads to a natural modification of the boltzmann machine learning rule

dynamical behaviour of a statistical manifold associated with correlated walks a riemann space associated with a correlated walk cw model is investigated by a numerical method which is applicable to many other stochastic processes the behaviour of the riemann curvature is shown to have a close relation to the stability of the cw system the riemann space is spanned by two parameters representing jump probabilities of the cw model the cw manifold evolves through some characteristic eras it starts at a point and changes to a line and then to a homogeneous spherical surface of the scalar curvature r 1 2 the homogeneous space immediately transforms to an inhomogeneous space the inhomogeneity gradually grows a region of the inhomogeneous space violently oscillates in time and another region expands fast as a whole the curvature of the space decreases and soon becomes negative the oscillations have already started to fade away the negative curvature goes on decreasing and finally the space converges to a homogeneous saddle surface of r 1 such a behaviour of the curvature is shown to be well understood by the terms of stability and order parameter of stochastic processes in other words the inhomogeneity the oscillation and the decrease behaviour of the curvature are closely related to stability and orderliness of the correlated system

asymptotic theory of sequential estimation differential geometrical approach sequential estimation continues observations until the observed sample satisfies a prescribed criterion its properties are superior on the average to those of nonsequential estimation in which the number of observations is fixed a priori a higher order asymptotic theory of sequential estimation is given in the framework of geometry of multidimensional curved exponential families this gives a design principle of the second order efficient sequential estimation procedure it is also shown that a sequential estimation can be designed to have a covariance stabilizing effect at the same time

dynamics of learning in multilayer perceptrons near singularities the dynamical behavior of learning is known to be very slow for the multilayer perceptron being often trapped in the plateau it has been recently understood that this is due to the singularity in the parameter space of perceptrons in which trajectories of learning are drawn the space is riemannian from the point of view of information geometry and contains singular regions where the riemannian metric or the fisher information matrix degenerates this paper analyzes the dynamics of learning in a neighborhood of the singular regions when the true teacher machine lies at the singularity we give explicit asymptotic analytical solutions trajectories both for the standard gradient sgd and natural gradient ngd methods it is clearly shown in the case of the sgd method that the plateau phenomenon appears in a neighborhood of the critical regions where the dynamical behavior is extremely slow the analysis of the ngd method is much more difficult because the inverse of the fisher information matrix diverges we conquer the difficulty by introducing the blow down technique used in algebraic geometry the ngd method works efficiently and the state converges directly to the true parameters very quickly while it staggers in the case of the sgd method the analytical results are compared with computer simulations showing good agreement the effects of singularities on learning are thus qualitatively clarified for both standard and ngd methods

a novel parameter estimation method for boltzmann machines we propose a novel estimator for a specific class of probabilistic models on discrete spaces such as the boltzmann machine the proposed estimator is derived from minimization of a convex risk function and can be constructed without calculating the normalization constant whose computational cost is exponential order we investigate statistical properties of the proposed estimator such as consistency and asymptotic normality in the framework of the estimating function small experiments show that the proposed estimator can attain comparable performance to the maximum likelihood expectation at a much lower computational cost and is applicable to high dimensional data

higher order boltzmann machines and entropy bounds we examine some aspects of the interface area between mathematical statistics and statistical physics relevant to the study of boltzmann machines the boltzmann machine learning algorithm is based on a variational principle gibbs lemma for relative entropy this fact suggests the possibility of a scheme of successive approximations here we consider successive approximations parametrized by the order of many body interactions among individual units we prove bounds on the gain in relative entropy in the crucial step of adding and estimating by hebb s rule a new parameter we address the problem of providing on the basis of local observations upper and lower bounds on the entropy while upper bounds are easily obtained by subadditivity lower bounds involve localization of hirschman bounds on a dual quantum system

natural gradient works efficiently in learning when a parameter space has a certain underlying structure the ordinary gradient of a function does not represent its steepest direction but the natural gradient does information geometry is used for calculating the natural gradients in the parameter space of perceptrons the space of matrices for blind source separation and the space of linear dynamical systems for blind source deconvolution the dynamical behavior of natural gradient online learning is analyzed and is proved to be fisher efficient implying that it has asymptotically the same performance as the optimal batch estimation of parameters this suggests that the plateau phenomenon which appears in the backpropagation learning algorithm of multilayer perceptrons might disappear or might not be so serious when the natural gradient is used an adaptive method of updating the learning rate is proposed and analyzed

approximate newton methods for policy search in markov decision processes approximate newton methods are standard optimization tools which aim to maintain the benefits of newton s method such as a fast rate of convergence while alleviating its drawbacks such as computationally expensive calculation or estimation of the inverse hessian in this work we investigate approximate newton methods for policy optimization in markov decision processes mdps we first analyse the structure of the hessian of the total expected reward which is a standard objective function for mdps we show that like the gradient the hessian exhibits useful structure in the context of mdps and we use this analysis to motivate two gauss newton methods for mdps like the gauss newton method for non linear least squares these methods drop certain terms in the hessian the approximate hessians possess desirable properties such as negative definiteness and we demonstrate several important performance guarantees including guaranteed ascent directions invariance to afine transformation of the parameter space and convergence guarantees we finally provide a unifying perspective of key policy search algorithms demonstrating that our second gauss newton algorithm is closely related to both the em algorithm and natural gradient ascent applied to mdps but performs significantly better in practice on a range of challenging domains

geometrical theory of higher order asymptotics of test interval estimator and conditional inference higher order asymptotic powers of one sided and two sided tests both conditional and unconditional are evaluated for a one parameter curved exponential family of distributions by using differential geometrical notions higher order asymptotic powers and the expected size of one sided and two sided interval estimators both conditional and unconditional are also obtained the tests and interval estimators which are third order most powerful at any arbitrary specified one point are explicitly designed with the help of the approximate ancillary statistic the characteristics of the conditional inference are elucidated by proving the equivalence up to the third order of the conditional inference and likelihood ratio inference geometrical notions such as curvatures and angles play a fundamental role in the present theory

forced oscillation of a class of neutral hyperbolic differential equations in this paper we study the following boundary value problem for a class of neutral hyperbolic differential equations 2 t2 u c t u x t a0 t u a1 t u x t ab q x t f u x g t d g x t x t r g u n v x t u 0 x t r a number of theorems for oscillatory solutions of the problem under two different cases are developed

representation of nonlinear random transformations by non gaussian stochastic neural networks the learning capability of neural networks is equivalent to modeling physical events that occur in the real environment several early works have demonstrated that neural networks belonging to some classes are universal approximators of input output deterministic functions recent works extend the ability of neural networks in approximating random functions using a class of networks named stochastic neural networks snn in the language of system theory the approximation of both deterministic and stochastic functions falls within the identification of nonlinear no memory systems however all the results presented so far are restricted to the case of gaussian stochastic processes sps only or to linear transformations that guarantee this property this paper aims at investigating the ability of stochastic neural networks to approximate nonlinear input output random transformations thus widening the range of applicability of these networks to nonlinear systems with memory in particular this study shows that networks belonging to a class named non gaussian stochastic approximate identity neural networks sainns are capable of approximating the solutions of large classes of nonlinear random ordinary differential transformations the effectiveness of this approach is demonstrated and discussed by some application examples

estimation in the presence of infinitely many nuisance parameters geometry of estimating functions when there exist nuisance parameters whose number increases in proportion to that of independent observations it is in general difficult to get a consistent or efficient estimator of a common structural parameter the present paper proposes a new theory based on a vector bundle consisting of certain random variables over the statistical model structures and properties of estimating functions are elucidated in the class of consistent estimators a necessary and sufficient condition is obtained for the existence of a consistent estimator given by an estimating function a necessary and sufficient condition is then given for the existence of the optimal estimator in this class which is further obtained when it exists in their derivations the concept of dual connections and parallel transports plays an essential role the results are applied to a special type of exponential family and the optimal estimators are explicitly obtained in some examples this explains the reason why the conditional score plays an important role

differential geometry of curved exponential families curvatures and information loss the differential geometrical framework is given for analyzing statistical problems related to multi parameter families of distributions the dualistic structures of the exponential families and curved exponential families are elucidated from the geometrical viewpoint the duality connected by the legendre transformation is thus extended to include two kinds of affine connections and two kinds of curvatures the second order information loss is calculated for fisher efficient estimators and is decomposed into the sum of two non negative terms one is related to the exponential curvature of the statistical model and the other is related to the mixture curvature of the estimator only the latter term depends on the estimator and vanishes for the maximum likelihood estimator a set of statistics which recover the second order information loss are given the second order efficiency also is obtained the differential geometry of the function space of distributions is discussed

estimating spiking irregularities under changing environments we considered a gammadistribution of interspike intervals as a statistical model for neuronal spike generation a gamma distribution is a natural extension of the poisson process taking the effect of a refractory period into account the model is specified by two parameters a time dependent firing rate and a shape parameter that characterizes spiking irregularities of individual neurons because the environment changes over time observed data are generated from a model with a time dependent firing rate which is an unknown function a statistical model with an unknown function is called a semiparametric model and is generally very difficult to solve we used a novel method of estimating functions in information geometry to estimate the shape parameter without estimating the unknown function we obtained an optimal estimating function analytically for the shape parameter independent of the functional form of the firing rate this estimation is efficient without fisher information loss and better than maximum likelihood estimation we suggest a measure of spiking irregularity based on the estimating function which may be useful for characterizing individual neurons in changing environments

a smooth nonparametric estimate of a mixing distribution using mixtures of gaussians abstract we propose a method of estimating mixing distributions using maximum likelihood over the class of arbitrary mixtures of gaussians subject to the constraint that the component variances be greater than or equal to some minimum value h this approach can lead to estimates of many shapes with smoothness controlled by parameter h we show that the resulting estimate will always be a finite mixture of gaussians each having variance h the nonparametric maximum likelihood estimate can be viewed as a special case with h 0 the method can be extended to estimate multivariate mixing distributions examples and the results of a simulation study are presented

tree based reparameterization framework for analysis of sum product and related algorithms we present a tree based reparameterization trp framework that provides a new conceptual view of a large class of algorithms for computing approximate marginals in graphs with cycles this class includes the belief propagation bp or sum product algorithm as well as variations and extensions of bp algorithms in this class can be formulated as a sequence of reparameterization updates each of which entails refactorizing a portion of the distribution corresponding to an acyclic subgraph i e a tree or more generally a hypertree the ultimate goal is to obtain an alternative but equivalent factorization using functions that represent exact or approximate marginal distributions on cliques of the graph our framework highlights an important property of the sum product algorithm and the larger class of reparameterization algorithms the original distribution on the graph with cycles is not changed the perspective of tree based updates gives rise to a simple and intuitive characterization of the fixed points in terms of tree consistency we develop interpretations of these results in terms of information geometry the invariance of the distribution in conjunction with the fixed point characterization enables us to derive an exact expression for the difference between the true marginals on an arbitrary graph with cycles and the approximations provided by belief propagation more broadly our analysis applies to any algorithm that minimizes the bethe free energy we also develop bounds on the approximation error which illuminate the conditions that govern their accuracy finally we show how the reparameterization perspective extends naturally to generalizations of bp e g kikuchi 1951 approximations and variants via the notion of hypertree reparameterization

information geometric measure for neural spikes this study introduces information geometric measures to analyze neural firing patterns by taking not only the second order but also higher order interactions among neurons into account information geometry provides useful tools and concepts for this purpose including the orthogonality of coordinate parameters and the pythagoras relation in the kullback leibler divergence based on this orthogonality we show a novel method for analyzing spike firing patterns by decomposing the interactions of neurons of various orders as a result purely pairwise triple wise and higher order interactions are singled out we also demonstrate the benefits of our proposal by using several examples

divergence function duality and convex analysis from a smooth strictly convex function rn r a parametric family of divergence function d may be introduced d x y 4 1 2 1 2 x 1 2 y 1 2x 1 2y for x y int dom rn and for r with d 1 defined through taking the limit of each member is shown to induce an independent riemannian metric as well as a pair of dual connections which are generally nonflat except for 1 in the latter case d 1 reduces to the nonparametric bregman divergence which is representable using and its convex conjugate and becomes the canonical divergence for dually flat spaces amari 1982 1985 amari nagaoka 2000 this formulation based on convex analysis naturally extends the information geometric interpretation of divergence functions eguchi 1983 to allow the distinction between two different kinds of duality referential duality and representational duality when applied to not necessarily normalized probability densities the concept of conjugated representations of densities is introduced so that connections defined on probability densities embody both referential and representational duality and are hence themselves bidual when restricted to a finite dimensional affine submanifold the natural parameters of a certain representation of densities and the expectation parameters under its conjugate representation form biorthogonal coordinates the alpha representation indexed by now 1 1 is shown to be the only measure invariant representation the resulting two parameter family of divergence functionals d 1 1 1 1 induces identical fisher information but bidual alpha connection pairs it reduces in form to amari s alpha divergence family when 1 or when 1 but to the family of jensen difference rao 1987 when 1

beyond the binary boltzmann machine the definition of the usual boltzmann machine is extended to allow for neurons with polytomous multicategory rather than simply binary responses updating rules are defined along with the associated stationary distributions and an alternating minimization method is described for the purposes of training emphasis is placed on the relevance of statistical ideas including polytomous logistic regression the iterative proportional fitting procedure and the em algorithm

semiparametric stochastic mixed models for longitudinal data abstract we consider inference for a semiparametric stochastic mixed model for longitudinal data this model uses parametric fixed effects to represent the covariate effects and an arbitrary smooth function to model the time effect and accounts for the within subject correlation using random effects and a stationary or nonstationary stochastic process we derive maximum penalized likelihood estimators of the regression coefficients and the nonparametric function the resulting estimator of the nonparametric function is a smoothing spline we propose and compare frequentist inference and bayesian inference on these model components we use restricted maximum likelihood to estimate the smoothing parameter and the variance components simultaneously we show that estimation of all model components of interest can proceed by fitting a modified linear mixed model we illustrate the proposed method by analyzing a hormone dataset and evaluate its performance through simulations

investigation of possible neural architectures underlying information geometric measures a novel analytical method based on information geometry was recently proposed and this method may provide useful insights into the statistical interactions within neural groups the link between informationgeometric measures and the structure of neural interactions has not yet been elucidated however because of the ill posed nature of the problem here possible neural architectures underlying information geometric measures are investigated using an isolated pair and an isolated triplet of model neurons by assuming the existence of equilibrium states we derive analytically the relationship between the information geometric parameters and these simple neural architectures for symmetric networks the first and second order information geometric parameters represent respectively the external input and the underlying connections between the neurons provided that the number of neurons used in the parameter estimation in the log linear model and the number of neurons in the network are the same for asymmetric networks however these parameters are dependent on both the intrinsic connections and the external inputs to each neuron in addition we derive the relation between the information geometric parameter corresponding to the two neuron interaction and a conventional cross correlation measure we also show that the information geometric parameters vary depending on the number of neurons assumed for parameter estimation in the log linear model this finding suggests a need to examine the information geometric method carefully a possible criterion for choosing an appropriate orthogonal coordinate is also discussed this article points out the importance of a model based approach and sheds light on the possible neural structure underlying the application of information geometry to neural network analysis