similarity metric learning for a variable kernel classifier nearest neighbor interpolation algorithms have many useful properties for applications to learning but they often exhibit poor generalization in this paper it is shown that much better generalization can be obtained by using a variable interpolation kernel in combination with conjugate gradient optimization of the similarity metric and kernel size the resulting method is called variable kernel similarity metric vsm learning it has been tested on several standard classification data sets and on these problems it shows better generalization than backpropagation and most other learning methods the number of parameters that must be determined through optimization are orders of magnitude less than for backpropagation or radial basis function rbf networks which may indicate that the method better captures the essential degrees of variation in learning other features of vsm learning are discussed that make it relevant to models for biological learning in the brain

limit cycles of planar quadratic differential equations abstract since hilbert posed the problem of systematically counting and locating the limite cycles of polynomial systems on the plane in 1900 much effort has been expended in its investigation a large body of literature chiefly by chinese and soviet authors has addressed this question in the context of differential equations whose field is specified by quadratic polynomials this paper considers the class of quadratic differential equations which admit a unique equilibrium state and establish sufficient conditions for the existence and uniqueness of limit cycles the work is based upon insights and techniques developed in an earlier analysis of such systems motivated by questions from mathematical control theory

a markov chain monte carlo em algorithm for analyzing interval censored data under the cox proportional hazards model summary this paper proposes a monte carlo em mcem algorithm for fitting the proportional hazards model for interval censored failure time data the algorithm generates orderings of the failures from their probability distribution under the model we maximize the average of the log likelihoods from these completed data sets to obtain updated parameter estimates as with the standard cox model this algorithm does not require the estimation of the baseline hazard function the performance of the algorithm is evaluated using simulations and the method is applied to data from aids and cancer studies our results indicate that our method produced more precise and unbiased estimates than methods of right and midpoint imputation

stabilizing controllers for bilinear systems the problem of finding stabilizing feedback controllers for continuous bilinear systems where the controls act additively and multiplicatively simultaneously is treated the applicability of the quadratic control law of jacobson 5 and others is extended to the case when the a matrix has arbitary eigenvalues under certain conditions a class of bilinear systems common among biochemical flow systems is defined the dyadic bilinear systems a control scheme the so called division controller for dyadic bilinear systems is suggested the practicality of the control schemes is demonstrated on the problem of controlling the neutron level in a fission reactor

on the stabilization of the homogeneous bilinear systems abstract the problem of stabilizing the single input homogeneous bilinear systems is considered a special class of these systems is considered the matrix of the linear i e control independent term of the right hand side is semitable while the matrix of the bilinear term is skew symmetric two types of stabilization are investigated constant feedback global asymptotic stabilization and practical stabilization by a family of the linear feedbacks it will be shown for the planar case that the above system is always practically stabilizable by a family of the linear feedbacks for n 3 the solution of both stabilization problems depends on the mutual position of the expanding direction of the linear part and the direction of the infinitesimal rotation defined by the bilinear term

inference based on imputed failure times for the proportional hazards model with interval censored data abstract we propose an approach to the proportional hazards model for interval censored data in which parameter estimates are obtained by solving estimating equations that are the partial likelihood score equations for the full data proportional hazards model averaged over all rankings of imputed failure times consistent with the observed censoring intervals imputed failure times are generated using a parametric estimate of the baseline distribution the parameters of the baseline distribution are estimated simultaneously with the proportional hazards regression parameters although a parametric form for the baseline distribution must be specified simulation studies show that the method performs well even when the baseline distribution is misspecified the estimating equations are solved using monte carlo techniques we present a recursive stochastic approximation scheme that converges to the zero of the estimating equations the solution has a random error that is asymptotically normally distributed w

varying coefficient models we explore a class of regression and generalized regression models in which the coefficients are allowed to vary as smooth functions of other variables general algorithms are presented for estimating the models flexibly and some examples are given this class of models ties together generalized additive models and dynamic generalized linear models into one common framework when applied to the proportional hazards model for survival data this approach provides a new way of modelling departures from the proportional hazards assumption

stabilizability of second order bilinear systems this note states necessary and sufficient conditions for the existence of a linear state feedback controller such that a second order bilinear system has a globally asymptotically stable closed loop a suitable controller is constructed for each system which satisfies the conditions

a monte carlo implementation of the em algorithm and the poor man s data augmentation algorithms abstract the first part of this article presents the monte carlo implementation of the e step of the em algorithm given the current guess to the maximizer of the posterior distribution latent data patterns are generated from the conditional predictive distribution the expected value of the augmented log posterior is then updated as a mixture of augmented log posteriors mixed over the generated latent data patterns multiple imputations in the m step of the algorithm this mixture is maximized to obtain the update to the maximizer of the observed posterior the gradient and hessian of the observed log posterior are also expressed as mixtures mixed over the multiple imputations the relation between the monte carlo em mcem algorithm and the data augmentation algorithm is noted two modifications to the mcem algorithm the poor man s data augmentation algorithms which allow for the calculation of the entire posterior are then presented these approximations serve as diagnostics for the validity o

singularity for static state feedback linearizable bilinear systems this paper deals with the problem of singularities for a class of static state feedback linearizable bilinear systems for this class of nonlinear systems the decoupling matrix is singular on an algebraic surface which contains the origin and this relates the static state feedback linearization to the difficult problem of the completeness of the trajectories of the closed loop system and or the boundedness of the feedback laws in this work we give a sufficient condition under which all the trajectories of the closed loop system are complete and the used feedback law is bounded on each of these trajectories

hierarchical knowledge gradient for sequential sampling we propose a sequential sampling policy for noisy discrete global optimization and ranking and selection in which we aim to efficiently explore a finite set of alternatives before selecting an alternative as best when exploration stops each alternative may be characterized by a multi dimensional vector of categorical and numerical attributes and has independent normal rewards we use a bayesian probability model for the unknown reward of each alternative and follow a fully sequential sampling policy called the knowledge gradient policy this policy myopically optimizes the expected increment in the value of sampling information in each time period we propose a hierarchical aggregation technique that uses the common features shared by alternatives to learn about many alternatives from even a single measurement this approach greatly reduces the measurement effort required but it requires some prior knowledge on the smoothness of the function in the form of an aggregation function and computational issues limit the number of alternatives that can be easily considered to the thousands we prove that our policy is consistent finding a globally optimal alternative when given enough measurements and show through simulations that it performs competitively with or significantly better than other policies

a bayesian justification of cox s partial likelihood summary in this paper we establish both naive and formal bayesian justifications of cox s 1975 partial likelihood and its various modifications we extend the original work of kalbfieisch 1978 who showed that the partial likelihood is a limiting marginal posterior under noninformative priors for baseline hazards we extend the result to scenarios with timedependent covariates and time varying regression parameters we establish results for continuous time as well as grouped survival data in addition we present a bayesian justification of a modified partial likelihood for handling ties we also present tools for simplification of the gibbs sampling algorithm for implementing partial likelihood based bayesian inference in various practical applications

a simple method for the adjustment of profile likelihoods summary we propose a simple adjustment for profile likelihoods the aim of the adjustment is to alleviate some of the problems inherent in the use of profile likelihoods such as bias inconsistency and overoptimistic variance estimates the adjustment is applied to the profile loglikelihood score function at each parameter value so that its mean is zero and its variance is the negative expected derivative matrix of the adjusted score function for cases in which explicit calculation of the adjustments is difficult we give two methods to simplify their computation an automatic simulation method that requires as input only the profile loglikelihood and its first few derivatives first order asymptotic expressions some examples are provided and a comparison is made with the conditional profile log likelihood of cox and reid

combining estimates in regression and classification abstract we consider the problem of how to combine a collection of general regression fit vectors to obtain a better predictive model the individual fits may be from subset linear regression ridge regression or something more complex like a neural network we develop a general framework for this problem and examine a cross validation based proposal called model mix or stacking in this context we also derive combination methods based on the bootstrap and analytic methods and compare them in examples finally we apply these ideas to classification problems where the estimated combination weights can yield insight into the structure of the problem

adaptive regression by mixing adaptation over different procedures is of practical importance different procedures perform well under different conditions in many practical situations it is rather hard to assess which conditions are approximately satisfied so as to identify the best procedure for the data at hand thus automatic adaptation over various scenarios is desirable a practically feasible method named adaptive regression by mixing arm is proposed to convexly combine general candidate regression procedures under mild conditions the resulting estimator is theoretically shown to perform optimally in rates of convergence without knowing which of the original procedures work the best simulations are conducted in several settings including comparing a parametric model with nonparametric alternatives comparing a neural network with a projection pursuit in multidimensional regression and combining bandwidths in kernel regression the results clearly support the theoretical property of arm the arm algorithm assigns we

rank based inference in the proportional hazards model for interval censored data summary a marginal likelihood approach to fitting the proportional hazards model to interval censored or grouped data is proposed this approach maximises a likelihood that is the sum over all rankings of the data that are consistent with the observed censoring intervals as in the usual proportional hazards model the method does not require specification of the baseline hazard function the score equations determining the maximum marginal likelihood estimator can be written as the expected value of the score of the usual proportional hazards model with respect to a certain distribution of rankings a gibbs sampling scheme is given to generate rankings from this distribution and stochastic approximation is used to solve the score equations simulation results under various censoring schemes give point estimates that are close to estimates obtained using actual failure times

asymptotic stabilization of the bilinear time invariant system via piecewise constant feedback abstract sufficient conditions for the asymptotic stabilization in a preassigned domain of the nonhomogeneous bilinear system bls via piecewise constant feedback are given based on those for an auxillary bls with additional control in the drift term application to the swing equation is also discussed

value function approximation using multiple aggregation for multiattribute resource management we consider the problem of estimating the value of a multiattribute resource where the attributes are categorical or discrete in nature and the number of potential attribute vectors is very large the problem arises in approximate dynamic programming when we need to estimate the value of a multiattribute resource from estimates based on monte carlo simulation these problems have been traditionally solved using aggregation but choosing the right level of aggregation requires resolving the classic tradeoff between aggregation error and sampling error we propose a method that estimates the value of a resource at different levels of aggregation simultaneously and then uses a weighted combination of the estimates using the optimal weights which minimizes the variance of the estimate while accounting for correlations between the estimates is computationally too expensive for practical applications we have found that a simple inverse variance formula adjusted for bias which effectively assumes the estimates are independent produces near optimal estimates we use the setting of two levels of aggregation to explain why this approximation works so well

the stability of second order quadratic differential equations this paper investigates the stability properties of second order systems dot x f x where f x contains either quadratic terms system 1 or linear and quadratic terms system 2 in x the principal contributions are summarized in two theorems which give necessary and sufficient conditions for stability and asymptotic stability in the large of systems 1 and 2 respectively

stability analysis of bilinear systems under aperiodic sampled data control this note considers the problem of local stability of bilinear systems with aperiodic sampled data linear state feedback control the sampling intervals are time varying and upper bounded it is shown that the feasibility of some linear matrix inequalities lmis implies the local asymptotic stability of the sampled data system in an ellipsoidal region containing the equilibrium the method is based on the analysis of contractive invariant sets and it is inspired by the dissipativity theory the results are illustrated by means of numerical examples

the output stabilization of siso bilinear systems a quite general approach to the problem of stabilizing a siso single input single output bilinear system by output feedback is presented following a lyapunov like method a family of admissible controls is defined and an easy way for deriving its general features is proposed previous results can be obtained as special cases of this approach numerical examples are included to show the feasibility of this algorithm in determining simple control laws

likelihood calculations for matched case control studies and survival studies with tied death times summary a recursion is presented which permits rapid computation of a conditional likelihood function which arises in matched case control studies and survival studies with tied death times an example with seven parameters and ten large strata the largest of which contains 29 cases and 52 controls illustrates the feasibility of the conditional maximum likelihood solution these methods may be specialized to permit rapid calculation of the conditional maximum likelihood estimate of the common odds ratio for several 2 x 2 tables

estimating the error rate of a prediction rule improvement on cross validation abstract we construct a prediction rule on the basis of some data and then wish to estimate the error rate of this rule in classifying future observations cross validation provides a nearly unbiased estimate using only the original data cross validation turns out to be related closely to the bootstrap estimate of the error rate this article has two purposes to understand better the theoretical basis of the prediction problem and to investigate some related estimators which seem to offer considerably improved estimation in small samples

maximizing generalized linear mixed model likelihoods with an automated monte carlo em algorithm summary two new implementations of the em algorithm are proposed for maximum likelihood fitting of generalized linear mixed models both methods use random independent and identically distributed sampling to construct monte carlo approximations at the e step one approach involves generating random samples from the exact conditional distribution of the random effects given the data by rejection sampling using the marginal distribution as a candidate the second method uses a multivariate t importance sampling approximation in many applications the two methods are complementary rejection sampling is more efficient when sample sizes are small whereas importance sampling is better with larger sample sizes monte carlo approximation using random samples allows the monte carlo error at each iteration to be assessed by using standard central limit theory combined with taylor series methods specifically we construct a sandwich variance estimate for the maximizer at each approximate e step this suggests a rule for automatically increasing the monte carlo sample size after iterations in which the true em step is swamped by monte carlo error in contrast techniques for assessing monte carlo error have not been developed for use with alternative implementations of monte carlo em algorithms utilizing markov chain monte carlo e step approximations three different data sets including the infamous salamander data of mccullagh and nelder are used to illustrate the techniques and to compare them with the alternatives the results show that the methods proposed can be considerably more efficient than those based on markov chain monte carlo algorithms however the methods proposed may break down when the intractable integrals in the likelihood function are of high dimension

remark on local and global stabilization of homogeneous bilinear systems in this paper we deal with the problem of stabilization of homogeneous bilinear systems the aim is to clarify some results on stabilizability of these systems

biometrika centenary survival analysis although papers relating to survival analysis have appeared in biometrika since the very first issue the current emphasis on the topic in this journal is fairly new especially as it relates to applications in biostatistics this survey traces the development of survival analysis throughout the twentieth century as reflected in the pages of biometrika focusing primarily on work published since 1980

maximization of the marginal likelihood of grouped survival data summary grouped failure time data occur in studies where subjects are monitored periodically to determine whether failure has occurred in the intervening interval here the model under consideration is cox s 1972 1975 proportional hazards model but the commonly used method of partial likelihood needs modification with grouped data due to a potentially large number of ties this paper demonstrates how the monte carlo em algorithm wei tanner 1990 can be used on grouped data to find the maximum likelihood estimate from the marginal likelihood based on the incomplete ranks of the event times the methodology is exemplified with a data set with precise failure times after artificially introducing grouping

on fusers that perform better than best sensor in a multiple sensor system sensor s sub i i 1 2 n outputs y sup i spl isin 0 1 according to an unknown probability distribution p y sup i x in response to input x spl isin 0 1 we choose a fuser that combines the outputs of sensors from a function class f f 0 1 sup n spl rarr 0 1 by minimizing empirical error based on an i i d sample if f satisfies the isolation property we show that the fuser performs at least as well as the best sensor in a probably approximately correct sense several well known fusers such as linear combinations special potential functions and certain feedforward networks satisfy the isolation property

an em algorithm for fitting two level structural equation models maximum likelihood is an important approach to analysis of two level structural equation models different algorithms for this purpose have been available in the literature in this paper we present a new formulation of two level structural equation models and develop an em algorithm for fitting this formulation this new formulation covers a variety of two level structural equation models as a result the proposed em algorithm is widely applicable in practice a practical example illustrates the performance of the em algorithm and the maximum likelihood statistic

multilevel models for survival analysis with random effects a method for modeling survival data with multilevel clustering is described the cox partial likelihood is incorporated into the generalized linear mixed model glmm methodology parameter estimation is achieved by maximizing a log likelihood analogous to the likelihood associated with the best linear unbiased prediction blup at the initial step of estimation and is extended to obtain residual maximum likelihood reml estimators of the variance component estimating equations for a three level hierarchical survival model are developed in detail and such a model is applied to analyze a set of chronic granulomatous disease cgd data on recurrent infections as an illustration with both hospital and patient effects being considered as random only the latter gives a significant contribution a simulation study is carried out to evaluate the performance of the reml estimators further extension of the estimation procedure to models with an arbitrary number of levels is also discussed

bayesian inference in econometric models using monte carlo integration methods for the systematic application of monte carlo integration with importance sampling to bayesian inference in econometric models are developed conditions under which the numerical approximation of a posterior moment converges almost surely to the true value as the number of monte carlo replications increases and the numerical accuracy of this approximation may be assessed reliably are set forth methods for the analytical verification of these conditions are discussed

double bagging combining classifiers by bootstrap aggregation the combination of classifiers leads to substantial reduction of misclassification error in a wide range of applications and benchmark problems we suggest using an out of bag sample for combining different classifiers in our setup a linear discriminant analysis is performed using the observations in the out of bag sample and the corresponding discriminant variables computed for the observations in the bootstrap sample are used as additional predictors for a classification tree two classifiers are combined and therefore method and variable selection bias is no problem for the corresponding estimate of misclassification error the need of an additional test sample disappears moreover the procedure performs comparable to the best classifiers used in a number of artificial examples and applications

estimation in the cox proportional hazards model with left truncated and interval censored data nous montrons que l estimateur non parametrique du maximum de vraisemblance npmle du coefficient de regression obtenu a partir de la vraisemblance jointe du coefficient de regression et de la survie de base est correct pour le modele de cox a hasards proportionnels avec des donnees tronquees a gauche et censurees par intervalle mais le npmle pent sous estimer la survie de base deux alternatives sont egalement etudiees premierement la vraisemblance marginale en etendant satten 1996 aux donnees tronquees la survie de base etant eliminee comme un parametre nuisible deuxiemement l estimateur monotone du maximum de vraisemblance qui maximise la vraisemblance jointe en supposant que la fonction de hasard est non decroissante pour la distribution de base ceci a ete initialement propose pour resoudre la sous estimation de la survie par le npmle dans le cas de donnees tronquees a gauche sans covariate tsai 1988 le boostrap est utilise a des fins d inference et des simulations sont realisees pour evaluer leur performance les methodes sont appliquees pour evaluer leur performance les methodes sont appliquees aux donnees de l etude du massachusetts health care panel afin de comparer chez les seniors en fonction du sexe les probabilites de perte d independance fonctionnelle

three step censored quantile regression and extramarital affairs this article suggests very simple three step estimators for censored quantile regression models with a separation restriction on the censoring probability the estimators are theoretically attractive i e asymptotically as efficient as the celebrated powell s censored least absolute deviation estimator at the same time they are conceptually simple and have trivial computational expenses they are especially useful in samples of small size or models with many regressors with desirable finite sample properties and small bias the separation restriction costs a small reduction of generality relative to the canonical censored regression quantile model yet its main plausible features remain intact the estimator can also be used to estimate a large class of traditional models including the normal amemiya tobin model and many accelerated failure and proportional hazard models we illustrate the approach with an extramarital affairs example and contrast our findings with those of fair

combining classifiers via discretization abstract i consider a method for combining different classifiers to develop more effective classification rules the proposed combined classifier which turns out to be strongly consistent is quite simple to use in real applications it is also shown that this combined classifier is strongly asymptotically at least as good as any one of the individual classifiers in addition if one of the individual classifiers is already bayes optimal asymptotically then so is the combined classifier

finding the observed information matrix when using the em algorithm a procedure is derived for extracting the observed information matrix when the em algorithm is used to find maximum likelihood estimates in incomplete data problems the technique requires computation of a complete data gradient vector or second derivative matrix but not those associated with the incomplete data likelihood in addition a method useful in speeding up the convergence of the em algorithm is developed two examples are presented

accelerated iterative reconstruction for positron emission tomography based on the em algorithm for maximum likelihood estimation the em method that was originally developed for maximum likelihood estimation in the context of mathematical statistics may be applied to a stochastic model of positron emission tomography pet the result is an iterative algorithm for image reconstruction that is finding increasing use in pet due to its attractive theoretical and practical properties its major disadvantage is the large amount of computation that is often required due to the algorithm s slow rate of convergence this paper presents an accelerated form of the em algorithm for pet in which the changes to the image as calculated by the standard algorithm are multiplied at each iteration by an overrelaxation parameter the accelerated algorithm retains two of the important practical properties of the standard algorithm namely the selfnormalization and nonnegativity of the reconstructed images experimental results are presented using measured data obtained from a hexagonal detector system for pet the likelihood function and the norm of the data residual were monitored during the iterative process according to both of these measures the images reconstructed at iterations 7 and 11 of the accelerated algorithm are similar to those at iterations 15 and 30 of the standard algorithm for two different sets of data important theoretical properties remain to be investigated namely the convergence of the accelerated algorithm and its performance as a maximum likelihood estimator