selection of important variables and determination of functional form for continuous predictors in multivariable model building in developing regression models data analysts are often faced with many predictor variables that may influence an outcome variable after more than half a century of research the best way of selecting a multivariable model is still unresolved it is generally agreed that subject matter knowledge when available should guide model building however such knowledge is often limited and data dependent model building is required we limit the scope of the modelling exercise to selecting important predictors and choosing interpretable and transportable functions for continuous predictors assuming linear functions stepwise selection and all subset strategies are discussed the key tuning parameters are the nominal p value for testing a variable for inclusion and the penalty for model complexity respectively we argue that stepwise procedures perform better than a literature based assessment would suggest concerning selection of functional form for continuous predictors the principal competitors are fractional polynomial functions and various types of spline techniques we note that a rigorous selection strategy known as multivariable fractional polynomials mfp has been developed no spline based procedure for simultaneously selecting variables and functional forms has found wide acceptance results of fp and spline modelling are compared in two data sets it is shown that spline modelling while extremely flexible can generate fitted curves with uninterpretable wiggles particularly when automatic methods for choosing the smoothness are employed we give general recommendations to practitioners for carrying out variable and function selection while acknowledging that further research is needed we argue why mfp is our preferred approach for multivariable model building with continuous covariates

hierarchical bayesian formulations for selecting variables in regression models the objective of finding a parsimonious representation of the observed data by a statistical model that is also capable of accurate prediction is commonplace in all domains of statistical applications the parsimony of the solutions obtained by variable selection is usually counterbalanced by a limited prediction capacity on the other hand methodologies that assure high prediction accuracy usually lead to models that are neither simple nor easily interpretable regularization methodologies have proven to be useful in addressing both prediction and variable selection problems the bayesian approach to regularization constitutes a particularly attractive alternative as it is suitable for high dimensional modeling offers valid standard errors and enables simultaneous estimation of regression coefficients and complexity parameters via computationally efficient mcmc techniques bayesian regularization falls within the versatile framework of bayesian hierarchical models which encompasses a variety of other approaches suited for variable selection such as spike and slab models and the mc3 approach in this article we review these bayesian developments and evaluate their variable selection performance in a simulation study for the classical small p large n setting the majority of the existing bayesian methodology for variable selection deals only with classical linear regression here we present two applications in the contexts of binary and survival regression where the bayesian approach was applied to select markers prognostically relevant for the development of rheumatoid arthritis and for overall survival in acute myeloid leukemia patients copyright 2012 john wiley sons ltd

nonparametric modeling of neural point processes via stochastic gradient boosting regression statistical nonparametric modeling tools that enable the discovery and approximation of functional forms e g tuning functions relating neural spiking activity to relevant covariates are desirable tools in neuroscience in this article we show how stochastic gradient boosting regression can be successfully extended to the modeling of spiking activity data while preserving their point process nature thus providing a robust nonparametric modeling tool we formulate stochastic gradient boosting in terms of approximating the conditional intensity function of a point process in discrete time and use the standard likelihood of the process to derive the loss function for the approximation problem to illustrate the approach we apply the algorithm to the modeling of primary motor and parietal spiking activity as a function of spiking history and kinematics during a two dimensional reaching task model selection goodness of fit via the time rescaling theorem model interpretation via partial dependence plots ranking of covariates according to their relative importance and prediction of peri event time histograms are illustrated and discussed additionally we use the tenfold cross validated log likelihood of the modeled neural processes 67 cells to compare the performance of gradient boosting regression to two alternative approaches standard generalized linear models glms and bayesian p splines with markov chain monte carlo mcmc sampling in our data set gradient boosting outperformed both bayesian p splines in approximately 90 of the cells and glms 100 because of its good performance and computational efficiency we propose stochastic gradient boosting regression as an off the shelf nonparametric tool for initial analyses of large neural data sets e g more than 50 cells more than 105 samples per cell with corresponding multidimensional covariate spaces e g more than four covariates in the cases where a functional form might be amenable to a more compact representation gradient boosting might also lead to the discovery of simpler parametric models

on the estimation of a probability density function by the maximum penalized likelihood method abstract a class of probability density estimates can be obtained by penalizing the likelihood by a functional which depends on the roughness of the logarithm of the density the limiting case of the estimates as the amount of smoothing increasing has a natural form which makes the method attractive for data analysis and which provides a rationale for a particular choice of roughness penalty the estimates are shown to be the solution of an unconstrained convex optimization problem and mild natural conditions are given for them to exist rates of consistency in various norms and conditions for asymptotic normality and approximation by a gaussian process are given thus breaking new ground in the theory of maximum penalized likelihood density estimation author

a bootstrap method for structure detection of narmax models many systems may be described by narmax models using only a few terms however depending on the order of the system the number of candidate terms can become very large selection of a subset of these candidate terms is necessary for an efficient system description this is an unresolved issue in system identification for over parameterized models therefore in this paper we develop a bootstrap structure detection bsd algorithm as a means of determining the structure of highly over parameterized models the performance of this bsd technique was evaluated by using it to estimate the structure of a 1 simple narmax model 2 moderately over parameterized narmax model and 3 highly over parameterized narmax model the results demonstrate that the bsd algorithm is a robust method for detecting the structure of narmax models this method provides accurate estimates of parameter statistics without relying on assumptions made by traditional procedures and yields a parsimonious description of the system

shrinkage estimators applied to prediction of french winter wheat yield summary the paper shows that it is possible to pool departmental information to predict regional wheat yield in an msep mean squared error of prediction context shrinkage estimation in linear agrometeorological models improves regional prediction cross validation is not precise enough to enable comparison between models the authors calculate the msep using the hypothesis of a multinormal distribution in simple cases the distribution parameters are estimated with data input and simulations are used for complex cases stein like estimators are shown to be a superior alternative to the conventional estimators usually applied for prediction

prediction error and its estimation for subset selected models strategies are compared for development of a linear regression model and the subsequent assessment of its predictive ability simulations were performed as a designed experiment over a range of data structures approaches using a forward selection of variables resulted in slightly smaller prediction errors and less biased estimators of predictive accuracy than all possible subsets selection but often did not improve on the full model random and balanced data splitting resulted in increased prediction errors and estimators with large mean squared error to maximize predictive accuracy while retaining a reliable estimate of that accuracy it is recommended that the entire sample usually he used for model development and assessment

cross validation in survival analysis the predictive value of a statistical model is conceptually different from the explained variation in this paper we construct a measure of the predictive value of the cox proportional hazards model computed from the leave one out regression coefficients these coefficients can also be used to calculate a shrinkage factor which can be applied to improve the predictions and that can be used in r2 type measures of the proportion of explained variation our methods are illustrated by a study of chemotherapy for advanced ovarian cancer

ridge regression biased estimation for nonorthogonal problems in multiple regression it is shown that parameter estimates based on minimum residual sum of squares have a high probability of being unsatisfactory if not incorrect if the prediction vectors are not orthogonal proposed is an estimation procedure based on adding small positive quantities to the diagonal of x x introduced is the ridge trace a method for showing in two dimensions the effects of nonorthogonality it is then shown how to augment x x to obtain biased estimates with smaller mean square error

sample size considerations and predictive performance of multinomial logistic prediction models multinomial logistic regression mlr has been advocated for developing clinical prediction models that distinguish between three or more unordered outcomes we present a full factorial simulation study to examine the predictive performance of mlr models in relation to the relative size of outcome categories number of predictors and the number of events per variable it is shown that mlr estimated by maximum likelihood yields overfitted prediction models in small to medium sized data in most cases the calibration and overall predictive performance of the multinomial prediction model is improved by using penalized mlr our simulation study also highlights the importance of events per variable in the multinomial context as well as the total sample size as expected our study demonstrates the need for optimism correction of the predictive performance measures when developing the multinomial logistic prediction model we recommend the use of penalized mlr when prediction models are developed in small data sets or in medium sized data sets with a small total sample size ie when the sizes of the outcome categories are balanced finally we present a case study in which we illustrate the development and validation of penalized and unpenalized multinomial prediction models for predicting malignancy of ovarian cancer

self adjointness of the operators in wigner s commutation relations both self adjointness and spectra of the momentum operator p and the hamiltonian h are studied p i d dx c x r h 2 1 d dx 2 c x2 c r x2 for a one dimensional harmonic oscillator governed by wigner s commutation relations here c is a real parameter and r x x first the essential self adjointness of p and h are shown to obtain that the spectrum of h consists of eigenvalues only moreover we clarify the meaning in l2 r of the classical form h 2 1 p2 x2 next it is shown that every real number is in the continuous spectrum of p by introducing a kind of generalized fourier transform b satisfying p b 1xb for the multiplication x in the case of c 0 b turns out to be the ordinary fourier transform and p becomes the well known operator by the methods stated above all the results of the recent papers where some restrictions on the parameter c were imposed are improved

estimation of the mean of a multivariate normal distribution estimation of the means of independent normal random variables is considered using sum of squared errors as loss an unbiased estimate of risk is obtained for an arbitrary estimate and certain special classes of estimates are then discussed the results are applied to smoothing by use of moving averages and to trimmed analogs of the james stein estimate a suggestion is made for calculating approximate confidence sets for the mean vector centered at an arbitrary estimate

inertia controlling methods for general quadratic programming active set quadratic programming qp methods use a working set to define the search direction and multiplier estimates in the method proposed by fletcher in 1971 and in several subsequent mathematically equivalent methods the working set is chosen to control the inertia of the reduced hessian which is never permitted to have more than one nonpositive eigenvalue such methods will be called inertia controlling this paper presents an overview of a generic inertia controlling qp method including the equations satisfied by the search direction when the reduced hessian is positive definite singular and indefinite recurrence relations are derived that define the search direction and lagrange multiplier vector through equations related to the karush kuhn tucker system discussion is included of connections with inertia controlling methods that maintain an explicit factorization of the reduced hessian matrix

an interpretation of partial least squares abstract univariate partial least squares pls is a method of modeling relationships between a y variable and other explanatory variables it may be used with any number of explanatory variables even far more than the number of observations a simple interpretation is given that shows the method to be a straightforward and reasonable way of forming prediction equations its relationship to multivariate pls in which there are two or more y variables is examined and an example is given in which it is compared by simulation with other methods of forming prediction equations with univariate pls linear combinations of the explanatory variables are formed sequentially and related to y by ordinary least squares regression it is shown that these linear combinations here called components may be viewed as weighted averages of predictors where each predictor holds the residual information in an explanatory variable that is not contained in earlier components and the quantity to be predicted is the vecto

classical and bayesian inference in neuroimaging theory this paper reviews hierarchical observation models used in functional neuroimaging in a bayesian light it emphasizes the common ground shared by classical and bayesian methods to show that conventional analyses of neuroimaging data can be usefully extended within an empirical bayesian framework in particular we formulate the procedures used in conventional data analysis in terms of hierarchical linear models and establish a connection between classical inference and parametric empirical bayes peb through covariance component estimation this estimation is based on an expectation maximization or em algorithm the key point is that hierarchical models not only provide for appropriate inference at the highest level but that one can revisit lower levels suitably equipped to make bayesian inferences bayesian inferences eschew many of the difficulties encountered with classical inference and characterize brain responses in a way that is more directly predicated on what one is interested in the motivation for bayesian approaches is reviewed and the theoretical background is presented in a way that relates to conventional methods in particular restricted maximum likelihood reml this paper is a technical and theoretical prelude to subsequent papers that deal with applications of the theory to a range of important issues in neuroimaging these issues include i estimating nonsphericity or variance components in fmri time series that can arise from serial correlations within subject or are induced by multisubject i e hierarchical studies ii spatiotemporal bayesian models for imaging data in which voxels specific effects are constrained by responses in other voxels iii bayesian estimation of nonlinear models of hemodynamic responses and iv principled ways of mixing structural and functional priors in eeg source reconstruction although diverse all these estimation problems are accommodated by the peb framework described in this paper

prediction based on mean subset shrinkage methods have traditionally been applied in prediction problems in this article we develop a shrinkage method mean subset that forms an average of regression coefficients from individual subsets of the explanatory variables a bayesian approach is taken to derive an expression of how the coefficient vectors from each subset should be weighted it is not computationally feasible to calculate the mean subset coefficient vector for larger problems and thus we suggest an algorithm to find an approximation to the mean subset coefficient vector in a comprehensive monte carlo simulation study it is found that the proposed mean subset method has superior prediction performance than prediction based on the best subset method and in some settings also better than the ridge regression and lasso methods the conclusions drawn from the monte carlo study is corroborated in an example in which prediction is made using spectroscopic data

meta analytical synthesis of regression coefficients under different categorization scheme of continuous covariates recently the number of clinical prediction models sharing the same regression task has increased in the medical literature however evidence synthesis methodologies that use the results of these regression models have not been sufficiently studied particularly in meta analysis settings where only regression coefficients are available one of the difficulties lies in the differences between the categorization schemes of continuous covariates across different studies in general categorization methods using cutoff values are study specific across available models even if they focus on the same covariates of interest differences in the categorization of covariates could lead to serious bias in the estimated regression coefficients and thus in subsequent syntheses to tackle this issue we developed synthesis methods for linear regression models with different categorization schemes of covariates a 2 step approach to aggregate the regression coefficient estimates is proposed the first step is to estimate the joint distribution of covariates by introducing a latent sampling distribution which uses one set of individual participant data to estimate the marginal distribution of covariates with categorization the second step is to use a nonlinear mixed effects model with correction terms for the bias due to categorization to estimate the overall regression coefficients especially in terms of precision numerical simulations show that our approach outperforms conventional methods which only use studies with common covariates or ignore the differences between categorization schemes the method developed in this study is also applied to a series of who epidemiologic studies on white blood cell counts

the extended stein procedure for simultaneous model selection and parameter estimation abstract a parameter truncation criterion developed in conjunction with stein s partitioned analog of the efron and morris limited translation estimator is used in an orthonormal linear statistical model setting as a basis for simultaneously selecting the relevant variables and estimating the unknown parameters using a squared error loss measure and making use of prior information concerning the coefficients of extraneous variables the sampling performance of the extended stein procedure esp is analyzed for two alternative structures of the parameter space and under normal and non normal errors

on practical conditions for the existence and uniqueness of solutions to the general equality quadratic programming problem we present practical conditions under which the existence and uniqueness of a finite solution to a given equality quadratic program may be examined different sets of conditions allow this examination to take place when null space range space or lagrangian methods are used to find stationary points for the quadratic program

abs algorithms for linear equations and optimization in this paper we review basic properties and the main achievements obtained by the class of abs methods developed since 1981 to solve linear and nonlinear algebraic equations and optimization problems

a goodness of fit test for binary regression models based on smoothing methods a new global test statistic for models with continuous covariates and binary response is introduced the test statistic is based on nonparametric kernel methods explicit expressions are given for the mean and variance of the test statistic asymptotic properties are considered and approximate corrections due to parameter estimation are presented properties of the test statistic are studied by simulation the goodness of fit method is illustrated on data from a dutch follow up study on preterm infants recommendations for practitioners are given

minimum sample size for developing a multivariable prediction model part ii binary and time to event outcomes when designing a study to develop a new prediction model with binary or time to event outcomes researchers should ensure their sample size is adequate in terms of the number of participants n and outcome events e relative to the number of predictor parameters p considered for inclusion we propose that the minimum values of n and e and subsequently the minimum number of events per predictor parameter epp should be calculated to meet the following three criteria i small optimism in predictor effect estimates as defined by a global shrinkage factor of 0 9 ii small absolute difference of 0 05 in the model s apparent and adjusted nagelkerke s r2 and iii precise estimation of the overall risk in the population criteria i and ii aim to reduce overfitting conditional on a chosen p and require prespecification of the model s anticipated cox snell r2 which we show can be obtained from previous studies the values of n and e that meet all three criteria provides the minimum sample size required for model development upon application of our approach a new diagnostic model for chagas disease requires an epp of at least 4 8 and a new prognostic model for recurrent venous thromboembolism requires an epp of at least 23 this reinforces why rules of thumb eg 10 epp should be avoided researchers might additionally ensure the sample size gives precise estimates of key predictor effects this is especially important when key categorical predictors have few events in some categories as this may substantially increase the numbers required

a dose finding approach based on shrunken predictive probability for combinations of two agents in phase i trials we develop a novel dose finding method for two agent combination phase i trials on the basis of the shrunken predictive probability of toxicity in this method a shrinkage logistic regression model that allows distinct shrinkage multipliers for the coefficients of the main effects of two agents and their interaction on the probability of toxicity constructs the toxicity outcome we also propose dose escalation de escalation decision rules on the basis of the shrunken predictive probability of toxicity simulation studies under various patterns of monotonic dose response relationships for combinations of two agents demonstrated that the proposed method performed no worse than the existing two dose finding methods we selected copyright 2013 john wiley sons ltd

regularized discriminant analysis abstract linear and quadratic discriminant analysis are considered in the small sample high dimensional setting alternatives to the usual maximum likelihood plug in estimates for the covariance matrices are proposed these alternatives are characterized by two parameters the values of which are customized to individual situations by jointly minimizing a sample based estimate of future misclassification risk computationally fast implementations are presented and the efficacy of the approach is examined through simulation studies and application to data these studies indicate that in many circumstances dramatic gains in classification accuracy can be achieved

general methods for analysing repeated measures we present an overview of issues and methods for analysing repeated measures of a continuous random variable we discuss modelling the mean vector and covariance structure statistical efficiency regression diagnostics and discrepancies between longitudinal and cross sectional methods we illustrate key points with examples and discuss areas requiring further development

bayes model averaging with selection of regressors when a number of distinct models contend for use in prediction the choice of a single model can offer rather unstable predictions in regression stochastic search variable selection with bayesian model averaging offers a cure for this robustness issue but at the expense of requiring very many predictors here we look at bayes model averaging incorporating variable selection for prediction this offers similar mean square errors of prediction but with a vastly reduced predictor space this can greatly aid the interpretation of the model it also reduces the cost if measured variables have costs the development here uses decision theory in the context of the multivariate general linear model in passing this reduced predictor space bayes model averaging is contrasted with single model approximations a fast algorithm for updating regressions in the markov chain monte carlo searches for posterior inference is developed allowing many more variables than observations to be contemplated we discuss the merits of absolute rather than proportionate shrinkage in regression especially when there are more variables than observations the methodology is illustrated on a set of spectroscopic data used for measuring the amounts of different sugars in an aqueous solution

risk scores from logistic regression unbiased estimates of relative and attributable risk in epidemiology the risk of disease in terms of a set of covariates is often modelled by logistic regression the resulting linear predictor can be used to define the extent of risk between extremes and to calculate an attributable risk for the covariates taken together as is well known straightforward use of the linear predictor on the sample from which it was derived to obtain estimates the relative and attributable risk will be biased often seriously use of the jack knife technique is extended to produce asymptotically unbiased estimates of relative and attributable risks the asymptotic variances associated with these estimates are derived by using the formulae of conditional variances they are applied to the results of a case control study of stomach cancer

a simulation study comparing properties of heterogeneity measures in meta analyses the assessment of heterogeneity or between study variance is an important issue in meta analysis it determines the statistical methods to be used and the interpretation of the results tests of heterogeneity may be misleading either due to low power for sparse data or to the detection of irrelevant amounts of heterogeneity when many studies are involved in the former case notable heterogeneity may remain unconsidered and an unsuitable model may be chosen and the latter case may lead to unnecessary complex analyses strategies measures of heterogeneity are better suited to determine appropriate analyses strategies we review two measures with different scaling and compare them with the heterogeneity test estimates of the within study variance are discussed and a new total information measure is introduced various properties of the quantities in question are assessed by a simulation study heterogeneity test and measures are not directly related to the amount of between study variance but to the relative increase of variance due to heterogeneity it is more favourable to base the within study variance estimate on the squared weights of individual studies than on the sum of weights a heterogeneity measure scaled to a fixed interval needs reference values for proper interpretation a measure defined by the relation of between to within study variance has a more natural interpretation but no upper limit both measures are quantifications of the impact of heterogeneity on the meta analysis result as both depend on the variance of the individual study effects and thus on the number of patients in the studies

a harmonic oscillator system with the generalized commutation relations the system of a one dimensional harmonic oscillator is discussed with the generalized commutation relations obtained by wigner the hamiltonian is shown to have self adjoint extensions the domain of self adjointness is explicitly specified in some cases the proof is carried out by the use of the lax milgram lemma a suitable rigged hilbert space is found for this system to reformulate the earlier arguments the main difficulty is that the momentum and hamiltonian operators contain the reflection r x x as well as singular terms

how many variables should be entered in a regression equation abstract the optimal number of regressors is determined to minimize mean squared prediction error and is shown to be a small fraction of the number of data points as the number of regressors grows large the sp criterion provides an asymptotically optimal rule for the number of variables to enter

zeroing neural network for solving time varying linear equation and inequality systems a typical recurrent neural network called zeroing neural network znn was developed for time varying problem solving in a previous study many applications result in time varying linear equation and inequality systems that should be solved in real time this paper provides a znn model for determining the solution of time varying linear equation and inequality systems by introducing a nonnegative slack variable the time varying linear equation and inequality systems are transformed into a mixed nonlinear system the znn model is established via the definition of an indefinite error function and the usage of an exponential decay formula theoretical results indicate the convergence property of the proposed znn model comparative simulation results prove the znn effectiveness and superiority for time varying linear equation and inequality systems furthermore the proposed znn model is employed to robot manipulators thus showing the znn applicability

solution of linear least squares via the abs algorithm the abs class for linear and nonlinear systems has been recently introduced by abaffy broyden galantai and spedicato here we consider various ways of applying these algorithms to the determination of the minimal euclidean norm solution of over determined linear systems in the least squares sense extensive numerical experiments show that the proposed algorithms are efficient and that one of them usually gives better accuracy than standard implementations of the qr orthogonalization algorithm with householder reflections

the quasi newton least squares method a new and fast secant method analyzed for linear systems we present a new quasi newton method that can solve systems of equations of which no information is known explicitly and which requires no special structure of the system matrix like positive definiteness or sparseness the method builds an approximate jacobian based on input output combinations of a black box system uses a rank one update of this jacobian after each iteration and satisfies the secant equation while it has originally been developed for nonlinear equations we analyze its properties and performance when applied to linear systems analytically the method is shown to be convergent in n 1 iterations n being the number of unknowns irrespective of the nature of the system matrix the performance of this method is greatly superior to other quasi newton methods and comparable with gmres when tested on a number of standardized test cases

adaptive prior weighting in generalized regression the prior distribution is a key ingredient in bayesian inference prior information on regression coefficients may come from different sources and may or may not be in conflict with the observed data various methods have been proposed to quantify a potential prior data conflict such as box s p value however there are no clear recommendations how to react to possible prior data conflict in generalized regression models to address this deficiency we propose to adaptively weight a prespecified multivariate normal prior distribution on the regression coefficients to this end we relate empirical bayes estimates of prior weight to box s p value and propose alternative fully bayesian approaches prior weighting can be done for the joint prior distribution of the regression coefficients or under prior independence separately for prespecified blocks of regression coefficients we outline how the proposed methodology can be implemented using integrated nested laplace approximations inla and illustrate the applicability with a bayesian logistic regression model for data from a cross sectional study we also provide a simulation study that shows excellent performance of our approach in the case of prior misspecification in terms of root mean squared error and coverage supplementary materials give details on software implementation and code and another application to binary longitudinal data from a randomized clinical trial using a bayesian generalized linear mixed model

predictive accuracy and explained variation in cox regression summary we suggest a new measure of the proportion of the variation of possibly censored survival times explained by a given proportional hazards model the proposed measure termed v shares several favorable properties with an earlier v1 but also improves the handling of censoring the statistic contrasts distance measures between individual 1 0 survival processes and fitted survival curves with and without covariate information these distance measures dx and d respectively are themselves informative as summaries of absolute rather than relative predictive accuracy we recommend graphical comparisons of survival curves for prognostic index groups to improve the understanding of obtained values for v dx and d their use and interpretation is exemplified for a yorkshire lung cancer study on survival from this and an overview for several well known clinical data sets we show that the likely amount of relative or absolute predictive accuracy is often low even if there are highly significant and relatively strong prognostic factors

the little bootstrap and other methods for dimensionality selection in regression x fixed prediction error abstract when a regression problem contains many predictor variables it is rarely wise to try to fit the data by means of a least squares regression on all of the predictor variables usually a regression equation based on a few variables will be more accurate and certainly simpler there are various methods for picking good subsets of variables and programs that do such procedures are part of every widely used statistical package the most common methods are based on stepwise addition or deletion of variables and on best subsets the latter refers to a search method that given the number of variables to be in the equation say five locates that regression equation based on five variables that has the lowest residual sum of squares among all five variable equations all of these procedures generate a sequence of regression equations the first based on one variable the next based on two variables and so on each member of this sequence is called a submodel and the number of variables in the e

bootstrap investigation of the stability of a cox regression model we describe a bootstrap investigation of the stability of a cox proportional hazards regression model resulting from the analysis of a clinical trial of azathioprine versus placebo in patients with primary biliary cirrhosis we have considered stability to refer both to the choice of variables included in the model and more importantly to the predictive ability of the model in stepwise cox regression analyses of 100 bootstrap samples using 17 candidate variables the most frequently selected variables were those selected in the original analysis and no other important variable was identified thus there was no reason to doubt the model obtained in the original analysis for each patient in the trial bootstrap confidence intervals were constructed for the estimated probability of surviving two years it is shown graphically that these intervals are markedly wider than those obtained from the original model

a diagnostic for cox regression and general conditional likelihoods abstract diagnostics for changes in the maximum likelihood estimate of due to deletion of single observations in linear and logistic regression models are equivalent to diagnostics obtained from fitting augmented regression models for the cox proportional hazards model and in conditional models for arbitrarily matched or stratified data diagnostics derived from augmented regression models are again useful and easy to compute these diagnostics are applicable to different formulations for the multiplicative covariable effects over time or over strata an exact or approximate treatment of tied failure times or multiple case matching and time dependent covariates

multiple imputation for an incomplete covariate that is a ratio we are concerned with multiple imputation of the ratio of two variables which is to be used as a covariate in a regression analysis if the numerator and denominator are not missing simultaneously it seems sensible to make use of the observed variable in the imputation model one such strategy is to impute missing values for the numerator and denominator or the log transformed numerator and denominator and then calculate the ratio of interest we call this passive imputation alternatively missing ratio values might be imputed directly with or without the numerator and or the denominator in the imputation model we call this active imputation in two motivating datasets one involving body mass index as a covariate and the other involving the ratio of total to high density lipoprotein cholesterol we assess the sensitivity of results to the choice of imputation model and as an alternative explore fully bayesian joint models for the outcome and incomplete ratio fully bayesian approaches using winbugs were unusable in both datasets because of computational problems in our first dataset multiple imputation results are similar regardless of the imputation model in the second results are sensitive to the choice of imputation model sensitivity depends strongly on the coefficient of variation of the ratio s denominator a simulation study demonstrates that passive imputation without transformation is risky because it can lead to downward bias when the coefficient of variation of the ratio s denominator is larger than about 0 1 active imputation or passive imputation after log transformation is preferable 2013 the authors statistics in medicine published by john wiley sons ltd

prognostic models based on literature and individual patient data in logistic regression analysis prognostic models can be developed with multiple regression analysis of a data set containing individual patient data often this data set is relatively small while previously published studies present results for larger numbers of patients we describe a method to combine univariable regression results from the medical literature with univariable and multivariable results from the data set containing individual patient data this adaptation method exploits the generally strong correlation between univariable and multivariable regression coefficients the method is illustrated with several logistic regression models to predict 30 day mortality in patients with acute myocardial infarction the regression coefficients showed considerably less variability when estimated with the adaptation method compared to standard maximum likelihood estimates also model performance as distinguished in calibration and discrimination improved clearly when compared to models including shrunk or penalized estimates we conclude that prognostic models may benefit substantially from explicit incorporation of literature data copyright 2000 john wiley sons ltd

the local convergence of abs methods for nonlinear algebraic equations in this paper we consider an extension to nonlinear algebraic systems of the class of algorithms recently proposed by abaffy broyden and spedicato for general linear systems we analyze the convergence properties showing that under the usual assumptions on the function and some mild assumptions on the free parameters available in the class the algorithm is locally convergent and has a superlinear rate of convergence per major iteration which is computationally comparable to a single newton s step some particular algorithms satisfying the conditions on the free parameters are considered

cross validation the jackknife and the bootstrap excess error estimation in forward logistic regression abstract given a prediction rule based on a set of patients what is the probability of incorrectly predicting the outcome of a new patient call this probability the true error an optimistic estimate is the apparent error or the proportion of incorrect predictions on the original set of patients and it is the goal of this article to study estimates of the excess error or the difference between the true and apparent errors i consider three estimates of the excess error cross validation the jackknife and the bootstrap using simulations and real data the three estimates for a specific prediction rule are compared when the prediction rule is allowed to be complicated overfitting becomes a real danger and excess error estimation becomes important the prediction rule chosen here is moderately complicated involving a variable selection procedure based on forward logistic regression

stability of multivariable fractional polynomial models with selection of variables and transformations a bootstrap investigation sauerbrei and royston have recently described an algorithm based on fractional polynomials for the simultaneous selection of variables and of suitable transformations for continuous predictors in a multivariable regression setting they illustrated the approach by analyses of two breast cancer data sets here we extend their work by considering how to assess possible instability in such multivariable fractional polynomial models we first apply the algorithm repeatedly in many bootstrap replicates we then use log linear models to investigate dependencies among the inclusion fractions for each predictor and among the simplified classes of fractional polynomial function chosen in the bootstrap samples to further evaluate the results we define measures of instability based on a decomposition of the variability of the bootstrap selected functions in relation to a reference function from the original model for each data set we are able to identify large reasonably stable subsets of the bootstrap replications in which the functional forms of the predictors appear fairly stable despite the considerable flexibility of the family of fractional polynomials and the consequent risk of overfitting when several variables are considered we conclude that the multivariable selection algorithm can find stable models

risk reducing shrinkage estimation for generalized linear models to generalized linear models in a manner retaining the original spirit of shrinkage estimation which is to reduce risk the investigation identifies two classes of simple all purpose prior distributions which supplement such non informative priors as jeffreys s prior with mechanisms for risk reduction one new class of priors is motivated as optimizers of a core component of asymptotic risk the methodology is evaluated in a numerical exploration and application to an existing data set

diagnostic and sampling models in screening summary screening methods are developed which raise the proportion of successes in the retained population to a predetermined level specification regions for the screening variables are derived within both the diagnostic and sampling paradigms dawid 1976 logistic and normal models are used for the two frameworks and the results are illustrated by an example of screening for malnourishment

representation and properties of para bose oscillator operators ii coherent states and the minimum uncertainty states the energy position and momentum eigenstates of a para bose oscillator system were considered in paper i here we consider the bargmann or the analytic function description of the para bose system this brings in in a natural way the coherent states z defined as the eigenstates of the annihilation operator the transformation functions relating this description to the energy position and momentum eigenstates are explicitly obtained possible resolution of the identity operator using coherent states is examined a particular resolution contains two integrals one containing the diagonal basis z z and the other containing the pseudodiagonal basis z z we briefly consider the normal and antinormal ordering of the operators and their diagonal and discrete diagonal coherent state approximations the problem of constructing states with a minimum value of the product of the position and momentum uncertainties and the possible dependence of this minimum value is considered

extension of the stein estimating procedure through the use of estimating functions abstract this article extends the stein method through the use of estimating functions godambe 1960 to address the simultaneous estimation of k population parameters 1 k in a mixed model setting the procedure generalizes the stein method by a allowing us to deal effectively with complications such as inequality of population variances that may arise in non gaussian mixed models b being appropriate for estimating i in populations of varying sizes and in particular populations of small sizes and c applying to situations where it cannot be assumed that the i s have unbiased estimators or even estimators of finite moment the focus of the article is on the quadratic variance function exponential family morris 1983b estimators for the parameters of the mixed model are developed in a regression model setting in which the i s are allowed to vary with a vector of covariates an application to incidence rates for the iceland breast cancer incidence data is presented for illustra

how biased is the apparent error rate of a prediction rule abstract a regression model is fitted to an observed set of data how accurate is the model for predicting future observations the apparent error rate tends to underestimate the true error rate because the data have been used twice both to fit the model and to check its accuracy we provide simple estimates for the downward bias of the apparent error rate the theory applies to general exponential family linear models and general measures of prediction error special attention is given to the case of logistic regression on binary data with error rates measured by the proportion of misclassified cases several connected ideas are compared mallows s cp cross validation generalized cross validation the bootstrap and akaike s information criterion

a likelihood approach to meta analysis with random effects in a meta analysis of a set of clinical trials a crucial but problematic component is providing an estimate and confidence interval for the overall treatment effect theta since in the presence of heterogeneity a fixed effect approach yields an artificially narrow confidence interval for theta the random effects method of dersimonian and laird which incorporates a moment estimator of the between trial components of variance sigma b2 has been advocated with the additional distributional assumptions of normality a confidence interval for theta may be obtained however this method does not provide a confidence interval for sigma b2 nor a confidence interval for theta which takes account of the fact that sigma b2 has to be estimated from the data we show how a likelihood based method can be used to overcome these problems and use profile likelihoods to construct likelihood based confidence intervals this approach yields an appropriately widened confidence interval compared with the standard random effects method examples of application to a published meta analysis and a multicentre clinical trial are discussed it is concluded that likelihood based methods are preferred to the standard method in undertaking random effects meta analysis when the value of sigma b2 has an important effect on the overall estimated treatment effect

generalized conjugate directions abstract this paper presents a simple unifying framework for a wide class of conjugate directions algorithms whose iterates minimize some quadratic functional over a subspace our approach is motivated by its advantages for nonlinear minimization but the purpose of this paper is to present the greatly simplified convergence analysis that results for the linear case

estimating the error rate of a prediction rule improvement on cross validation abstract we construct a prediction rule on the basis of some data and then wish to estimate the error rate of this rule in classifying future observations cross validation provides a nearly unbiased estimate using only the original data cross validation turns out to be related closely to the bootstrap estimate of the error rate this article has two purposes to understand better the theoretical basis of the prediction problem and to investigate some related estimators which seem to offer considerably improved estimation in small samples

validation and updating of predictive logistic regression models a study on sample size and shrinkage a logistic regression model may be used to provide predictions of outcome for individual patients at another centre than where the model was developed when empirical data are available from this centre the validity of predictions can be assessed by comparing observed outcomes and predicted probabilities subsequently the model may be updated to improve predictions for future patients as an example we analysed 30 day mortality after acute myocardial infarction in a large data set gusto i n 40 830 we validated and updated a previously published model from another study timi ii n 3339 in validation samples ranging from small 200 patients 14 deaths to large 10 000 patients 700 deaths updated models were tested on independent patients updating methods included re calibration re estimation of the intercept or slope of the linear predictor and more structural model revisions re estimation of some or all regression coefficients model extension with more predictors we applied heuristic shrinkage approaches in the model revision methods such that regression coefficients were shrunken towards their re calibrated values parsimonious updating methods were found preferable to more extensive model revisions which should only be attempted with relatively large validation samples in combination with shrinkage copyright 2004 john wiley sons ltd

on the robustness of shrinkage predictors in regression to differences between past and future data on etudie la robustesse des predicteurs de copas 1983 vis a vis des ecarts a la distribution

cross validation shrinkage of regression predictors summary copas 1983 suggests that shrinkage of predictors can be related to the least squares slope of actual on predicted values in a new set of data considering multiple regression in a cross validatory setting this shrinkage slope is approximated by an estimable function of the fitted residuals patterns of heteroscedasticity of residual variance can lead to more or less shrinkage of least squares a new predictor using a non parametric shrinkage multiplier is proposed

a numerically stable dual method for solving strictly convex quadratic programs an efficient and numerically stable dual algorithm for positive definite quadratic programming is described which takes advantage of the fact that the unconstrained minimum of the objective function can be used as a starting point its implementation utilizes the cholesky and qr factorizations and procedures for updating them the performance of the dual algorithm is compared against that of primal algorithms when used to solve randomly generated test problems and quadratic programs generated in the course of solving nonlinear programming problems by a successive quadratic programming code the principal motivation for the development of the algorithm these computational results indicate that the dual algorithm is superior to primal algorithms when a primal feasible point is not readily available the algorithm is also compared theoretically to the modified simplex type dual methods of lemke and van de panne and whinston and it is illustrated by a numerical example

prognostic modelling with logistic regression analysis a comparison of selection and estimation methods in small data sets logistic regression analysis may well be used to develop a prognostic model for a dichotomous outcome especially when limited data are available it is difficult to determine an appropriate selection of covariables for inclusion in such models also predictions may be improved by applying some sort of shrinkage in the estimation of regression coefficients in this study we compare the performance of several selection and shrinkage methods in small data sets of patients with acute myocardial infarction where we aim to predict 30 day mortality selection methods included backward stepwise selection with significance levels alpha of 0 01 0 05 0 157 the aic criterion or 0 50 and the use of qualitative external information on the sign of regression coefficients in the model estimation methods included standard maximum likelihood the use of a linear shrinkage factor penalized maximum likelihood the lasso or quantitative external information on univariable regression coefficients we found that stepwise selection with a low alpha for example 0 05 led to a relatively poor model performance when evaluated on independent data substantially better performance was obtained with full models with a limited number of important predictors where regression coefficients were reduced with any of the shrinkage methods incorporation of external information for selection and estimation improved the stability and quality of the prognostic models we therefore recommend shrinkage methods in full models including prespecified predictors and incorporation of external information when prognostic models are constructed in small data sets

steepest edge simplex algorithms for linear programming we present several new steepest edge simplex algorithms for solving linear programming problems including variants of both the primal and the dual simplex method these algorithms differ depending upon the space in which the problem is viewed as residing and include variants in which this space varies dynamically we present computational results comparing steepest edge simplex algorithms and approximate versions of them against simplex algorithms that use standard pivoting rules on truly large scale realworld linear programs with as many as tens of thousands of rows and columns these results demonstrate unambiguously the superiority of steepest edge pivot selection criteria to other pivot selection criteria in the simplex method

statistical treatment rules for heterogeneous populations an important objective of empirical research on treatment response is to provide decision makers with information useful in choosing treatments this paper studies minimax regret treatment choice using the sample data generated by a classical randomized experiment consider a utilitarian social planner who must choose among the feasible statistical treatment rules these being functions that map the sample data and observed covariates of population members into a treatment allocation if the planner knew the population distribution of treatment response the optimal treatment rule would maximize mean welfare conditional on all observed covariates the appropriate use of covariate information is a more subtle matter when only sample data on treatment response are available i consider the class of conditional empirical success rules that is rules assigning persons to treatments that yield the best experimental outcomes conditional on alternative subsets of the observed covariates i derive a closed form bound on the maximum regret of any such rule comparison of the bounds for rules that condition on smaller and larger subsets of the covariates yields sufficient sample sizes for productive use of covariate information when the available sample size exceeds the sufficiency boundary a planner can be certain that conditioning treatment choice on more covariates is preferable in terms of minimax regret to conditioning on fewer covariates

a predictive view of the detection and characterization of influential observations in regression analysis abstract assuming a set of observations is available from the general linear model and assuming prior information about parameters we propose a method of assessing the influence of specified subsets of the data when the goal is to predict future observations

bounds on reciprocal moments with applications and developments in stein estimation and post stratification summary bounds on the kth reciprocal moment of a distribution are given in terms of the mean variance and left extremity of the distribution a methodology for obtaining distribution specific bounds with improved accuracy is outlined some natural approximations are noted applications are given in stein estimation and in classical and bayesian post stratification

bootstrap and cross validation estimates of the prediction error for linear regression models different estimates of the mean squared error of prediction for linear regression models are derived by the bootstrap and cross validation approaches a comparison is made under normal error distributions especially by the biases and the mean square errors the results indicate that the bias corrected bootstrap estimator is best unbiased and should be the first choice while its simulated variant has approximately the same behaviour on the other hand if only a comparison between uncorrected estimators is made with implications for nonlinear regression models in mind then other variants of bootstrap estimates are preferable for a large or a small dimension of the model parameter for a small dimension the cross validation estimate and sometimes grouped variants of it seem also to be acceptable if the model error is known to be small

strictly proper scoring rules prediction and estimation scoring rules assess the quality of probabilistic forecasts by assigning a numerical score based on the predictive distribution and on the event or value that materializes a scoring rule is proper if the forecaster maximizes the expected score for an observation drawn from the distributionf if he or she issues the probabilistic forecast f rather than g f it is strictly proper if the maximum is unique in prediction problems proper scoring rules encourage the forecaster to make careful assessments and to be honest in estimation problems strictly proper scoring rules provide attractive loss and utility functions that can be tailored to the problem at hand this article reviews and develops the theory of proper scoring rules on general probability spaces and proposes and discusses examples thereof proper scoring rules derive from convex functions and relate to information measures entropy functions and bregman divergences in the case of categorical variables we prove a rigorous version of the

penalized likelihood in cox regression in a cox regression model instability of the estimated regression coefficients can be reduced by maximizing a penalized partial log likelihood where a penalty function of the regression coefficients is substracted from the partial log likelihood in this paper we choose the optimal weight of the penalty function by maximizing the predictive value of the model as measured by the crossvalidated partial log likelihood our methods are illustrated by a study of ovarian cancer survival and by a study of centre effects in kidney graft survival

mixtures of g priors in generalized linear models abstractmixtures of zellner s g priors have been studied extensively in linear models and have been shown to have numerous desirable properties for bayesian variable selection and model averaging

predictive value of statistical models a review is given of different ways of estimating the error rate of a prediction rule based on a statistical model a distinction is drawn between apparent optimum and actual error rates moreover it is shown how cross validation can be used to obtain an adjusted predictor with smaller error rate a detailed discussion is given for ordinary least squares logistic regression and cox regression in survival analysis finally the splitsample approach is discussed and demonstrated on two data sets

logistic regression diagnostics a maximum likelihood fit of a logistic regression model and other similar models is extremely sensitive to outlying responses and extreme points in the design space we develop diagnostic measures to aid the analyst in detecting such observations and in quantifying their effect on various aspects of the maximum likelihood fit the elements of the fitting process which constitute the usual output parameter estimates standard errors residuals etc will be used for this purpose with a properly designed computing package for fitting the usual maximum likelihood model the diagnostics are essentially free for the asking in particular good data analysis for logistic regression models need not be expensive or time consuming

a class of direct methods for linear systems a class of methods of direct type for solving determined or underdetermined full rank or deficient rank linear systems is presented and theoretically analyzed the class can be considered as a generalization of the methods of brent and brown as restricted to linear systems and implicitly contains orthogonal lu andll t factorization methods

the bootstrap and identification of prognostic factors via cox s proportional hazards regression model this paper describes the use of the bootstrap 1 2 a new computer based statistical methodology to help validate a regression model resulting from the fitting of cox s proportional hazards model3 to a set of censored survival data as an example we define a prognostic model for outcome in childhood acute lymphocytic leukemia with the cox model and use of a training set of 224 patients to validate the accuracy of the model we use a bootstrap resampling technique to mimic the population under study in two stages first we select the important prognostic factors via a stepwise regression procedure with 100 bootstrap samples secondly we estimate the corresponding regression parameters for these important factors with 400 bootstrap samples the bootstrap result suggests that the model constructed from the training set is reasonable

an optimal selection of regression variables summary an asymptotically optimal selection of regression variables is proposed the key assumption is that the number of control variables is infinite or increases with the sample size it is also shown that mallows s qp akaike s fpe and aic methods are all asymptotically equivalent to this method

the horseshoe estimator for sparse signals this paper proposes a new approach to sparsity called the horseshoe estimator the horseshoe is a close cousin of other widely used bayes rules arising from for example double exponential and cauchy priors in that it is a member of the same family of multivariate scale mixtures of normals but the horseshoe enjoys a number of advantages over existing approaches including its robustness its adaptivity to dierent sparsity patterns and its analytical tractability we prove two theorems that formally characterize both the horseshoe s adeptness at large outlying signals and its super ecient rate of convergence to the correct estimate of the sampling density in sparse situations finally using a combination of real and simulated data we show that the horseshoe estimator corresponds quite closely to the answers one would get by pursuing a full bayesian model averaging approach using a discrete mixture prior to model signals and noise

more comments on c p we discuss the interpretation of c p plots and show how they can be calibrated in several ways we comment on the practice of using the display as a basis for formal selection of a subset regression model and extend the range of application of the device to encompass arbitrary linear estimates of the regression coefficients for example ridge estimates

penalized partial likelihood regression for right censored data with bootstrap selection of the penalty parameter le modele de cox a risque proportionnel est souvent utilise pour estimer l association entre des covariables et un delai de survenue d un evenement qui peut etre censure les estimateurs bases sur la vraisemblance partielle sont utilises pour l estimation et la prediction du risque relatif de l evenement cependant ces estimateurs sont instables et ont une grande variance en cas de colinearite entre les covariables ou quand le nombre d evenements n est pas nettement plus grand que le nombre de covariables une vraisemblance partielle penalisee est proposee pour donner des estimateurs du risque relatif plus precis nous montrons qu asymptotiquement il existe toujours un parametre de penalite pour la vraisemblance partielle penalisee qui reduit le carre moyen de l erreur d estimation du logarithme du risque relatif nous proposons une methode de reechantillonnage pour choisir le parametre de penalite des simulations et un exemple montrent que des estimateurs bases sur la vraisemblance partielle penalisee et la methode du bootstrap peuvent dans certaines circonstances avoir un biais plus petit que celui des estimateurs bases sur la vraisemblance partielle ainsi qu un carre moyeu plus petit de l erreur d estimation et de prediction du logarithme du risque relatif nous illustrons ces methodes avec des donnees d une etude sur le myelome multiple du eastern cooperative oncology group ecog