fungible weights in multiple regression every set of alternate weights i e nonleast squares weights in a multiple regression analysis with three or more predictors is associated with an infinite class of weights all members of a given class can be deemed fungiblebecause they yield identical sse sum of squared errors and r 2 values equations for generating fungible weights are reviewed and an example is given that illustrates how fungible weights can be profitably used to evaluate parameter sensitivity in multiple regression

on multivariate ridge regression summary a multivariate linear regression model with q responses as a linear function of p independent variables is considered with a p x q parameter matrix b the least squares or normal theory maximum likelihood estimate of b is deficient in that it takes no account of the across regression correlations and ignores the stein effect a remedy was offered by brown zidek 1980 in the form of a multivariate ridge estimator a richer class of estimators is obtained here by casting the model in a linear hierarchical framework obtaining the brown zidek multivariate ridge estimates efron morris s estimates of several normal mean vectors and fearn s bayesian estimates of growth curves as special cases the unknown covariance case results in an identifiability problem which can be overcome by a bayesian approach using conjugate priors for the unidentified covariance matrices

ridge regression biased estimation for nonorthogonal problems in multiple regression it is shown that parameter estimates based on minimum residual sum of squares have a high probability of being unsatisfactory if not incorrect if the prediction vectors are not orthogonal proposed is an estimation procedure based on adding small positive quantities to the diagonal of x x introduced is the ridge trace a method for showing in two dimensions the effects of nonorthogonality it is then shown how to augment x x to obtain biased estimates with smaller mean square error

penalized estimating equations les modeles avec penalisation comme l estimateur ridge l estimateur de stein l estimateur en pont et le lasso ont ete proposes pour prendre en compte la collineariteen regression le lasso a ete applique au modele lineaire a la regression logistique au modele des hasards proportionnels de cox et aux reseaux de neurones cet article etudie le modele avec penalisation en pont pour une penalite j j applique aux equations d estimation et applique ce modele avec penalisation aux equations d estimation generalisees gee dans des etudes longitudinales le defaut de vraisemblance jointe dans les gee est surmonte par les equations d estimation penalisees pour lesquelles aucune vraisemblance jointe n est requise on donne les resultats asymptotiques pour l estimateur sons penalite on montre par des simulations et une application que les cee avec penalisation ameliorent potentiellement les resultats de l estimateur gee et presentent les memes proprietes que les modeles lineaires penalises

nonparametric modeling of longitudinal covariance structure in functional mapping of quantitative trait loci estimation of the covariance structure of longitudinal processes is a fundamental prerequisite for the practical deployment of functional mapping designed to study the genetic regulation and network of quantitative variation in dynamic complex traits we present a nonparametric approach for estimating the covariance structure of a quantitative trait measured repeatedly at a series of time points specifically we adopt huang et al s 2006 biometrika 93 85 98 approach of invoking the modified cholesky decomposition and converting the problem into modeling a sequence of regressions of responses a regularized covariance estimator is obtained using a normal penalized likelihood with an l 2 penalty this approach embedded within a mixture likelihood framework leads to enhanced accuracy precision and flexibility of functional mapping while preserving its biological relevance simulation studies are performed to reveal the statistical properties and advantages of the proposed method a real example from a mouse genome project is analyzed to illustrate the utilization of the methodology the new method will provide a useful tool for genome wide scanning for the existence and distribution of quantitative trait loci underlying a dynamic trait important to agriculture biology and health sciences

robust mean squared error estimation of multiple signals in linear systems affected by model and noise uncertainties this paper is a continuation of the work in 11 and 2 on the problem of estimating by a linear estimator n unobservable input vectors undergoing the same linear transformation from noise corrupted observable output vectors whereas in the aforementioned papers only the matrix representing the linear transformation was assumed uncertain here we are concerned with the case in which the second order statistics of the noise vectors i e their covariance matrices are also subjected to uncertainty we seek a robust mean squared error estimator immuned against both sources of uncertainty we show that the optimal robust mean squared error estimator has a special form represented by an elementary block circulant matrix and moreover when the uncertainty sets are ellipsoidal like the problem of finding the optimal estimator matrix can be reduced to solving an explicit semidefinite programming problem whose size is independent of n

kernel methods in machine learning we review machine learning methods employing positive definite kernels these methods formulate learning and estimation problems in a reproducing kernel hilbert space rkhs of functions defined on the data domain expanded in terms of a kernel working in linear spaces of function has the benefit of facilitating the construction and analysis of learning algorithms while at the same time allowing large classes of functions the latter include nonlinear functions as well as functions defined on nonvectorial data we cover a wide range of methods ranging from binary classifiers to sophisticated methods for estimation with structured data

a geometric analysis of when fixed weighting schemes will outperform ordinary least squares many researchers have demonstrated that fixed exogenously chosen weights can be useful alternatives to ordinary least squares ols estimation within the linear model e g dawes am psychol 34 571 582 1979 einhorn hogarth org behav human perform 13 171 192 1975 wainer psychol bull 83 213 217 1976 generalizing the approach of davis stober dana and budescu psychometrika 75 521 541 2010b i present an analytic method to determine when a choice of fixed weights will incur less mean squared error than ols as a function of sample size error variance and model predictability geometrically i solve for the region of population that favors a choice of fixed weights over ols i derive closed form upper and lower bounds on the volume of this region giving tight bounds on the proportion of population favoring a choice of fixed weights i illustrate this methodology with several examples and provide a matlab the mathworks matlab software version 2009b 2010 programming implementation of the major results

sparse kernel regression modeling using combined locally regularized orthogonal least squares and d optimality experimental design the note proposes an efficient nonlinear identification algorithm by combining a locally regularized orthogonal least squares lrols model selection with a d optimality experimental design the proposed algorithm aims to achieve maximized model robustness and sparsity via two effective and complementary approaches the lrols method alone is capable of producing a very parsimonious model with excellent generalization performance the d optimality design criterion further enhances the model efficiency and robustness an added advantage is that the user only needs to specify a weighting for the d optimality cost in the combined model selecting criterion and the entire model construction procedure becomes automatic the value of this weighting does not influence the model selection procedure critically and it can be chosen with ease from a wide range of values

on some bayesian regularization methods for image restoration methods are reviewed for choosing regularized restorations in image processing in particular a method developed by galatsanos and katsaggelos see ibid vol 1 p 322 336 1992 is given a bayesian interpretation and is compared with other bayesian and non bayesian alternatives a small illustrative example is provided and a complement is provided for the discussion of noise variance estimation of galatsanos et al

maximum set estimators with bounded estimation error we consider the linear regression problem of estimating a deterministic parameter vector x from observations y hx w where h is known and w is additive noise we seek an estimator whose estimation error does not exceed a given maximum error for as wide a range of conditions as possible the maximum error is a design choice and is generally lower than the error provided by the well known least squares ls estimator we develop estimators guaranteeing the required error for as large a parameter set as possible and for as large a noise level as possible we discuss methods for finding these estimators and demonstrate that in many cases the proposed estimators outperform the ls estimator

a statistical framework to infer delay and direction of information flow from measurements of complex systems in neuroscience data are typically generated from neural network activity the resulting time series represent measurements from spatially distributed subsystems with complex interactions weakly coupled to a high dimensional global system we present a statistical framework to estimate the direction of information flow and its delay in measurements from systems of this type informed by differential topology gaussian process regression is employed to reconstruct measurements of putative driving systems from measurements of the driven systems these reconstructions serve to estimate the delay of the interaction by means of an analytical criterion developed for this purpose the model accounts for a range of possible sources of uncertainty including temporally evolving intrinsic noise while assuming complex nonlinear dependencies furthermore we show that if information flow is delayed this approach also allows for inference in strong coupling scenarios of systems exhibiting synchronization phenomena the validity of the method is demonstrated with a variety of delay coupled chaotic oscillators in addition we show that these results seamlessly transfer to local field potentials in cat visual cortex

multiclass classifiers based on dimension reduction with generalized lda linear discriminant analysis lda has been widely used for dimension reduction of data sets with multiple classes the lda has been recently extended to various generalized lda methods that are applicable regardless of the relative sizes between the data dimension and the number of data items in this paper we propose several multiclass classifiers based on generalized lda glda algorithms taking advantage of the dimension reducing transformation matrix without requiring additional training or parameter optimization a marginal linear discriminant classifier mldc a bayesian linear discriminant classifier bldc and a one dimensional bldc are introduced for multiclass classification our experimental results illustrate that these classifiers produce higher ten fold cross validation accuracy than knn and centroid based classifiers in the reduced dimensional space obtained from glda

bayes model averaging with selection of regressors when a number of distinct models contend for use in prediction the choice of a single model can offer rather unstable predictions in regression stochastic search variable selection with bayesian model averaging offers a cure for this robustness issue but at the expense of requiring very many predictors here we look at bayes model averaging incorporating variable selection for prediction this offers similar mean square errors of prediction but with a vastly reduced predictor space this can greatly aid the interpretation of the model it also reduces the cost if measured variables have costs the development here uses decision theory in the context of the multivariate general linear model in passing this reduced predictor space bayes model averaging is contrasted with single model approximations a fast algorithm for updating regressions in the markov chain monte carlo searches for posterior inference is developed allowing many more variables than observations to be contemplated we discuss the merits of absolute rather than proportionate shrinkage in regression especially when there are more variables than observations the methodology is illustrated on a set of spectroscopic data used for measuring the amounts of different sugars in an aqueous solution

existence and modulation of traveling waves in particles chains we consider an infinite particle chain whose dynamics are governed by the following system of differential equations where qn t is the displacement of the nth particle at time t along the chain axis and denotes differentiation with respect to time we assume that all particles have unit mass and that the interaction potential v between adjacent particles is a convex c function for this system we prove the existence of c time periodic traveling wave solutions of the form qn t q wt kn where q is a periodic function q z q z 1 the period is normalized to equal 1 and k are respectively the frequency and the wave number is the mean particle spacing and can be chosen to be an arbitrary parameter we present two proofs one based on a variational principle and the other on topological methods in particular degree theory for small amplitude waves based on perturbation techniques we describe the form of the traveling waves and we derive the weakly nonlinear dispersion relation for the fully nonlinear case when the amplitude of the waves is high we use numerical methods to compute the traveling wave solution and the non linear dispersion relation we finally apply whitham s method of averaged lagrangian to derive the modulation equations for the wave parameters k and 1999 john wiley sons inc

model sparsity and brain pattern interpretation of classification models in neuroimaging interest is increasing in applying discriminative multivariate analysis techniques to the analysis of functional neuroimaging data model interpretation is of great importance in the neuroimaging context and is conventionally based on a brain map derived from the classification model in this study we focus on the relative influence of model regularization parameter choices on both the model generalization the reliability of the spatial patterns extracted from the classification model and the ability of the resulting model to identify relevant brain networks defining the underlying neural encoding of the experiment for a support vector machine logistic regression and fisher s discriminant analysis we demonstrate that selection of model regularization parameters has a strong but consistent impact on the generalizability and both the reproducibility and interpretable sparsity of the models for both 2 and 1 regularization importantly we illustrate a trade off between model spatial reproducibility and prediction accuracy we show that known parts of brain networks can be overlooked in pursuing maximization of classification accuracy alone with either 2 and or 1 regularization this supports the view that the quality of spatial patterns extracted from models cannot be assessed purely by focusing on prediction accuracy our results instead suggest that model regularization parameters must be carefully selected so that the model and its visualization enhance our ability to interpret the brain

forecasting using random subspace methods textabstractrandom subspace methods are a novel approach to obtain accurate forecasts in high dimensional regression settings we provide a theoretical justification of the use of random subspace methods and show their usefulness when forecasting monthly macroeconomic variables we focus on two approaches the first is random subset regression where random subsets of predictors are used to construct a forecast second we discuss random projection regression where artificial predictors are formed by randomly weighting the original predictors using recent results from random matrix theory we obtain a tight bound on the mean squared forecast error for both randomized methods we identify settings in which one randomized method results in more precise forecasts than the other and than alternative regularization strategies such as principal component regression partial least squares lasso and ridge regression the predictive accuracy on the high dimensional macroeconomic fred md data set increases substantially when using the randomized methods with random subset regression outperforming any one of the above mentioned competing methods for at least 66 of the series

outlier induced collinearities when an observation in a regression analysis has very large values on two or more predictor variables artificial collinearities can be induced the effects of such collinearities on a regression analysis are not well documented although they can be shown to be similar in many respects to those resulting from approximate linear dependencies among the columns of predictor variable values the purpose of this article is to explore the effects of outlier induced collinearities on the estimation of regression coefficients

shock waves in the dissipative toda lattice we consider the propagation of a shock wave sw in the damped toda lattice the sw is a moving boundary between two semi infinite lattice domains with different densities a steadily moving sw may exist if the damping in the lattice is represented by an inner friction which is a discrete analogue of the second viscosity in hydrodynamics the problem can be considered analytically in the continuum approximation and the analysis produces an explicit relation between the sw s velocity and the densities of the two phases numerical simulations of the lattice equations of motion demonstrate that a stable sw establishes if the initial velocity is directed towards the less dense phase in the opposite case the wave gradually spreads out the numerically found equilibrium velocity of the sw turns out to be in very good agreement with the analytical formula even in a strongly discrete case if the initial velocity is essentially different from the one determined by the densities but has the correct sign the velocity does not alter significantly but instead the sw adjusts itself to the given velocity by sending another sw in the opposite direction

on an additive partial correlation operator and nonparametric estimation of graphical models we introduce an additive partial correlation operator as an extension of partial correlation to the nonlinear setting and use it to develop a new estimator for nonparametric graphical models our graphical models are based on additive conditional independence a statistical relation that captures the spirit of conditional independence without having to resort to high dimensional kernels for its estimation the additive partial correlation operator completely characterizes additive conditional independence and has the additional advantage of putting marginal variation on appropriate scales when evaluating interdependence which leads to more accurate statistical inference we establish the consistency of the proposed estimator through simulation experiments and analysis of the dream4 challenge dataset we demonstrate that our method performs better than existing methods in cases where the gaussian or copula gaussian assumption does not hold and that a more appropriate scaling for our method further enhances its performance

comment ridge regression and regularization of large matrices we view ridge regression through the lens of eigenvalue shrinkage and consider its influence on two modern problems in high dimensional statistical inference covariance estimation and community d

principal component estimation for generalized linear regression summary the generalized linear model nelder wedderburn 1972 has become an elegant and practical option to classical least squares linear model building we consider the specific problem of generalized linear regression utilizing a set of continuous explanatory variables to model an exponential family response it is the objective of this paper to develop and present an asymptotically biased principal component parameter estimation technique as an option to traditional maximum likelihood estimation for generalized linear regression both iterative and one step principal component estimators are developed directly compared and can be particularly useful with the presence of an ill conditioned information matrix the bias variance and mean squared error of principal component estimation will be quantified generalizations for rules of deletion of components will be examined lastly an example employs principal component estimation for poisson response data

recursive green s function method for multi terminal nanostructures we present and review an efficient method to calculate the retarded green s function in multi terminal nanostructures which is needed in order to calculate the conductance through the system and the local particle densities within it the method uses the recursive green s function method after the discretized hamilton matrix has been properly partitioned we show that this method the circular slicing scheme can be modified to accommodate multi terminal systems as well as the traditional two terminal systems furthermore we show that the performance and robustness of the circular slicing scheme is on par with other advanced methods and is well suited for large variety of multi terminal geometries we end by giving an example of how the method can be used to calculate transport in a non trivial multi terminal geometry

algebraic relationships between classical regression and total least squares estimation abstract the total least squares tls technique which is well known in numerical linear algebra and able to compute strongly consistent estimators of the parameters in a linear errors in variables model is compared algebraically with the classical regression estimators using the singular value decomposition and geometric concepts algebraic equivalences and important relationships between the classical regression techniques and tls estimation are established with special reference to problems of collinearity the equivalence between principal component and latent root regression in collinearity problems is proven and the difference between latent root regression and tls estimation is clarified

risk reducing shrinkage estimation for generalized linear models to generalized linear models in a manner retaining the original spirit of shrinkage estimation which is to reduce risk the investigation identifies two classes of simple all purpose prior distributions which supplement such non informative priors as jeffreys s prior with mechanisms for risk reduction one new class of priors is motivated as optimizers of a core component of asymptotic risk the methodology is evaluated in a numerical exploration and application to an existing data set

active learning and basis selection for kernel based linear models a bayesian perspective we develop an active learning algorithm for kernel based linear regression and classification the proposed greedy algorithm employs a minimum entropy criterion derived using a bayesian interpretation of ridge regression we assume access to a matrix bbrn n for which the i j th element is defined by the kernel function k i j bbr with the observed data i bbrd we seek a model m i yi where yi is a real valued response or integer valued label which we do not have access to a priori to achieve this goal a submatrix il ib bbrn m is sought that corresponds to the intersection of n rows and m columns of indexed by the sets il and ib respectively typically m n and n n we have two objectives i determine the m columns of indexed by the set ib that are the most informative for building a linear model m 1 i ib t yi without any knowledge of yi i 1n and ii using active learning sequentially determine which subset of n elements of yi i 1n should be acquired both stopping values ib m and il n are also to be inferred from the data these steps are taken with the goal of minimizing the uncertainty of the model parameters x as measured by the differential entropy of its posterior distribution the parameter vector x bbrm as well as the model bias bbr is then learned from the resulting problem yil il ibx 1 the remaining n n responses labels not included in yil can be inferred by applying x to the remaining n n rows of ib we show experimental results for several regression and classification problems and compare to other active learning methods

least squares ridge regression and goal programming constrained regression alternatives abstract ridge regression has been extensively reviewed in the literature of applied statistics as a method for dealing with problems of collinearity in least squares regressions in this paper ordinary least squares and least squares combined with ridge regressions are examined and compared with ordinary goal programming and goal programming combined with constraints on regressand values that admissible statistical estimates must satisfy the comparisons are effected by means of an example in chemical processing that has been extensively treated in the ridge regression literature

a constrained linear estimator for multiple regression improper linear models see dawes am psychol 34 571 582 1979 such as equal weighting have garnered interest as alternatives to standard regression models we analyze the general circumstances under which these models perform well by recasting a class of improper linear models as proper statistical models with a single predictor we derive the upper bound on the mean squared error of this estimator and demonstrate that it has less variance than ordinary least squares estimates we examine common choices of the weighting vector used in the literature e g single variable heuristics and equal weighting and illustrate their performance in various test cases

two nonparametric methods for identifying the impulse response of linear systems this paper addresses the problem of impulse response identification using nonparametric methods although the techniques developed herein apply to the truncated untruncated and circulant models we focus on the truncated model which is useful in certain applications two methods of impulse response identification will be presented the first is based on the minimization of the c l statistic which is an estimate of the mean square prediction error the second is a bayesian approach for both of these methods we consider the effects of using both the identity matrix and the laplacian matrix as weights on the energy in the impulse response in addition we present a method for estimating the effective length of the impulse response estimating the length is particularly important in the truncated case finally we develop a method for estimating the noise variance at the output which is needed in the approach involving the c l statistic often prior information on the noise variance is not available and a good estimate is crucial to the success of estimating the impulse response with a nonparametric technique

variable selection in a log linear birnbaum saunders regression model for high dimensional survival data via the elastic net and stochastic em the birnbaum saunders bs distribution is broadly used to model failure times in reliability and survival analysis in this article we propose a simultaneous parameter estimation and variable selection procedure in a log linear bs regression model for high dimensional survival data to deal with censored survival data we iteratively run a combination of the stochastic em algorithm sem and variable selection procedure to generate pseudo complete data and select variables until convergence treating pseudo complete data as uncensored data via sem makes it possible to incorporate iterative penalized least squares and simplify computation we demonstrate the efficacy of our method using simulated and real datasets

piecewise linear regularized solution paths we consider the generic regularized optimization problem argmin l y x j efron hastie johnstone and tibshirani ann statist 32 2004 407 499 have shown that for the lasso that is if l is squared error loss and j 1 is the l 1 norm of the optimal coefficient path is piecewise linear that is is piecewise constant we derive a general characterization of the properties of loss l penalty j pairs which give piecewise linear coefficient paths such pairs allow for efficient generation of the full regularized coefficient paths we investigate the nature of efficient path following algorithms which arise we use our results to suggest robust versions of the lasso for regression and classification and to develop new efficient algorithms for existing problems in the literature including mammen and van de geer s locally adaptive regression splines

orthogonalizing em a design based least squares algorithm we introduce an efficient iterative algorithm intended for various least squares problems based on a design of experiments perspective the algorithm called orthogonalizing em oem works for ordinary least squares ols and can be easily extended to penalized least squares the main idea of the procedure is to orthogonalize a design matrix by adding new rows and then solve the original problem by embedding the augmented design in a missing data framework we establish several attractive theoretical properties concerning oem for the ols with a singular regression matrix an oem sequence converges to the moore penrose generalized inverse based least squares estimator for ordinary and penalized least squares with various penalties it converges to a point having grouping coherence for fully aliased regression matrices convergence and the convergence rate of the algorithm are examined finally we demonstrate that oem is highly efficient for large scale least squares and penalized least squares proble

s values conventional context minimal measures of the sturdiness of regression coefficients this paper proposes a context minimal range of alternative regression models that is used to generate a range of alternative estimates a prior distribution is assumed with a zero mean but an ambiguous covariance matrix the choice of the prior covariance matrix is facilitated by transformation to standardized variables which makes the prior expected r2 equal to the sum of the prior variances three different ranges of the prior expected r2 are used to define three different intervals of prior covariance matrices which are used to produce three different sets of s values

grafting fast incremental feature selection by gradient descent in function space we present a novel and flexible approach to the problem of feature selection called grafting rather than considering feature selection as separate from learning grafting treats the selection of suitable features as an integral part of learning a predictor in a regularized learning framework to make this regularized learning process sufficiently fast for large scale problems grafting operates in an incremental iterative fashion gradually building up a feature set while training a predictor model using gradient descent at each iteration a fast gradient based heuristic is used to quickly assess which feature is most likely to improve the existing model that feature is then added to the model and the model is incrementally optimized using gradient descent the algorithm scales linearly with the number of data points and at most quadratically with the number of features grafting can be used with a variety of predictor model classes both linear and non linear and can be used for both classification and regression experiments are reported here on a variant of grafting for classification using both linear and non linear models and using a logistic regression inspired loss function results on a variety of synthetic and real world data sets are presented finally the relationship between grafting stagewise additive modelling and boosting is explored

on dissipationless shock waves in a discrete nonlinear schrodinger equation it is shown that the generalized discrete nonlinear schr odinger equation in a small amplitude approximation is reduced to a number of basic nonlinear integrable equations such as the kdv mkdv and kdv 2 equations or to the fifth order kdv equation depending on the values of the parameters in the dispersionless limit these equations lead to the wave breaking phenomenon for general enough initial conditions and after taking into account small dispersion effects result in the formation of dissipationless shock waves the whitham theory of modulations of nonlinear waves is used for an analytical description of such waves numerical simulations are used to obtain diferent types of bright and dark shocks

a hybrid method for the parallel computation of green s functions quantum transport models for nanodevices using the non equilibrium green s function method require the repeated calculation of the block tridiagonal part of the green s and lesser green s function matrices this problem is related to the calculation of the inverse of a sparse matrix because of the large number of times this calculation needs to be performed this is computationally very expensive even on supercomputers the classical approach is based on recurrence formulas which cannot be efficiently parallelized this practically prevents the solution of large problems with hundreds of thousands of atoms we propose new recurrences for a general class of sparse matrices to calculate green s and lesser green s function matrices which extend formulas derived by takahashi and others we show that these recurrences may lead to a dramatically reduced computational cost because they only require computing a small number of entries of the inverse matrix then we propose a parallelization strategy for block tridiagonal matrices which involves a combination of schur complement calculations and cyclic reduction it achieves good scalability even on problems of modest size

a center sliding bayesian binary classifier adopting orthogonal polynomials a center sliding bayesian design adopting orthogonal polynomials for binary pattern classification is studied in this paper essentially a bayesian weight solution is coupled with a center sliding scheme in feature space which provides an easy tuning capability for binary classification the proposed method is compared with several state of the art binary classifiers in terms of their solution forms decision thresholds and decision boundaries based on the center sliding bayesian framework a novel orthogonal polynomial classifier is subsequently developed the orthogonal polynomial classifier is evaluated using two representative orthogonal polynomials for feature mapping our experimental results show promising potential of the orthogonal polynomial classifier since it achieves both desired accuracy and computational efficiency highlightsproposed a center sliding bayesian classifier with deterministic solution proposed to adopt orthogonal polynomials for efficient feature expansion developed an easy tuning mechanism for the classification framework provided an extensive evaluation of the proposed classifier

locating the extrema of fungible regression weights in a multiple regression analysis with three or more predictors every set of alternate weights belongs to an infinite class of fungible weights waller psychometrica in press that yields identical sse sum of squared errors and r 2 values when the r 2 using the alternate weights is a fixed value fungible weights a i that yield the maximum or minimum cosine with an ols weight vector b are called fungible extrema we describe two methods for locating fungible extrema and we report r code r development core team 2007 for one of the methods we then describe a new approach for populating a class of fungible weights that is derived from the geometry of alternate regression weights finally we illustrate how fungible weights can be profitably used to gauge parameter sensitivity in linear models by locating the fungible extrema of a regression model of executive compensation horton guerard commun stat simul comput 14 441 448 1985

improved recursive green s function formalism for quasi one dimensional systems with realistic defects abstract we derive an improved version of the recursive green s function formalism rgf which is a standard tool in the quantum transport theory we consider the case of disordered quasi one dimensional materials where the disorder is applied in form of randomly distributed realistic defects leading to partly periodic hamiltonian matrices the algorithm accelerates the common rgf in the recursive decimation scheme using the iteration steps of the renormalization decimation algorithm this leads to a smaller effective system which is treated using the common forward iteration scheme the computational complexity scales linearly with the number of defects instead of linearly with the total system length for the conventional approach we show that the scaling of the calculation time of the green s function depends on the defect density of a random test system furthermore we discuss the calculation time and the memory requirement of the whole transport formalism applied to defective carbon nanotubes

on the sensitivity of a composite to its weights a method is presented for investigating two aspects of the sensitivity of a linear composite to the values of its weights

covariance regularized regression and classification for high dimensional problems in recent years many methods have been developed for regression in high dimensional settings we propose covariance regularized regression a family of methods that use a shrunken estimate of the inverse covariance matrix of the features in order to achieve superior prediction an estimate of the inverse covariance matrix is obtained by maximizing its log likelihood under a multivariate normal model subject to a constraint on its elements this estimate is then used to estimate coefficients for the regression of the response onto the features we show that ridge regression the lasso and the elastic net are special cases of covariance regularized regression and we demonstrate that certain previously unexplored forms of covariance regularized regression can outperform existing methods in a range of situations the covariance regularized regression framework is extended to generalized linear models and linear discriminant analysis and is used to analyze gene expression data sets with multiple class and survival outcomes

functional multiple set canonical correlation analysis we propose functional multiple set canonical correlation analysis for exploring associations among multiple sets of functions the proposed method includes functional canonical correlation analysis as a special case when only two sets of functions are considered as in classical multiple set canonical correlation analysis computationally the method solves a matrix eigen analysis problem through the adoption of a basis expansion approach to approximating data and weight functions we apply the proposed method to functional magnetic resonance imaging fmri data to identify networks of neural activity that are commonly activated across subjects while carrying out a working memory task

a group bridge approach for variable selection in multiple regression problems when covariates can be naturally grouped it is important to carry out feature selection at the group and within group individual variable levels simultaneously the existing methods including the lasso and group lasso are designed for either variable selection or group selection but not for both we propose a group bridge approach that is capable of simultaneous selection at both the group and within group individual variable levels the proposed approach is a penalized regularization method that uses a specially designed group bridge penalty it has the oracle group selection property in that it can correctly select important groups with probability converging to one in contrast the group lasso and group least angle regression methods in general do not possess such an oracle property in group selection simulation studies indicate that the group bridge has superior performance in group and individual variable selection relative to several existing methods

optimal weighting of a priori statistics in linear estimation theory abstract many scientific and engineering applications require the determination of the state of a dynamical system at some reference time a common solution technique is to perform a taylor series expansion about a nominal solution and compute linear corrections to this nominal by the use of a set of discrete observations of the dynamical system in cases where the observational data set is ill conditioned statistical information on the nominal solution is commonly incorporated into the solution process in order to provide stability in the computation of the linear corrections this statistical information is generally the uncertainty in the nominal solution known as the a priori covariance this a priori information is typically weighted equally with the observational data in conventional minimum variance or maximum likelihood solution methods often the a priori information available for certain types of dynamical systems is inaccurate or generated by inadequate means it has been shown that the accuracy of the estimates is highly dependent on the choice of the a priori covariance inaccurate a priori information can contribute to large errors in the estimates in ill conditioned problems methods developed herein provide for the optimal weighting of the a priori covariance used in the solution of the linear correction to the nominal solution these methods are demonstrated by their application to a space mechanics simulation in which a satellite s orbit must be precisely determined results are compared to those obtained by conventional minimum variance solution techniques in order to demonstrate the possible improvement in the estimate accuracy achievable by the use of such optimal weighting techniques

the soliton birth rate in the forced toda lattice the soliton birth rate in the semi infinite toda lattice is studied the lattice is forced by driving the zeroth particle with a constant velocity into the remainder of the lattice an approximate solution for the soliton birth rate is derived and it is shown to compare quite favorably with the actual birth rate

simultaneous regression shrinkage variable selection and supervised clustering of predictors with oscar variable selection can be challenging particularly in situations with a large number of predictors with possibly high correlations such as gene expression data in this article a new method called the oscar octagonal shrinkage and clustering algorithm for regression is proposed to simultaneously select variables while grouping them into predictive clusters in addition to improving prediction accuracy and interpretation these resulting groups can then be investigated further to discover what contributes to the group having a similar behavior the technique is based on penalized least squares with a geometrically intuitive penalty function that shrinks some coefficients to exactly zero additionally this penalty yields exact equality of some coefficients encouraging correlated predictors that have a similar effect on the response to form predictive clusters represented by a single coefficient the proposed procedure is shown to compare favorably to the existing shrinkage and variable selection techniques in terms of both prediction error and model complexity while yielding the additional grouping information

on a relationship between minimax linear and admissible estimators in linear regression minimax linear estimation with respect to the quadratic risk is considered among the class of linear estimators of under the linear regression model m y x 2i new classes of minimax linear estimators of are derived among certain subsets of linear estimators which are simple from the point of view of minimax estimation the admissible linear estimators of under m are then characterized via these classes of minimax linear estimators and the relationship between the minimax linear and admissible estimators of is examined various properties of linear admissible estimators follow readily from this new characterization

correlation weights in multiple regression a general theory on the use of correlation weights in linear prediction has yet to be proposed in this paper we take initial steps in developing such a theory by describing the conditions under which correlation weights perform well in population regression models using ols weights as a comparison we define cases in which the two weighting systems yield maximally correlated composites and when they yield minimally similar weights we then derive the least squares weights for any set of predictors that yield the largest drop in r 2 the coefficient of determination when switching to correlation weights our findings suggest that two characteristics of a model data combination are especially important in determining the effectiveness of correlation weights 1 the condition number of the predictor correlation matrix r xx and 2 the orientation of the correlation weights to the latent vectors of r xx

optimal block tridiagonalization of matrices for coherent charge transport numerical quantum transport calculations are commonly based on a tight binding formulation a wide class of quantum transport algorithms require the tight binding hamiltonian to be in the form of a block tridiagonal matrix here we develop a matrix reordering algorithm based on graph partitioning techniques that yields the optimal block tridiagonal form for quantum transport the reordered hamiltonian can lead to significant performance gains in transport calculations and allows to apply conventional two terminal algorithms to arbitrarily complex geometries including multi terminal structures the block tridiagonalization algorithm can thus be the foundation for a generic quantum transport code applicable to arbitrary tight binding systems we demonstrate the power of this approach by applying the block tridiagonalization algorithm together with the recursive green s function algorithm to various examples of mesoscopic transport in two dimensional electron gases in semiconductors and graphene

the forced toda lattice an example of an almost integrable system a method for solving forced integrable systems is presented the method requires the knowledge of at least one piece of information about the solution once this is known one may then construct the remainder of the solution in this sense these systems are almost integrable the forced semi infinite toda lattice is used as an example and to illustrate the method

bayesian sparse polynomial chaos expansion for global sensitivity analysis polynomial chaos expansions are frequently used by engineers and modellers for uncertainty and sensitivity analyses of computer models they allow representing the input output relations of computer models usually only a few terms are really relevant in such a representation it is a challenge to infer the best sparse polynomial chaos expansion of a given model input output data set in the present article sparse polynomial chaos expansions are investigated for global sensitivity analysis of computer model responses a new bayesian approach is proposed to perform this task based on the kashyap information criterion for model selection the efficiency of the proposed algorithm is assessed on several benchmarks before applying the algorithm to identify the most relevant inputs of a double diffusive convection model

nonlinear schro dinger solitons in the forced toda lattice abstract i demonstrate that the pulsating oscillations observed by hill and knopoff in the forced toda lattice may be interpreted as nonlinear schrodinger solitons this is accomplished by taking the small amplitude and the continuum limit of the forced toda lattice equation this limit is shown to be the stable form of the nonlinear schrodinger equation however dark solitons can be expected to occur for this form since in this case the nonlinear schrodinger field can be consider to approach a nonzero value at infinity i then demonstrate that one can interpret these pulsating oscillations in terms of these dark nonlinear schrodinger solitons

a penalized method for multivariate concave least squares with application to productivity analysis abstract we propose a penalized method for the least squares estimator of a multivariate concave regression function this estimator is formulated as a quadratic programing qp problem with o n2 constraints where n is the number of observations computing such an estimator is a very time consuming task and the computational burden rises dramatically as the number of observations increases by introducing a quadratic penalty function we reformulate the concave least squares estimator as a qp with only non negativity constraints this reformulation can be adapted for estimating variants of shape restricted least squares i e the monotonic concave convex least squares the experimental results and an empirical study show that the reformulated problem and its dual are solved significantly faster than the original problem the matlab and r codes for implementing the penalized problems are provided in the paper

penalized likelihood in cox regression in a cox regression model instability of the estimated regression coefficients can be reduced by maximizing a penalized partial log likelihood where a penalty function of the regression coefficients is substracted from the partial log likelihood in this paper we choose the optimal weight of the penalty function by maximizing the predictive value of the model as measured by the crossvalidated partial log likelihood our methods are illustrated by a study of ovarian cancer survival and by a study of centre effects in kidney graft survival

the regularized iteratively reweighted mad method for change detection in multi and hyperspectral data this paper describes new extensions to the previously published multivariate alteration detection mad method for change detection in bi temporal multi and hypervariate data such as remote sensing imagery much like boosting methods often applied in data mining work the iteratively reweighted ir mad method in a series of iterations places increasing focus on difficult observations here observations whose change status over time is uncertain the mad method is based on the established technique of canonical correlation analysis for the multivariate data acquired at two points in time and covering the same geographical region we calculate the canonical variates and subtract them from each other these orthogonal differences contain maximum information on joint change in all variables spectral bands the change detected in this fashion is invariant to separate linear affine transformations in the originally measured variables at the two points in time such as 1 changes in gain and offset in the measuring device used to acquire the data 2 data normalization or calibration schemes that are linear affine in the gray values of the original variables or 3 orthogonal or other affine transformations such as principal component pc or maximum autocorrelation factor maf transformations the ir mad method first calculates ordinary canonical and original mad variates in the following iterations we apply different weights to the observations large weights being assigned to observations that show little change i e for which the sum of squared standardized mad variates is small and small weights being assigned to observations for which the sum is large like the original mad method the iterative extension is invariant to linear affine transformations of the original variables to stabilize solutions to the ir mad problem some form of regularization may be needed this is especially useful for work on hyperspectral data this paper describes ordinary two set canonical correlation analysis the mad transformation the iterative extension and three regularization schemes a simple case with real landsat thematic mapper tm data at one point in time and partly constructed data at the other point in time that demonstrates the superiority of the iterative scheme over the original mad method is shown also examples with spot high resolution visible data from an agricultural region in kenya and hyperspectral airborne hymap data from a small rural area in southeastern germany are given the latter case demonstrates the need for regularization

prediction by supervised principal components in regression problems where the number of predictors greatly exceeds the number of observations conventional regression techniques may produce unsatisfactory results we describe a technique called supervised principal components that can be applied to this type of problem supervised principal components is similar to conventional principal components analysis except that it uses a subset of the predictors selected based on their association with the outcome supervised principal components can be applied to regression and generalized regression problems such as survival analysis it compares favorably to other techniques for this type of problem and can also account for the effects of other covariates and help identify which predictor variables are most important we also provide asymptotic consistency results to help support our empirical findings these methods could become important tools for dna microarray data where they may be used to more accurately diagnose and treat cancer

universal weighted mse improvement of the least squares estimator since the seminal work of stein in the 1950s there has been continuing research devoted to improving the total mean squared error mse of the least squares ls estimator in the linear regression model however a drawback of these methods is that although they improve the total mse they do so at the expense of increasing the mse of some of the individual signal components here we consider a framework for developing linear estimators that outperform the ls strategy over bounded norm signals under all weighted mse measures this guarantees for example that both the total mse and the mse of each of the elements will be smaller than that resulting from the ls approach we begin by deriving an easily verifiable condition on a linear method that ensures ls domination for every weighted mse we then suggest a minimax estimator that minimizes the worst case mse over all weighting matrices and bounded norm signals subject to the universal weighted mse domination constraint

a fast algorithm for sparse matrix computations related to inversion we have developed a fast algorithm for computing certain entries of the inverse of a sparse matrix such computations are critical to many applications such as the calculation of non equilibrium green s functions g r and g for nano devices the find fast inverse using nested dissection algorithm is optimal in the big o sense however in practice find suffers from two problems due to the width 2 separators used by its partitioning scheme one problem is the presence of a large constant factor in the computational cost of find the other problem is that the partitioning scheme used by find is incompatible with most existing partitioning methods and libraries for nested dissection which all use width 1 separators our new algorithm resolves these problems by thoroughly decomposing the computation process such that width 1 separators can be used resulting in a significant speedup over find for realistic devices up to twelve fold in simulation the new algorithm also has the added advantage that desired off diagonal entries can be computed for free consequently our algorithm is faster than the current state of the art recursive methods for meshes of any size furthermore the framework used in the analysis of our algorithm is the first attempt to explicitly apply the widely used relationship between mesh nodes and matrix computations to the problem of multiple eliminations with reuse of intermediate results this framework makes our algorithm easier to generalize and also easier to compare against other methods related to elimination trees finally our accuracy analysis shows that the algorithms that require back substitution are subject to significant extra round off errors which become extremely large even for some well conditioned matrices or matrices with only moderately large condition numbers when compared to these back substitution algorithms our algorithm is generally a few orders of magnitude more accurate and our produced round off errors stay at a reasonable level

maximin effects in inhomogeneous large scale data large scale data are often characterized by some degree of inhomogeneity as data are either recorded in different time regimes or taken from multiple sources we look at regression models and the effect of randomly changing coefficients where the change is either smoothly in time or some other dimension or even without any such structure fitting varying coefficient models or mixture models can be appropriate solutions but are computationally very demanding and often return more information than necessary if we just ask for a model estimator that shows good predictive properties for all regimes of the data then we are aiming for a simple linear model that is reliable for all possible subsets of the data we propose the concept of maximin effects and a suitable estimator and look at its prediction accuracy from a theoretical point of view in a mixture model with known or unknown group structure under certain circumstances the estimator can be computed orders of magnitudes faster than standard penalized regression estimators making computations on large scale data feasible empirical examples complement the novel methodology and theory

penalized partial likelihood regression for right censored data with bootstrap selection of the penalty parameter le modele de cox a risque proportionnel est souvent utilise pour estimer l association entre des covariables et un delai de survenue d un evenement qui peut etre censure les estimateurs bases sur la vraisemblance partielle sont utilises pour l estimation et la prediction du risque relatif de l evenement cependant ces estimateurs sont instables et ont une grande variance en cas de colinearite entre les covariables ou quand le nombre d evenements n est pas nettement plus grand que le nombre de covariables une vraisemblance partielle penalisee est proposee pour donner des estimateurs du risque relatif plus precis nous montrons qu asymptotiquement il existe toujours un parametre de penalite pour la vraisemblance partielle penalisee qui reduit le carre moyen de l erreur d estimation du logarithme du risque relatif nous proposons une methode de reechantillonnage pour choisir le parametre de penalite des simulations et un exemple montrent que des estimateurs bases sur la vraisemblance partielle penalisee et la methode du bootstrap peuvent dans certaines circonstances avoir un biais plus petit que celui des estimateurs bases sur la vraisemblance partielle ainsi qu un carre moyeu plus petit de l erreur d estimation et de prediction du logarithme du risque relatif nous illustrons ces methodes avec des donnees d une etude sur le myelome multiple du eastern cooperative oncology group ecog