streamwise feature selection in streamwise feature selection new features are sequentially considered for addition to a predictive model when the space of potential features is large streamwise feature selection offers many advantages over traditional feature selection methods which assume that all features are known in advance features can be generated dynamically focusing the search for new features on promising subspaces and overfitting can be controlled by dynamically adjusting the threshold for adding features to the model in contrast to traditional forward feature selection algorithms such as stepwise regression in which at each step all possible features are evaluated and the best one is selected streamwise feature selection only evaluates each feature once when it is generated we describe information investing and investing two adaptive complexity penalty methods for streamwise feature selection which dynamically adjust the threshold on the error reduction required for adding a new feature these two methods give false discovery rate style guarantees against overfitting they differ from standard penalty methods such as aic bic and ric which always drastically over or under fit in the limit of infinite numbers of non predictive features empirical results show that streamwise regression is competitive with on small data sets and superior to on large data sets much more compute intensive feature selection methods such as stepwise regression and allows feature selection on problems with millions of potential features

ideal spatial adaptation by wavelet shrinkage summary with ideal spatial adaptation an oracle furnishes information about how best to adapt a spatially variable estimator whether piecewise constant piecewise polynomial variable knot spline or variable bandwidth kernel to the unknown function estimation with the aid of an oracle offers dramatic advantages over traditional linear estimation by nonadaptive kernels however it is a priori unclear whether such performance can be obtained by a procedure relying on the data alone we describe a new principle for spatially adaptive estimation selective wavelet reconstruction we show that variable knot spline fits and piecewise polynomial fits when equipped with an oracle to select the knots are not dramatically more powerful than selective wavelet reconstruction with an oracle we develop a practical spatially adaptive method riskshrink which works by shrinkage of empirical wavelet coefficients riskshrink mimics the performance of an oracle for selective wavelet reconstruction as well as it is possible to do so a new inequality in multivariate normal decision theory which we call the oracle inequality shows that attained performance differs from ideal performance by at most a factor of approximately 2 log n where n is the sample size moreover no estimator can give a better guarantee than this within the class of spatially adaptive procedures riskshrink is essentially optimal relying only on the data it comes within a factor log 2 n of the performance of piecewise polynomial and variableknot spline methods equipped with an oracle in contrast it is unknown how or if piecewise polynomial methods could be made to function this well when denied access to an oracle and forced to rely on data alone

flexural damage index equations of a plate a new nondestructive damage evaluation method is introduced the method is able to detect locate and size structural damage in a plate like structure using measured modal parameters of a structure the solution procedure consists of two steps first the method requires constructing a set of flexural damage index equations that represent a rigorous mechanical relationship between damage and curvature of modal flexibility second a pseudo inverse solution to the resulting system of the over determined equation nondestructively evaluates damage in a plate like structure this study confirms that there is a strong linear relationship between the curvature of flexibility of a structure and flexural damage in consequence the proposed method introduces a way to avoid the singularity problem and mode selection problem of the mode shape curvature method and damage index method

adapting to unknown sparsity by controlling the false discovery rate we attempt to recover an n dimensional vector observed in white noise where n is large and the vector is known to be sparse but the degree of sparsity is unknown we consider three different ways of defining sparsity of a vector using the fraction of nonzero terms imposing power law decay bounds on the ordered entries and controlling the lp norm for p small we obtain a procedure which is asymptotically minimax for l r loss simultaneously throughout a range of such sparsity classes the optimal procedure is a data adaptive thresholding scheme driven by control of the false discovery rate fdr fdr control is a relatively recent innovation in simultaneous testing ensuring that at most a certain fraction of the rejected null hypotheses will correspond to false rejections in our treatment the fdr control parameter qn also plays a determining role in asymptotic minimaxity if q lim qn 0 1 2 and also qn log n we get sharp asymptotic minimaxity simultaneously over a wide range of sparse parameter spaces and loss functions on the other hand q lim qn 1 2 1 forces the risk to exceed the minimax risk by a factor growing with q to our knowledge this relation between ideas in simultaneous inference and asymptotic decision theory is new our work provides a new perspective on a class of model selection rules which has been introduced recently by several authors these new rules impose complexity penalization of the form 2 log potential model size actual model size we exhibit a close connection with fdr controlling procedures under stringent control of the false discovery rate

estimation of reciprocal residual flexibility from experimental modal data a technique is presented for estimating the residual flexibility between nonexcited structural degrees of freedom from experimental structural vibration data using this method one can include the residual flexibility estimated from modal measurements in the computation of measured flexibility for experiments with incomplete reciprocity i e when the response and excitation measurement sensors are not fully collocated the method can also be used to estimate the unknown entries in the residual flexibility matrix for experimental component mode synthesis when excitations are not provided at all interface degrees of freedom a general solution is presented that contains an unknown positive semidefinite contribution the general solution satisfies modal orthogonality in the limit that all of the structural degrees of freedom are instrumented and when the positive semidefinite contribution lies in a nullspace defined by the stiffness matrix and the modal flexibility with a limited number of measurements modal orthogonality is shown to be satisfied to the extent that the measured modes are preserved by static condensation a rank deficient solution is presented that allows the residual to be used in the computation of the flexibility matrix without further modeling assumptions numerical and experimental results that demonstrate the application of the method to both flexibility matrix convergence and experimental component mode synthesis are presented

the variable selection problem abstract the problem of variable selection is one of the most pervasive model selection problems in statistical applications often referred to as the problem of subset selection it arises when one wants to model the relationship between a variable of interest and a subset of potential explanatory variables or predictors but there is uncertainty about which subset to use this vignette reviews some of the key developments that have led to the wide variety of approaches for this problem

variational learning of a dirichlet process of generalized dirichlet distributions for simultaneous clustering and feature selection this paper introduces a novel enhancement for unsupervised feature selection based on generalized dirichlet gd mixture models our proposal is based on the extension of the finite mixture model previously developed in 1 to the infinite case via the consideration of dirichlet process mixtures which can be viewed actually as a purely nonparametric model since the number of mixture components can increase as data are introduced the infinite assumption is used to avoid problems related to model selection i e determination of the number of clusters and allows simultaneous separation of data in to similar clusters and selection of relevant features our resulting model is learned within a principled variational bayesian framework that we have developed the experimental results reported for both synthetic data and real world challenging applications involving image categorization automatic semantic annotation and retrieval show the ability of our approach to provide accurate models by distinguishing between relevant and irrelevant features without over or under fitting the data

detection of spring support locations in elastic structures using a gradient based finite element model updating technique a methodology is presented for detection of the location of simple translational spring supports in elastic structures by non destructive and non intrusive means based on measured natural frequencies of the structure a gradient based finite element fe model updating technique has been used for such detection measured modal data along with an initially correlated fe model are used the present study is a new application of an existing fe model updating technique the proposed method involves the detection of spring support locations by updating the position parameters of the support in the fe model through the optimization of an error criterion based on the difference between measured and computed natural frequencies of the structural system the location of the support appears explicitly as an updating parameter in the formulation of the problem three different numerical schemes for computation of the gradient of the eigenvalue derivative with respect to the support location have been evaluated the results and various limitations observed while developing the current method have been presented through numerical examples and a simple experimental example

method for determining minimum order mass and stiffness matrices from modal test data a new method for determining mass and stiffness matrices from modal test data is presented the method builds on the identified modes and the mass normalized mode shapes at the sensor locations and is not limited by either the number of driving points or measurement points and so it is applicable to most general test settings a mixed coordinate basis is defined for the mass and stiffness matrices which is analogous to the craig bampton component mode synthesis method for finite element models the resultant mass and stiffness are of minimal order necessary to span the measured modes and the resulting generalized coordinates provide an objective basis for the test derived matrices to be used as if they are component mode synthesized finite element matrices inclusion of rigid body modes and the relationship of the new method to traditional physical parameter computations based on mobility curves is considered examples of the method as applied to numerical and experimental data are provided

extraction of substructural flexibility from global frequencies and mode shapes a computational procedure for extracting substructure by substructure flexibility properties from global modal parameters is presented the present procedure consists of two key features an element based direct flexibility method which uniquely determines the global flexibility without resorting to case dependent redundancy selections and the projection of cinematically inadmissible modes that are contained in the iterated substructural matrices the direct flexibility method is used as the basis of an inverse problem whose goal is to determine substructural flexibilities given the global flexibility geometrically determined substructural rigid body modes and the local to global assembly operators the resulting procedure given accurate global flexibility extracts the exact element by element substructural flexibilities for determinate structures for indeterminate structures the accuracy depends on the iteration tolerance limits the procedure is illustrated using both simple and complex numerical examples and appears to be effective for structural applications such as damage localization and finite element model reconciliation

needles and straw in haystacks empirical bayes estimates of possibly sparse sequences an empirical bayes approach to the estimation of possibly sparse sequences observed in gaussian white noise is set out and investigated the prior considered is a mixture of an atom of probability at zero and a heavy tailed density y with the mixing weight chosen by marginal maximum likelihood in the hope of adapting between sparse and dense sequences if estimation is then carried out using the posterior median this is a random thresholding procedure other thresholding rules employing the same threshold can also be used probability bounds on the threshold chosen by the marginal maximum likelihood approach lead to overall risk bounds over classes of signal sequences of length n allowing for sparsity of various kinds and degrees the signal classes considered are nearly black sequences where only a proportion is allowed to be nonzero and sequences with normalized p norm bounded by for 0 and 0 1 simulations show excellent performance for appropriately chosen functions y the method is computationally tractable and software is available the extension to a modified thresholding method relevant to the estimation of very sparse sequences is also considered

a note on visualizing response transformations in regression a new graphical method for assessing parametric transformations of the response in linear regression is given we simply regress the response variable y on the predictors find the fitted values and then dynamically plot the transformed response y against those fitted values by varying the transformation parameter until the plot is linear the method can be used also to assess the success of numerical response transformation methods and to discover influential observations modifications using robust estimators can be used as well

parallel maximum likelihood estimator for multiple linear regression models consistency and run time are important questions in performing multiple linear regression models in response we introduce a new parallel maximum likelihood estimator for multiple linear models we first provide an equivalent condition between the method and the generalized least squares estimator we also consider the rank of projections and the eigenvalue we then present consistency when a stable solution exists in this paper we describe several consistency theorems and perform experiments on consistency outlier and scalability finally we fit the proposed method onto bankruptcy data

multivariate statistical process control using lasso this article develops a new multivariate statistical process control spc methodology based on adapting the lasso variable selection method to the spc problem the lasso method has the sparsity property of being able to select exactly the set of nonzero regression coefficients in multivariate regression modeling which is especially useful in cases where the number of nonzero coefficients is small in multivariate spc applications process mean vectors often shift in a small number of components our primary goals are to detect such a shift as soon as it occurs and to identify the shifted mean components using this connection between the two problems we propose a lasso based multivariate test statistic and then integrate this statistic into the multivariate ewma charting scheme for phase ii multivariate process monitoring we show that this approach balances protection against various shift levels and shift directions and thus provides an effective tool for multivariate spc applications this article

inverse groundwater modeling for hydraulic conductivity estimation using bayesian model averaging and variance window 1 this study proposes a bayesian model averaging bma method to address parameter estimation uncertainty arising from nonuniqueness in parameterization methods bma is able to incorporate multiple parameterization methods for prediction through the law of total probability and to obtain an ensemble average of hydraulic conductivity estimates two major issues in applying bma to hydraulic conductivity estimation are discussed the first problem is using occam s window in usual bma applications to measure approximated posterior model probabilities occam s window only accepts models in a very narrow range tending to single out the best method and discard other good methods we propose a variance window to replace occam s window to cope with this problem the second problem is the kashyap information criterion kic in the approximated posterior model probabilities which tends to prefer highly uncertain parameterization methods by considering the fisher information matrix with sufficient amounts of observation data the bayesian information criterion bic is a good approximation and is able to avoid controversial results from using kic this study adopts multiple generalized parameterization gp methods such as the bma models to estimate spatially correlated hydraulic conductivity numerical examples illustrate the issues of using kic and occam s window and show the advantages of using bic and the variance window in bma application finally we apply bma to the hydraulic conductivity estimation of the 1500 foot sand in east baton rouge parish louisiana

controlling variable selection by the addition of pseudovariables we propose a new approach to variable selection designed to control the false selection rate fsr defined as the proportion of uninformative variables included in selected models the method works by adding a known number of pseudovariables to the real dataset running a variable selection procedure and monitoring the proportion of pseudovariables falsely selected information obtained from bootstrap like replications of this process is used to estimate the proportion of falsely selected real variables and to tune the selection procedure to control the fsr

model update using modal contribution to static flexibility error a new technique for the parametric correction of full order analytical stiffness matrices from reduced order modal measurements is presented the proposed algorithm corrects model parameters by minimizing a matrix residual formed at measurement degrees of freedom only the residual is the difference between the measured modal contribution to the static flexibility matrix and the corresponding analytical modal contribution to the static flexibility matrix analytical expressions are developed for the flexibility matrix error residual gradient in terms of modal sensitivities found via nelson s method by utilizing a flexibility based matrix error residual the effect of poorly modeled inertia properties on stiffness parameter estimates is greatly reduced unlike technlques that utilize only static data this technique is suitable for use on unconstrained structures in addition the method avoids the problems of mode selection determining modal correspondence and eigenvector expansion or model reduction numerical simulation results are presented for kabe s model as well as a finite element model of a welded frame structure

quadratic programming feature selection identifying a subset of features that preserves classification accuracy is a problem of growing importance because of the increasing size and dimensionality of real world data sets we propose a new feature selection method named quadratic programming feature selection qpfs that reduces the task to a quadratic optimization problem in order to limit the computational complexity of solving the optimization problem qpfs uses the nystrom method for approximate matrix diagonalization qpfs is thus capable of dealing with very large data sets for which the use of other methods is computationally expensive in experiments with small and medium data sets the qpfs method leads to classification accuracy similar to that of other successful techniques for large data sets qpfs is superior in terms of computational efficiency

model free variable selection computing power has encouraged the modelling of data sets of ever increasing size data mining applications in finance marketing and bioinformatics are obvious examples a limitation of nearly all existing variable selection methods is the need to specify the correct model before selection when the number of predictors is large model formulation and validation can be difficult or even infeasible on the basis of the theory of sufficient dimension reduction we propose a new class of model free variable selection approaches the methods proposed assume no model of any form require no nonparametric smoothing and allow for general predictor effects the efficacy of the methods proposed is demonstrated via simulation and an empirical example is given

bayesian regression modeling with interactions and smooth effects abstract there have been many recent suggestions as to how to build and estimate flexible bayesian regression models using constructs such as trees neural networks and gaussian processes although there is much to commend these methods their implementation and interpretation can be daunting for practitioners this article presents a spline based methodology for flexible bayesian regression that is quite simple in terms of computation and interpretation smooth bivariate interactions are modeled in an economical and apparently novel way and prior distributions that penalize complexity are used predictions can be based on either model selection or model averaging taking computation interpretation and predictive performance into account the method is seen to perform well when applied to simulated and real data

improved heteroscedasticity consistent covariance matrix estimators summary the heteroscedasticity consistent covariance matrix estimator proposed by white 1980 is commonly used in practical applications and is implemented into a number of pieces of statistical software however although consistent it can display substantial bias in small to moderately large samples as shown by monte carlo simulations elsewhere this paper defines modified white estimators which are approximately bias free numerical results show that the modified estimators display much smaller bias than white s estimator in small samples we also show that the bias correction leads to some variance inflation in hypothesis testing based on heteroscedasticity consistent covariance matrix estimators numerical results suggest that tests based on the proposed bias corrected estimators typically display smaller size distortions

sparse estimators and the oracle property or the return of hodges estimator abstract we point out some pitfalls related to the concept of an oracle property as used in fan and li 2001 variable selection via nonconcave penalized likelihood and its oracle properties journal of the american statistical association 96 1348 1360 2002 variable selection for cox s proportional hazards model and frailty model annals of statistics 30 74 99 2004 new estimation and model selection procedures for semiparametric modeling in longitudinal data analysis journal of the american statistical association 99 710 723 which are reminiscent of the well known pitfalls related to hodges estimator the oracle property is often a consequence of sparsity of an estimator we show that any estimator satisfying a sparsity property has maximal risk that converges to the supremum of the loss function in particular the maximal risk diverges to infinity whenever the loss function is unbounded for ease of presentation the result is set in the framework of a linear regression model but generalizes far beyond that setting in a monte carlo study we also assess the extent of the problem in finite samples for the smoothly clipped absolute deviation scad estimator introduced in fan and li 2001 variable selection via nonconcave penalized likelihood and its oracle properties journal of the american statistical association 96 1348 1360 we find that this estimator can perform rather poorly in finite samples and that its worst case performance relative to maximum likelihood deteriorates with increasing sample size when the estimator is tuned to sparsity

inference for a proton accelerator using convolution models proton beams present difficulties in analysis because of the limited data that can be collected the study of such beams must depend on complex computer simulators that incorporate detailed physical equations the statistical problem of interest is to infer the initial state of the beam from the limited data collected as the beam passes through a series of focusing magnets we are thus faced with a classic inverse problem where the computer simulator links the initial state to the observables we propose a new model for the initial distribution that is derived from the discretized process convolution approach this model provides a computationally tractable method for this highly challenging problem taking a bayesian perspective allows better estimation of the uncertainty and propagation of this uncertainty

boosted classification trees and class probability quantile estimation the standard by which binary classifiers are usually judged misclassification error assumes equal costs of misclassifying the two classes or equivalently classifying at the 1 2 quantile of the conditional class probability function p y 1 x boosted classification trees are known to perform quite well for such problems in this article we consider the use of standard off the shelf boosting for two more general problems 1 classification with unequal costs or equivalently classification at quantiles other than 1 2 and 2 estimation of the conditional class probability function p y 1 x we first examine whether the latter problem estimation of p y 1 x can be solved with logitboost and with adaboost when combined with a natural link function the answer is negative both approaches are often ineffective because they overfit p y 1 x even though they perform well as classifiers a major negative point of the present article is the disconnect between class probability estimation and classification next we consider the practice of over under sampling of the two classes we present an algorithm that uses adaboost in conjunction with over under sampling and jittering of the data jous boost this algorithm is simple yet successful and it preserves the advantage of relative protection against overfitting but for arbitrary misclassification costs and equivalently arbitrary quantile boundaries we then use collections of classifiers obtained from a grid of quantiles to form estimators of class probabilities the estimates of the class probabilities compare favorably to those obtained by a variety of methods across both simulated and real data sets

variable selection in data mining building a predictive model for bankruptcy we predict the onset of personal bankruptcy using least squares regression although well publicized only 2 244 bankruptcies occur in our dataset of 2 9 million months of credit card activity we use stepwise selection to find predictors of these from a mix of payment history debt load demographics and their interactions this combination of rare responses and over 67 000 possible predictors leads to a challenging modeling question how does one separate coincidental from useful predictors we show that three modifications turn stepwise regression into an effective methodology for predicting bankruptcy our version of stepwise regression 1 organizes calculations to accommodate interactions 2 exploits modern decision theoretic criteria to choose predictors and 3 conservatively estimates p values to handle sparse data and a binary response omitting any one of these leads to poor performance a final step in our procedure calibrates regression predictions with these modifications stepwise regr

a flexibility based continuum damage identification approach the present work presents a flexibility based continuum damage identification approach here it is considered that the current integrity state of a structure is described by means of a continuum damage model this model enables one to parameterize the stiffness matrix of the finite element fe model of the system such that in an updating process properties such as symmetry and sparsity of the original fe model are naturally preserved the damage identification is accomplished by minimizing the frobenius norm of the difference between the experimental and analytical flexibility matrix only related to the measured degrees of freedom dof of the structure by utilizing an error only related to the measured dof one avoids the need for model expansion or model reduction difficulties commonly present in the model updating process the modal information required for the experimental flexibility matrix calculation are obtained by means of eigensystem realization algorithm era and common based structure identification cbsi the assessment of the present approach has been performed by means of simulations on a beam like structure considering typical shortcomings occurring in real applications such as small number of sensors different levels of signal to noise ratio and limited spectral information it has also been analyzed the use of a regularization term in the error function to be minimized in order to improve the performance of the proposed method

calibration and empirical bayes variable selection for the problem of variable selection for the normal linear model selection criteria such as aic c p bic and ric have fixed dimensionality penalties such criteria are shown to correspond to selection of maximum posterior models under implicit hyperparameter choices for a particular hierarchical bayes formulation based on this calibration we propose empirical bayes selection criteria that use hyperparameter estimates instead of fixed choices for obtaining these estimates both marginal and conditional maximum likelihood methods are considered as opposed to traditional fixed penalty criteria these empirical bayes criteria have dimensionality penalties that depend on the data their performance is seen to approximate adaptively the performance of the best fixed penalty criterion across a variety of orthogonal and nonorthogonal set ups including wavelet regression empirical bayes shrinkage estimators of the selected coefficients are also proposed

mixtures of g priors for bayesian model averaging with economic application this paper examines the issue of variable selection in linear regression modeling where there is a potentially large amount of possible covariates and economic theory offers insufficient guidance on how to select the appropriate subset in this context bayesian model averaging presents a formal bayesian solution to dealing with model uncertainty the main interest here is the effect of the prior on the results such as posterior inclusion probabilities of regressors and predictive performance the authors combine a binomial beta prior on model size with a g prior on the coefficients of each model in addition they assign a hyperprior to g as the choice of g has been found to have a large impact on the results for the prior on g they examine the zellner siow prior and a class of beta shrinkage priors which covers most choices in the recent literature the authors propose a benchmark beta prior inspired by earlier findings with fixed g and show it leads to consistent model selection inference is conducted through a markov chain monte carlo sampler over model space and g the authors examine the performance of the various priors in the context of simulated and real data for the latter they consider two important applications in economics namely cross country growth regression and returns to schooling recommendations for applied users are provided

some comments on cp i study the typical configuration of a cp plot when the number of variables in the regression problem is large and there are many weak effects i show that a particular configuration that is very commonly seen can arise in a simple way i give a formula by means of which the risk incurred by the minimum cp rule can be estimated

benchmark priors for bayesian model averaging in contrast to a posterior analysis given a particular sampling model posterior model probabilities in the context of model uncertainty are typically rather sensitive to the speciflcation of the prior in particular difiuse priors on model speciflc parameters can lead to quite unexpected consequences here we focus on the practically relevant situation where we need to entertain a large number of sampling models and we have or wish to use little or no subjective prior information we aim at providing an automatic or benchmark prior structure that can be used in such cases we focus on the normal linear regression model with uncertainty in the choice of re gressors we propose a partly noninformative prior structure related to a natural conjugate g prior speciflcation where the amount of subjective information requested from the user is limited to the choice of a single scalar hyperparameter g0j the consequences of difierent choices for g0j are examined we investigate theoretical properties such as consistency of the implied bayesian procedure links with classical information criteria are provided in addition we examine the flnite sample implications of several choices of g0j in a simulation study the use of the mc 3 algorithm of madigan and york 1995 combined with e cient coding in fortran makes it feasible to conduct large simulations in addition to posterior criteria we shall also compare the predictive performance of difierent priors a classic exam ple concerning the economics of crime will also be provided and contrasted with results in the literature the main flndings of the paper will lead us to propose a benchmark prior speciflcation in a linear regression context with model uncertainty

the risk inflation criterion for multiple regression a new criterion is proposed for the evaluation of variable selection procedures in multiple regression this criterion which we call the risk inflation is based on an adjustment to the risk essentially the risk inflation is the maximum increase in risk due to selecting rather than knowing the correct predictors a new variable selection procedure is obtained which in the case of orthogonal predictors substantially improves on aic c p and bic and is close to optimal in contrast to aic c p and bic which use dimensionality penalties of 2 2 and log n respectively this new procedure uses a penalty 2 log p where p is the number of available predictors for the case of nonorthogonal predictors bounds for the optimal penalty are obtained

damage estimation on beam like structures using the multi resolution analysis abstract this work proposes a vibration based damage evaluation method that can detect locate and size damage utilizing only a few of the lower mode shapes the proposed method is particularly advantageous for beam like structures with uncertain applied axial load mass density and foundation stiffness based on a small damage assumption a linear relationship between damaged and undamaged curvatures is revealed in the context of elasticity it turns out that the resulting damage index equation inherently suffers from singularities near inflection nodes the transformation of the problem into the multi resolution wavelet domain provides a set of coupled linear equations with the aid of the singular value decomposition technique the solution to the damage index equation is achieved in the wavelet space next the desired physical solution to the damage index equation is reconstructed from the one in the wavelet space the performance of the proposed method is compared with two existing damage detection methods using a set of numerical simulations the proposed method attempts to resolve the mode selection problem the singularity problem the axial force problem and the absolute severity estimation problem all of which remained unsolved by earlier attempts

local asymptotic coding and the minimum description length local asymptotic arguments imply that parameter selection via the minimum description length mdl resembles a traditional hypothesis test a common approximation for mdl estimates the cost of adding a parameter at about 1 2 log n bits for a model fit to n observations while accurate for parameters which are large on a standardized scale this approximation overstates the parameter cost near zero we find that encoding the parameter produces a shorter description length when the corresponding estimator is about two standard errors away from zero as in a traditional statistical hypothesis test

local causal and markov blanket induction for causal discovery and feature selection for classification part i algorithms and empirical evaluation we present an algorithmic framework for learning local causal structure around target variables of interest in the form of direct causes effects and markov blankets applicable to very large data sets with relatively small samples the selected feature sets can be used for causal discovery and classification the framework generalized local learning or gll can be instantiated in numerous ways giving rise to both existing state of the art as well as novel algorithms the resulting algorithms are sound under well defined sufficient conditions in a first set of experiments we evaluate several algorithms derived from this framework in terms of predictivity and feature set parsimony and compare to other local causal discovery methods and to state of the art non causal feature selection methods using real data a second set of experimental evaluations compares the algorithms in terms of ability to induce local causal neighborhoods using simulated and resimulated data and examines the relation of predictivity with causal induction performance our experiments demonstrate consistently with causal feature selection theory that local causal feature selection methods under broad assumptions encompassing appropriate family of distributions types of classifiers and loss functions exhibit strong feature set parsimony high predictivity and local causal interpretability although non causal feature selection methods are often used in practice to shed light on causal relationships we find that they cannot be interpreted causally even when they achieve excellent predictivity therefore we conclude that only local causal techniques should be used when insight into causal structure is sought in a companion paper we examine in depth the behavior of gll algorithms provide extensions and show how local techniques can be used for scalable and accurate global causal graph learning

fast robust model selection in large datasets large datasets are increasingly common in many research fields in particular in the linear regression context it is often the case that a huge number of potential covariates are available to explain a response variable and the first step of a reasonable statistical analysis is to reduce the number of covariates this can be done in a forward selection procedure that includes selecting the variable to enter deciding to retain it or stop the selection and estimating the augmented model least squares plus t tests can be fast but the outcome of a forward selection might be suboptimal when there are outliers in this article we propose a complete algorithm for fast robust model selection including considerations for huge sample sizes because simply replacing the classical statistical criteria with robust ones is not computationally possible we develop simplified robust estimators selection criteria and testing procedures for linear regression the robust estimator is a one step weighted m estimato

markov random field models for high dimensional parameters in simulations of fluid flow in porous media we give an approach for using flow information from a system of wells to characterize hydrologic properties of an aquifer in particular we consider experiments where an impulse of tracer fluid is injected along with the water at the input wells and its concentration is recorded over time at the uptake wells we focus on characterizing the spatially varying permeability field which is a key attribute of the aquifer for determining flow paths and rates for a given flow experiment as is standard for estimation from such flow data we use complicated subsurface flow code that simulates the fluid flow through the aquifer for a particular well configuration and aquifer specification in particular the permeability field over a grid the solution to this ill posed problem requires that some regularity conditions be imposed on the permeability field typically this regularity is accomplished by specifying a stationary gaussian process model for the permeability field here we use an intrinsically stationary

controlling the false discovery rate a practical and powerful approach to multiple testing summary the common approach to the multiplicity problem calls for controlling the familywise error rate fwer this approach though has faults and we point out a few a different approach to problems of multiple significance testing is presented it calls for controlling the expected proportion of falsely rejected hypotheses the false discovery rate this error rate is equivalent to the fwer when all hypotheses are true but is smaller otherwise therefore in problems where the control of the false discovery rate rather than that of the fwer is desired there is potential for a gain in power a simple sequential bonferronitype procedure is proved to control the false discovery rate for independent test statistics and a simulation study shows that the gain in power is substantial the use of the new procedure and the appropriateness of the criterion are illustrated with examples

an optimal selection of regression variables summary an asymptotically optimal selection of regression variables is proposed the key assumption is that the number of control variables is infinite or increases with the sample size it is also shown that mallows s qp akaike s fpe and aic methods are all asymptotically equivalent to this method

indicator and stratification methods for missing explanatory variables in multiple linear regression abstract the statistical literature and folklore contain many methods for handling missing explanatory variable data in multiple linear regression one such approach is to incorporate into the regression model an indicator variable for whether an explanatory variable is observed another approach is to stratify the model based on the range of values for an explanatory variable with a separate stratum for those individuals in which the explanatory variable is missing for a least squares regression analysis using either of these two missing data approaches the exact biases of the estimators for the regression coefficients and the residual variance are derived and reported the complete case analysis in which individuals with any missing data are omitted is also investigated theoretically and is found to be free of bias in many situations though often wasteful of information a numerical evaluation of the bias of two missing indicator methods and the complete case analysis is reported the missing indi

bayesian calibration of computer models we consider prediction and uncertainty analysis for systems which are approximated using complex mathematical models such models implemented as computer codes are often generic in the sense that by a suitable choice of some of the model s input parameters the code can be used to predict the behaviour of the system in a variety of specific applications however in any specific application the values of necessary parameters may be unknown in this case physical observations of the system in the specific context are used to learn about the unknown parameters the process of fitting the model to the observed data by adjusting the parameters is known as calibration calibration is typically effected by ad hoc fitting and after calibration the model is used with the fitted input values to predict the future behaviour of the system we present a bayesian calibration technique which improves on this traditional approach in two respects first the predictions allow for all sources of uncertainty including the remaining uncertainty over the fitted parameters second they attempt to correct for any inadequacy of the model which is revealed by a discrepancy between the observed data and the model predictions from even the best fitting parameter values the method is illustrated by using data from a nuclear radiation release at tomsk and from a more complex simulated nuclear accident exercise