stabilized nearest neighbor classifier and its statistical properties abstractthe stability of statistical analysis is an important indicator for reproducibility which is one main principle of the scientific method it entails that similar statistical conclusions can be reached based on independent samples from the same underlying population in this article we introduce a general measure of classification instability cis to quantify the sampling variability of the prediction made by a classification method interestingly the asymptotic cis of any weighted nearest neighbor classifier turns out to be proportional to the euclidean norm of its weight vector based on this concise form we propose a stabilized nearest neighbor snn classifier which distinguishes itself from other nearest neighbor classifiers by taking the stability into consideration in theory we prove that snn attains the minimax optimal convergence rate in risk and a sharp convergence rate in cis the latter rate result is established for general plug in classifiers under a low noise condition ext

new estimation and model selection procedures for semiparametric modeling in longitudinal data analysis semiparametric regression models are very useful for longitudinal data analysis the complexity of semiparametric models and the structure of longitudinal data pose new challenges to parametric inferences and model selection that frequently arise from longitudinal data analysis in this article two new approaches are proposed for estimating the regression coefficients in a semiparametric model the asymptotic normality of the resulting estimators is established an innovative class of variable selection procedures is proposed to select significant variables in the semiparametric models the proposed procedures are distinguished from others in that they simultaneously select significant variables and estimate unknown parameters rates of convergence of the resulting estimators are established with a proper choice of regularization parameters and penalty functions the proposed variable selection procedures are shown to perform as well as an oracle estimator a robust standard error formula is derived usi

estimation and variable selection for generalized additive partial linear models we study generalized additive partial linear models proposing the use of polynomial spline smoothing for estimation of nonparametric functions and deriving quasi likelihood based estimators for the linear parameters we establish asymptotic normality for the estimators of the parametric components the procedure avoids solving large systems of equations as in kernel based procedures and thus results in gains in computational simplicity we further develop a class of variable selection procedures for the linear parameters by employing a nonconcave penalized quasi likelihood which is shown to have an asymptotic oracle property monte carlo simulations and an empirical example are presented for illustration

smooth blockwise iterative thresholding a smooth fixed point estimator based on the likelihood s block gradient the proposed smooth blockwise iterative thresholding estimator sbite is a model selection technique defined as a fixed point reached by iterating a likelihood gradient based thresholding function the smooth james stein thresholding function has two regularization parameters and and a smoothness parameter s it enjoys smoothness like ridge regression and selects variables like lasso focusing on gaussian regression we show that sbite is uniquely defined and that its stein unbiased risk estimate is a smooth function of and for better selection of the two regularization parameters we perform a monte carlo simulation to investigate the predictive and oracle properties of this smooth version of adaptive lasso the motivation is a gravitational wave burst detection problem from several concomitant time series a nonparametric wavelet based estimator is developed to combine information from all captors by block thresholding multiresolution coefficients we study how the smoothness parameter s tem

on the non negative garrotte estimator summary we study the non negative garrotte estimator from three different aspects consistency computation and flexibility we argue that the non negative garrotte is a general procedure that can be used in combination with estimators other than the original least squares estimator as in its original form in particular we consider using the lasso the elastic net and ridge regression along with ordinary least squares as the initial estimate in the non negative garrotte we prove that the non negative garrotte has the nice property that with probability tending to 1 the solution path contains an estimate that correctly identifies the set of important variables and is consistent for the coefficients of the important variables whereas such a property may not be valid for the initial estimators in general we show that the non negative garrotte can turn a consistent estimate into an estimate that is not only consistent in terms of estimation but also in terms of variable selection we also show that the non negative garrotte has a piecewise linear solution path using this fact we propose an efficient algorithm for computing the whole solution path for the non negative garrotte simulations and a real example demonstrate that the non negative garrotte is very effective in improving on the initial estimator in terms of variable selection and estimation accuracy

sequence folding lattice tiling and multidimensional coding folding a sequence b into a multidimensional box is a well known method which is used as a multidimensional coding technique the operation of folding is generalized in a way that the sequence b can be folded into various shapes and not just a box the novel definition of folding is based on a lattice tiling for the given shape s and a direction in the d dimensional integer grid necessary and sufficient conditions that a lattice tiling for s combined with a direction define a folding of a sequence into s are derived the immediate and most impressive applications are some new lower bounds on the number of dots in two dimensional synchronization patterns asymptotically optimal such patterns were known only for rectangular shapes we show asymptotically optimal such patterns for a large family of hexagons this is also generalized for multidimensional synchronization patterns the best known patterns in terms of dots for circles and other polygons are also given the technique and its application for two dimensional synchronization patterns raises some interesting problems in discrete geometry we will also discuss these problems it is also shown how folding can be used to construct multidimensional error correcting codes finally by using the new definition of folding new types of multidimensional pseudo random arrays with various shapes are generated

forecasting with model uncertainty representations and risk reduction we consider forecasting with uncertainty about the choice of predictor variables the researcher wants to select a model estimate the parameters and use the parameter estimates for forecasting we investigate the distributional properties of a number of different schemes for model choice and parameter estimation in sample model selection using the akaike information criterion out ofsample model selection and splitting the data into subsamples for model selection and parameter estimation using a weak predictor local asymptotic scheme we provide a representation result that facilitates comparison of the distributional properties of the procedures and their associated forecast risks we develop a simulation procedure that improves the accuracy of the out of sample and split sample methods uniformly over the local parameter space we also examine how bootstrap aggregation bagging affects the local asymptotic risk of the estimators and their associated forecasts numerically we find that for many values of the local parameter the out of sample and split sample schemes perform poorly if implemented in the conventional way but they perform well if implemented in conjunction with our risk reduction method or bagging

on the kms condition for chiral conformal field theories using the kubo martin schwinger kms condition and exchange algebras we discuss how to compute correlation functions for the conformal exchange field theories in particular we derive the verlinde algebra from modular invariance of the conformal field theories we also show how the verlinde matrices can be related to the kms matrices

heuristics of instability and stabilization in model selection in model selection usually a best predictor is chosen from a collection hat mu cdot s of predictors where hat mu cdot s is the minimum least squares predictor in a collection mathsf u s of predictors here s is a complexity parameter that is the smaller s the lower dimensional smoother the models in mathsf u s if mathsf l is the data used to derive the sequence hat mu cdot s the procedure is called unstable if a small change in mathsf l can cause large changes in hat mu cdot s with a crystal ball one could pick the predictor in hat mu cdot s having minimum prediction error without prescience one uses test sets cross validation and so forth the difference in prediction error between the crystal ball selection and the statistician s choice we call predictive loss for an unstable procedure the predictive loss is large this is shown by some analytics in a simple case and by simulation results in a more complex comparison of four different linear regression methods unstable procedures can be stabilized by perturbing the data getting a new predictor sequence hat mu cdot s and then averaging over many such predictor sequences

variable selection for multivariate failure time data in this paper we proposed a penalised pseudo partial likelihood method for variable selection with multivariate failure time data with a growing number of regression coefficients under certain regularity conditions we show the consistency and asymptotic normality of the penalised likelihood estimators we further demonstrate that for certain penalty functions with proper choices of regularisation parameters the resulting estimator can correctly identify the true model as if it were known in advance based on a simple approximation of the penalty function the proposed method can be easily carried out with the newton raphson algorithm we conduct extensive monte carlo simulation studies to assess the finite sample performance of the proposed procedures we illustrate the proposed method by analysing a dataset from the framingham heart study

prediction based on mean subset shrinkage methods have traditionally been applied in prediction problems in this article we develop a shrinkage method mean subset that forms an average of regression coefficients from individual subsets of the explanatory variables a bayesian approach is taken to derive an expression of how the coefficient vectors from each subset should be weighted it is not computationally feasible to calculate the mean subset coefficient vector for larger problems and thus we suggest an algorithm to find an approximation to the mean subset coefficient vector in a comprehensive monte carlo simulation study it is found that the proposed mean subset method has superior prediction performance than prediction based on the best subset method and in some settings also better than the ridge regression and lasso methods the conclusions drawn from the monte carlo study is corroborated in an example in which prediction is made using spectroscopic data

braid matrices and structure constants for minimal conformal models using the feigin fuchs representation of minimal conformal models in a form introduced recently by one of us the braid group representation matrices describing the analytic continuation properties of conformal blocks are computed in a suitable normalization their matrix elements are shown to essentially factorize into pairs of boltzmann weights of critical rsos models in a certain limit of the spectral parameter these boltzmann weights are related to quantum groupr matrices by the vertex sos transformation we show that the crossing symmetry of the four point function in left right symmetric models follows from a quantum group relation also called crossing symmetry this observation gives a simple way to evaluate the structure constants

analysis of variance of cross validation estimators of the generalization error this paper brings together methods from two different disciplines statistics and machine learning we address the problem of estimating the variance of cross validation cv estimators of the generalization error in particular we approach the problem of variance estimation of the cv estimators of generalization error as a problem in approximating the moments of a statistic the approximation illustrates the role of training and test sets in the performance of the algorithm it provides a unifying approach to evaluation of various methods used in obtaining training and test sets and it takes into account the variability due to different training and test sets for the simple problem of predicting the sample mean and in the case of smooth loss functions we show that the variance of the cv estimator of the generalization error is a function of the moments of the random variables y card sj sj and y card sjc sj c where sj sj are two training sets and sjc sj c are the corresponding test sets we prove that the distribution of y and y is hypergeometric and we compare our estimator with the one proposed by nadeau and bengio 2003 we extend these results in the regression case and the case of absolute error loss and indicate how the methods can be extended to the classification case we illustrate the results through simulation

combining classification trees using mle we propose a probability distribution for an equivalence class of classification trees that is those that ignore the value of the cutpoints but retain tree structure this distribution is parameterized by a central tree structure representing the true model and a precision or concentration coefficient representing the variability around the central tree we use this distribution to model an observed set of classification trees exhibiting variability in tree structure we propose the maximum likelihood estimate of the central tree as the best tree to represent the set this mle retains the interpretability of a single tree model and has excellent generalizability we implement an ascent search for the mle tree structure using a data set of 13 classification trees that predict the presence or absence of cancer based on immune system parameters

bounds for the size of radar arrays etablissement de bornes ameliorees superieure et inferieure pour la taille des tableaux radars formes de matrices rectangulaires a elements 0 et 1

the exchange algebra for zamolodchikov and fateev s parafermionic theories the concept of an exchange algebra has recently been introduced by rehren and schroer 1989 in the context of two dimensional conformal field theories to give an algebraic setting to both the dynamics and the locality requirement labelling the conformal families with two indices and assuming an interpolating scheme for one of the fields it is shown that the braiding matrices for a subset of fields in zamolodchikov s and fateev s 1986 parafermionic theories containing all the order parameters are identical to those of the diagonal minimal models the authors recover the full spectrum of these theories modulo integers from the phase condition of the exchange algebra even though the subset does not include the parafermionic currents

james stein type estimators in large samples with application to the least absolute deviations estimator we explore the extension of james stein type estimators in a direction that enables them to preserve their superiority when the sample size goes to infinity instead of shrinking a base estimator toward a fixed point we shrink it toward a data dependent point we provide an analytic expression for the asymptotic risk and bias of james stein type estimators shrunk toward a data dependent point and prove that they have smaller asymptotic risk than the base estimator shrinking an estimator toward a data dependent point turns out to be equivalent to combining two random variables using the james stein rule we propose a general combination scheme that includes random combination the james stein combination and the usual nonrandom combination as special cases as an example we apply our method to combine the least absolute deviations estimator and the least squares estimator our simulation study indicates that the resulting combination estimators have desirable finite sample properties when errors are dr

bayes model averaging with selection of regressors when a number of distinct models contend for use in prediction the choice of a single model can offer rather unstable predictions in regression stochastic search variable selection with bayesian model averaging offers a cure for this robustness issue but at the expense of requiring very many predictors here we look at bayes model averaging incorporating variable selection for prediction this offers similar mean square errors of prediction but with a vastly reduced predictor space this can greatly aid the interpretation of the model it also reduces the cost if measured variables have costs the development here uses decision theory in the context of the multivariate general linear model in passing this reduced predictor space bayes model averaging is contrasted with single model approximations a fast algorithm for updating regressions in the markov chain monte carlo searches for posterior inference is developed allowing many more variables than observations to be contemplated we discuss the merits of absolute rather than proportionate shrinkage in regression especially when there are more variables than observations the methodology is illustrated on a set of spectroscopic data used for measuring the amounts of different sugars in an aqueous solution

local overfitting control via leverages we present a novel approach to dealing with overfitting in black box models it is based on the leverages of the samples that is on the influence that each observation has on the parameters of the model since overfitting is the consequence of the model specializing on specific data points during training we present a selection method for nonlinear models based on the estimation of leverages and confidence intervals it allows both the selection among various models of equivalent complexities corresponding to different minima of the cost function e g neural nets with the same number of hidden units and the selection among models having different complexities e g neural nets with different numbers of hidden units a complete model selection methodology is derived

multivariate statistical process control using lasso this article develops a new multivariate statistical process control spc methodology based on adapting the lasso variable selection method to the spc problem the lasso method has the sparsity property of being able to select exactly the set of nonzero regression coefficients in multivariate regression modeling which is especially useful in cases where the number of nonzero coefficients is small in multivariate spc applications process mean vectors often shift in a small number of components our primary goals are to detect such a shift as soon as it occurs and to identify the shifted mean components using this connection between the two problems we propose a lasso based multivariate test statistic and then integrate this statistic into the multivariate ewma charting scheme for phase ii multivariate process monitoring we show that this approach balances protection against various shift levels and shift directions and thus provides an effective tool for multivariate spc applications this article

locality of conformal fields in two dimensions exchange algebra on the light cone we discuss the exchange algebra of light cone operators as the fundamental structure of two dimensional conformal quantum field theory it is necessary in order to account for the locality properties of wightman functions of conformal fields we discuss the consistency requirements of this new type of algebra and obtain a classification containing the well known minimal models

inverse random under sampling for class imbalance problem and its application to multi label classification in this paper a novel inverse random under sampling irus method is proposed for the class imbalance problem the main idea is to severely under sample the majority class thus creating a large number of distinct training sets for each training set we then find a decision boundary which separates the minority class from the majority class by combining the multiple designs through fusion we construct a composite boundary between the majority class and the minority class the proposed methodology is applied on 22 uci data sets and experimental results indicate a significant increase in performance when compared with many existing class imbalance learning methods we also present promising results for multi label classification a challenging research problem in many modern applications such as music text and image categorization

methods for causal inference from gene perturbation experiments and validation inferring causal effects from observational and interventional data is a highly desirable but ambitious goal many of the computational and statistical methods are plagued by fundamental identifiability issues instability and unreliable performance especially for large scale systems with many measured variables we present software and provide some validation of a recently developed methodology based on an invariance principle called invariant causal prediction icp the icp method quantifies confidence probabilities for inferring causal structures and thus leads to more reliable and confirmatory statements for causal relations and predictions of external intervention effects we validate the icp method and some other procedures using large scale genome wide gene perturbation experiments in saccharomyces cerevisiae the results suggest that prediction and prioritization of future experimental interventions such as gene deletions can be improved by using our statistical inference techniques

a practical bayesian framework for backpropagation networks a quantitative and practical bayesian framework is described for learning of mappings in feedforward networks the framework makes possible 1 objective comparisons between solutions using alternative network architectures 2 objective stopping rules for network pruning or growing procedures 3 objective choice of magnitude and type of weight decay terms or additive regularizers for penalizing large weights etc 4 a measure of the effective number of well determined parameters in a model 5 quantified estimates of the error bars on network parameters and on network output and 6 objective comparisons with alternative learning and interpolation models such as splines and radial basis functions the bayesian evidence automatically embodies occam s razor penalizing overflexible and overcomplex models the bayesian approach helps detect poor underlying assumptions in learning models for learning models well matched to a problem a good correlation between generalization ability and the bayesian evidence is obtained

perturbation theory around two dimensional critical systems through holomorphic decomposition the authors consider perturbations by relevant interactions of two dimensional conformally invariant rational field theories the meromorphic structure with respect to the scaling dimensions of the perturbing interactions of the correlation functions is made explicit through a successive application of stoke s theorem the resulting decomposition of the amplitudes into holomorphic and antiholomorphic factors yields a representation of the meromorphic structure in terms of the basic data of the rational field theory which are scaling dimensions operator product coefficients and braid matrices they exemplify the general deduction by a concrete calculation to the third order in the coupling constant of the perturbing interaction

the little bootstrap and other methods for dimensionality selection in regression x fixed prediction error abstract when a regression problem contains many predictor variables it is rarely wise to try to fit the data by means of a least squares regression on all of the predictor variables usually a regression equation based on a few variables will be more accurate and certainly simpler there are various methods for picking good subsets of variables and programs that do such procedures are part of every widely used statistical package the most common methods are based on stepwise addition or deletion of variables and on best subsets the latter refers to a search method that given the number of variables to be in the equation say five locates that regression equation based on five variables that has the lowest residual sum of squares among all five variable equations all of these procedures generate a sequence of regression equations the first based on one variable the next based on two variables and so on each member of this sequence is called a submodel and the number of variables in the e

stability of randomized learning algorithms we extend existing theory on stability namely how much changes in the training data influence the estimated models and generalization performance of deterministic learning algorithms to the case of randomized algorithms we give formal definitions of stability for randomized algorithms and prove non asymptotic bounds on the difference between the empirical and expected error as well as the leave one out and expected error of such algorithms that depend on their random stability the setup we develop for this purpose can be also used for generally studying randomized learning algorithms we then use these general results to study the effects of bagging on the stability of a learning method and to prove non asymptotic bounds on the predictive performance of bagging which have not been possible to prove with the existing theory of stability for deterministic learning algorithms

nonconcave penalized likelihood with a diverging number of parameters a class of variable selection procedures for parametric models via nonconcave penalized likelihood was proposed by fan and li to simultaneously estimate parameters and select important variables they demonstrated that this class of procedures has an oracle property when the number of parameters is finite however in most model selection problems the number of parameters should be large and grow with the sample size in this paper some asymptotic properties of the nonconcave penalized likelihood are established for situations in which the number of parameters tends to as the sample size increases under regularity conditions we have established an oracle property and the asymptotic normality of the penalized likelihood estimators furthermore the consistency of the sandwich formula of the covariance matrix is demonstrated nonconcave penalized likelihood ratio statistics are discussed and their asymptotic distributions under the null hypothesis are obtained by imposing some mild conditions on the penalty functions the asymptotic results are augmented by a simulation study and the newly developed methodology is illustrated by an analysis of a court case on the sexual discrimination of salary

joint variable selection for fixed and random effects in linear mixed effects models it is of great practical interest to simultaneously identify the important predictors that correspond to both the fixed and random effects components in a linear mixed effects lme model typical approaches perform selection separately on each of the fixed and random effect components however changing the structure of one set of effects can lead to different choices of variables for the other set of effects we propose simultaneous selection of the fixed and random factors in an lme model using a modified cholesky decomposition our method is based on a penalized joint log likelihood with an adaptive penalty for the selection and estimation of both the fixed and random effects it performs model selection by allowing fixed effects or standard deviations of random effects to be exactly zero a constrained expectation maximization algorithm is then used to obtain the final estimates it is further shown that the proposed penalized estimator enjoys the oracle property in that asymptotically it performs as well as if the true model was known beforehand we demonstrate the performance of our method based on a simulation study and a real data example

combining linear regression models when and how model combining i e mixing methods have been proposed in recent years to deal with uncertainty in model selection even though advantages of model combining over model selection have been demonstrated in simulations and data examples it is still unclear to a large extent when model combining should be preferred in this work first we propose an instability measure to capture the uncertainty of model selection in estimation called perturbation instability in estimation pie based on perturbation of the sample we demonstrate that estimators from model selection can have large pie values and that model combining substantially reduces the instability for such cases second we propose a model combining method adaptive regression by mixing with model screening arms and derive a theoretical property in arms a screening step is taken to narrow down the list of candidate models before combining which not only saves computing time but also can improve estimation accuracy third we compare arms w

new bounds for the sizes of radar arrays improved upper bounds for the size of radar arrays are found by using the window method new general constructions and computer searched examples are presented the radar array problem is generalized to include arrays with sidelobes greater than 1

factor profiled sure independence screening we propose a method of factor profiled sure independence screening for ultrahigh dimensional variable selection the objective of this method is to identify nonzero components consistently from a sparse coefficient vector the new method assumes that the correlation structure of the high dimensional data can be well represented by a set of low dimensional latent factors which can be estimated consistently by eigenvalue eigenvector decomposition the estimated latent factors should then be profiled out from both the response and the predictors such an operation referred to as factor profiling produces uncorrelated predictors therefore sure independence screening can be applied subsequently and the resulting screening result is consistent for model selection a major advantage that standard sure independence screening does not share we refer to the new method as factor profiled sure independence screening numerical studies confirm its outstanding performance copyright 2012 oxford university press

sparsenet coordinate descent with nonconvex penalties we address the problem of sparse selection in linear models a number of nonconvex penalties have been proposed in the literature for this purpose along with a variety of convex relaxation algorithms for finding good solutions in this article we pursue a coordinate descent approach for optimization and study its convergence properties we characterize the properties of penalties suitable for this approach study their corresponding threshold functions and describe a df standardizing reparametrization that assists our pathwise algorithm the mc penalty is ideally suited to this task and we use it to demonstrate the performance of our algorithm certain technical derivations and experiments related to this article are included in the supplementary materials section

model selection in estimating equations summary model selection is a necessary step in many practical regression analyses but for methods based on estimating equations such as the quasi likelihood and generalized estimating equation gee approaches there seem to be few well studied model selection techniques in this article we propose a new model selection criterion that minimizes the expected predictive bias epb of estimating equations a bootstrap smoothed cross validation bcv estimate of epb is presented and its performance is assessed via simulation for overdispersed generalized linear models for illustration the method is applied to a real data set taken from a study of the development of ewe embryos

jacobian conditioning analysis for model validation our aim is to stress the importance of jacobian matrix conditioning for model validation wealso comment on monari and dreyfus 2002 where following rivals and personnaz 2000 it is proposed to discard neural candidates that are likely to overfit and or for which quantities of interest such as confidence intervals cannot be computed accurately in rivals and personnaz 2000 we argued that such models are to be discarded on the basis of the condition number of their jacobian matrix but monari and dreyfus 2002 suggest making the decision on the basis of the computed values of the leverages the diagonal elements of the projection matrix on the range of the jacobian or hat matrix they propose to discard a model if computed leverages are outside some theoretical bounds pretending that it is the symptom of the jacobian rank deficiency we question this proposition because theoretically the hat matrix is defined whatever the rank of the jacobian and because in practice the computed leverages of very ill conditioned networks may respect their theoretical bounds while confidence intervals cannot be estimated accurately enough two facts that have escaped monari and dreyfus s attention wealso recall the most accurate way to estimate the leverages and the properties of these estimations finally we make an additional comment concerning the performance estimation in monari and dreyfus 2002

stability and generalization we define notions of stability for learning algorithms and show how to use these notions to derive generalization error bounds based on the empirical error and the leave one out error the methods we use can be applied in the regression framework as well as in the classification one when the classifier is obtained by thresholding a real valued function we study the stability properties of large classes of learning algorithms such as regularization based algorithms in particular we focus on hilbert space regularization and kullback leibler regularization we demonstrate how to apply the results to svm for regression and classification

variable selection in a log linear birnbaum saunders regression model for high dimensional survival data via the elastic net and stochastic em the birnbaum saunders bs distribution is broadly used to model failure times in reliability and survival analysis in this article we propose a simultaneous parameter estimation and variable selection procedure in a log linear bs regression model for high dimensional survival data to deal with censored survival data we iteratively run a combination of the stochastic em algorithm sem and variable selection procedure to generate pseudo complete data and select variables until convergence treating pseudo complete data as uncensored data via sem makes it possible to incorporate iterative penalized least squares and simplify computation we demonstrate the efficacy of our method using simulated and real datasets

bagging binary and quantile predictors for time series abstract bootstrap aggregating or bagging introduced by breiman 1996a bagging predictors machine learning 24 123 140 has been proved to be effective to improve on unstable forecast theoretical and empirical works using classification regression trees variable selection in linear and non linear regression have shown that bagging can generate substantial prediction gain however most of the existing literature on bagging has been limited to the cross sectional circumstances with symmetric cost functions in this paper we extend the application of bagging to time series settings with asymmetric cost functions particularly for predicting signs and quantiles we use quantile predictions to construct a binary predictor and the majority voted bagging binary prediction we show that bagging may improve the binary prediction in small sample but it does not improve in large sample various bagging forecast combination weights are used such as equal weighted and bayesian model averaging bma weighted combinations for demonstration we present results from monte carlo experiments and from empirical applications using monthly s p500 and nasdaq stock index returns

variable selection in finite mixture of regression models in the applications of finite mixture of regression fmr models often many covariates are used and their contributions to the response variable vary from one component to another of the mixture model this creates a complex variable selection problem existing methods such as the akaike information criterion and the bayes information criterion are computationally expensive as the number of covariates and components in the mixture model increases in this article we introduce a penalized likelihood approach for variable selection in fmr models the new method introduces penalties that depend on the size of the regression coefficients and the mixture structure the new method is shown to be consistent for variable selection a data adaptive method for selecting tuning parameters and an em algorithm for efficient numerical computations are developed simulations show that the method performs very well and requires much less computing power than existing methods the new method is illustrated by analyzin

a systematic construction for radar arrays the radar array problem arises from the need to design frequency hopping sequences with small out of phase autocorrelations it assumes the reflected signals have negligible doppler shifts so the correlations are calculated along the time axis only in this correspondence a systematic construction for radar arrays is provided by means of homogeneous uniform difference matrices a systematic construction for properly centered permutation matrices a special kind of homogeneous uniform difference matrices is also provided which partially solves the open problems posed by zhang and tu

anisotropic noise injection for input variables relevance determination there are two archetypal ways to control the complexity of a flexible regressor subset selection and ridge regression in neural networks jargon they are respectively known as pruning and weight decay these techniques may also be adapted to estimate which features of the input space are relevant for predicting the output variables relevance is given by a binary indicator for subset selection and by a continuous rating for ridge regression this paper shows how to achieve such a rating for a multilayer perceptron trained with noise or jitter noise injection ni is modified in order to penalize heavily irrelevant features the proposed algorithm is attractive as it requires the tuning of a single parameter this parameter controls the complexity of the model effective number of parameters together with the rating of feature relevances effective input space dimension bounds on the effective number of parameters support that the stability of this adaptive scheme is enforced by the constraints applied to the admissible set of relevance indices the good properties of the algorithm are confirmed by satisfactory experimental results on simulated data sets

on the structure of unitary conformal field theory ii representation theoretic approach two dimensional unitary rational conformal field theory is studied from the point of view of the representation theory of chiral algebras chiral algebras are equipped with a family of co multiplications which serve to define tensor product representations chiral vertices arise as clebsch gordan operators from tensor product representations to irreducible subrepresentations of a chiral algebra the algebra of chiral vertices is studied and shown to give rise to representations of the braid groups determined by yang baxter braid matrices chiral fusion is analyzed it is shown that the braid and fusion matrices determine invariants of knots and links connections between the representation theories of chiral algebras and of quantum groups are sketched finally it is shown how the local fields of a conformal field theory can be reconstructed from the chiral vertices of two chiral algebras

braiding matrices modular transformations and topological field theories in 2 1 dimensions relations between 3d topological field theories and rational conformal field theories are discussed in the former framework we can define the generalized verlinde operators using these operators we find modular transformations for conformal blocks of one point functions and two point functions on the torus the result is generalized to higher genus the correctness of our formulae is illustrated by some examples we also emphasize the importance of the fusion algebra

model selection and estimation in the gaussian graphical model summary we propose penalized likelihood methods for estimating the concentration matrix in the gaussian graphical model the methods lead to a sparse and shrinkage estimator of the concentration matrix that is positive definite and thus conduct model selection and estimation simultaneously the implementation of the methods is nontrivial because of the positive definite constraint on the concentration matrix but we show that the computation can be done effectively by taking advantage of the efficient maxdet algorithm developed in convex optimization we propose a bic type criterion for the selection of the tuning parameter in the penalized likelihood methods the connection between our methods and existing methods is illustrated simulations and real examples demonstrate the competitive performance of the new methods

variable selection for partially linear models with measurement errors this article focuses on variable selection for partially linear models when the covariates are measured with additive errors we propose two classes of variable selection procedures penalized least squares and penalized quantile regression using the nonconvex penalized principle the first procedure corrects the bias in the loss function caused by the measurement error by applying the so called correction for attenuation approach whereas the second procedure corrects the bias by using orthogonal regression the sampling properties for the two procedures are investigated the rate of convergence and the asymptotic normality of the resulting estimates are established we further demonstrate that with proper choices of the penalty functions and the regularization parameter the resulting estimates perform asymptotically as well as an oracle procedure as proposed by fan and li choice of smoothing parameters is also discussed finite sample performance of the proposed variable selection procedures is asse

duality and modular invariance in rational conformal field theories we investigate the polynomial equations which should be satisfied by the duality data for a rational conformal field theory we show that by these duality data we can construct some vector spaces which are isomorphic to the spaces of conformal blocks one can construct explicitly the inner product for the former if one deals with a unitary theory these vector spaces endowed with an inner product are the algebraic reminiscences of the hilbert spaces in a chern simons theory as by products we show that the polynomial equations involving the modular transformations for the one point blocks on the torus are not independent we discuss the solution of structure constants for a physical theory making some assumption we obtain a neat solution and this solution in turn implies that the quantum groups of the left sector and of the right sector must be the same although the chiral algebras need not be the same some examples are given finally we discuss the reconstruction of the quantum group in a rational conformal theory

no unbiased estimator of the variance of k fold cross validation most machine learning researchers perform quantitative experiments to estimate generalization error and compare the performance of different algorithms in particular their proposed algorithm in order to be able to draw statistically convincing conclusions it is important to estimate the uncertainty of such estimates this paper studies the very commonly used k fold cross validation estimator of generalization performance the main theorem shows that there exists no universal valid under all distributions unbiased estimator of the variance of k fold cross validation the analysis that accompanies this result is based on the eigen decomposition of the covariance matrix of errors which has only three different eigenvalues corresponding to three degrees of freedom of the matrix and three components of the total variance this analysis helps to better understand the nature of the problem and how it can make naive estimators that don t take into account the error correlations due to the overlap between training and test sets grossly underestimate variance this is confirmed by numerical experiments in which the three components of the variance are compared when the difficulty of the learning problem and the number of folds are varied

the quantum group structure of 2d gravity and minimal models ii the genus zero chiral bootstrap the chiral operator algebra of the quantum group covariant operators of vertex type is completely worked out by making use of the operator approach suggested by the liouville theory where the quantum group symmetry is explicit this completes earlier articles along the same line the relationship between the quantum group invariant of irf type and quantum group covariant of vertex type chiral operator algebras is fully clarified and connected with the transition to the shadow world for quantum group symbols the corresponding 3 j symbol dressing is shown to reduce to the simpler transformation of babelon and one of the authors j l g in a suitable infinite limit defined by analytic continuation the above two types of operators are found to coincide when applied to states with liouville momenta going to in a suitable way the introduction of quantum group covariant operators in the three dimensional picture gives a generalization of the quantum group version of discrete three dimensional gravity that includes tetrahedra associated with 3 j symbols and universalr matrix elements altogether the present work and a previous parallel article gives the concrete realization of moore and seiberg s scheme that describes the chiral operator algebra of two dimensional gravity and minimal models

discrimination with many variables abstract many statistical methods for discriminant analysis do not adapt well or easily to situations where the number of variables is large possibly even exceeding the number of cases in the training set we explore a variety of methods for providing robust identification of future samples in this situation we develop a range of flexible bayesian methods and primarily a new hierarchical covariance compromise method akin to regularized discriminant analysis although the methods are much more widely applicable the motivating problem was that of discriminating between groups of samples on the basis of their near infrared spectra here the ability of the bayesian methods to take account of continuity of the spectra may be beneficial the spectra may consist of absorbances or reflectances at as many as 1 000 wavelengths and yet there may be only tens or hundreds of training samples in which both sample spectrum and group identity are known such problems arise in the food and pharmaceutical industries

ranking a random feature for variable and feature selection we describe a feature selection method that can be applied directly to models that are linear with respect to their parameters and indirectly to others it is independent of the target machine it is closely related to classical statistical hypothesis tests but it is more intuitive hence more suitable for use by engineers who are not statistics experts furthermore some assumptions of classical tests are relaxed the method has been used successfully in a number of applications that are briefly described

parametric or nonparametric a parametricness index for model selection in model selection literature two classes of criteria perform well asymptotically in different situations bayesian information criterion bic as a representative is consistent in selection when the true model is finite dimensional parametric scenario akaike s information criterion aic performs well in an asymptotic efficiency when the true model is infinite dimensional nonparametric scenario but there is little work that addresses if it is possible and how to detect the situation that a specific model selection problem is in in this work we differentiate the two scenarios theoretically under some conditions we develop a measure parametricness index pi to assess whether a model selected by a potentially consistent procedure can be practically treated as the true model which also hints on aic or bic is better suited for the data for the goal of estimating the regression function a consequence is that by switching between aic and bic based on the pi the resulting regression estimator is simultaneously asymptotically efficient for both parametric and nonparametric scenarios in addition we systematically investigate the behaviors of pi in simulation and real data and show its usefulness

unified low rank matrix estimate via penalized matrix least squares approximation low rank matrix estimation arises in a number of statistical and machine learning tasks in particular the coefficient matrix is considered to have a low rank structure in multivariate linear regression and multivariate quantile regression in this paper we propose a method called penalized matrix least squares approximation pmlsa toward a unified yet simple low rank matrix estimate specifically pmlsa can transform many different types of low rank matrix estimation problems into their asymptotically equivalent least squares forms which can be efficiently solved by a popular matrix fast iterative shrinkage thresholding algorithm furthermore we derive analytic degrees of freedom for pmlsa with which a bayesian information criterion bic type criterion is developed to select the tuning parameters the estimated rank based on the bic type criterion is verified to be asymptotically consistent with the true rank under mild conditions extensive experimental studies are performed to confirm our assertion

partially linear structure selection in cox models with varying coefficients summary to explore the nonlinear interactions between covariates and an index variable partially linear proportional hazards models have been proposed for censored survival data however specification of the partially linear structure was usually carried out in an ad hoc manner by first fitting a full varying coefficient model and visually examining the resulting fit to identify the linear part in this article we consider the problem of coefficient estimation and constant coefficient identification based on a double shrinkage approach variable selection is also considered in a coherent estimation framework resulting in a double penalization procedure under the mild assumptions we establish asymptotic properties for the procedure such as consistency sparesistency constansistency and asymptotic normality we evaluate the performance of the proposed method by numerical simulations and demonstrate its application using a breast cancer data set

weighted wilcoxon type smoothly clipped absolute deviation method summary shrinkage type variable selection procedures have recently seen increasing applications in biomedical research however their performance can be adversely influenced by outliers in either the response or the covariate space this article proposes a weighted wilcoxon type smoothly clipped absolute deviation ww scad method which deals with robust variable selection and robust estimation simultaneously the new procedure can be conveniently implemented with the statistical software r we establish that the ww scad correctly identifies the set of zero coefficients with probability approaching one and estimates the nonzero coefficients with the rate n 1 2 moreover with appropriately chosen weights the ww scad is robust with respect to outliers in both the x and y directions the important special case with constant weights yields an oracle type estimator with high efficiency in the presence of heavier tailed random errors the robustness of the ww scad is partly justified by its asymptotic performance under local shrinking contamination we propose a bayesian information criterion type tuning parameter selector for the ww scad the performance of the ww scad is demonstrated via simulations and by an application to a study that investigates the effects of personal characteristics and dietary factors on plasma beta carotene level

some notes on the nonnegative garrote some notes on the use of the nonnegative garrote ng are given in this article the main result is that compared with other penalized least squares methods the ng has a natural selection of penalty function according to an estimator of prediction risk furthermore two natural estimators of the tuning parameter which only involve very simple computations are proposed corresponding to the akaike information criterion aic and bayesian information criterion bic respectively this indicates that to select tuning parameters it may be unnecessary to optimize a model selection criterion repeatedly several reasonable ng estimators with natural tuning parameters are proposed for multicollinearity problems and other problems simulation results and a real data analysis are reported for testing the results obtained previously

high dimensional odes coupled with mixed effects modeling techniques for dynamic gene regulatory network identification gene regulation is a complicated process the interaction of many genes and their products forms an intricate biological network identification of this dynamic network will help us understand the biological processes in a systematic way however the construction of a dynamic network is very challenging for a high dimensional system in this article we propose to use a set of ordinary differential equations ode coupled with dimensional reduction by clustering and mixed effects modeling techniques to model the dynamic gene regulatory network grn the ode models allow us to quantify both positive and negative gene regulation as well as feedback effects of genes in a functional module on the dynamic expression changes of genes in another functional module which results in a directed graph network a five step procedure clustering smoothing regulation identification parameter estimates refining and function enrichment analysis csief is developed to identify the ode based dynamic grn in the prop

tuning parameter selectors for the smoothly clipped absolute deviation method summary the penalized least squares approach with smoothly clipped absolute deviation penalty has been consistently demonstrated to be an attractive regression shrinkage and selection method it not only automatically and consistently selects the important variables but also produces estimators which are as efficient as the oracle estimator however these attractive features depend on appropriate choice of the tuning parameter we show that the commonly used generalized crossvalidation cannot select the tuning parameter satisfactorily with a nonignorable overfitting effect in the resulting model in addition we propose a bic tuning parameter selector which is shown to be able to identify the true model consistently simulation studies are presented to support theoretical findings and an empirical example is given to illustrate its use in the female labor supply data

two dimensional patterns with distinct differences constructions bounds and maximal anticodes a two dimensional 2 d grid with dots is called a configuration with distinct differences if any two lines which connect two dots are distinct either in their length or in their slope these configurations are known to have many applications such as radar sonar physical alignment and time position synchronization rather than restricting dots to lie in a square or rectangle as previously studied we restrict the maximum distance between dots of the configuration the motivation for this is a new application of such configurations to key distribution in wireless sensor networks we consider configurations in the hexagonal grid as well as in the traditional square grid with distances measured both in the euclidean metric and in the manhattan or hexagonal metrics we note that these configurations are confined inside maximal anticodes in the corresponding grid we classify maximal anticodes for each diameter in each grid we present upper bounds on the number of dots in a pattern with distinct differences contained in these maximal anticodes our bounds settle in the negative a question of golomb and taylor on the existence of honeycomb arrays of arbitrarily large size we present constructions and lower bounds on the number of dots in configurations with distinct differences contained in various 2 d shapes such as anticodes by considering periodic configurations with distinct differences in the square grid

golomb rectangles there are a number of communications problems involving two dimensional arrays with constrained correlation functions golomb s two classes of radar arrays and sonar arrays are examined improved bounds optimal cases and computational procedures are reported

regression coefficient and autoregressive order shrinkage and selection via the lasso summary the least absolute shrinkage and selection operator lasso has been widely used in regression shrinkage and selection we extend its application to the regression model with autoregressive errors two types of lasso estimators are carefully studied the first is similar to the traditional lasso estimator with only two tuning parameters one for regression coefficients and the other for autoregression coefficients these tuning parameters can be easily calculated via a data driven method but the resulting lasso estimator may not be fully efficient to overcome this limitation we propose a second lasso estimator which uses different tuning parameters for each coefficient we show that this modified lasso can produce the estimator as efficiently as the oracle moreover we propose an algorithm for tuning parameter estimates to obtain the modified lasso estimator simulation studies demonstrate that the modified estimator is superior to the traditional estimator one empirical example is also presented to illustrate the usefulness of lasso estimators the extension of the lasso to the autoregression with exogenous variables model is briefly discussed

variable selection in semiparametric regression modeling in this paper we are concerned with how to select significant variables in semiparametric modeling variable selection for semiparametric regression models consists of two components model selection for nonparametric components and select significant variables for parametric portion thus it is much more challenging than that for parametric models such as linear models and generalized linear models because traditional variable selection procedures including stepwise regression and the best subset selection require model selection to nonparametric components for each submodel this leads to very heavy computational burden in this paper we propose a class of variable selection procedures for semiparametric regression models using nonconcave penalized likelihood the newly proposed procedures are distinguished from the traditional ones in that they delete insignificant variables and estimate the coefficients of significant variables simultaneously this allows us to establish the sampling properties of the resulting estimate we first establish the rate of convergence of the resulting estimate with proper choices of penalty functions and regularization parameters we then establish the asymptotic normality of the resulting estimate and further demonstrate that the proposed procedures perform as well as an oracle procedure semiparametric generalized likelihood ratio test is proposed to select significant variables in the nonparametric component we investigate the asymptotic behavior of the proposed test and demonstrate its limiting null distribution follows a chi squared distribution which is independent of the nuisance parameters extensive monte carlo simulation studies are conducted to examine the finite sample performance of the proposed variable selection procedures