dimension reduction and coefficient estimation in multivariate linear regression summary we introduce a general formulation for dimension reduction and coefficient estimation in the multivariate linear model we argue that many of the existing methods that are commonly used in practice can be formulated in this framework and have various restrictions we continue to propose a new method that is more flexible and more generally applicable the method proposed can be formulated as a novel penalized least squares estimate the penalty that we employ is the coefficient matrix s ky fan norm such a penalty encourages the sparsity among singular values and at the same time gives shrinkage coefficient estimates and thus conducts dimension reduction and coefficient estimation simultaneously in the multivariate linear model we also propose a generalized cross validation type of criterion for the selection of the tuning parameter in the penalized least squares simulations and an application in financial econometrics demonstrate competitive performance of the new method an extension to the non parametric factor model is also discussed

dynamics of the fractional oscillator abstract the integral equation of motion of a simple harmonic oscillator is generalized by taking the integral to be of arbitrary order according to the methods of fractional calculus to yield the equation of motion of a fractional oscillator the solution is obtained in terms of mittag leffler functions using laplace transforms the expressions for the generalized momentum and the total energy of the fractional oscillator are also obtained numerical application and the phase plane representation of the dynamics are discussed

analysis of fractional order bonhoeffer van der pol oscillator abstract we investigate a bonhoeffer van der pol dynamical system with fractional derivatives of different orders spectral analysis is fulfilled analytically for certain relationships between derivative orders and numerically for any relation between them it is shown that such a system could be more unstable than the system with integer derivatives even for fractional order indices less than one different types of oscillations appear as a result of this instability computer simulation of the typical oscillations demonstrating the observed effects are performed

gibbs posterior for variable selection in high dimensional classification and data mining in the popular approach of bayesian variable selection bvs one uses prior and posterior distributions to select a subset of candidate variables to enter the model a completely new direction will be considered here to study bvs with a gibbs posterior originating in statistical mechanics the gibbs posterior is constructed from a risk function of practical interest such as the classification error and aims at minimizing a risk function without modeling the data probabilistically this can improve the performance over the usual bayesian approach which depends on a probability model which may be misspecified conditions will be provided to achieve good risk performance even in the presence of high dimensionality when the number of candidate variables k can be much larger than the sample size n in addition we develop a convenient markov chain monte carlo algorithm to implement bvs with the gibbs posterior

controllability of fractional damped dynamical systems in this paper we study the controllability of linear and nonlinear fractional damped dynamical systems which involve fractional caputo derivatives with different order in finite dimensional spaces using the mittag leffler matrix function and the iterative technique a numerical example is provided to illustrate the theory

the choice of variables in multivariate regression a non conjugate bayesian decision theory approach we consider the choice of explanatory variables in multivariate linear regression our approach balances prediction accuracy against costs attached to variables in a multivariate version of a decision theory approach pioneered by lindley 1968 we also employ a non conjugate proper prior distribution for the parameters of the regression model extending the standard normal inverse wishart by adding a component of error which is unexplainable by any number of predictor variables thus avoiding the determinism identified by dawid 1988 simulated annealing and fast updating algorithms are used to search for good subsets when there are very many regressors the technique is illustrated on a near infrared spectroscopy example involving 39 observations and 300 explanatory variables this demonstrates the effectiveness of multivariate regression as opposed to separate univariate regressions it also emphasises that within a bayesian framework more variables than observations can be utilised

prediction via orthogonalized model mixing abstract we introduce an approach and algorithms for model mixing in large prediction problems with correlated predictors we focus on the choice of predictors in linear models and mix over possible subsets of candidate predictors our approach is based on expressing the space of models in terms of an orthogonalization of the design matrix advantages are both statistical and computational statistically orthogonalization often leads to a reduction in the number of competing models by eliminating correlations computationally large model spaces cannot be enumerated recent approaches are based on sampling models with high posterior probability via markov chains based on orthogonalization of the space of candidate predictors we can approximate the posterior probabilities of models by products of predictor specific terms this leads to an importance sampling function for sampling directly from the joint distribution over the model space without resorting to markov chains compared to the latter ortho

bayes model averaging with selection of regressors when a number of distinct models contend for use in prediction the choice of a single model can offer rather unstable predictions in regression stochastic search variable selection with bayesian model averaging offers a cure for this robustness issue but at the expense of requiring very many predictors here we look at bayes model averaging incorporating variable selection for prediction this offers similar mean square errors of prediction but with a vastly reduced predictor space this can greatly aid the interpretation of the model it also reduces the cost if measured variables have costs the development here uses decision theory in the context of the multivariate general linear model in passing this reduced predictor space bayes model averaging is contrasted with single model approximations a fast algorithm for updating regressions in the markov chain monte carlo searches for posterior inference is developed allowing many more variables than observations to be contemplated we discuss the merits of absolute rather than proportionate shrinkage in regression especially when there are more variables than observations the methodology is illustrated on a set of spectroscopic data used for measuring the amounts of different sugars in an aqueous solution

optimal bayesian design by inhomogeneous markov chain simulation we consider decision problems defined by a utility function and an underlying probability model for all unknowns the utility function quantifies the decision maker s preferences over consequences the optimal decision maximizes the expected utility function where the expectation is taken with respect to all unknowns that is future data and parameters in many problems the solution is not analytically tractable for example the utility function might involve moments that can be computed only by numerical integration or simulation also the nature of the decision space i e the set of all possible actions might have a shape or dimension that complicates the maximization the motivating application for this discussion is the choice of a monitoring network when the optimization is performed over the high dimensional set of all possible locations of monitoring stations possibly including choice of the number of locations we propose an approach to optimal bayesian design based on inhomogeneous markov

fractional oscillator driven by a gaussian noise abstract the time dependent response is studied of a fractional oscillator driven by a gaussian white noise applying the method of characteristic functionals combined with the technique of integral transforms we derive explicit formulas for the variances of position and velocity and the asymptotic power spectrum of the oscillator the effects of the order of fractional derivative on statistical characteristics of the oscillator are analyzed numerically

bayesian variable selection in linear regression abstract this article is concerned with the selection of subsets of predictor variables in a linear regression model for the prediction of a dependent variable it is based on a bayesian approach intended to be as objective as possible a probability distribution is first assigned to the dependent variable through the specification of a family of prior distributions for the unknown parameters in the regression model the method is not fully bayesian however because the ultimate choice of prior distribution from this family is affected by the data it is assumed that the predictors represent distinct observables the corresponding regression coefficients are assigned independent prior distributions for each regression coefficient subject to deletion from the model the prior distribution is a mixture of a point mass at 0 and a diffuse uniform distribution elsewhere that is a spike and slab distribution the random error component is assigned a normal distribution with mean 0 and standard deviation

coefficient sign prediction methods for model selection summary we consider a bayesian model selection strategy based on predicting the signs of the coefficients in a regression model i e we consider identification of coefficients in a full or encompassing model for which we can confidently predict whether they are positive or negative this is useful when our main purpose in doing model selection is interpretation since the sign of a coefficient is often of primary importance for this task in the case of a linear model with standard non informative prior we connect our sign coefficient prediction approach to the classical zheng loh procedure for model selection one advantage of our approach is that only specification of a prior on the full model is required unlike standard bayesian variable selection approaches which require specification of prior distributions on parameters in all submodels and specification of a prior on the model itself we consider applying our method with proper hierarchical shrinkage priors which makes the procedure more useful in large p small n regression problems with more predictors than observations and in problems involving multicollinearity in these problems we may wish to do prediction by using shrinkage methods in the full model but interpreting which variables are important is also of interest we compare selection by using our coefficient sign prediction approach with the recently proposed elastic net procedure of zou and hastie and observe that our method shares some of the features of the elastic net such as a group selection property the method can be extended to more complex model selection problems such as selection on variance components in random effects models for selection on variance components where the parameter of interest is non negative and hence prediction of the sign of the parameter not the appropriate way to proceed we consider instead prediction of the sign of the score component for the parameter at zero obtaining a method that is related to classical score tests on variance components

expansion and the fractional oscillator abstract we use the e expansion method to obtain expressions for the position momenta and the total energy of a fractional oscillator we prove that the total energy is a decreasing function of time we also compute the intrinsic damping force

twist of fractional oscillations abstract using the method of the laplace transform we consider fractional oscillations they are obtained by the time clock randomization of ordinary harmonic vibrations in contrast to sine and cosine the functions describing the fractional oscillations exhibit a finite number of damped oscillations with an algebraic decay their fractional differential equation is derived

bayesian model averaging for linear regression models abstract we consider the problem of accounting for model uncertainty in linear regression models conditioning on a single selected model ignores model uncertainty and thus leads to the underestimation of uncertainty when making inferences about quantities of interest a bayesian solution to this problem involves averaging over all possible models i e combinations of predictors when making inferences about quantities of interest this approach is often not practical in this article we offer two alternative approaches first we describe an ad hoc procedure occam s window which indicates a small set of models over which a model average can be computed second we describe a markov chain monte carlo approach that directly approximates the exact solution in the presence of model uncertainty both of these model averaging procedures provide better predictive performance than any single model that might reasonably have been selected in the extreme case where there are many candidate predictors but

bayesian wavelet regression on curves with application to a spectroscopic calibration problem motivated by calibration problems in near infrared nir spectroscopy we consider the linear regression setting in which the many predictor variables arise from sampling an essentially continuous curve at equally spaced points and there may be multiple predictands we tackle this regression problem by calculating the wavelet transforms of the discretized curves then applying a bayesian variable selection method using mixture priors to the multivariate regression of predictands on wavelet coefficients for prediction purposes we average over a set of likely models applied to a particular problem in nir spectroscopy this approach was able to find subsets of the wavelet coefficients with overall better predictive performance than the more usual approaches in the application the available predictors are measurements of the nir reflectance spectrum of biscuit dough pieces at 256 equally spaced wavelengths the aim is to predict the composition i e the fat flour sugar and water content of the doug

spectral decomposition of nonlinear systems with memory we present an alternative approach to the analysis of nonlinear systems with long term memory that is based on the koopman operator and a levy transformation in time memory effects are considered to be the result of interactions between a system and its surrounding environment the analysis leads to the decomposition of a nonlinear system with memory into modes whose temporal behavior is anomalous and lacks a characteristic scale on average the time evolution of a mode follows a mittag leffler function and the system can be described using the fractional calculus the general theory is demonstrated on the fractional linear harmonic oscillator and the fractional nonlinear logistic equation when analyzing data from an ill defined black box system the spectral decomposition in terms of mittag leffler functions that we propose may uncover inherent memory effects through identification of a small set of dynamically relevant structures that would otherwise be obscured by conventional spectral methods consequently the theoretical concepts we present may be useful for developing more general methods for numerical modeling that are able to determine whether observables of a dynamical system are better represented by memoryless operators or operators with long term memory in time when model details are unknown

dynamics with low level fractionality abstract the notion of fractional dynamics is related to equations of motion with one or a few terms with derivatives of a fractional order this type of equation appears in the description of chaotic dynamics wave propagation in fractal media and field theory for the fractional linear oscillator the physical meaning of the derivative of order 2 is dissipation in systems with many spacially coupled elements oscillators the fractional derivative along the space coordinate corresponds to a long range interaction we discuss a method of constructing a solution using an expansion in n with small and positive integer n the method is applied to the fractional linear and nonlinear oscillators and to fractional ginzburg landau or parabolic equations

nonholonomic constraints with fractional derivatives we consider the fractional generalization of nonholonomic constraints defined by equations with fractional derivatives and provide some examples the corresponding equations of motion are derived using variational principle we prove that fractional constraints can be used to describe the evolution of dynamical systems in which some coordinates and velocities are related to velocities through a power law memory function

the fourier expansion of eisenstein series for 3 the fourier expansions of eisenstein series for gl 3 z are obtained by two methods one analogous to the classical method used by many number theorists including weber in his derivation of the kronecker limit formula the other method is analogous to that used by siegel to obtain fourier expansions of eisenstein series for the siegel modular group the expansions involve matrix argument k bessel functions recently studied by tom bengtson these k bessel functions are natural generalizations of the ordinary k bessel function which arise when considering harmonic analysis on the symmetric space of the general linear group using a certain system of coordinates

some matrix variate distribution theory notational considerations and a bayesian application summary we introduce and justify a convenient notation for certain matrix variate distributions which by its emphasis on the important underlying parameters and the theory on which it is based eases greatly the task of manipulating such distributions important examples include the matrix variate normal t f and beta and the wishart and inverse wishart distributions the theory is applied to compound matrix distributions and to bayesian prediction in the multivariate linear model

discrimination with many variables abstract many statistical methods for discriminant analysis do not adapt well or easily to situations where the number of variables is large possibly even exceeding the number of cases in the training set we explore a variety of methods for providing robust identification of future samples in this situation we develop a range of flexible bayesian methods and primarily a new hierarchical covariance compromise method akin to regularized discriminant analysis although the methods are much more widely applicable the motivating problem was that of discriminating between groups of samples on the basis of their near infrared spectra here the ability of the bayesian methods to take account of continuity of the spectra may be beneficial the spectra may consist of absorbances or reflectances at as many as 1 000 wavelengths and yet there may be only tens or hundreds of training samples in which both sample spectrum and group identity are known such problems arise in the food and pharmaceutical industries

simultaneous variable selection we propose a new method for selecting a common subset of explanatory variables where the aim is to model several response variables the idea is a natural extension of the lasso technique proposed by tibshirani 1996 and is based on the joint residual sum of squares while constraining the parameter estimates to lie within a suitable polyhedral region the properties of the resulting convex programming problem are analyzed for the special case of an orthonormal design for the general case we develop an efficient interior point algorithm the method is illustrated on a dataset with infrared spectrometry measurements on 14 qualitatively different but correlated responses using 770 wavelengths the aim is to select a subset of the wavelengths suitable for use as predictors for as many of the responses as possible

subordination to periodic processes and synchronization abstract we study the subordination to a process that is periodic in the natural time scale and equivalent to a clock with n states the rationale for this investigation is given by a set of many interacting clocks with n states the natural time scale representation corresponds to the dynamics of an individual clock with no interaction with the other clocks of this set we argue that the cooperation among the clocks of this set has the effect of generating a global clock whose times of sojourn in each of its n states are described by a distribution density with an inverse power law form and power index 2 this is equivalent to extending the widely used subordination method from fluctuation dissipation processes to periodic processes thereby raising the question of whether special conditions exist of perfect synchronization signaled by regular oscillations and especially by oscillations with no damping we study first the case of a poisson subordination function we show that in spite of the random nature of the subordination method the procedure has the effect of creating damped oscillations whose damping vanishes in the limiting case of n 1 thereby suggesting a condition of perfect synchronization in this limit the bateman s mathematical arguments h bateman higher transcendental functions vol iii robert k krieger publishing company inc krim fr drive malabar fl copyright 1953 by mcgraw hill book company inc indicate that the condition of perfect synchronization is possible also in the non poisson case with 2 although it may lie beyond the range of computer simulation to make the theoretical predictions accessible to numerical simulation we use a subordination function whose survival probability is a mittag leffler exponential function this method prevents us from directly establishing the macroscopic coherence emerging from 2 which generates a perfect form of 1 f noise however it affords indirect evidence that perfect synchronization signaled by undamped regular oscillations may be produced in this case furthermore we explore a condition characterized by an excellent agreement between theory and numerical simulation where the long time region relaxation with a perfect inverse power law decay emerging from the subordination to ordinary fluctuation dissipation processes is replaced by exponentially damped regular oscillations

multivariate bayesian variable selection and prediction the multivariate regression model is considered with p regressors a latent vector with p binary entries serves to identify one of two types of regression coefficients those close to 0 and those not specializing our general distributional setting to the linear model with gaussian errors and using natural conjugate prior distributions we derive the marginal posterior distribution of the binary latent vector fast algorithms aid its direct computation and in high dimensions these are supplemented by a markov chain monte carlo approach to sampling from the known posterior distribution problems with hundreds of regressor variables become quite feasible we give a simple method of assigning the hyperparameters of the prior distribution the posterior predictive distribution is derived and the approach illustrated on compositional analysis of data involving three sugars with 160 near infra red absorbances as regressors

comparing stochastic optimization methods for variable selection in binary outcome prediction with application to health policy traditional variable selection strategies in generalized linear models glms seek to optimize a measure of predictive accuracy without regard for the cost of data collection when the purpose of such model building is the creation of predictive scales to be used in future studies with constrained budgets the standard approach may not be optimal we propose a bayesian decision theoretic framework for variable selection in binary outcome glms where the budget for data collection is constrained and potential predictors may vary considerably in cost the method is illustrated using data from a large study of quality of hospital care in the u s in the 1980s especially when the number of available predictors p is large it is important to use an appropriate technique for optimization e g in an application presented here where p 83 the space over which we search has 283 1025 elements which is too large to explore using brute force enumeration specifically we investigate simulated annealing sa

methods and criteria for model selection model selection is an important part of any statistical analysis and indeed is central to the pursuit of science in general many authors have examined the question of model selection from both frequentist and bayesian perspectives and many tools for selecting the best model have been suggested in the literature this paper considers the various proposals from a bayesian decision theoretic perspective