a general framework for updating belief distributions we propose a framework for general bayesian inference we argue that a valid update of a prior belief distribution to a posterior can be made for parameters which are connected to observations through a loss function rather than the traditional likelihood function which is recovered as a special case modern application areas make it increasingly challenging for bayesians to attempt to model the true data generating mechanism for instance when the object of interest is low dimensional such as a mean or median it is cumbersome to have to achieve this via a complete model for the whole data distribution more importantly there are settings where the parameter of interest does not directly index a family of density functions and thus the bayesian approach to learning about such parameters is currently regarded as problematic our framework uses loss functions to connect information in the data to functionals of interest the updating of beliefs then follows from a decision theoretic approach involving cumulative loss functions importantly the procedure coincides with bayesian updating when a true likelihood is known yet provides coherent subjective inference in much more general settings connections to other inference frameworks are highlighted

observer based feedback boundary stabilization of the navier stokes equations abstract this paper aims at designing an observer based feedback law which locally stabilizes the solution to the two dimensional navier stokes equations with mixed boundary conditions we consider a finite number of controls acting on a portion of the boundary through robin boundary conditions and construct a linear luenberger observer based on the point observations of the linearized navier stokes equations the sensor location for the point observations is determined by the response of feedback functional gains we prove that the nonlinear system coupled with the observer through the feedback law is locally exponentially stable numerical experiments based on a taylor hood finite element method are presented to illustrate the design for different reynolds numbers

gibbs posterior for variable selection in high dimensional classification and data mining in the popular approach of bayesian variable selection bvs one uses prior and posterior distributions to select a subset of candidate variables to enter the model a completely new direction will be considered here to study bvs with a gibbs posterior originating in statistical mechanics the gibbs posterior is constructed from a risk function of practical interest such as the classification error and aims at minimizing a risk function without modeling the data probabilistically this can improve the performance over the usual bayesian approach which depends on a probability model which may be misspecified conditions will be provided to achieve good risk performance even in the presence of high dimensionality when the number of candidate variables k can be much larger than the sample size n in addition we develop a convenient markov chain monte carlo algorithm to implement bvs with the gibbs posterior

a new actuator activation policy for performance enhancement of controlled diffusion processes the present work aims at the development of a framework within which a moving actuator activation policy and feedback controller synthesis are integrated for diffusion processes modelled by parabolic partial differential equations it is assumed that the process of interest has either i multiple actuators and the desirable arrangement is to activate only one while keeping the remaining ones dormant or ii a single actuator capable of moving at a priori selected positions within the spatial domain practical advantages associated with arrangement i are energy cost savings and simplification of the local controller synthesis and the overall switching scheme and with ii is enhanced spatiotemporal disturbance compensation feedback controller synthesis methods based on linear matrix inequality techniques are employed for a finite dimensional galerkin approximation of the original distributed parameter system along with standard controllability criteria additional conditions are imposed that ensure robustness with respect to a certain class of disturbances the value of a performance functional is then explicitly calculated by solving a location parameterized family of lyapunov matrix equations and then optimized with respect to the set of admissible actuator locations finally a case study of a moving bed adsorber is presented where the performance enhancing capabilities of the proposed method is evaluated through simulation studies

a smoothed maximum score estimator for the binary response model this paper describes a semiparametric estimator for binary response models in which there may be arbitrary heteroskedasticity of unknown form the estimator is obtained by maximizing a smoothed version of the objective function of c manski s maximum score estimator the smoothing procedure is similar to that used in kernel nonparametric density estimation the resulting estimator s rate of convergence in probability is the fastest possible under the assumptions that are made the centered normalized estimator is asymptotically normally distributed methods are given for consistently estimating the parameters of the limiting distribution and for selecting the bandwidth required by the smoothing procedure copyright 1992 by the econometric society

the choice of variables in multivariate regression a non conjugate bayesian decision theory approach we consider the choice of explanatory variables in multivariate linear regression our approach balances prediction accuracy against costs attached to variables in a multivariate version of a decision theory approach pioneered by lindley 1968 we also employ a non conjugate proper prior distribution for the parameters of the regression model extending the standard normal inverse wishart by adding a component of error which is unexplainable by any number of predictor variables thus avoiding the determinism identified by dawid 1988 simulated annealing and fast updating algorithms are used to search for good subsets when there are very many regressors the technique is illustrated on a near infrared spectroscopy example involving 39 observations and 300 explanatory variables this demonstrates the effectiveness of multivariate regression as opposed to separate univariate regressions it also emphasises that within a bayesian framework more variables than observations can be utilised

efficient agnostic learning of neural networks with bounded fan in we show that the class of two layer neural networks with bounded fan in is efficiently learnable in a realistic extension to the probably approximately correct pac learning model in this model a joint probability distribution is assumed to exist on the observations and the learner is required to approximate the neural network which minimizes the expected quadratic error as special cases the model allows learning real valued functions with bounded noise learning probabilistic concepts and learning the best approximation to a target function that cannot be well approximated by the neural network the networks we consider have real valued inputs and outputs an unlimited number of threshold hidden units with bounded fan in and a bound on the sum of the absolute values of the output weights the number of computation steps of the learning algorithm is bounded by a polynomial in 1 spl epsiv 1 spl delta n and b where spl epsiv is the desired accuracy spl delta is the probability that the algorithm fails n is the input dimension and b is the bound on both the absolute value of the target which may be a random variable and the sum of the absolute values of the output weights in obtaining the result we also extended some results on iterative approximation of functions in the closure of the convex hull of a function class and on the sample complexity of agnostic learning with the quadratic loss function

from ɛ entropy to kl entropy analysis of minimum information complexity density estimation we consider an extension of e entropy to a kl divergence based complexity measure for randomized density estimation methods based on this extension we develop a general information theoretical inequality that measures the statistical complexity of some deterministic and randomized density estimators consequences of the new inequality will be presented in particular we show that this technique can lead to improvements of some classical results concerning the convergence of minimum description length and bayesian posterior distributions moreover we are able to derive clean finite sample convergence bounds that are not obtainable using previous approaches

learning without concentration we obtain sharp bounds on the estimation error of the empirical risk minimization procedure performed in a convex class and with respect to the squared loss without assuming that class members and the target are bounded functions or have rapidly decaying tails rather than resorting to a concentration based argument the method used here relies on a small ball assumption and thus holds for classes consisting of heavy tailed functions and for heavy tailed targets the resulting estimates scale correctly with the noise level of the problem and when applied to the classical bounded scenario always improve the known bounds

on the bayes risk consistency of regularized boosting methods the probability of error of classification methods based on convex combinations of simple base classifiers by boosting algorithms is investigated the main result of the paper is that certain regularized boosting algorithms provide bayes risk consistent classifiers under the sole assumption that the bayes classifier may be approximated by a convex combination of the base classifiers nonasymptotic distribution free bounds are also developed which offer interesting new insight into how boosting works and help explain its success in practical classification problems

nonparametric regression using bayesian variable selection abstract this paper estimates an additive model semiparametrically while automatically selecting the significant independent variables and the appropriate power transformation of the dependent variable the nonlinear variables are modeled as regression splines with significant knots selected from a large number of candidate knots the estimation is made robust by modeling the errors as a mixture of normals a bayesian approach is used to select the significant knots the power transformation and to identify outliers using the gibbs sampler to carry out the computation empirical evidence is given that the sampler works well on both simulated and real examples and that in the univariate case it compares favorably with a kernel weighted local linear smoother the variable selection algorithm in the paper is substantially faster than previous bayesian variable selection algorithms

output feedback stabilization of an unstable wave equation we consider the problem of stabilization of a one dimensional wave equation that contains instability at its free end and control on the opposite end in contrast to classical collocated boundary damper feedbacks for the neutrally stable wave equations with one end satisfying a homogeneous boundary condition the controllers and the associated observers designed in the paper are more complex due to the open loop instability of the plant the controller and observer gains are designed using the method of backstepping which results in explicit formulae for the gain functions we prove exponential stability and the existence and uniqueness of classical solutions for the closed loop system we also derive the explicit compensators in frequency domain the results are illustrated with simulations

misspecification in infinite dimensional bayesian statistics we consider the asymptotic behavior of posterior distributions if the model is misspecified given a prior distribution and a random sample from a distribution p 0 which may not be in the support of the prior we show that the posterior concentrates its mass near the points in the support of the prior that minimize the kullback leibler divergence with respect to p 0 an entropy condition and a prior mass condition determine the rate of convergence the method is applied to several examples with special interest for infinite dimensional models these include gaussian mixtures nonparametric regression and parametric models

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

nonconcave penalized likelihood with a diverging number of parameters a class of variable selection procedures for parametric models via nonconcave penalized likelihood was proposed by fan and li to simultaneously estimate parameters and select important variables they demonstrated that this class of procedures has an oracle property when the number of parameters is finite however in most model selection problems the number of parameters should be large and grow with the sample size in this paper some asymptotic properties of the nonconcave penalized likelihood are established for situations in which the number of parameters tends to as the sample size increases under regularity conditions we have established an oracle property and the asymptotic normality of the penalized likelihood estimators furthermore the consistency of the sandwich formula of the covariance matrix is demonstrated nonconcave penalized likelihood ratio statistics are discussed and their asymptotic distributions under the null hypothesis are obtained by imposing some mild conditions on the penalty functions the asymptotic results are augmented by a simulation study and the newly developed methodology is illustrated by an analysis of a court case on the sexual discrimination of salary

minimum cost input output design for large scale linear structural systems in this paper we provide optimal solutions to two different but related input output design problems involving large scale linear dynamical systems where the cost associated to each directly actuated measured state variable can take different values but is independent of the input output performing the task under these conditions we first aim to determine and characterize the input output placement that incurs in the minimum cost while ensuring that the resulting placement achieves structural controllability observability further we address a constrained variant of the above problem in which we seek to determine the minimum cost placement configuration among all possible input output placement configurations that ensures structural controllability observability with the lowest number of directly actuated measured state variables we develop new graph theoretical characterizations of cost constrained input selections for structural controllability and properties that enable us to address both problems by reduction to a weighted maximum matching problem efficiently addressed by algorithms with polynomial time complexity in the number of state variables finally we illustrate the obtained results with an example

assigning a value to a power likelihood in a general bayesian model summarybayesian robustness under model misspecification is a current area of active research among recent ideas is that of raising the likelihood function to a power in this paper we discuss the choice of appropriate power and provide examples

h optimal actuator location in many control applications there is freedom to place the actuators the actuator location should be chosen to optimize certain performance objectives in this paper h performance with state feedback is considered that is both the controller and the actuator locations are chosen to minimize the effect of disturbances on the output a framework for calculating h optimal actuator locations is developed conditions for well posedness of the h optimal actuator location problem are presented many optimal actuator problems involve systems modelled by partial differential equations and conditions under which approximations yield reliable results are given a derivative free optimization algorithm to calculate h optimal actuator locations is described the results are illustrated using several examples

sensor placement for optimal estimation of vector valued diffusion processes abstract diffusion processes are commonplace in many scientific disciplines as they describe a broad range of physical phenomenon consider a diffusion process observed through linear sensors with additive white noise we derive the optimal placement of these sensors for estimating this process where optimality is defined in terms of the mean squared estimation error mse of the state given past observations we consider two cases first we assume the sensors to be orthogonal we show in this case that the minimum mse is related to the nuclear norm of the system matrix of the process second we remove the orthogonality constraint and show that the mse is related to the schatten p norm of the system matrix of the process and the optimal sensors are proportional its matrix cube root we present simulation results illustrating the fact that the gain afforded by optimizing the choice of sensors depends on the ratio p n where n is the dimension of the system and p the dimension of the wiener processes driving it and this gain is in general large especially when p n is small

linear quadratic optimal actuator location in control of vibrations diffusion and many other problems governed by partial differential equations there is freedom in the choice of actuator location the actuator location should be chosen to optimize performance objectives in this paper we consider linear quadratic performance two types of cost are considered the choice depends on whether the response to the worst initial condition is to be minimized or whether the initial condition is regarded as random in practice approximations are used in controller design and thus in selection of the actuator locations the optimal cost and location of the approximating sequence should converge to the exact optimal cost and location in this work conditions for this convergence are given in the case of linear quadratic control examples are provided to illustrate that convergence may fail when these conditions are not satisfied

weak eigenfunctions of two interval sturm liouville problems together with interaction conditions in this study we consider a new type boundary value problem consisting of a sturm liouville equation on two disjoint intervals together with interaction conditions and with eigenvalue parameter in the boundary conditions we suggest a special technique to reduce the considered problem into an integral equation by the use of which we define a new concept the so called weak eigenfunction for the considered problem then we construct some hilbert spaces and define some self adjoint compact operators in these spaces in such a way that the considered problem can be interpreted as a self adjoint operator pencil equation finally it is shown that the spectrum is discrete and the set of weak eigenfunctions form a riesz basis of the suitable hilbert space

high dimensional classification using features annealed independence rules classification using high dimensional features arises frequently in many contemporary statistical studies such as tumor classification using microarray or other high throughput data the impact of dimensionality on classifications is largely poorly understood in a seminal paper bickel and levina 2004 show that the fisher discriminant performs poorly due to diverging spectra and they propose to use the independence rule to overcome the problem we first demonstrate that even for the independence classification rule classification using all the features can be as bad as the random guessing due to noise accumulation in estimating population centroids in high dimensional feature space in fact we demonstrate further that almost all linear discriminants can perform as bad as the random guessing thus it is paramountly important to select a subset of important features for high dimensional classification resulting in features annealed independence rules fair the conditions under which all the important features can be selected by the two sample t statistic are established the choice of the optimal number of features or equivalently the threshold value of the test statistics are proposed based on an upper bound of the classification error simulation studies and real data analysis support our theoretical results and demonstrate convincingly the advantage of our new classification procedure

best subset selection persistence in high dimensional statistical learning and optimization under l1 constraint let y x 1 x m be a random vector it is desired to predict y based on x 1 x m examples of prediction methods are regression classification using logistic regression or separating hyperplanes and so on we consider the problem of best subset selection and study it in the context m n a 1 where n is the number of observations we investigate procedures that are based on empirical risk minimization it is shown that in common cases we should aim to find the best subset among those of size which is of order o n log n it is also shown that in some asymptotic sense when assuming a certain sparsity condition there is no loss in letting m be much larger than n for example m n a 1 this is in comparison to starting with the best subset of size smaller than n and regardless of the value of a we then study conditions under which empirical risk minimization subject to l 1 constraint yields nearly the best subset these results extend some recent results obtained by greenshtein and ritov finally we present a high dimensional simulation study of a boosting type classification procedure

a continuous analogue of sturm sequences in the context of sturm liouville equation a shooting method is presented for finding the nth eigenvalue and eigenfunction of a sturm liouville equation in which the eigenvalue occurs nonlinearly the method is analyzed in two ways by applying the sturm comparison and oscillation theorems to the continuous problem and by applying sturm sequences to a discretization the method works for general separated boundary conditions and provides an a posteriors error estimate for the approximate eigenvalue a related method which involves critical lengths in the invariant imbedding method is shown to be incorrect for general boundary conditions

posterior consistency of nonparametric conditional moment restricted models this paper addresses the estimation of the nonparametric conditional moment restricted model that involves an infinite dimensional parameter g0 we estimate it in a quasi bayesian way based on the limited information likelihood and investigate the impact of three types of priors on the posterior consistency i truncated prior priors supported on a bounded set ii thin tail prior a prior that has very thin tail outside a growing bounded set and iii normal prior with nonshrinking variance in addition g0 is allowed to be only partially identified in the frequentist sense and the parameter space does not need to be compact the posterior is regularized using a slowly growing sieve dimension and it is shown that the posterior converges to any small neighborhood of the identified region we then apply our results to the nonparametric instrumental regression model finally the posterior consistency using a random sieve dimension parameter is studied

robust bayesian inference via coarsening the standard approach to bayesian inference is based on the assumption that the distribution of the data belongs to the chosen model class however even a small violation of this assumption can ha

computational design of optimal output feedback controllers we consider the problem of designing feedback control laws when a complete set of state variables is not available for linear autonomous systems with quadratic performance criterion the design problem consists of choosing an appropriate matrix of feedback gains according to a certain objective function in the literature the performance of quasi newton methods has been reported to be substandard we try to explain some of these observations and to propose structured quasi newton updates these methods which take into account the special structure of the problem show considerable improvement in the convergence using test examples from optimal output feedback design we also can verify these results numerically

bayesian variable selection for high dimensional generalized linear models convergence rates of the fitted densities bayesian variable selection has gained much empirical success recently in a variety of applications when the number k of explanatory variables x 1 ldots x k is possibly much larger than the sample size n for generalized linear models if most of the x j s have very small effects on the response y we show that it is possible to use bayesian variable selection to reduce overfitting caused by the curse of dimensionality k gg n in this approach a suitable prior can be used to choose a few out of the many x j s to model y so that the posterior will propose probability densities p that are often close to the true density p in some sense the closeness can be described by a hellinger distance between p and p that scales at a power very close to n 1 2 which is the finite dimensional rate corresponding to a low dimensional situation these findings extend some recent work of jiang technical report 05 02 2005 dept statistics northwestern univ on consistency of bayesian variable selection for binary classification

regular third order boundary value problems abstract in this paper we consider some boundary value problems generated by third order formally symmetric self adjoint regular differential expression and separated real coupled and complex coupled boundary conditions it is shown that these problems generate self adjoint operators moreover the dependence of eigenvalues of these problems on the data are studied and some derivatives of the eigenvalues with respect to some elements of data are introduced

backstepping observers for linear pdes on higher dimensional spatial domains the extension of the backstepping based state observer design is considered for systems governed by parabolic pdes defined on a higher dimensional cuboidal spatial domain with the measured output being restricted to a single boundary hyperplane thereby a suitable coordinate and state transformation is employed which allows for the application of the backstepping method for the determination of a luenberger type state observer with observer corrections entering the pde and the boundary condition this ensures that the observer error dynamics follows the behavior of a predefined exponentially stable target system in view of a practical realization of the proposed observer approach the idealized system output previously assumed as measurable on the entire boundary hyperplane is reconstructed from a limited set of optimally located measurements by means of a least squares approximation the observer error convergence and the applicability of the proposed approach are confirmed analytically and are evaluated in numerical simulations

on the properties of variational approximations of gibbs posteriors the pac bayesian approach is a powerful set of techniques to derive nonasymptotic risk bounds for random estimators the corresponding optimal distribution of estimators usually called the gibbs posterior is unfortunately often intractable one may sample from it using markov chain monte carlo but this is usually too slow for big datasets we consider instead variational approximations of the gibbs posterior which are fast to compute we undertake a general study of the properties of such approximations our main finding is that such a variational approximation has often the same rate of convergence as the original pac bayesian procedure it approximates in addition we show that when the risk function is convex a variational approximation can be obtained in polynomial time using a convex solver we give finite sample oracle inequalities for the corresponding estimator we specialize our results to several learning tasks classification ranking matrix completion discuss how to implement a variational approximation in each case and illustrate the good properties of said approximation on real datasets

population theory for boosting ensembles tree ensembles are looked at in distribution space that is the limit case of infinite sample size it is shown that the simplest kind of trees is complete in d dimensional l 2 p space if the number of terminal nodes t is greater than d for such trees we show that the adaboost algorithm gives an ensemble converging to the bayes risk

optimal actuator sensor location for active noise regulator and tracking control problems in this paper we investigate the problem of finding the optimal location of sensors and actuators to achieve reduction of the noise field in an acoustic cavity we offer two control strategies the first is based on linear quadratic tracking where the offending noise is tracked and the second considers the formulation of the harmonic control strategy as a periodic static output feedback control problem the first method which is based on full state information is suitable for optimal location of actuators while the second strategy can extend the results to finding optimal location of sensors as well as actuators for both methods we consider the optimization of an appropriate quadratic performance criterion with respect to the location of the actuators and or the sensors numerical examples are presented to compare the effectiveness of each control strategy and also the effect of optimal placement of actuators and sensors

eigenparameter dependent sturm liouville problems in boundary conditions with transmission conditions abstract in this paper we investigate the nonselfadjoint dissipative boundary value transmission problems in weyl s limit circle case at first using the method of operator theoretic formulation we pass to a new operator after showing that this new operator is a maximal dissipative operator we construct a selfadjoint dilation of the maximal dissipative operator using the equivalence of the lax phillips scattering function and the sz nagy foias characteristic function we show that all eigenfunctions and associated functions are complete in the space l w 2