the construction of preconditioners for elliptic problems by substructuring i in earlier parts of this series of papers we constructed preconditioners for the discrete systems of equations arising from the numerical approximation of elliptic boundary value problems the resulting algorithms are well suited for implementation on computers with parallel architecture in this paper we will develop a technique which utilizes these earlier methods to derive even more efficient preconditioners the iterative algorithms using these new preconditioners converge to the solution of the discrete equations with a rate that is independent of the number of unknowns these preconditioners involve an incomplete chebyshev iteration for boundary interface conditions which results in a negligible increase in the amount of computational work theoretical estimates and the results of numerical experiments are given which demonstrate the effectiveness of the methods

improved calibration of high dimensional precision matrices estimation of a precision matrix i e the inverse covariance matrix is a fundamental problem in statistical signal processing applications when the observation dimension is of the same order of magnitude as the number of samples the conventional estimators of covariance matrix and its inverse perform poorly in order to obtain well behaved estimators in high dimensional settings we consider a general class of estimators of covariance matrices and precision matrices based on weighted sampling and linear shrinkage the estimation error is measured in terms of both quadratic loss and stein s loss and these loss functions are used to calibrate the set of parameters defining our proposed estimator in an asymptotic setting where the observation dimension is comparable in magnitude to the number of samples we provide estimators of the precision matrix that are as good as their oracle counterparts we test our estimators with both synthetic data and financial market data and monte carlo simulations show the advantage of our precision matrix estimator compared with well known estimators in finite sample size settings

second order analysis for optimal control problems improving results expected from abstract theory an abstract optimization problem of minimizing a functional on a convex subset of a banach space is considered we discuss natural assumptions on the functional that permit establishing sufficient second order optimality conditions with minimal gap with respect to the associated necessary ones though the two norm discrepancy is taken into account the obtained results exhibit the same formulation as the classical ones known from finite dimensional optimization we demonstrate that these assumptions are fulfilled in particular by important optimal control problems for partial differential equations we prove that in contrast to a widespread common belief the standard second order conditions formulated for these control problems imply strict local optimality of the controls not only in the sense of l infty but also of l 2

bayesian minimum mean square error estimation for classification error part i definition and the bayesian mmse error estimator for discrete classification with the advent of high throughput genomic and proteomic technologies in conjunction with the difficulty in obtaining even moderately sized samples small sample classifier design has become a major issue in the biological and medical communities with small samples training data error estimation becomes mandatory yet none of the popular error estimation techniques have been rigorously designed based on statistical inference and optimization in this investigation we place classifier error estimation into the framework of optimal mean square error mse signal estimation in the presence of uncertainty which results in a bayesian approach to error estimation based on a parameterized family of feature label distributions with the prior distribution of the parameters governing the choice of feature label distribution these bayesian error estimators are optimal when averaged over a given family of distributions unbiased when averaged over a given family and all samples and analytically address a trade off between robustness modeling assumptions and accuracy minimum mean square error in this paper part i of a two part study we define the minimum mean square error mmse error estimator discuss its basic properties provide closed form analytic estimator representation for discrete classifiers with both non informative and informative prior distributions and examine the performance and robustness of the mmse error estimator via simulations in part ii of this paper in this same issue of ieee transactions on signal processing we address all of these issues in particular closed form representation for linear classification in the gaussian model with known and unknown covariance matrices for both the discrete and gaussian cases the mmse error estimator has especially good performance for distributions having moderate true errors

second order optimality conditions for semilinear elliptic control problems with finitely many state constraints this paper deals with necessary and sufficient optimality conditions for control problems governed by semilinear elliptic partial differential equations with finitely many equality and inequality state constraints some recent results on this topic for optimal control problems based upon results for abstract optimization problems are compared with some new results using methods adapted to the control problems meanwhile the lagrangian formulation is followed to provide the optimality conditions in the first case the lagrangian and hamiltonian functions are used in the second statement finally we prove the equivalence of both formulations

approximations of very weak solutions to boundary value problems standard weak solutions to the poisson problem on a bounded domain have square integrable derivatives which limits the admissible regularity of inhomogeneous data the concept of solution may be further weakened in order to define solutions when data is rough such as for inhomogeneous dirichlet data that is only square integrable over the boundary such very weak solutions satisfy a nonstandard variational form u v g v a galerkin approximation combined with an approximation of the right hand side g defines a finite element approximation of the very weak solution applying conforming linear elements leads to a discrete solution equivalent to the text book finite element solutionto the poisson problem in which the boundary data is approximated by l2 projections the l2 convergence rate of the discrete solution is o hs for some s in 0 1 2 that depends on the shape of the domain assuming a polygonal two dimensional or polyhedral three dimensional domain without slits and only square integr

sparse linear discriminant analysis by thresholding for high dimensional data in many social economical biological and medical studies one objective is to classify a subject into one of several classes based on a set of variables observed from the subject because the probability distribution of the variables is usually unknown the rule of classification is constructed using a training sample the well known linear discriminant analysis lda works well for the situation where the number of variables used for classification is much smaller than the training sample size because of the advance in technologies modern statistical studies often face classification problems with the number of variables much larger than the sample size and the lda may perform poorly we explore when and why the lda has poor performance and propose a sparse lda that is asymptotically optimal under some sparsity conditions on the unknown parameters for illustration of application we discuss an example of classifying human cancer into two classes of leukemia based on a set of 7 129 genes and a training sample of size 72 a simulation is also conducted to check the performance of the proposed method

monomer dimer problem on random planar honeycomb lattice we consider the monomer dimer md problem on a random planar honeycomb lattice model namely the random multiple chain this is a lattice system with non periodic boundary condition whose generating process is inspired by the growth of single walled zigzag carbon nanotubes by applying algebraic and combinatorial techniques we establish a calculating expression of the md partition function for bipartite graphs which corresponds to the permanent of a matrix further by using the transfer matrix argument we show that the computing problem of the permanent of high order matrix can be converted into some lower order matrices for this family of lattices based on which we derive an explicit recurrence formula for evaluating the md partition function of multiple chains and random multiple chains finally we analyze the expectation of the number of monomer dimer arrangements on a random multiple chain and the asymptotic behavior of the annealed md entropy when the multiple chain becomes infinite in width and length respectively

an energy space finite element approach for elliptic dirichlet boundary control problems in this paper we present a finite element analysis for a dirichlet boundary control problem where the dirichlet control is considered in a convex closed subspace of the energy space h 1 2 gamma h 1 2 as an equivalent norm in h 1 2 gamma h 1 2 we use the energy norm induced by the so called steklov poincare operator which realizes the dirichlet to neumann map and which can be implemented by using standard finite element methods the presented stability and error analysis of the discretization of the resulting variational inequality is based on the mapping properties of the solution operators related to the primal and adjoint boundary value problems and their finite element approximations some numerical results are given which confirm on one hand the theoretical estimates but on the other hand indicate the differences when modelling the control in l 2 gamma l 2

a new approach for mutual information analysis of large dimensional multi antenna channels this paper adresses the behavior of the mutual information of correlated multiple input multiple output mimo rayleigh channels when the numbers of transmit and receive antennas converge to infin at the same rate using a new and simple approach based on poincare nash inequality and on an integration by parts formula it is rigorously established that the mutual information when properly centered and rescaled converges to a standard gaussian random variable simple expressions for the centering and scaling parameters are provided these results confirm previous evaluations based on the powerful but nonrigorous replica method it is believed that the tools that are used in this paper are simple robust and of interest for the communications engineering community

optimal control problems with partially polyhedric constraints this paper discusses a class of state constrained optimal control problems for which it is possible to formulate second order necessary or sufficient conditions for local optimality or quadratic growth that do not involve all curvature terms for the constraints this kind of result is classical in the case of polyhedric control constraints our theory of optimization problems with partially polyhedric constraints allows to extend these results to the case when the control constraints are polyhedric in the presence of state constraints satisfying some specific hypotheses the analysis is based on the assumption that some strict semilinearized qualification condition is satisfied we apply the theory to some optimal control problems of elliptic equations with state and control constraints

analytic study of performance of error estimators for linear discriminant analysis we derive double asymptotic analytical expressions for the first moments second moments and cross moments with the actual error for the resubstitution and leave one out error estimators in the case of linear discriminant analysis in the multivariate gaussian model under the assumption of a common known covariance matrix and a fixed mahalanobis distance as dimensionality approaches infinity sample sizes for the two classes need not be the same they are only assumed to reach a fixed but arbitrary asymptotic ratio with the dimensionality from the asymptotic moment representations we directly obtain double asymptotic expressions for the bias variance and rms of the error estimators the asymptotic expressions presented here generally provide good small sample approximations as demonstrated via numerical experiments the applicability of the theoretical results is illustrated by finding the minimum sample size to bound the rms in gene expression classification

generalization error of linear discriminant analysis in spatially correlated sensor networks generalization error the probability of error of a detection rule learned from training samples on new unseen samples is a fundamental quantity to be characterized however characterizations of generalization error in the statistical learning theory literature are often loose and practically unusable for optimizing detection systems in this work focusing on learning linear discriminant analysis detection rules from spatially correlated sensor measurements a tight generalization error approximation is developed that can be used to optimize the parameters of a sensor network detection system as such the approximation is used to optimize network settings the approximation is also used to derive a detection error exponent and select an optimal subset of deployed sensor nodes a gauss markov random field is used to model correlation and weak laws of large numbers in geometric probability are employed in the analysis

error estimates for variational normal derivatives and dirichlet control problems with energy regularization this article deals with error estimates for the finite element approximation of variational normal derivatives and as a consequence error estimates for the finite element approximation of dirichlet boundary control problems with energy regularization the regularity of the solution is carefully carved out exploiting weighted sobolev and holder spaces this allows to derive a sharp relation between the convergence rates for the approximation and the structure of the geometry more precisely the largest opening angle at the vertices of polygonal domains numerical experiments confirm that the derived convergence rates are sharp

error analysis for a finite element approximation of elliptic dirichlet boundary control problems we consider the galerkin finite element approximation of an elliptic dirichlet boundary control model problem governed by the laplacian operator the analytical setting of this problem uses l 2 controls and a very weak formulation of the state equation however the corresponding finite element approximation uses standard continuous trial and test functions for this approximation we derive a priori error estimates of optimal order which are confirmed by numerical experiments the proofs employ duality arguments and known results from the l p error analysis for the finite element dirichlet projection

finite element approximation of parabolic time optimal control problems the numerical approximation of a parabolic time optimal control problem via piecewise linear splines is considered at each stage of the approximation a bang bang approximate control is selected by solving for its switching times as the solution of a constrained nonlinear least squares optimization problem the well posedness of the approximation scheme is shown and the rate of convergence to the exact solution investigated numerical results for some one and two dimensional parabolic control problems are given

on the regularity of the solutions of dirichlet optimal control problems in polygonal domains a linear quadratic dirichlet control problem governed by an elliptic equation posed on a possibly nonconvex polygonal domain is analyzed detailed regularity results are provided in classical sobolev slobodetski spaces in particular it is proved that in the presence of pointwise control constraints the optimal control is continuous despite the nonconvexity of the domain

propagation observation and control of waves approximated by finite difference methods this paper surveys several topics related to the observation and control of wave propagation phenomena modeled by finite difference methods the main focus is on the property of observability corresponding to the question of whether the total energy of solutions can be estimated from partial measurements on a subregion of the domain or boundary the mathematically equivalent property of controllability corresponds to the question of whether wave propagation behavior can be controlled using forcing terms on that subregion as is often desired in engineering applications observability controllability of the continuous wave equation is well understood for the scalar linear constant coefficient case that is the focus of this paper however when the wave equation is discretized by finite difference methods the control for the discretized model does not necessarily yield a good approximation to the control for the original continuous problem in other words the classical convergence consistency stability property of a numerical scheme does not suffice to guarantee its suitability for providing good approximations to the controls that might be needed in applications observability controllability may be lost under numerical discretization as the mesh size tends to zero due to the existence of high frequency spurious solutions for which the group velocity vanishes this phenomenon is analyzed and several remedies are suggested including filtering tychonoff regularization multigrid methods and mixed finite element methods we also briefly discuss these issues for the heat beam and schrodinger equations to illustrate that diffusive and dispersive effects may help to retain the observability controllability properties at the discrete level we conclude with a list of open problems and future subjects for research

bayesian minimum mean square error estimation for classification error part ii linear classification of gaussian models in this paper part ii of a two part study we derive a closed form analytic representation of the bayesian minimum mean square error mmse error estimator for linear classification assuming gaussian models this is presented in a general framework permitting a structure on the covariance matrices and a very flexible class of prior parameter distributions with four free parameters closed form solutions are provided for known scaled identity and arbitrary covariance matrices we examine performance in small sample settings via simulations on both synthetic and real genomic data and demonstrate the robustness of these error estimators to false gaussian modeling assumptions by applying them to johnson distributions

multiplier methods for nonlinear optimal control error estimates are derived for an augmented lagrangian approximation to an optimal control problem convex control constraints are treated explicitly while a lagrange multiplier is introduced for the nonlinear system dynamics the nonlinear optimal control problem does not fit the classical theory for estimating the error in multiplier approximations since the natural coercivity assumption is formulated in a hilbert space where the cost is not differentiable this discrepancy between the function space setting needed for coercivity and that needed for differentiability is compensated for by regularity results associated with the necessary conditions the paper concludes with an analysis of the optimal penalty parameter corresponding to a given finite element discretization

ritz galerkin approximation of the time optimal boundary control problem for parabolic systems with dirichlet boundary conditions this paper deals with ritz galerkin approximations of the following two problems i boundary value problems with l 2 boundary data given in the form of dirichlet boundary conditions ii time optimal control problems for parabolic systems with control acting on the boundary in both cases optimal rates of convergence of approximation are obtained also the specialization is considered to approximating with nonconformal elements which are not required to satisfy boundary conditions corresponding rates of convergence are also proved

a parabolic inverse convection diffusion reaction problem solved using space time localization and adaptivity this paper investigates the solution of a parabolic inverse problem based upon the convection diffusion reaction equation which can be used to estimate both water and air pollution we will consider both known and unknown source location while in the first case the problem is solved using a projected damped gauss newton pdgn in the second one it is ill posed and an adaptive parametrization with space time localization will be adopted to regularize it

the euler approximation in state constrained optimal control we analyze the euler approximation to a state constrained control problem we show that if the active constraints satisfy an independence condition and the lagrangian satisfies a coercivity condition then locally there exists a solution to the euler discretization and the error is bounded by a constant times the mesh size the proof couples recent stability results for state constrained control problems with results established here on discrete time regularity the analysis utilizes mappings of the discrete variables into continuous spaces where classical finite element estimates can be invoked

a high accuracy nfem for constrained optimal control problems governed by elliptic equations in this paper a new nonconforming finite element method nfem is proposed for the constrained optimal control problems ocps governed by elliptic equations the state and co state are approximated by the nonconforming eq 1 rot element and the control is approximated by the piecewise constant element respectively the superclose properties for the above three variables are obtained moreover for the state and co state the superconvergence results are achieved in a broken energy norm by using the post processing technique lastly some numerical examples are provided to verify the theoretical analysis

moments and root mean square error of the bayesian mmse estimator of classification error in the gaussian model the most important aspect of any classifier is its error rate because this quantifies its predictive capacity thus the accuracy of error estimation is critical error estimation is problematic in small sample classifier design because the error must be estimated using the same data from which the classifier has been designed use of prior knowledge in the form of a prior distribution on an uncertainty class of feature label distributions to which the true but unknown feature distribution belongs can facilitate accurate error estimation in the mean square sense in circumstances where accurate completely model free error estimation is impossible this paper provides analytic asymptotically exact finite sample approximations for various performance metrics of the resulting bayesian minimum mean square error mmse error estimator in the case of linear discriminant analysis lda in the multivariate gaussian model these performance metrics include the first second and cross moments of the bayesian mmse error estimator with the true error of lda and therefore the root mean square rms error of the estimator we lay down the theoretical groundwork for kolmogorov double asymptotics in a bayesian setting which enables us to derive asymptotic expressions of the desired performance metrics from these we produce analytic finite sample approximations and demonstrate their accuracy via numerical examples various examples illustrate the behavior of these approximations and their use in determining the necessary sample size to achieve a desired rms the supplementary material contains derivations for some equations and added figures highlightsthe bayesian minimum mean square error bmmse error estimator of lda is studied the first second and cross moments of the bmmse estimator with true error are considered both conditional and unconditional performance metrics are analyzed for the first time the kolmogorov double asymptotic is used in a bayesian setting asymptotically exact finite sample approximations of performance metrics are derived

finite sample linear filter optimization in wireless communications and financial systems we study the problem of linear filter optimization with finite sample size which has wide applications such as beamformer design in wireless communications and portfolio optimization in finance traditional methods in both fields are not robust against the imprecise channel vector and the noise covariance matrix or the mean return and the covariance of assets in finance due to finite sample size we consider estimation errors both in the channel vector and the noise covariance matrix or the mean return and the covariance simultaneously we resort to high dimensional asymptotics to account for the fact that the observation dimension is of the same order of magnitude as the number of samples and use the diagonal loading method or the shrinkage estimator to improve the robustness the channel vector or mean return and the noise covariance matrix are estimated from the training data and then corrected under several widely used criteria in an asymptotic setting where the number of samples is comparable to the observation dimension we obtain linear filters that are as good as the optimal filters with a shrinkage structure and a perfect channel vector or mean return under different criteria monte carlo simulations show the advantage of our linear filters in the finite sample size regime

first order tree type dependence between variables and classification performance structuralization of the covariance matrix reduces the number of parameters to be estimated from the training data and does not affect an increase in the generalization error asymptotically as both the number of dimensions and training sample size grow a method to benefit from approximately correct assumptions about the first order tree dependence between components of the feature vector is proposed we use a structured estimate of the covariance matrix to decorrelate and scale the data and to train a single layer perceptron in the transformed feature space we show that training the perceptron can reduce negative effects of inexact a priori information experiments performed with 13 artificial and 10 real world data sets show that the first order tree type dependence model is the most preferable one out of two dozen of the covariance matrix structures investigated

weak boundary penalization for dirichlet boundary control problems governed by elliptic equations abstract this paper concerns the finite element approximation of dirichlet boundary control problems governed by elliptic equations different from the existing literatures in which standard finite element method mixed finite element method or robin penalization method are used to deal with the underlying problems we adopt an alternative penalization approach introduced by nitsche called weak boundary penalization compared with the above methods our discrete scheme not only keeps consistency and avoids penalization error but also can be analyzed and computed conveniently as neumann boundary control problems based on the weak boundary penalization method we establish a finite element approximation to the dirichlet boundary control problems and derive the a priori error estimates for the control state and adjoint state numerical experiments are provided to confirm our theoretical results

second order runge kutta approximations in control constrained optimal control in this paper we analyze second order runge kutta approximations to a nonlinear optimal control problem with control constraints if the optimal control has a derivative of bounded variation and a coercivity condition holds we show that for a special class of runge kutta schemes the error in the discrete approximating control is o h2 where h is the mesh spacing

generalized consistent error estimator of linear discriminant analysis a classifier is epistemologically vacuous without an accurate estimate of its true error rate in situations where the number of sample points is of the same order of magnitude as the dimension of observations serious issues arise with respect to the performance of error estimators in this paper we place the problem of synthesizing an error rate estimator of a common linear classifier in an asymptotic setting in which the number of sample points is kept comparable in magnitude to the dimension of observations double asymptotic we construct a generalized consistent estimator of the true error rate for linear discriminant analysis in the multivariate gaussian model under the assumption of a common covariance matrix in other words the estimator converges to true error rate in the double asymptotic sense we employ simulations using both synthetic and real data to compare the performance of the new estimator to the classical consistent estimator of the true error plug in estimator as well as other well known estimators we observe that the constructed estimator can outperform other estimators of the true error in many situations in terms of bias and root mean square rms error

error estimates for the numerical approximation of dirichlet boundary control for semilinear elliptic equations we study the numerical approximation of boundary optimal control problems governed by semilinear elliptic partial differential equations with pointwise constraints on the control the control is the trace of the state on the boundary of the domain which is assumed to be a convex polygonal open set in mathbb r 2 piecewise linear finite elements are used to approximate the control as well as the state we prove that the error estimates are of order o h 1 1 p for some p 2 which is consistent with the w 1 1 p p gamma regularity of the optimal control

a discontinuous galerkin time stepping scheme for the velocity tracking problem the velocity tracking problem for the evolutionary navier stokes equations in two dimensions is studied the controls are of distributed type and are submitted to bound constraints first and second order necessary and sufficient conditions are proved a fully discrete scheme based on the discontinuous in time galerkin approach combined with conforming finite element subspaces in space is proposed and analyzed provided that the time and space discretization parameters tau and h respectively satisfy tau leq ch 2 then l 2 error estimates of order o h are proved for the difference between the locally optimal controls and their discrete approximations

second order analysis for bang bang control problems of pdes in this paper we derive some sufficient second order optimality conditions for control problems of partial differential equations pdes when the cost functional does not involve the usual quadratic term for the control or higher nonlinearities for it though not always in this situation the optimal control is typically bang bang two different control problems are studied the second differs from the first in the presence of the l 1 norm of the control this term leads to optimal controls that are sparse and usually take only three different values we call them bang bang bang controls though the proofs are detailed in the case of a semilinear elliptic state equation the approach can be extended to parabolic control problems some hints are provided in the last section to extend the results

superconvergence properties of optimal control problems an optimal control problem for a two dimensional 2 d elliptic equation is investigated with pointwise control constraints this paper is concerned with discretization of the control by piecewise constant functions the state and the adjoint state are discretized by linear finite elements approximations of the optimal solution of the continuous optimal control problem will be constructed by a projection of the discrete adjoint state it is proved that these approximations have convergence order h2