testing for non nested conditional moment restrictions using unconditional empirical likelihood we propose non nested tests for competing conditional moment resctriction models using a method of empirical likelihood our tests are based on the method of conditional empirical likelihood developed by kitamura tripathi and ahn 2004 and zhang and gijbels 2003 by using the conditional implied probabilities we develop three non nested tests the moment encompassing cox type and effcient score encompassing tests compared to the existing non nested tests which mainly focus on testing unconditional moment restrictions our approach directly tests conditional moment restrictions which imply the infinite number of unconditional moment restrictions we derive the null distributions and power properties of the proposed tests simulation experiments show that our tests have reasonable finite sample properties

surrogate programming and multipliers in quasi convex programming a result due to luenberger on the existence of multipliers in a quasi convex programming problem is extended to the case of constraints given by an arbitrary convex cone under a constraint qualification condition more general than slater s condition the existence of solutions is not assumed we point out links with even convexity in the sense of fenchel and quasi subdifferentiability in the sense of greenberg pierskalla and we observe that the couples of primal dual optimal solutions reduce to saddle points of a suitable lagrangian function

facial reduction in partially finite convex programming we consider the problem of minimizing an extended valued convex function on a locally convex space subject to a finite number of linear in equalities when the standard constraint qualification fails a reduction technique is needed to derive necessary optimality conditions facial reduction is usually applied in the range of the constraints in this paper it is applied in the domain space thus maintaining any structure and in particular lattice properties of the underlying domain applications include constrained approximation and best entropy estimation

spectrum estimation using multirate observations in this paper we are interested in estimating the power spectral density of a stationary random signal x n when the signal itself is not available but some low resolution measurements derived from it are observed we consider a model where x n is being measured using a set of linear multirate sensors each sensor outputs a measurement signal v sub i n whose sampling rate is only a fraction of the sampling rate assumed for the original signal based on this model we pose the following problem given certain autocorrelation coefficients of the observable signals v sub i n estimate the power spectral density of the original signal x n it turns out that this problem is ill posed we suggest to resolve this issue by using the principle of maximum entropy me we address technical difficulties associated with the me solution and then devise a practical algorithm for its approximate computation we demonstrate the viability of this algorithm through simulation examples

asymptotic behavior of interior point methods a view from semi infinite programming we study the asymptotic behavior of interior point methods for linear programming problems attempts to solve larger problems using interior point methods lead to the question of how these algorithms behave as n the number of variables goes to infinity here we take a different point of view and investigate what happens when n is infinite motivated by this approach we study the limits of search directions potential functions and central paths we also suggest that the complexity of some linear programming problems may depend on the smoothness of the given problem rather than the number of variables we prove that when n is infinite for some subclasses of problems one can still obtain a bound on the number of iterations required in terms of the smoothness of the problem and the desired accuracy

constructing narmax models using armax models this paper outlines how it is possible to decompose a complex non linear modelling problem into a set of simpler linear modelling problems local armax models valid within certain operating regimes are interpolated to construct a global narmax non linear narmax model knowledge of the system behaviour in terms of operating regimes is the primary basis for building such models hence it should not be considered as a pure black box approach but as an approach that utilizes a limited amount of a priori system knowledge it is shown that a large class of non linear systems can be modelled in this way and indicated how to decompose the systems range of operation into operating regimes standard system identification algorithms can be used to identify the narmax model and several aspects of the system identification problem are discussed and illustrated by a simulation example

bayesian methods and maximum entropy for ill posed inverse problems in this paper we study linear inverse problems where some generalized moments of an unknown positive measure are observed we introduce a new construction called the maximum entropy on the mean method mem which relies on a suitable sequence of finite dimensional discretized inverse problems its advantage is threefold it allows us to interpret all usual deterministic methods as bayesian methods it gives a very convenient way of taking into account prior information it also leads to new criteria for the existence question concerning the linear inverse problem which will be a starting point for the investigation of superresolution phenomena the key tool in this work is the large deviations property of some discrete random measure connected with the reconstruction procedure

moment matching and best entropy estimation abstract given the first n moments of an unknown function x on the unit interval a common estimate of x is n where n is a polynomial of degree n taking values in a prescribed interval is a given monotone function and n is chosen so that the moments of n equal those of x this moment matching procedure is closely related to best entropy estimation of x two classical cases arise when is the exponential function corresponding to the boltzmann shannon entropy and the reciprocal function corresponding to the burg entropy general conditions ensuring the existence and uniqueness of n are given using convex programming duality techniques and it is shown that the estimate n converges uniformly to x providing x is sufficiently smooth

multivariate adaptive regression splines a new method is presented for flexible regression modeling of high dimensional data the model takes the form of an expansion in product spline basis functions where the number of basis functions as well as the parameters associated with each one product degree and knot locations are automatically determined by the data this procedure is motivated by the recursive partitioning approach to regression and shares its attractive properties unlike recursive partitioning however this method produces continuous models with continuous derivatives it has more power and flexibility to model relationships that are nearly additive or involve interactions in at most a few variables in addition the model can be represented in a form that separately identifies the additive contributions and those associated with the different multivariable interactions

partially finite programming in l_1 and the existence of maximum entropy estimates best entropy estimation is a technique that has been widely applied in many areas of science it consists of estimating an unknown density from some of its moments by maximizing some measure of the entropy of the estimate this problem can be modelled as a partially finite convex program with an integrable function as the variable a complete duality and existence theory is developed for this problem and for an associated extended problem which allows singular measure theoretic solutions this theory explains the appearance of singular components observed in the literature when the burg entropy is used it also provides a unified treatment of existence conditions when the burg boltzmann shannon or some other entropy is used as the objective some examples are discussed

asymptotic optimality of empirical likelihood for testing moment restrictions we show by example that empirical likelihood and other commonly used tests for moment restrictions are unable to control the exponential rate at which the probability of a type i error tends to zero unless the possible distributions for the observed data are restricted appropriately from this it follows that for the optimality claim for empirical likelihood in kitamura 2001 to hold additional assumptions and qualifications are required under stronger assumptions than those in kitamura 2001 we establish the following optimality result i empirical likelihood controls the rate at which the probability of a type i error tends to zero and ii among all procedures for which the probability of a type i error tends to zero at least as fast empirical likelihood maximizes the rate at which the probability of a type ii error tends to zero for most alternatives this result further implies that empirical likelihood maximizes the rate at which the probability of a type ii error tends to zero for all alternatives among a class of tests that satisfy a weaker criterion for their type i error probabilities

on two dimensional spectral realization reconstruction of a spectral density function from a finite set of covariances can be performed by maximizing an entropy functional the method of the maximum entropy on the mean is used for computing a discrete version of this spectral density and allows one to give a new interpretation of these reconstruction methods in fact the authors show that the choice of the entropy is directly related to a prior distribution in particular they consider processes on z sup 2 steepest descent procedures permit the numerical computation of discrete realizations for a wide class of entropies to ensure the nonnegativity of the solution related to the burg entropy they present a new algorithm based on a fixed point method and the yule walker equations to compute this solution then the solution of the dual problem is obtained as the limit of the trajectory of an ordinary differential equation

an unconstrained convex programming approach to linear semi infinite programming in this paper an unconstrained convex programming dual approach for solving a class of linear semi infinite programming problems is proposed both primal and dual convergence results are established under some basic assumptions numerical examples are also included to illustrate this approach

an adaptive multi scale basis method for modelling discrete time nonlinear dynamical systems based on locally applicable autoregressive moving average with exogenous armax models this paper proposes two new adaptive modelling schemes for discrete time non linear dynamic systems based on the concept of a multiscale which originates in approximation theory a general result on the convergence of modelling based on the multiscale basis functions with a least square estimated is derived with the advantage of the multiresolution nature of the multiscale basis the new modelling scheme can be used to capture both the global and local characteristics of a non linear system over distinct scales the algorithms also provide a trade off between the model structure model size and the modelling error one of which emphasizes the modelling error with variable model structure whilst the other stresses a desired model size with better modelling error data based modelling examples are used to demonstrate the effectiveness of the multiscale modelling schemes with different multiscale basis

painless nonorthogonal expansions in a hilbert space h discrete families of vectors hj with the property that f j hj f hj for every f in h are considered this expansion formula is obviously true if the family is an orthonormal basis of h but also can hold in situations where the hj are not mutually orthogonal and are overcomplete the two classes of examples studied here are i appropriate sets of weyl heisenberg coherent states based on certain non gaussian fiducial vectors and ii analogous families of affine coherent states it is believed that such quasiorthogonal expansions will be a useful tool in many areas of theoretical physics and applied mathematics

dual methods in entropy maximization application to some problems in crystallography this paper is devoted to some infinite dimensional optimization problems with finitely many constraints these deal with entropy maximization and this paper is particularly concerned with those or

multiscale decompositions on bounded domains a construction of multiscale decompositions relative to domains rd is given multiscale spaces are constructed on which retain the important features of univariate multiresolution analysis including local polynomial reproduction and locally supported stable bases

non asymptotic analysis of a new bandit algorithm for semi bounded rewards in this paper we consider a stochastic multiarmed bandit problem it is known in this problem that deterministic minimum empirical divergence dmed policy achieves the asymptotic theoretical bound for the model where each reward distribution is supported in a known bounded interval say 0 1 however the regret bound of dmed is described in an asymptotic form and the performance in finite time has been unknown we modify this policy and derive a finite time regret bound for the new policy indexed minimum empirical divergence imed by refining large deviation probabilities to a simple nonasymptotic form further the refined analysis reveals that the finite time regret bound is valid even in the case that the reward is not bounded from below therefore our finite time result applies to the case that the minimum reward that is the maximum loss is unknown or unbounded we also present some simulation results which shows that imed much improves dmed and performs competitively to other state of the art policies

hodges lehmann optimality for testing moment conditions this paper studies the hodges and lehmann 1956 optimality of tests in a general setup the tests are compared by the exponential rates of growth to one of the power functions evaluated at a fixed alternative while keeping the asymptotic sizes bounded by some constant we present two sets of sufficient conditions for a test to be hodges lehmann optimal these new conditions extend the scope of the hodges lehmann optimality analysis to setups that cannot be covered by other conditions in the literature the general result is illustrated by our applications of interest testing for moment conditions and overidentifying restrictions in particular we show that i the empirical likelihood test does not necessarily satisfy existing conditions for optimality but does satisfy our new conditions and ii the generalized method of moments gmm test and the generalized empirical likelihood gel tests are hodges lehmann optimal under mild primitive conditions these results support the belief that the hodges lehmann optimality is a weak asymptotic requirement

c 1 spline wavelets on triangulations in this paper we investigate spline wavelets on general triangulations in particular we are interested in c 1 wavelets generated from piecewise quadratic polynomials by using the powell sabin elements we set up a nested family of spaces of c 1 quadratic splines which are suitable for multiresolution analysis of besov spaces consequently we construct c 1 wavelet bases on general triangulations and give explicit expressions for the wavelets on the three direction mesh a general theory is developed so as to verify the global stability of these wavelets in besov spaces the wavelet bases constructed in this paper will be useful for numerical solutions of partial differential equations

recursive hybrid algorithm for non linear system identification using radial basis function networks recursive identification of non linear systems is investigated using radial basis function networks a novel approach is adopted which employs a hybrid clustering and least squares algorithm the recursive clustering algorithm adjusts the centres of the radial basis function network while the recursive least squares algorithm estimates the connection weights of the network because these two recursive learning rules are both linear rapid convergence is guaranteed and this hybrid algorithm significantly enhances the real time or adaptive capability of radial basis function models the application to simulated real data are included to demonstrate the effectiveness of this hybrid approach

nonlinear black box models in system identification mathematical foundations we discuss several aspects of the mathematical foundations of the nonlinear black box identification problem we shall see that the quality of the identification procedure is always a result of a certain trade off between the expressive power of the model we try to identify the larger the number of parameters used to describe the model the more flexible is the approximation and the stochastic error which is proportional to the number of parameters a consequence of this trade off is the simple fact that a good approximation technique can be the basis of a good identification algorithm from this point of view we consider different approximation methods and pay special attention to spatially adaptive approximants we introduce wavelet and neuron approximations and show that they are spatially adaptive then we apply the acquired approximation experience to estimation problems finally we consider some implications of these theoretical developments for the practically implemented versions of the spatially adaptive algorithms

wavelet methods for pdes some recent developments abstract this paper is concerned with recent developments of wavelet schemes for the numerical treatment of operator equations with special emphasis on two issues adaptive solution concepts and nontrivial domain geometries after describing a general multiresolution framework the key features of wavelet bases are highlighted namely locality norm equivalences and cancellation properties assuming first that wavelet bases with these properties are available on the relevant problem domains the relevance of these features for a wide class of stationary problems is explained in subsequent sections the main issues are preconditioning and the efficient adaptive application of wavelet representations of the involved operators we indicate then how these ingredients combined with concepts from nonlinear or best n term approximation culminate in an adaptive wavelet scheme for elliptic selfadjoint problems covering boundary value problems as well as boundary integral equations these schemes can be shown to exhibit convergence rates that are in a certain sense asymptotically optimal we conclude this section with some brief remarks on data structures and implementation interrelations with regularity in a certain scale of besov spaces and strategies of extending such schemes to unsymmetric or indefinite problems we address then the adaptive evaluation of nonlinear functionals of wavelet expansions as a central task arising in connection with nonlinear problems wavelet bases on nontrivial domains are discussed next the main issues are the development of fourier free construction principles and criteria for the validity of norm equivalences finally we indicate possible combinations of wavelet concepts with conventional discretizations such as finite element or finite volume schemes in connection with convection dominated and hyperbolic problems

nonlinear wavelet image processing variational problems compression and noise removal through wavelet shrinkage this paper examines the relationship between wavelet based image processing algorithms and variational problems algorithms are derived as exact or approximate minimizers of variational problems in particular we show that wavelet shrinkage can be considered the exact minimizer of the following problem given an image f defined on a square i minimize over all g in the besov space b11 l1 i the functional f g l2 i 2 g b11 l1 i we use the theory of nonlinear wavelet image compression in l2 i to derive accurate error bounds for noise removal through wavelet shrinkage applied to images corrupted with i i d mean zero gaussian noise a new signal to noise ratio snr which we claim more accurately reflects the visual perception of noise in images arises in this derivation we present extensive computations that support the hypothesis that near optimal shrinkage parameters can be derived if one knows or can estimate only two parameters about an image f the largest for which f bq lq i 1 q 2 1 2 and the norm f bq lq i both theoretical and experimental results indicate that our choice of shrinkage parameters yields uniformly better results than donoho and johnstone s visushrink procedure an example suggests however that donoho and johnstone s 1994 1995 1996 sureshrink method which uses a different shrinkage parameter for each dyadic level achieves a lower error than our procedure

an elementary survey of general duality theory in mathematical programming we survey some recent developments in duality theory with the idea of explaining and unifying certain basic duality results in both nonlinear and integer programming the idea of replacing dual variables prices by price functions suggested by everett and developed by gould is coupled with an appropriate dual problem with the consequence that many of the results resemble those used in linear programming the dual problem adopted has a traditional economic interpretation and dual feasibility then provides a simple alternative to concepts such as conjugate functions or subdifferentials used in the study of optimality in addition we attempt to make precise the relationship between primal dual and saddlepoint results in both the traditional lagrangean and the more general duality theories and to see the implications of passing from prices to price functions finally and perhaps surprisingly it appears that all the standard algorithms terminate by constructing primal and dual feasible solutions of equal value i e by satisfying generalised optimality conditions

large deviations for random power moment problem we consider the set mn of all n truncated power moment sequences of probability measures on 0 1 we endow this set with the uniform probability picking randomly a point in mn we show that the upper canonical measure associated with this point satisfies a large deviation principle moderate deviation are also studied completing earlier results on asymptotic normality given by chang kemperman and studden ann probab 21 1993 1295 1309 surprisingly our large deviations results allow us to compute explicitly the n 1 th moment range size of the set of all probability measures having the same n first moments the main tool to obtain these results is the representation of mn on canonical moments see the book of dette and studden

on the geometry of maximum entropy problems we show that a simple geometric result suffices to derive the form of the optimal solution in a large class of finite and infinite dimensional maximum entropy problems concerning probability distributions spectral densities and covariance matrices these include burg s spectral estimation method and dempster s covariance completion as well as various recent generalizations of the above we then apply this orthogonality principle to the new problem of completing a block circulant covariance matrix when an a priori estimate is available

superresolution in the markov moment problem abstract suppose that bar x is a characteristic function and a 1 a 2 a n are weight functions on a finite measure space recent work of gamboa and gassiat observed conditions guaranteeing that x bar x 1 is small whenever 0 x 1 a e and the moment errors int x bar x a i are small i 1 2 n using concise and elementary techniques we obtain similar results under very mild assumptions we also provide precise error bounds

on the importance of combining wavelet based nonlinear approximation with coding strategies this paper provides a mathematical analysis of transform compression in its relationship to linear and nonlinear approximation theory contrasting linear and nonlinear approximation spaces we show that there are interesting classes of functions random processes which are much more compactly represented by wavelet based nonlinear approximation these classes include locally smooth signals that have singularities and provide a model for many signals encountered in practice in particular for images however we also show that nonlinear approximation results do not always translate to efficient compress on strategies in a rate distortion sense based on this observation we construct compression techniques and formulate the family of functions stochastic processes for which they provide efficient descriptions in a rate distortion sense we show that this family invariably leads to besov spaces yielding a natural relationship among besov smoothness linear nonlinear approximation order and compression performance in a rate distortion sense the designed compression techniques show similarities to modern high performance transform codecs allowing us to establish relevant rate distortion estimates and identify performance limits

triangular wavelet based finite elements via multivalued scaling equations this paper develops a class of triangular finite elements based on l2 rn orthonormal compactly supported wavelets as in the case of recently derived tensor product wavelet elements the triangular elements are defined by deriving an algebraic eigenvalue problem characterizing the entries of the elemental matrices these entries are uniquely defined by normalization conditions derived from the polynomial reproducing properties of the wavelet scaling functions the work herein generalizes the development of the tensor product wavelet elements in that the geometry of the triangular elemental domain is characterized using a multivalued scaling equation the multivalued scaling function is used to represent the self similar tiling of rn onto itself by triangular domains

a dual approach to multidimensional l p spectral estimation problems a complete duality theory is presented for the multidimensional l p spectral estimation problem the authors use a new constraint qualification bwcq for infinite dimensional convex programs with linear type constraints recently introduced in borwein and wolkowicz math programming 35 1986 pp 83 96 this allows direct derivation of the explicit optimal solution of the problem as presented in goodrich and steinhardt siam j appl math 46 1986 pp 417 426 and establishment of the existence of a simple and computationally tractable unconstrained lagrangian dual problem moreover the results illustrate that bwcq is more appropriate to spectral estimation problems than the traditional slater condition which may only be applied after transformation of the problem into an l p space goodrich and steinhardt op cit and which therefore yields only necessary conditions

interpolation of besov spaces we investigate besov spaces and their connection with dyadic spline approximation in lp q 0 p oo our main results are the determination of the interpolation spaces between a pair of besov spaces an atomic decomposition for functions in a besov space the characterization of the class of functions which have certain prescribed degree of approximation by dyadic splines

approximation by smooth multivariate splines abstract one of the important properties of univariate splines is that in most senses smooth splines approximate just as well as do piecewise polynomials on the same mesh this report shows this to be untrue in the multivariate setting in particular it details the cost in approximating power one may have to pay for the luxury of a smooth piecewise polynomial approximant in an extreme case piecewise polynomials of total degree r on a rectangular grid with all derivatives of order or rho continuous will fail to approximate certain smooth functions at all as the grid goes to zero unless rho is kept below r 3 2 during the analysis of approximation on a certain regular triangular grid a novel kind of bivariate b spline is introduced this b spline in contrast to the established multivariate b spline derived from a simplex can be made to have all its breaklines in a given regular grid this makes it a prime candidate for use in the construction of smooth multivariate piecewise polynomial approximation and its properties will be explored further

on the stability in weak topology of the set of global solutions to the navier stokes equations let x be a suitable function space and let mathcal g subset x be the set of divergence free vector fields generating a global smooth solution to the incompressible homogeneous three dimensional navier stokes equations we prove that a sequence of divergence free vector fields converging in the sense of distributions to an element of mathcal g belongs to mathcal g if n is large enough provided the convergence holds anisotropically in frequency space typically this excludes self similar type convergence anisotropy appears as an important qualitative feature in the analysis of the navier stokes equations it is also shown that initial data which do not belong to mathcal g hence which produce a solution blowing up in finite time cannot have a strong anisotropy in their frequency support

a general theory of surrogate dual and perturbational extended surrogate dual optimization problems abstract for the optimization problem p inf h g where g o is a subset of a locally convex space f and h f r we introduce and study two general concepts of dual problems encompassing the classical surrogate dual problem the first one involves only a family of surrogate constraints sets g f w where w r x x being a locally convex space the second one uses a perturbation functional f x r and a family of sets gd f x0 f x w where w r x we give duality theorems introduce lagrangians and show some relations between these problems and the dual problems to p defined with the aid of a perturbation and a concept of conjugation of functionals

which is the better entropy expression for speech processing s log s or log s in maximum entropy spectral analysis mesa one maximizes the integral of logs f where s f is a power spectrum the resulting spectral estimate which is equivalent to that obtained by linear prediction and other methods is popular in speech processing applications an alternative expression s f logs f is used in optical processing and elsewhere this paper considers whether the alternative expression leads to spectral estimates useful in speech processing we investigate the question both theoretically and empirically the theoretical investigation is based on generalizations of file two estimates the generalizations take into account prior estimates of the unknown power spectrum it is shown that both estimates result from applying a generalized version of the principle of maximum entropy but they differ concerning the quantities that are treated as random variables the empirical investigation is based on speech synthesized using the different spectral estimates although both estimates lead to intelligible speech speech based on the mesa estimate is qualitatively superior

a discrete transform and decompositions of distribution spaces abstract we study a representation formula of the form q q q for a distribution on r n this formula is obtained by discretizing and localizing a standard littlewood paley decomposition the map taking to the sequence q q with q running over the dyadic cubes in r n is called the transform the functions q and q have a particularly simple form moreover most of the familiar distribution spaces lp spaces 1 r n we also consider pointwise multipliers for the characteristic function of a domain we obtain boundedness results for a general class of domains which properly includes lipschitz domains several interpolation methods are easily analyzed via the sequence spaces for real interpolation we obtain among other things an extension to the case p 0 this in turn gives a new approach to the traditional atomic decomposition of hardy spaces

wavelet neural networks for function learning a wavelet based neural network is described the structure of this network is similar to that of the radial basis function rbf network except that in the present paper the radial basis functions are replaced by orthonormal scaling functions that are not necessarily radial symmetric the efficacy of this type of network in function learning and estimation is demonstrated through theoretical analysis and experimental results in particular it has been shown that the wavelet network has universal and l sup 2 approximation properties and is a consistent function estimator convergence rates associated with these properties are obtained for certain function classes where the rates avoid the curse of dimensionality in the experiments the wavelet network performed well and compared favorably to the mlp and rbf networks

from projection pursuit and cart to adaptive discriminant analysis while many efforts have been put into the development of nonlinear approximation theory and its applications to signal and image compression encoding and denoising there seems to be very few theoretical developments of adaptive discriminant representations in the area of feature extraction selection and signal classification in this paper we try to advocate the idea that such developments and efforts are worthwhile based on the theoretical study of a data driven discriminant analysis method on a simple yet instructive example we consider the problem of classifying a signal drawn from a mixture of two classes using its projections onto low dimensional subspaces unlike the linear discriminant analysis lda strategy which selects subspaces that do not depend on the observed signal we consider an adaptive sequential selection of projections in the spirit of nonlinear approximation and classification and regression trees cart at each step the subspace is enlarged in a direction that maximizes the mutual information with the unknown class we derive explicit characterizations of this adaptive discriminant analysis ada strategy in two situations when the two classes are gaussian with the same covariance matrix but different means the adaptive subspaces are actually nonadaptive and can be computed with an algorithm similar to orthonormal matching pursuit when the classes are centered gaussians with different covariances the adaptive subspaces are spanned by eigen vectors of an operator given by the covariance matrices just as could be predicted by regular lda however we prove that the order of observation of the components along these eigen vectors actually depends on the observed signal numerical experiments on synthetic data illustrate how data dependent features can be used to outperform lda on a classification task and we discuss how our results could be applied in practice

surrogate dual problems and surrogate lagrangians abstract by a surrogate dual problem to an optimization problem p embedded into a family of perturbed problems px x x we mean here any quasi convex dual 2 or any pseudo dual 30 problem to p with respect to the embedding these include in particular the usual surrogate dual problems we give some new results on surrogate dual problems and surrogate lagrangians which we introduce in this general framework also we determine the surrogate dual problems and surrogate lagrangians for various optimization problems for norm minimization on linear manifolds we find an optimal perturbation for pseudo duality i e which yields the gap 0

quasiconjugates of functions duality relationship between quasiconvex minimization under a reverse convex constraint and quasiconvex maximization under a convex constraint and applications abstract in this paper we introduce a concept of quasiconjugate for functions defined on rn whose values are in r the conjugacy correspondence between functions and their quasiconjugates is one to one and symmetric in a class of quasiconvex functions whose minimizer on rn is located at the origin by using the concept of quasiconjugate we obtain a duality relationship between quasiconvex minimization under a reverse convex constraint and quasiconvex maximization under a convex constraint this duality relationship allows us to establish a primal dual pair in a class of nonconvex optimization problems without the duality gap several applications are given

duality theorems for perturbed convex optimization abstract we show that if f x are two locally convex spaces and h f r f x r are two convex functionals satisfying h y y x0 y f for some x0 x then under suitable assumptions the computation of inf h f can be reduced to the computation of inf h on certain hyperplanes h of f x we give some applications

adaptive wavelet estimation a block thresholding and oracle inequality approach we study wavelet function estimation via the approach of block thresh olding and ideal adaptation with oracle oracle inequalities are derived and serve as guides for the selection of smoothing parameters based on an oracle inequality and motivated by the data compression and localization properties of wavelets an adaptive wavelet estimator for nonparametric re gression is proposed and the optimality of the procedure is investigated we show that the estimator achieves simultaneously three objectives adaptiv ity spatial adaptivity and computational efficiency specifically it is proved that the estimator attains the exact optimal rates of convergence over a range of besov classes and the estimator achieves adaptive local minimax rate for estimating functions at a point the estimator is easy to imple ment at the computational cost of on simulation shows that the es timator has excellent numerical performance relative to more traditional wavelet estimators 1 introduction wavelet methods have demonstrated considerable suc cess in nonparametric function estimation in terms of spatial adaptivity com putational efficiency and asymptotic optimality in contrast to the traditional linear procedures wavelet methods achieve near optimal convergence rates over large function classes such as besov classes and enjoy excellent mean squared error properties when used to estimate functions that are spatially inhomogeneous for example as shown by donoho and johnstone 1998 wavelet methods can outperform optimal linear methods even at the level of convergence rate over certain besov classes standard wavelet methods achieve adaptivity through term by term thresholding of the empirical wavelet coefficients there each individual empirical wavelet coefficient is compared with a predetermined threshold a wavelet coefficient is retained if its magnitude is above the threshold level and is discarded otherwise a well known example of term by term thresholding is donoho and johnstone s visushrink donoho and johnstone 1994 visushrink is spatially adaptive and the estimator is within a log arithmic factor of the optimal convergence rate over a wide range of besov classes visushrink achieves a degree of tradeoff between variance and bias contributions to the mean squared error however the tradeoff is not optimal visushrink reconstruction is often over smoothed hall kerkyacharian and picard 1999 considered block thresholding for wavelet function estimation which thresholds empirical wavelet coefficients in

automatic generation of rbf networks using wavelets abstract learning can be viewed as mapping from an input space to an output space examples of these mappings are used to construct a continuous function that approximates the given data and generalizes for intermediate instances radial basis function rbf networks are used to formulate this approximating function a novel method is introduced that automatically constructs a generalized radial basis function grbf network for a given mapping and error bound this network is shown to be the smallest network within the error bound for the given mapping the integral wavelet transform is used to determine the parameters of the network simple one dimensional examples are used to demonstrate how the network constructed using the transform is superior to that constructed using standard ad hoc optimization techniques the paper concludes with the automatic generation of grbf networks for a multi dimensional problem namely real time 3d object recognition and pose estimation the results of this application are favorable

representations of non linear systems the narmax model abstract input output representations of non linear discrete time systems are discussed it is shown that the narmax non linear autoregressive moving average with exogenous inputs model is a general and natural representation of non linear systems and contains as special cases several existing non linear models the problem of approximating non linear input output systems is also addressed and several properties of non linear models are highlighted using simple examples

some general lagrangian duality theorems abstract we give some lagrangian duality theorems for optimization problems in terms of non linear separation properties involving epigraphs or related sets some applications are also given

a simple constraint qualification in infinite dimensional programming a new simple constraint qualification for infinite dimensional programs with linear programming type constraints is used to derive the dual program see theorem 3 1 applications include a proof of the explicit solution of the best interpolation problem presented in 8

some relations between dualities polarities coupling functionals and conjugations on etablit certaines relations entre dualites au sens de evers et van maaren 1981 polarites comme celles utilisees dans griffin araoz et edmonds 1982 fonctionnelles de couplage au sens de moreau 1966 1967 1970 et conjugaisons comme celles introduites par l auteur 1983

multidimensional mem spectral estimation the problem of multidimensional maximum entropy method mem spectral estimation from nonuniformly spaced correlation measurements is investigated a necessary and sufficient condition is derived for the existence and uniqueness of the mem spectral estimate in its usual form it is shown that this condition is not satisfied in many multidimensional problems of interest although it is satisfied in the important practical case of spectral supports composed of a finite number of points when the existence condition is satisfied calculation of the mem estimate reduces to the solution of a finite dimensional convex optimization problem the application of standard optimization techniques to this problem results in iterative computational algorithms which are guaranteed to converge the algorithms so obtained are compared to those previously proposed and a spectral estimation example is presented

adaptive transient solution of nonuniform multiconductor transmission lines using wavelets this paper presents a highly adaptive algorithm for the transient simulation of nonuniform interconnects loaded with arbitrary nonlinear and dynamic terminations the discretization of the governing equations is obtained through a weak formulation using biorthogonal wavelet bases as trial and test functions it is shown how the multiresolution properties of wavelets lead to very sparse approximations of the voltages and currents in typical transient analyzes a simple yet effective time space adaptive algorithm capable of selecting the minimal number of unknowns at each time iteration is described numerical results show the high degree of adaptivity of the proposed scheme

maximum entropy reconstruction using derivative information part 2 computational results summary maximum entropy density estimation a technique for reconstructing an unknown density function on the basis of certain measurements has applications in various areas of applied physical sciences and engineering here we present numerical results for the maximum entropy inversion program based on a new class of information measures which are designed to control derivative values of the unknown densities

maximum entropy solution to ill posed inverse problems with approximately known operator we consider the linear inverse problem of reconstructing an unknown finite measure from a noisy observation of a generalized moment of defined as the integral of a continuous and bounded operator with respect to motivated by various applications we focus on the case where the operator is unknown instead only an approximation m to it is available an approximate maximum entropy solution to the inverse problem is introduced in the form of a minimizer of a convex functional subject to a sequence of convex constraints under several assumptions on the convex functional the convergence of the approximate solution is established

an efficient computational procedure for solving entropy optimization problems with infinitely many linear constraints a cutting plane type algorithm for solving entropy optimization problems with a finite number of variables but an infinite number of linear constraints is proposed in this paper in each iteration we solve a finite entropy optimization problem and add one more constraint the iterative process ends when an optimal solution is identified a convergence proof under some mild conditions is given an efficient implementation based on a dual approach is also included our preliminary computational experience confirms the efficiency of the proposed method and shows its potential for solving large scale problems

maximum entropy in the problem of moments the maximum entropy approach to the solution of underdetermined inverse problems is studied in detail in the context of the classical moment problem in important special cases such as the hausdorff moment problem we establish necessary and sufficient conditions for the existence of a maximum entropy solution and examine the convergence of the resulting sequence of approximations a number of explicit illustrations are presented in addition to some elementary examples we analyze the maximum entropy reconstruction of the density of states in harmonic solids and of dynamic correlation functions in quantum spin systems we also briefly indicate possible applications to the lee yang theory of ising models to the summation of divergent series and so on the general conclusion is that maximum entropy provides a valuable approximation scheme a serious competitor of traditional pade like procedures

a hybrid learning scheme combining em and masmod algorithms for fuzzy local linearization modeling fuzzy local linearization fll is a useful divide and conquer method for coping with complex problems such as modeling unknown nonlinear systems from data for state estimation and control based on a probabilistic interpretation of fll the paper proposes a hybrid learning scheme for fll modeling which uses a modified adaptive spline modeling masmod algorithm to construct the antecedent parts membership functions in the fll model and an expectation maximization em algorithm to parameterize the consequent parts local linear models the hybrid method not only has an approximation ability as good as most neuro fuzzy network models but also produces a parsimonious network structure gain from masmod and provides covariance information about the model error gain from em which is valuable in applications such as state estimation and control numerical examples on nonlinear time series analysis and nonlinear trajectory estimation using fll models are presented to validate the derived algorithm

a block spin construction of ondelettes part i lemarie functions using block spin assignments we construct anl2 orthonormal basis consisting of dyadic scalings and translates of just a finite number of functions these functions have exponential localization and for even values of a construction parameterm one can make them classcm 1 with vanishing moments up to orderm inclusive such a basis has an important application to phase cell cluster expansions in quantum field theory

the maximum entropy method on the mean applications to linear programming and superresolution in this paper we give two different results we propose new methods to solve classical optimization problems in linear programming we also obtain precise quantitative results for the superresolution phenomenon as observed earlier by practical searchers on specific algorithms the common background of our work is the generalized moment problem which is known to be connected with linear programming and superresolution we describe the maximum entropy method on the mean that provides solution to the problem and leads to computational criteria to decide the existence of solutions or not

on besov regularity of solutions to nonlinear elliptic partial differential equations abstract in this paper we study the regularity of the solutions of some nonlinear elliptic equations in kondratiev spaces on certain domains of polyhedral type general embedding theorems between kondratiev spaces and besov spaces will allow to avoid drawbacks to the standard sobolev regularity theory for those nonsmooth domains this will give us the opportunity to derive optimal rates for certain nonlinear approximation schemes

robustness infinitesimal neighborhoods and moment restrictions this paper is concerned with robust estimation under moment restrictions a moment restriction model is semiparametric and distribution free therefore it imposes mild assumptions yet it is reasonable to expect that the probability law of observations may have some deviations from the ideal distribution being modeled due to various factors such as measurement errors it is then sensible to seek an estimation procedure that is robust against slight perturbation in the probability measure that generates observations this paper considers local deviations within shrinking topological neighborhoods to develop its large sample theory so that both bias and variance matter asymptotically the main result shows that there exists a computationally convenient estimator that achieves optimal minimax robust properties it is semiparametrically efficient when the model assumption holds and at the same time it enjoys desirable robust properties when it does not

a construction of interpolating wavelets on invariant sets we introduce the concept of a refinable set relative to a family of contractive mappings on a metric space and demonstrate how such sets are useful to recursively construct interpolants which have a multiscale structure the notion of a refinable set parallels that of a refinable function which is the basis of wavelet construction the interpolation points we recursively generate from a refinable set by a set theoretic multiresolution are analogous to multiresolution for functions used in wavelet construction we then use this recursive structure for the points to construct multiscale interpolants several concrete examples of refinable sets which can be used for generating interpolatory wavelets are included

the lower semi continuous quasi convex hull as a normalized second conjugate let e be a real locally convex space and e its conjugate i e the linear space of all continuous linear functionals e i cq it is well known that if f e r u m or iffz a thenf e i qz 3 1 the lower semi continuous convex hull of f i e the greatest lower semi continuous convex functional majorized by f coincides with f the second convex conjugate off since this fact has important applications in optimization theory several attempts have been made to generalize it to f4 the lower semi continuous quasi convex hull off the greatest lower semi continuous quasi convex functional majorized by f i e to represent f as a second conjugate off in some sense thus alternatively to the a quasi conjugate f of greenberg and pierskalla 5 w ic we shall recall below crouzeix h h has introduced in 2 chapter ii for any f e r and a e r another a quasi conjugate of f as a certain functional f f e i and has shown that f sup f f and fq inf f f

local linear independence of refinable vectors of functions this paper is devoted to a study of local linear independence of refinable vectors of functions a vector of functions 1 r c ir r is said to be refinable if it satisfies the vector refinement equation x z s a 2x where a is a finitely supported sequence of r r matrices called the refinement mask a complete characterization for the local linear independence of the shifts of 1 r is given strictly in terms of the mask several examples are provided to illustrate the general theory this investigation is important for construction of wavelets on bounded domains and nonlinear approximation by wavelets local linear independence of refinable vectors of functions

probability distributions of assets inferred from option prices via the principle of maximum entropy this article revisits the maximum entropy algorithm in the context of recovering the probability distribution of an asset from the prices of finitely many associated european call options via partially finite convex programming we are able to provide an effective characterization of the constraint qualification under which the problem reduces to optimizing an explicit function in finitely many variables we also prove that the value or objective function is lower semicontinuous on its domain reference is given to a website which exploits these ideas for the efficient computation of the maximum entropy solution mes

asmo dan algorithm for adaptive spline modelling of observation data nonlinear system identification by modelling the underlying relationships in observation data is an important application area for artificial neural networks and other learning paradigms splines have been used for scattered data interpolation but the applications have mainly been restricted to low dimensional input spaces this paper describes asmod a new learning paradigm for higher dimensional data 3 based on b spline interpolation the models can be trained online and a method for step wise model refinement is applied during model training for gradually increasing the modelling capability until the desired or best possible accuracy is obtained for every refinement step a number of possible refinement actions are evaluated and the one that gives the highest improvement of the model accuracy is chosen the model structure is hence adapted to the modelling problem giving a model of small size and high accuracy asmod has very efficient implementations on serial computers the scheme has been

on the duality of a non convex optimization problem and the strong unicity constant in linear chebyshev approximation abstract the main goal of this paper is to define a dual problem for a special non convex global optimization problem and to show that a duality gap may not occur the proof is based on the convergence of a cutting plane algorithm the results can be applied to a dual characterization of the strong unicity constant in linear chebyshev approximation and the algorithm can be used to compute this constant

el inference for partially identified models large deviations optimality and bootstrap validity this paper addresses the issue of optimal inference for parameters that are partially identied in models with moment inequalities there currently exists a variety of inferential methods for use in this setting however the question of choosing optimally among contending procedures is unresolved in this paper i rst consider a canonical large deviations criterion for optimality and show that inference based on the empirical likelihood ratio statistic is optimal this nding is a direct analog to that in kitamura 2001 for moment equality models second i introduce a new empirical likelihood bootstrap that provides a valid resampling method for moment inequality models and overcomes the implementation challenges that arise as a result of non pivotal limit distributions lastly i analyze the nite sample properties of the proposed framework using monte carlo simulations the

convergence of best entropy estimates given a finite number of moments of an unknown density on a finite measure space the best entropy estimate that nonnegative density x with the given moments which minimizes the boltzmann shannon entropy i x x log x is considered a direct proof is given that i has the kadec property in l if yn converges weakly to 37 and i yn converges to i 37 then yn converges to 37 in norm as a corollary it is obtained that as the number of given moments increases the best entropy estimates converge in li norm to the best entropy estimate of the limiting problem which is simply in the determined case furthermore for classical moment problems on intervals with strictly positive and sufficiently smooth error bounds and uniform convergence are actually obtained

minimization of entropy functionals entropy i e convex integral functionals and extensions of these functionals are minimized on convex sets this paper is aimed at reducing as much as possible the assumptions on the constraint set primal attainment dual equalities dual attainment and characterizations of the minimizers are obtained with weak constraint qualifications these results improve several aspects of the literature on the subject

solution of the general moment problem via a one parameter imbedding this paper presents a computational theory for the general scalar moment problem the formalism is sufficiently general to encompass problems in sensor arrays with arbitrary geometry and dynamics and in nonuniform multidimensional sampling given a finite set of moments the theory provides a test for the existence of a positive measure which is consistent with such data at the same time the theory also provides a characterization of all such consistent positive measures it should be noted that classical results e g in the theory of the trigonometric moment problem hamburger stieljes nevanlinna pick interpolation etc are not applicable to the general setting sought herein where there is no natural shift operator in the space spanned by the integration kernels the centerpiece of the theory is a differential equation which depends on the given finite set of moments and on an arbitrary positive function spl psi which plays the role of a free parameter the differential equation has an exponentially attractive point of equilibrium if and only if there exists a consistent positive measure for each spl psi the fixed point determines a corresponding measure suitable choice of spl psi allows recovering any measure which is consistent with the data the fixed point of the differential equation corresponds to an extremum of an entropy like functional and the differential equation is constructed via an appropriate homotopy that follows changes in the lagrange multipliers from a convenient starting value to a value for the multipliers that corresponds to the given moments

multilevel preconditioning this paper is concerned with multilevel techniques for preconditioning linear systems arising from galerkin methods for elliptic boundary value problems a general estimate is derived which is based on the characterization of besov spaces in terms of weighted sequence norms related to corresponding multilevel expansions the result brings out clearly how the various ingredients of a typical multilevel setting affect the growth rate of the condition numbers in particular our analysis indicates how to realize even uniformly bounded condition numbers for example the general results are used to show that the bramble pasciak xu preconditioner for piecewise linear finite elements gives rise to uniformly bounded condition numbers even when the refinements of the underlying triangulations are highly nonuniform furthermore they are applied to a general multivariate setting of refinable shift invariant spaces in particular covering those induced by various types of wavelets

a general theory of dual optimization problems we construct a unified theory of dual optimization problems which encompasses as particular cases the known dual problems for each concept of dual problem we define first an unperturbational version from which we deduce via a certain scheme the corresponding perturbational version we generalize simultaneously the lagrangian and surrogate cases using coupling functionals we study the connections between dual problems and define lagrangian functionals for them we study the class of perturbation functionals which can be written as the upper sum of the primal objective functional h and a functional with values not depending on h

regular families of kernels for nonlinear approximation abstract this article studies sufficient conditions on families of approximating kernels which provide n term approximation errors from an associated nonlinear approximation space which match the best known orders of n term wavelet expansion these conditions provide a framework which encompasses some notable approximation kernels including splines cardinal functions and many radial basis functions such as the gaussians and general multiquadrics examples of such kernels are given to justify the criteria additionally the techniques involved allow for some new results on n term greedy interpolation of sobolev functions via radial basis functions

nonlinear black box modeling in system identification a unified overview a nonlinear black box structure for a dynamical system is a model structure that is prepared to describe virtually any nonlinear dynamics there has been considerable recent interest in this area with structures based on neural networks radial basis networks wavelet networks and hinging hyperplanes as well as wavelet transform based methods and models based on fuzzy sets and fuzzy rules this paper describes all these approaches in a common framework from a user s perspective it focuses on what are the common features in the different approaches the choices that have to be made and what considerations are relevant for a successful system identification application of these techniques it is pointed out that the nonlinear structures can be seen as a concatenation of a mapping form observed data to a regression vector and a nonlinear mapping from the regressor space to the output space these mappings are discussed separately the latter mapping is usually formed as a basis function expansion the basis functions are typically formed from one simple scalar function which is modified in terms of scale and location the expansion from the scalar argument to the regressor space is achieved by a radial or a ridge type approach basic techniques for estimating the parameters in the structures are criterion minimization as well as two step procedures where first the relevant basis functions are determined using data and then a linear least squares step to determine the coordinates of the function approximation a particular problem is to deal with the large number of potentially necessary parameters this is handled by making the number of used parameters considerably less than the number of offered parameters by regularization shrinking pruning or regressor selection

adaptive wavelet methods for elliptic operator equations convergence rates this paper is concerned with the construction and analysis of wavelet based adaptive algorithms for the numerical solution of elliptic equations these algorithms approximate the solution u of the equation by a linear combination of n wavelets therefore a benchmark for their performance is provided by the rate of best approximation to u by an arbitrary linear combination of n wavelets so called n term approximation which would be obtained by keeping the n largest wavelet coefficients of the real solution which of course is unknown the main result of the paper is the construction of an adaptive scheme which produces an approximation to u with error o n s in the energy norm whenever such a rate is possible by n term approximation the range of s 0 for which this holds is only limited by the approximation properties of the wavelets together with their ability to compress the elliptic operator moreover it is shown that the number of arithmetic operations needed to compute the approximate solution stays proportional to n the adaptive algorithm applies to a wide class of elliptic problems and wavelet bases the analysis in this paper puts forward new techniques for treating elliptic problems as well as the linear systems of equations that arise from the wavelet discretization

integral operators on sparse grids in this paper we are concerned with the construction and use of wavelet approximation spaces for the fast evaluation of integral expressions the spaces are based on biorthogonal anisotropic tensor product wavelets we introduce sparse grid hyperbolic cross approximation spaces which are adapted not only to the smoothness of the kernel but also to the norm in which the error is measured furthermore we introduce compression schemes for the corresponding discretizations numerical examples for the laplace equation with dirichlet boundary conditions and an additional integral term with a smooth kernel demonstrate the validity of our theoretical results