ideal spatial adaptation by wavelet shrinkage summary with ideal spatial adaptation an oracle furnishes information about how best to adapt a spatially variable estimator whether piecewise constant piecewise polynomial variable knot spline or variable bandwidth kernel to the unknown function estimation with the aid of an oracle offers dramatic advantages over traditional linear estimation by nonadaptive kernels however it is a priori unclear whether such performance can be obtained by a procedure relying on the data alone we describe a new principle for spatially adaptive estimation selective wavelet reconstruction we show that variable knot spline fits and piecewise polynomial fits when equipped with an oracle to select the knots are not dramatically more powerful than selective wavelet reconstruction with an oracle we develop a practical spatially adaptive method riskshrink which works by shrinkage of empirical wavelet coefficients riskshrink mimics the performance of an oracle for selective wavelet reconstruction as well as it is possible to do so a new inequality in multivariate normal decision theory which we call the oracle inequality shows that attained performance differs from ideal performance by at most a factor of approximately 2 log n where n is the sample size moreover no estimator can give a better guarantee than this within the class of spatially adaptive procedures riskshrink is essentially optimal relying only on the data it comes within a factor log 2 n of the performance of piecewise polynomial and variableknot spline methods equipped with an oracle in contrast it is unknown how or if piecewise polynomial methods could be made to function this well when denied access to an oracle and forced to rely on data alone

hilbert schmidt lower bounds for estimators on matrix lie groups for atr deformable template representations of observed imagery model the variability of target pose via the actions of the matrix lie groups on rigid templates in this paper we study the construction of minimum mean squared error estimators on the special orthogonal group so n for pose estimation due to the nonflat geometry of so n the standard bayesian formulation of optimal estimators and their characteristics requires modifications by utilizing hilbert schmidt metric defined on gl n a larger group containing so n a mean squared criterion is defined on so n the hilbert schmidt estimate hse is defined to be a minimum mean squared error estimator restricted to so n the expected error associated with the hse is shown to be a lower bound called the hilbert schmidt bound hsb on the error incurred by any other estimator analysis and algorithms are presented for evaluating the hse and the hsb in cases of both ground based and airborne targets

a moving finite element method with error estimation and refinement for one dimensional time dependent partial differential equations we discuss a moving finite element method for solving vector systems of time dependent partial differential equations in one space dimension the mesh is moved so as to equidistribute the spatial component of the discretization error in h 1 we present a method of estimating this error by using p hierarchic finite elements the error estimate is also used in an adaptive mesh refinement procedure to give an algorithm that combines mesh movement and refinement we discretize the partial differential equations in space using a galerkin procedure with piecewise linear elements to approximate the solution and quadratic elements to estimate the error a system of ordinary differential equations for mesh velocities are used to control element motions we use existing software for stiff ordinary differential equations for the temporal integration of the solution the error estimate and the mesh motion computational results using a code based on our method are presented for several examples

diffusions for global optimizations we seek a global minimum of u 0 1 n to r the solution to d dt x t nabla u x t will find local minima the solution to dx t nabla u x t dt sqrt 2t dw t where w is standard n dimensional brownian motion and the boundaries are reflecting will concentrate near the global minima of u at least when temperature t is small the equilibrium distribution for x t is gibbs with density pi t x alpha exp u x t this suggests setting t t t downarrow 0 to find the global minima of u we give conditions on u x and t t such that the solution to dx t nabla u x t dt sqrt 2t dw t converges weakly to a distribution concentrated on the global minima of u

iterative image reconstruction algorithms based on cross entropy minimization the related problems of minimizing the functionals f x alpha kl y px 1 alpha kl p x and g x alpha kl px y 1 alpha kl x p respectively over the set of vectors x or 0 are considered kl a b is the cross entropy or kullback leibler distance between two nonnegative vectors a and b iterative algorithms for minimizing both functionals using the method of alternating projections are derived a simultaneous version of the multiplicative algebraic reconstruction technique mart algorithm called smart is introduced and its convergence is proved

analysis of multiresolution image denoising schemes using generalized gaussian and complexity priors research on universal and minimax wavelet shrinkage and thresholding methods has demonstrated near ideal estimation performance in various asymptotic frameworks however image processing practice has shown that universal thresholding methods are outperformed by simple bayesian estimators assuming independent wavelet coefficients and heavy tailed priors such as generalized gaussian distributions ggds in this paper we investigate various connections between shrinkage methods and maximum a posteriori map estimation using such priors in particular we state a simple condition under which map estimates are sparse we also introduce a new family of complexity priors based upon rissanen s universal prior on integers one particular estimator in this class outperforms conventional estimators based on earlier applications of the minimum description length mdl principle we develop analytical expressions for the shrinkage rules implied by ggd and complexity priors this allows us to show the equivalence between universal hard thresholding map estimation using a very heavy tailed ggd and mdl estimation using one of the new complexity priors theoretical analysis supported by numerous practical experiments shows the robustness of some of these estimates against mis specifications of the prior a basic concern in image processing applications

stability of moving mesh systems of partial differential equations moving mesh methods based on the equidistribution principle ep are studied from the viewpoint of stability of the moving mesh system of differential equations for fine spatial grids the moving mesh system inherits the stability of the original discretized partial differential equation pde unfortunately for some pdes the moving mesh methods require so many spatial grid points that they no longer appear to be practical failures and successes of the moving mesh method applied to three reaction diffusion problems are explained via an analysis of the stability and accuracy of the moving mesh pde

penalized maximum likelihood image reconstruction using space alternating generalized em algorithms most expectation maximization em type algorithms for penalized maximum likelihood image reconstruction converge slowly particularly when one incorporates additive background effects such as scatter random coincidences dark current or cosmic radiation in addition regularizing smoothness penalties or priors introduce parameter coupling rendering intractable the m steps of most em type algorithms this paper presents space alternating generalized em sage algorithms for image reconstruction which update the parameters sequentially using a sequence of small hidden data spaces rather than simultaneously using one large complete data space the sequential update decouples the m step so the maximization can typically be performed analytically we introduce new hidden data spaces that are less informative than the conventional complete data space for poisson data and that yield significant improvements in convergence rate this acceleration is due to statistical considerations not numerical overrelaxation methods so monotonic increases in the objective function are guaranteed we provide a general global convergence proof for sage methods with nonnegativity constraints

a row action alternative to the em algorithm for maximizing likelihood in emission tomography the maximum likelihood ml approach to estimating the radioactive distribution in the body cross section has become very popular among researchers in emission computed tomography ect since it has been shown to provide very good images compared to those produced with the conventional filtered backprojection fbp algorithm the expectation maximization em algorithm is an often used iterative approach for maximizing the poisson likelihood in ect because of its attractive theoretical and practical properties its major disadvantage is that due to its slow rate of convergence a large amount of computation is often required to achieve an acceptable image here the authors present a row action maximum likelihood algorithm ramla as an alternative to the em algorithm for maximizing the poisson likelihood in ect the authors deduce the convergence properties of this algorithm and demonstrate by way of computer simulations that the early iterates of ramla increase the poisson likelihood in ect at an order of magnitude faster that the standard em algorithm specifically the authors show that from the point of view of measuring total radionuclide uptake in simulated brain phantoms iterations 1 2 3 and 4 of ramla perform at least as well as iterations 45 60 70 and 80 respectively of em moreover the authors show that iterations 1 2 3 and 4 of ramla achieve comparable likelihood values as iterations 45 60 70 and 80 respectively of em the authors also present a modified version of a recent fast ordered subsets em os em algorithm and show that ramla is a special case of this modified os em furthermore the authors show that their modification converges to a ml solution whereas the standard os em does not

mesh independent spectra in the moving finite element equations we derive the moving finite element mfe equations for the solution of a scalar evolutionary equation in d space dimensions d geqq 1 and introduce the elementwise approach to mfe this approa

deblurring subject to nonnegativity constraints csiszar s i divergence is used as a discrepancy measure for deblurring subject to the constraint that all functions involved are nonnegative an iterative algorithm is proposed for minimizing this measure it is shown that every function in the sequence is nonnegative and the sequence converges monotonically to a global minimum other properties of the algorithm are shown including lower bounds on the improvement in the i divergence at each step of the algorithm and on the difference between the i difference at step k and at the limit point a method for regularizing the solution is proposed

general entropy criteria for inverse problems with applications to data compression pattern classification and cluster analysis minimum distance approaches are considered for the reconstruction of a real function from finitely many linear functional values an optimal class of distances satisfying an orthogonality condition analogous to that enjoyed by linear projections in hilbert space is derived these optimal distances are related to measures of distances between probability distributions recently introduced by c r rao and t k nayak 1985 and possess the geometric properties of cross entropy useful in speech and image compression pattern classification and cluster analysis several examples from spectrum estimation and image processing are discussed

moving mesh methods based on moving mesh partial differential equations several versions of a moving mesh method are developed based on a mesh spatial smoothing technique and on the moving mesh pdes derived in a previous paper these versions are quite simple and easy to program they are applied to three bench mark one dimensional problems which show different solution behaviour the numerical results clearly demonstrate that the present methods are capable of accurately tracking rapid spatial and temporal transitions

corrections for accidental coincidences and attenuation in maximum likelihood image reconstruction for positron emission tomography reconstruction procedures that account for attenuation in forming maximum likelihood estimates of activity distributions in positron emission tomography are extended to include regularization constraints and accidental coincidences a mathematical model is used for these effects the corrections are incorporated into the iterations of an expectation maximization algorithm for numerically producing the maximum likelihood estimate of the distribution of radioactivity within a patient the images reconstructed with this procedure are unbiased and exhibit lower variance than those reconstructed from precorrected data

multiple target direction of arrival tracking a new algorithm is presented for estimating the directions of arrival of signals from an unknown number of moving signal sources the parameter space is a countable union of cartesian products of the torus each product space corresponding to a different number of signals reaching the sensor array a bayesian posterior probability measure is defined on this parameter space by combining the sensor array manifold models with the von mises prior on source motion the estimates are generated empirically using a random sampling algorithm based on jump diffusion processes and the results are presented from an implementation on a dap510 massively parallel computer

iterative deblurring for ct metal artifact reduction iterative deblurring methods using the expectation maximization em formulation and the algebraic reconstruction technique art respectively are adapted for metal artifact reduction in medical computed tomography ct in experiments with synthetic noise free and additive noisy projection data of dental phantoms it is found that both simultaneous iterative algorithms produce superior image quality as compared to filtered backprojection after linearly fitting projection gaps furthermore the em type algorithm converges faster than the art type algorithm in terms of either the i divergence or euclidean distance between ideal and reprojected data in the authors simulation also for a given iteration number the em type deblurring method produces better image clarity but stronger noise than the art type reconstruction the computational complexity of em and art based iterative deblurring is essentially the same dominated by reprojection and backprojection relevant practical and theoretical issues are discussed

accelerated image reconstruction using ordered subsets of projection data the authors define ordered subset processing for standard algorithms such as expectation maximization em for image restoration from projections ordered subsets methods group projection data into an ordered sequence of subsets or blocks an iteration of ordered subsets em is defined as a single pass through all the subsets in each subset using the current estimate to initialize application of em with that data subset this approach is similar in concept to block kaczmarz methods introduced by eggermont et al 1981 for iterative reconstruction simultaneous iterative reconstruction sirt and multiplicative algebraic reconstruction mart techniques are well known special cases ordered subsets em os em provides a restoration imposing a natural positivity condition and with close links to the em algorithm os em is applicable in both single photon spect and positron emission tomography pet in simulation studies in spect the os em algorithm provides an order of magnitude acceleration over em with restoration quality maintained

a recursive algorithm for computing cramer rao type bounds on estimator covariance we give a recursive algorithm to calculate submatrices of the cramer rao cr matrix bound on the covariance of any unbiased estimator of a vector parameter spl theta our algorithm computes a sequence of lower bounds that converges monotonically to the cr bound with exponential speed of convergence the recursive algorithm uses an invertible splitting matrix to successively approximate the inverse fisher information matrix we present a statistical approach to selecting the splitting matrix based on a complete data incomplete data formulation similar to that of the well known em parameter estimation algorithm as a concrete illustration we consider image reconstruction from projections for emission computed tomography

stabilization of daes and invariant manifolds many methods have been proposed for the stabilization of higher index differential algebraic equations daes such methods often involve constraint differentiation and problem stabilization thus obtaining a stabilized index reduction a popular method is baumgarte stabilization but the choice of parameters to make it robust is unclear in practice here we explain why the baumgarte method may run into trouble we then show how to improve it we further develop a unifying theory for stabilization methods which includes many of the various techniques proposed in the literature our approach is to i consider stabilization of odes with invariants ii discretize the stabilizing term in a simple way generally different from the ode discretization and iii use orthogonal projections whenever possible the best methods thus obtained are related to methods of coordinate projection we discuss them and make concrete algorithmic suggestions

maximum likelihood reconstruction for emission tomography previous models for emission tomography et do not distinguish the physics of et from that of transmission tomography we give a more accurate general mathematical model for et where an unknown emission density x y z generates and is to be reconstructed from the number of counts n d in each of d detector units d within the model we give an algorithm for determining an estimate of which maximizes the probability p n of observing the actual detector count data n over all possible densities let independent poisson variables n b with unknown means b b 1 b represent the number of unobserved emissions in each of b boxes pixels partitioning an object containing an emitter suppose each emission in box b is detected in detector unit d with probability p b d d 1 d with p b d a one step transition matrix assumed known we observe the total number n n d of emissions in each detector unit d and want to estimate the unknown b b 1 b for each the observed data n has probability or likelihood p n the em algorithm of mathematical statistics starts with an initial estimate 0 and gives the following simple iterative procedure for obtaining a new estimate new from an old estimate old to obtain k k 1 2 new b old b dd 1 n d p b d old b p b d b 1 b

an algorithm for maximizing expected log investment return let the random stock market vector x geq 0 be drawn according to a known distribution function f x x in r m a log optimal portfolio b ast is any portfolio b achieving maximal expected log return w ast sup b e ln b t x where the supremum is over the simplex b geq 0 sum i 1 m b i 1 an algorithm is presented for finding b ast the algorithm consists of replacing the portfolio b by the expected portfolio b b i e b i x i b t x corresponding to the expected proportion of holdings in each stock after one market period the improvement in w b after each iteration is lower bounded by the kullback leibler information number d b b between the current and updated portfolios thus the algorithm monotonically improves the return w an upper bound on w ast is given in terms of the current portfolio and the gradient and the convergence of the algorithm is established

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

information theoretic analysis of interscale and intrascale dependencies between image wavelet coefficients this paper presents an information theoretic analysis of statistical dependencies between image wavelet coefficients the dependencies are measured using mutual information which has a fundamental relationship to data compression estimation and classification performance mutual information is computed analytically for several statistical image models and depends strongly on the choice of wavelet filters in the absence of an explicit statistical model a method is studied for reliably estimating mutual information from image data the validity of the model based and data driven approaches is assessed on representative real world photographic images our results are consistent with empirical observations that coding schemes exploiting inter and intrascale dependencies alone perform very well whereas taking both into account does not significantly improve coding performance a similar observation applies to other image processing applications

a vector quantization approach to universal noiseless coding and quantization a two stage code is a block code in which each block of data is coded in two stages the first stage codes the identity of a block code among a collection of codes and the second stage codes the data using the identified code the collection of codes may be noiseless codes fixed rate quantizers or variable rate quantizers we take a vector quantization approach to two stage coding in which the first stage code can be regarded as a vector quantizer that quantizes the input data of length n to one of a fixed collection of block codes we apply the generalized lloyd algorithm to the first stage quantizer using induced measures of rate and distortion to design locally optimal two stage codes on a source of medical images two stage variable rate vector quantizers designed in this way outperform standard one stage fixed rate vector quantizers by over 9 db the tail of the operational distortion rate function of the first stage quantizer determines the optimal rate of convergence of the redundancy of a universal sequence of two stage codes we show that there exist two stage universal noiseless codes fixed rate quantizers and variable rate quantizers whose per letter rate and distortion redundancies converge to zero as k 2 n sup 1 log n when the universe of sources has finite dimension k this extends the achievability part of rissanen s theorem from universal noiseless codes to universal quantizers further we show that the redundancies converge as o n sup 1 when the universe of sources is countable and as o n sup 1 spl epsiv when the universe of sources is infinite dimensional under appropriate conditions

spectrum estimation from quantum limited interferograms a quantitative model for interferogram data collected in a quantum limited hyperspectral imaging system is derived this model accounts for the geometry of the interferometer the poisson noise and the parameterization of the mean of the noise in terms of the autocorrelation function of the incident optical signal the crame spl acute r rao bound on the variance of unbiased spectrum estimates is derived and provides an explanation for what is often called the multiplex disadvantage in interferometer based methods three spectrum estimation algorithms are studied maximum likelihood via the expectation maximization em algorithm least squares ls and the fast fourier transform fft with data precorrection extensive simulation results reveal advantages and disadvantages with all three methods in different signal to noise ratio snr regimes

moving finite elements i we present new and general numerical methods for dealing with problems whose solutions develop sharp transition layers or near shocks these methods allow many nodes automatically to concentrate in the critical regions and move with them for clarity of exposition we concentrate on the space of piecewise linear functions with movable nodes with burgers equation as our test equation but the generalization to much more general spaces and equations including even certain previous moving vorticity blobs of the first author and s doss for the navier stokes equations becomes clear in this paper we present the theoretical and computational details of our scheme along with computational trial runs for burgers equation showing that the nodes do concentrate sharply and move with the shocks as desired the conclusiveness of these preliminary numerical trials is marred somewhat by the fact that we never successfully debugged a newton s method for our implicit stiff ode solver and were thus limited to ver

lower bounds for parametric estimation with constraints a chapman robbins form of the barankin bound is used to derive a multiparameter cramer rao cr type lower bound on estimator error covariance when the parameter theta in r sup n is constrained to lie in a subset of the parameter space a simple form for the constrained cr bound is obtained when the constraint set theta sub c can be expressed as a smooth functional inequality constraint it is shown that the constrained cr bound is identical to the unconstrained cr bound at the regular points of theta sub c i e where no equality constraints are active on the other hand at those points theta in theta sub c where pure equality constraints are active the full rank fisher information matrix in the unconstrained cr bound must be replaced by a rank reduced fisher information matrix obtained as a projection of the full rank fisher matrix onto the tangent hyperplane of the full rank fisher matrix onto the tangent hyperplane of the constraint set at theta a necessary and sufficient condition involving the forms of the constraint and the likelihood function is given for the bound to be achievable and examples for which the bound is achieved are presented in addition to providing a useful generalization of the cr bound the results permit analysis of the gain in achievable mse performance due to the imposition of particular constraints on the parameter space without the need for a global reparameterization

radio astronomical imaging in the presence of strong radio interference radio astronomical observations are increasingly contaminated by interference and suppression techniques become essential a powerful candidate for interference mitigation is adaptive spatial filtering we study the effect of spatial filtering techniques on radio astronomical imaging current deconvolution procedures such as clean are shown to be unsuitable for spatially filtered data and the necessary corrections are derived to that end we reformulate the imaging deconvolution calibration process as a sequential estimation of the locations of astronomical sources this not only leads to an extended clean algorithm but also the formulation allows the insertion of other array signal processing techniques for direction finding and gives estimates of the expected image quality and the amount of interference suppression that can be achieved finally a maximum likelihood ml procedure for the imaging is derived and an approximate ml image formation technique is proposed to overcome the computational burden involved some of the effects of the new algorithms are shown in simulated images

a two dimensional mesh moving technique for time dependent partial differential equations abstract we discuss an adaptive mesh moving technique that can be used with a finite difference or finite element scheme to solve initial boundary value problems for systems of partial differential equations in two space dimensions and time the mesh moving technique is based on an algebraic node movement function determined from the geometry and propagation of regions having significant discretization error indicators our procedure is designed to be flexible so that it can be used with many existing finite difference and finite element methods to test the mesh moving algorithm we implemented it in a system code with an initial mesh generator and a maccormack finite difference scheme on quadrilateral cells for hyperbolic vector systems of conservation laws results are presented for several computational examples the moving mesch scheme reduces dispersive errors near shocks and wave fronts and thereby reduces grid requirements necessary to compute accurate solutions while increasing computational efficiency

information theoretic image formation the emergent role of information theory in image formation is surveyed unlike the subject of information theoretic communication theory information theoretic imaging is far from a mature subject the possible role of information theory in problems of image formation is to provide a rigorous framework for defining the imaging problem for defining measures of optimality used to form estimates of images for addressing issues associated with the development of algorithms based on these optimality criteria and for quantifying the quality of the approximations the definition of the imaging problem consists of an appropriate model for the data and an appropriate model for the reproduction space which is the space within which image estimates take values each problem statement has an associated optimality criterion that measures the overall quality of an estimate the optimality criteria include maximizing the likelihood function and minimizing mean squared error for stochastic problems and minimizing squared error and discrimination for deterministic problems the development of algorithms is closely tied to the definition of the imaging problem and the associated optimality criterion algorithms with a strong information theoretic motivation are obtained by the method of expectation maximization related alternating minimization algorithms are discussed in quantifying the quality of approximations global and local measures are discussed global measures include the mean squared error and discrimination between an estimate and the truth and probability of error for recognition or hypothesis testing problems local measures include fisher information

roughness penalties on finite domains a class of penalty functions for use in estimation and image regularization is proposed these penalty functions are defined for vectors whose indexes are locations in a finite lattice as the discrepancy between the vector and a shifted version of itself after motivating this class of penalty functions their relationship to markov random field priors is explored one of the penalty functions proposed a divergence roughness penalty is shown to be a discretization of a penalty proposed by good and gaskins 1971 for use in density estimation one potential use in estimation problems is explored an iterative algorithm that takes advantage of induced neighborhood structures is proposed and convergence of the algorithm is proven under specified conditions examples in emission tomographic imaging and radar imaging are given

a statistical multiscale framework for poisson inverse problems this paper describes a statistical multiscale modeling and analysis framework for linear inverse problems involving poisson data the framework itself is founded upon a multiscale analysis associated with recursive partitioning of the underlying intensity a corresponding multiscale factorization of the likelihood induced by this analysis and a choice of prior probability distribution made to match this factorization by modeling the splits in the underlying partition the class of priors used here has the interesting feature that the noninformative member yields the traditional maximum likelihood solution other choices are made to reflect prior belief as to the smoothness of the unknown intensity adopting the expectation maximization em algorithm for use in computing the maximum a posteriori map estimate corresponding to our model we find that our model permits remarkably simple closed form expressions for the em update equations the behavior of our em algorithm is examined and it is shown that convergence to the global map estimate can be guaranteed applications in emission computed tomography and astronomical energy spectral analysis demonstrate the potential of the new approach

stability of computational methods for constrained dynamics systems many methods have been proposed for numerically integrating the differential algebraic systems arising from the euler lagrange equations for constrained motion these are based on various problem formulations and discretizations we offer a critical evaluation of these methods from the standpoint of stability nconsidering a linear model we first give conditions under which the differential algebraic problem is well conditioned this involves the concept of an essential underlying ode we review a variety of reformulations which have been proposed in the literature and show that most of them preserve the well conditioning of the original problem then we consider stiff and nonstiff discretizations of such reformulated models in some cases the same implicit discretization may behave in a very different way when applied to different problem formulations acting as a stiff integrator on some formulations and as a nonstiff integrator on others we present the approach of projected invariants as a method for yielding problem reformulations which are desirable in this sense

a method of sieves for multiresolution spectrum estimation and radar imaging a method of sieves using splines is proposed for regularizing maximum likelihood estimates of power spectra this method has several important properties including the flexibility to be used at multiple resolution levels the resolution level is defined in terms of the support of the polynomial b splines used an expression for the optimal rate of growth of the sieve is derived using a discrepancy measure derived from the kullback leibler divergence of parameterized density functions while the sieves may be defined on nonuniform grids in the case of uniform grids the optimal sieve size corresponds to an optimal resolution iterative algorithms for obtaining the maximum likelihood sieve estimates are derived applications to spectrum estimation and radar imaging are proposed

moving finite element methods for evolutionary problems ii applications abstract in this the second of two papers on the subject we present applications of the moving finite element method to a number of test problems key features are linear elements a direct approach to parallelism and node overtaking avoiding penalty functions rapid inversion of the mass matrix by preconditioned conjugate gradients and explicit euler time stepping the resulting codes are fast and efficient and are able to follow fronts and similar features with great accuracy the paper includes a substantial section on changes of dependent variable and front tracking techniques for non linear diffusion problems test problems include non linear hyperbolic conservation laws and non linear parabolic equations in one and two dimensions

space alternating generalized expectation maximization algorithm the expectation maximization em method can facilitate maximizing likelihood functions that arise in statistical estimation problems in the classical em paradigm one iteratively maximizes the conditional log likelihood of a single unobservable complete data space rather than maximizing the intractable likelihood function for the measured or incomplete data em algorithms update all parameters simultaneously which has two drawbacks 1 slow convergence and 2 difficult maximization steps due to coupling when smoothness penalties are used the paper describes the space alternating generalized em sage method which updates the parameters sequentially by alternating between several small hidden data spaces defined by the algorithm designer the authors prove that the sequence of estimates monotonically increases the penalized likelihood objective derive asymptotic convergence rates and provide sufficient conditions for monotone convergence in norm two signal processing applications illustrate the method estimation of superimposed signals in gaussian noise and image reconstruction from poisson measurements in both applications the sage algorithms easily accommodate smoothness penalties and converge faster than the em algorithms

conditional mean estimation via jump diffusion processes in multiple target tracking recognition a new algorithm is presented for generating the conditional mean estimates of functions of target positions orientations and type in recognition and tracking of an unknown number of targets and target types taking a bayesian approach a posterior measure is defined on the tracking target parameter space by combining a narrowband sensor array manifold model with a high resolution imaging model and a prior based on airplane dynamics the newtonian force equations governing rigid body dynamics are utilized to form the prior density on airplane motion the conditional mean estimates are generated using a random sampling algorithm based on jump diffusion processes for empirically generating mmse estimates of functions of these random target positions orientations and type under the posterior measure results are presented on target tracking and identification from an implementation of the algorithm on a networked silicon graphics workstation and decmpp maspar parallel machine

on the convergence properties of the em algorithm two convergence aspects of the em algorithm are studied i does the em algorithm find a local maximum or a stationary value of the incomplete data likelihood function ii does the sequence of parameter estimates generated by em converge several convergence results are obtained under conditions that are applicable to many practical situations two useful special cases are a if the unobserved complete data specification can be described by a curved exponential family with compact parameter space all the limit points of any em sequence are stationary points of the likelihood function b if the likelihood function is unimodal and a certain differentiability condition is satisfied then any em sequence converges to the unique maximum likelihood estimate a list of key properties of the algorithm is included

the discrimination power of projection pursuit with different density estimators summary we explore the properties of projection pursuit discriminant analysis this discriminant method is very powerful but relies heavily on a univariate density estimate we show that the procedure based on wavelets maintains the same rate of convergence as with univariate wavelet density estimation we also show the bayes risk strong consistency of both the kernel and wavelet based methods simulated data and real data concerning character recognition show that the method is effective and robust against the curse of dimensionality the wavelet alternative seems more likely than the kernel counterpart to find an interesting projection wavelets are often criticised for giving too wiggly an estimate and for being too localised to give good global properties in the above context these potential drawbacks do not weaken the method but the use of wavelets seems to enhance it a multiple projection generalisation is also considered

inversion of large support ill posed linear operators using a piecewise gaussian mrf we propose a method for the reconstruction of signals and images observed partially through a linear operator with a large support e g a fourier transform on a sparse set this inverse problem is ill posed and we resolve it by incorporating the prior information that the reconstructed objects are composed of smooth regions separated by sharp transitions this feature is modeled by a piecewise gaussian pg markov random field mrf known also as the weak string in one dimension and the weak membrane in two dimensions the reconstruction is defined as the maximum a posteriori estimate the prerequisite for the use of such a prior is the success of the optimization stage the posterior energy corresponding to a pg mrf is generally multimodal and its minimization is particularly problematic in this context general forms of simulated annealing rapidly become intractable when the observation operator extends over a large support global optimization is dealt with by extending the graduated nonconvexity gnc algorithm to ill posed linear inverse problems gnc has been pioneered by blake and zisserman 1987 in the field of image segmentation the resulting algorithm is mathematically suboptimal but it is seen to be very efficient in practice we show that the original gnc does not correctly apply to ill posed problems our extension is based on a proper theoretical analysis which provides further insight into the gnc the performance of the proposed algorithm is corroborated by a synthetic example in the area of diffraction tomography

image restoration by the method of convex projections part 1ߞtheory a projection operator onto a closed convex set in hilbert space is one of the few examples of a nonlinear map that can be defined in simple abstract terms moreover it minimizes distance and is nonexpansive and therefore shares two of the more important properties of ordinary linear orthogonal projections onto closed linear manifolds in this paper we exploit the properties of these operators to develop several iterative algorithms for image restoration from partial data which permit any number of nonlinear constraints of a certain type to be subsumed automatically their common conceptual basis is as follows every known property of an original image f is envisaged as restricting it to lie in a well defined closed convex set thus m such properties place f in the intersection e0 ei of the corresponding closed convex sets e1 e2 em given only the projection operators pi onto the individual ei s i 1 m we restore f by recursive means clearly in this approach the realization of the pi s in a hilbert space setting is one of the major synthesis problems section i describes the geometrical significance of the three main theorems in considerable detail and most of the underlying ideas are illustrated with the aid of simple diagrams section ii presents rules for the numerical implementation of 11 specific projection operators which are found to occur frequently in many signal processing applications and the appendix contains proofs of all the major results

recursive algorithms for computing the cramer rao bound computation of the cramer rao bound crb on estimator variance requires the inverse or the pseudo inverse fisher information matrix fim direct matrix inversion can be computationally intractable when the number of unknown parameters is large in this correspondence we compare several iterative methods for approximating the crb using matrix splitting and preconditioned conjugate gradient algorithms for a large class of inverse problems we show that nonmonotone gauss seidel and preconditioned conjugate gradient algorithms require significantly fewer flops for convergence than monotone bound preserving algorithms

stability analysis of the equilibrium of a constrained mechanical system this paper studies the local stability properties of the motion of mechanical systems with holonomic constraints a set of linear differential algebraic equations is used to characterize local constrained motion and to determine local stability properties of an equilibrium of the nonlinear differential algebraic equations stability conditions are developed for linear differential algebraic equations examples are shown to demonstrate that the constraint functions play an important role in stability properties of an equilibrium conditions are given that guarantee that constraints will not affect local stability properties

image restoration by the method of convex projections part 2 applications and numerical results the image restoration theory discussed in a previous paper by youla and webb 1 is applied to a simulated image and the results compared with the well known method known as the gerchberg papoulis algorithm the results show that the method of image restoration by projection onto convex sets by providing a convenient technique for utilizing a priori information performs significantly better than the gerchberg papoulis method

the moving finite element method applications to general partial differential equations with multiple large gradients abstract the moving finite element mfe method has been reduced to practice in the automatic solution program dyla for general systems of transient partial differential equations pdes in 1 d several test examples are presented which illustrate the unique node movement and systematic control features which are intrinsic in the mfe method computational dilemmas of numerical diffusion gibbs overshooting and undershooting zone tangling and grid remap or re connection aliasing which occur frequently in conventional pde methods are essentially eliminated in the mfe method arbitrarily large gradients or shocks can be solved with extremely high resolution and accuracy for non coincident or even counterdirected propagating wavefronts boundary layers of arbitrarily small dimensions are solved with high accuracy simultaneously with the large scale structures in reactive and non reactive fluid calculations the mfe method requires a small fraction of the grid nodes which are used in conventional pde solution methods because the nodes migrate continuously and systematically to those positions where they are most needed in order to yield accurate pde solutions on entire problem domains courant friedrichs lewy time step limits are exceeded by wide margins by factors of two to several thousand finally the extension of the mfe method to 2 d is briefly discussed

order parameters for detecting target curves in images when does high level knowledge help many problems in vision can be formulated as bayesian inference it is important to determine the accuracy of these inferences and how they depend on the problem domain in this paper we provide a theoretical framework based on bayesian decision theory which involves evaluating performance based on an ensemble of problem instances we pay special attention to the task of detecting a target in the presence of background clutter this framework is then used to analyze the detectability of curves in images we restrict ourselves to the case where the probability models are ergodic both for the geometry of the curve and for the imaging these restrictions enable us to use techniques from large deviation theory to simplify the analysis we show that the detectability of curves depend on a parameter k which is a function of the probability distributions characterizing the problem at critical values of k the target becomes impossible to detect on average our framework also enables us to determine whether a simpler approximate model is sufficient to detect the target curve and hence clarify how much information is required to perform specific tasks these results generalize our previous work yuille a l and coughlan j m 2000 pattern analysis and machine intelligence pami 22 2 160 173 by placing it in a bayesian decision theory framework by extending the class of probability models which can be analyzed and by analysing the case where approximate models are used for inference

a statistical model for positron emission tomography abstract positron emission tomography pet still in its research stages is a technique that promises to open new medical frontiers by enabling physicians to study the metabolic activity of the body in a pictorial manner much as in x ray transmission tomography and other modes of computerized tomography the quality of the reconstructed image in pet is very sensitive to the mathematical algorithm to be used for reconstruction in this article we tailor a mathematical model to the physics of positron emissions and we use the model to describe the basic image reconstruction problem of pet as a standard problem in statistical estimation from incomplete data we describe various estimation procedures such as the maximum likelihood ml method using the em algorithm the method of moments and the least squares method a computer simulation of a pet experiment is then used to demonstrate the ml and the least squares reconstructions the main purposes of this article are to report on what we believe is an

alternating minimization algorithms for transmission tomography a family of alternating minimization algorithms for finding maximum likelihood estimates of attenuation functions in transmission x ray tomography is described the model from which the algorithms are derived includes polyenergetic photon spectra background events and nonideal point spread functions the maximum likelihood image reconstruction problem is reformulated as a double minimization of the i divergence a novel application of the convex decomposition lemma results in an alternating minimization algorithm that monotonically decreases the objective function each step of the minimization is in closed form the family of algorithms includes variations that use ordered subset techniques for increasing the speed of convergence simulations demonstrate the ability to correct the cupping artifact due to beam hardening and the ability to reduce streaking artifacts that arise from beam hardening and background events

a multiscale relaxation algorithm for snr maximization in nonorthogonal subband coding develops a technique for improving the applicability of complete nonorthogonal multiresolution transforms to image coding as is well known the l sup 2 norm of the quantization errors is not preserved by nonorthogonal transforms so the l sup 2 reconstruction error may be unacceptably large however given the quantizers and synthesis filters the authors show that this artifact can be eliminated by formulating the coding problem as that of minimizing the l sup 2 reconstruction error over the set of possible encoded images with this new formulation the coding problem becomes a high dimensional discrete optimization problem and features a coupling between the redundancy removing and quantization operations a practical solution to the optimization problem is presented in the form of a multiscale relaxation algorithm using inter and intrascale quantization noise feedback filters bounds on the coding gain over the standard coding technique are derived a simple extension of the algorithm allows for the use of a weighted l sup 2 error criterion and deadband non mmse quantizers experiments using biorthogonal spline filter banks demonstrate appreciable snr gains over the standard coding technique and comparable visual improvements