convergence studies on iterative algorithms for image reconstruction we introduce a general iterative scheme for image reconstruction based on landweber s method in our configuration a sequential block iterative seqbi version can be readily formulated from a simultaneous block iterative simbi version and vice versa this provides a mechanism to derive new algorithms from known ones it is shown that some widely used iterative algorithms such as the algebraic reconstruction technique art simultaneous art sart cimmino s and the recently designed diagonal weighting and component averaging algorithms are special examples of the general scheme we prove convergence of the general scheme under conditions more general than assumed in earlier studies for its seqbi and simbi versions in the consistent and inconsistent cases respectively our results suggest automatic relaxation strategies for the seqbi and simbi versions and characterize the dependence of the limit image on the initial guess it is found that in all cases the limit is the sum of the minimum norm solution of a weighted least squares problem and an oblique projection of the initial image onto the null space of the system matrix

component averaged row projections a robust block parallel scheme for sparse linear systems a new method for the parallel solution of large sparse linear systems is introduced it proceeds by dividing the equations into blocks and operating in block parallel iterative mode i e all the blocks are processed in parallel and the partial results are merged to form the next iterate the new scheme performs kaczmarz row projections within the blocks and merges the results by certain component averaging operations hence it is called component averaged row projections or carp the system matrix can be general nonsymmetric and ill conditioned and the division into blocks is unrestricted for partial differential equations pdes if the blocks are domain based then only variables at the boundaries between domains are averaged thereby minimizing data transfer between processors carp is very robust its application to test cases of linear systems derived from pdes shows that it converges in difficult cases where state of the art methods fail it is also very memory efficient and exhibits an almost linear speedup ratio with efficiency greater than unity in some cases a formal proof of convergence is presented it is shown that the component averaging operations are equivalent to row projections in a certain superspace so the convergence properties of carp are identical to those of kaczmarz s algorithm in the superspace carp and its convergence proof also apply to the consistent convex feasibility problem

on projection algorithms for solving convex feasibility problems due to their extraordinary utility and broad applicability in many areas of classical mathematics and modern physical sciences most notably computerized tomography algorithms for solving convex feasibility problems continue to receive great attention to unify generalize and review some of these algorithms a very broad and flexible framework is investigated several crucial new concepts which allow a systematic discussion of questions on behaviour in general hilbert spaces and on the quality of convergence are brought out numerous examples are given

signal recovery by best feasible approximation the objective of set theoretical signal recovery is to find a feasible signal in the form of a point in the intersection of s of sets modeling the information available about the problem for problems in which the true signal is known to lie near a reference signal r the solution should not be any feasible point but one which best approximates r i e a projection of r onto s such a solution cannot be obtained by the feasibility algorithms currently in use e g the method of projections onto convex sets pocs and its offsprings methods for projecting a point onto the intersection of closed and convex sets in a hilbert space are introduced and applied to signal recovery by best feasible approximation of a reference signal these algorithms are closely related to the above projection methods to which they add little computational complexity

dual coordinate ascent methods for non strictly convex minimization we consider a dual method for solving non strictly convex programs possessing a certain separable structure this method may be viewed as a dual version of a block coordinate ascent method studied by auslender 1 section 6 we show that the decomposition methods of han 6 7 and the method of multipliers may be viewed as special cases of this method we also prove a convergence result for this method which can be applied to sharpen the available convergence results for han s methods

an iterative procedure for obtaining i projections onto the intersection of convex sets abstract a frequently occuring problem is to find that probability distribution pd lying within a convex set which minimizes the i divergence between it and given pd r this document proposes an iterative procedure which requires only that the set be convex and not necessarily linear which under general conditions will converge to the desired i projection

extensions of hildreth s row action method for quadratic programming an extended version of hildreth s iterative quadratic programming algorithm is presented geometrically interpreted and proved to produce a sequence of iterates that i converges to the solution and ii has an important intermediate optimality property this extended hildreth algorithm is cast into a new form which more pronouncedly brings out its primal dual nature the application of the algorithm may be governed by an index sequence which is more general than a cyclic sequence namely by an almost cyclic control and a sequence of relaxation parameters is incorporated without ruining convergence the algorithm is a row action method which is particularly suitable for handling large or huge and sparse systems

a successive projection method it is of both theoretical and practical interest to find the projection of a point to the intersection of a finite number of closed convex sets by a sequence of projections to the individual sets successively in this paper we study such a method and analyze its convergence properties a main feature of the method is its capability to decompose the projection problem into several small ones for some structured sparse problems these small subproblems can be solved independently and the presented method has a potential use in parallel computation

non linear reconstruction of delta sigma modulated signals randomized surrogate constraint decoding algorithm delta sigma a d conversion is a popular technique used to achieve high resolution data conversion for low to moderate bandwidth applications it is commonly accepted that the digital circuitry of the a d converters should perform a linear time invariant lti filtering on the output of the modulator in order to reconstruct the input signal however it has been shown that when higher order a d converters are operated at higher oversampling ratios osrs non linear reconstruction algorithms extract more information about the input signal than lti methods still use of non linear algorithms has been limited due to their complexity and stability issues two practical non linear methods presented in the literature are fast projection onto convex sets pocs and direct projection dp in this work we show that the non linear reconstruction process for modulated sequences can be treated as a linear feasibility problem lfp we then show that if the reconstruction process is defined as an lfp its implementation is less complex than in the case of the dp method we further describe how the lfp can be well conditioned and demonstrate the reconstruction process by using the surrogate constraint algorithm with random constraint selection the described algorithm considers a small number of constraints at a time which makes it suitable for serial implementation

on the behavior of subgradient projections methods for convex feasibility problems in euclidean spaces we study some methods of subgradient projections for solving a convex feasibility problem with general not necessarily hyperplanes or half spaces convex sets in the inconsistent case and propose a strategy that controls the relaxation parameters in a specific self adapting manner this strategy leaves enough user flexibility but gives a mathematical guarantee for the algorithm s behavior in the inconsistent case we present the numerical results of computational experiments that illustrate the computational advantage of the new method

markovian structures in biological sequence alignments abstract the alignment of multiple homologous biopolymer sequences is crucial in research on protein modeling and engineering molecular evolution and prediction in terms of both gene function and gene product structure in this article we provide a coherent view of the two recent models used for multiple sequence alignment the hidden markov model hmm and the block based motif model to develop a set of new algorithms that have both the sensitivity of the block based model and the flexibility of the hmm in particular we decompose the standard hmm into two components the insertion component which is captured by the so called propagation model and the deletion component which is described by a deletion vector such a decomposition serves as a basis for rational compromise between biological specificity and model flexibility furthermore we introduce a bayesian model selection criterion that in combination with the propagation model genetic algorithm and other computational aspects forms the cor

two generalizations of dykstra s cyclic projections algorithm dykstra s cyclic projections algorithm allows one to compute best approximations to any pointx in a hilbert space from the intersectionc lrci of a finite number of closed convex setsci by reducing it to a sequence of best approximation problems from theindividual setsci here we present two generalizations of this algorithm first we allow the number of setsci to beinfinite rather than finite secondly we allow arandom rather than cyclic ordering of the setsci

exact filtering for partially observed continuous time models summary the forward backward algorithm is an exact filtering algorithm which can efficiently calculate likelihoods and which can be used to simulate from posterior distributions using a simple result which relates gamma random variables with different rates we show how the forward backward algorithm can be used to calculate the distribution of a sum of gamma random variables and to simulate from their joint distribution given their sum one application is to calculating the density of the time of a specific event in a markov process as this time is the sum of exponentially distributed interevent times this enables us to apply the forward backward algorithm to a range of new problems we demonstrate our method on three problems calculating likelihoods and simulating allele frequencies under a non neutral population genetic model analysing a stochastic epidemic model and simulating speciation times in phylogenetics

computational acceleration of projection algorithms for the linear best approximation problem abstract this is an experimental computational account of projection algorithms for the linear best approximation problem we focus on the sequential and simultaneous versions of dykstra s algorithm and the halpern lions wittmann bauschke algorithm for the best approximation problem from a point to the intersection of closed convex sets in the euclidean space these algorithms employ different iterative approaches to reach the same goal but no mathematical connection has yet been found between their algorithmic schemes we compare these algorithms on linear best approximation test problems that we generate so that the solution will be known a priori and enable us to assess the relative computational merits of these algorithms for the simultaneous versions we present a new component averaging variant that substantially accelerates their initial behavior for sparse systems

on diagonally relaxed orthogonal projection methods we propose and study a block iterative projection method for solving linear equations and or inequalities the method allows diagonal componentwise relaxation in conjunction with orthogonal projections onto the individual hyperplanes of the system and is thus called diagonally relaxed orthogonal projections drop diagonal relaxation has proven useful in accelerating the initial convergence of simultaneous and block iterative projection algorithms but until now it was available only in conjunction with generalized oblique projections in which there is a special relation between the weighting and the oblique projections drop has been used by practitioners and in this paper a contribution to its convergence theory is provided the mathematical analysis is complemented by some experiments in image reconstruction from projections which illustrate the performance of drop

a simultaneous iterative method for computing projections on polyhedra a simultaneous version of hildreth s iterative algorithm for norm minimization over linear inequalities is presented proofs are given showing that the algorithm converges from any starting point to its projection on the linear constraints set in the feasible case and to the nearest least squares solution in the general case

new class of running wave solutions of the 2 1 dimensional sine gordon equation a new class of running wave solutions of the 2 1 dimensional sine gordon equation is investigated the obtained waves require two spatial dimensions for their propagation i e they generalize solutions of the 2 0 dimensional sine gordon equation the parameters of the waves strongly depend on the wave amplitude and there exist forbidden areas for the wavenumber and frequency the obtained solutions describe a new class of josephson waves whose velocity is smaller than the swihart velocity if omega 0 the running waves are reduced to the self consistent phase current and magnetic field distributions in a large two dimensional josephson junction the self restriction coefficient for the josephson current corresponding to one of the structures is calculated

an algorithm for restricted least squares regression abstract a commonly occurring problem in statistics is that of minimizing a least squares expression subject to side constraints here a simple iterative algorithm is presented and shown to converge to the desired solution several examples are presented including finding the closest concave convex function to a set of points and other general quadratic programming problems the dual problem to the basic problem is also discussed and a solution for it is given in terms of the algorithm finally extensions to expressions other than least squares are given

on the convergence of han s method for convex programming with quadratic objective we present an iterative method for minimizing strictly convex quadratic functions over the intersection of a finite number of convex sets the method consists in computing projections onto the individual sets simultaneously and the new iterate is a convex combination of those projections we give convergence proofs even for the inconsistent case i e when the intersection of the sets is empty

bayesian tree structured image modeling using wavelet domain hidden markov models wavelet domain hidden markov models have proven to be useful tools for statistical signal and image processing the hidden markov tree hmt model captures the key features of the joint probability density of the wavelet coefficients of real world data one potential drawback to the hmt framework is the need for computationally expensive iterative training to fit an hmt model to a given data set e g using the expectation maximization algorithm we greatly simplify the hmt model by exploiting the inherent self similarity of real world images the simplified model specifies the hmt parameters with just nine meta parameters independent of the size of the image and the number of wavelet scales we also introduce a bayesian universal hmt uhmt that fixes these nine parameters the uhmt requires no training of any kind while extremely simple we show using a series of image estimation denoising experiments that these new models retain nearly all of the key image structure modeled by the full hmt finally we propose a fast shift invariant hmt estimation algorithm that outperforms other wavelet based estimators in the current literature both visually and in mean square error

bayesian methods for hidden markov models markov chain monte carlo mcmc sampling strategies can be used to simulate hidden markov model hmm parameters from their posterior distribution given observed data some mcmc methods used in practice for computing likelihood conditional probabilities of hidden states and the most likely sequence of states can be improved by incorporating established recursive algorithms the most important of these is a set of forward backward recursions calculating conditional distributions of the hidden states given observed data and model parameters i show how to use the recursive algorithms in an mcmc context and demonstrate mathematical and empirical results showing a gibbs sampler using the forward backward recursions mixes more rapidly than another sampler often used for hmms iintroduce an augmented variables technique for obtaining unique state labels in hmms and finite mixture models i show how recursive computing allows the statistically efficient use of mcmc output when estimating the hidden states i

an exact gibbs sampler for the markov modulated poisson process a markov modulated poisson process mmpp is a poisson process whose intensity varies according to a markov process we present a novel technique for simulating from the exact distribution of a continuous time markov chain over an interval given the start and end states and the infinitesimal generator and use this to create a gibbs sampler which samples from the exact distribution of the hidden markov chain in an mmpp we apply the gibbs sampler to modelling the occurrence of a rare dna motif the chi site and to inferring regions of the genome with evidence of high or low intensities for occurrences of this site

a parallel algorithm for a class of convex programs a parallel algorithm is proposed in this paper for solving the problem min q x x in c 1 cap cdots cap c m where q is an uniformly convex function and c i are closed convex sets in r n in each iteration of the method we solve in parallel m independent subproblems each minimizing a definite quadratic function over an individual set c i the method has attractive convergence properties and can be implemented as parallel algorithms for tackling definite quadratic programs linear programs systems of linear equations and systems of generalized nonlinear inequalities

a weak to strong convergence principle for feje monotone methods in hilbert spaces we consider a wide class of iterative methods arising in numerical mathematics and optimization that are known to converge only weakly exploiting an idea originally proposed by haugazeau we present a simple modification of these methods that makes them strongly convergent without additional assumptions several applications are discussed

robust stopping criteria for dykstra s algorithm dykstra s algorithm is a suitable alternating projection scheme for solving the optimization problem of finding the closest point to one given in the intersection of a finite number of closed and convex sets it has been recently used in a wide variety of applications however in practice the commonly used stopping criteria are not robust and could stop the iterative process prematurely at a point that does not solve the optimization problem in this work we present a counterexample to illustrate the weakness of the commonly used criteria and then we develop robust stopping rules additional experimental results are shown to illustrate the advantages of this new stopping criteria including their associated computational cost

the approximation of fixed points of compositions of nonexpansive mappings in hilbert space determining fixed points of nonexpansive mappings is a frequent problem in mathematics and physical sciences an algorithm for finding common fixed points of nonexpansive mappings in hilbert space essentially due to halpern is analyzed the main theorem extends wittmann s recent work and partially generalizes a result by lions algorithms of this kind have been applied to the convex feasibility problem