penalty barrier multiplier methods for convex programming problems we study a class of methods for solving convex programs which are based on nonquadratic augmented lagrangians for which the penalty parameters are functions of the multipliers this gives rise to lagrangians which are nonlinear in the multipliers each augmented lagrangian is specified by a choice of a penalty function varphi and a penalty updating function pi the requirements on varphi are mild and allow for the inclusion of most of the previously suggested augmented lagrangians more importantly a new type of penalty barrier function having a logarithmic branch glued to a quadratic branch is introduced and used to construct an efficient algorithm convergence of the algorithms is proved for the case of pi being a sublinear function of the dual multipliers the algorithms are tested on large scale quadratically constrained problems arising in structural optimization

lipschitz continuity of solutions of linear inequalities programs and complementarity problems it is shown that solutions of linear inequalities linear programs and certain linear complementarity problems e g those with p matrices or z matrices but not semidefinite matrices are lipschitz continuous with respect to changes in the right hand side data of the problem solutions of linear programs are not lipschitz continuous with respect to the coefficients of the objective function the lipschitz constant given here is a generalization of the role played by the norm of the inverse of a nonsingular matrix in bounding the perturbation of the solution of a system of equations in terms of a right hand side perturbation

separation of nonconvex sets with general augmenting functions in this paper we consider two geometric optimization problems that are dual to each other and characterize conditions under which the optimal values of the two problems are equal this characterization relies on establishing separation results for nonconvex sets using general concave surfaces defined in terms of convex augmenting functions we prove separation results for augmenting functions that are bounded from below unbounded augmenting functions and asymptotic augmenting functions

monotonic optimization problems and solution approaches problems of maximizing or minimizing monotonic functions of n variables under monotonic constraints are discussed a general framework for monotonic optimization is presented in which a key role is given to a property analogous to the separation property of convex sets the approach is applicable to a wide class of optimization problems including optimization problems dealing with functions representable as differences of increasing functions d i functions

numerical methods for solving a two dimensional variable order anomalous subdiffusion equation australian research council dp0559807 dp0986766 national natural science foundation of china 10271098 natural science foundation of fujian province 2009j01014

nonlinear proximal point algorithms using bregman functions with applications to convex programming a bregman function is a strictly convex differentiable function that induces a well behaved distance measure or d function on euclidean space this paper shows that for every bregman function there exists a nonlinear version of the proximal point algorithm and presents an accompanying convergence theory applying this generalization of the proximal point algorithm to convex programming one obtains the d function proximal minimization algorithm of censor and zenios and a wide variety of new multiplier methods these multiplier methods are different from those studied by kort and bertsekas and include nonquadratic variations on the proximal method of multipliers

a network of oscillators a model of neuronal behaviour capable of accounting for the oscillatory plateau and rebound properties of biological neurons is derived discussed and analysed the model is based on a piecewise linear form of the fitzhugh nagumo equations but reduces to a set of maps very similar to those of the hopfield model 1982 in particular the binary description of individual neurons and the well studied form of the synaptic current sigma jijsj are preserved although the model is capable of reproducing behaviours on the slow timescales characteristic of plateau and oscillation by coupling two model cells together a mutually inhibitory or half centred oscillator and an oscillator fixed phase follower pairs are constructed

regularity theory for a nonlinear fractional reaction diffusion equation abstract this paper is dedicated to the study of a nonlinear fractional reaction diffusion equation we analyze the behavior of the resolvent family associated with the problem in the scale of fractional power spaces associated with the laplace operator we ensure the existence and uniqueness of regular mild solutions to the problem in the l q setting furthermore we consider global existence or non continuation by a blow up of such solutions

perturbation analysis of a condition number for convex inequality systems and global error bounds for analytic systems in this paper several types of perturbations on a convex inequality system are considered and conditions are obtained for the system to be well conditioned under these types of perturbations where the well conditionedness of a convex inequality system is defined in terms of the uniform boundedness of condition numbers under a set of perturbations it is shown that certain types of perturbations can be used to characterize the well conditionedness of a convex inequality system in which either the system has a bounded solution set and satisfies the slater condition or an associated convex inequality system which defines the recession cone of the solution set for the system satisfies the slater condition finally sufficient conditions are given for the existence of a global error bound for an analytic system it is shown that such a global error bound always holds for any inequality system defined by finitely many convex analytic functions when the zero vector is in the relative interior of the domain of an associated convex conjugate function

interior gradient and epsilon subgradient descent methods for constrained convex minimization we extend epsilon subgradient descent methods for unconstrained nonsmooth convex minimization to constrained problems over polyhedral sets in particular over r p this is done by replacing the usual squared quadratic regularization term used in subgradient schemes by the logarithmic quadratic distancelike function recently introduced by the authors we then obtain interior e subgradient descent methods which allow us to provide a natural extension of bundle methods and polyak s subgradient projection methods for nonsmooth convex minimization furthermore similar extensions are considered for smooth constrained minimization to produce interior gradient descent methods global convergence as well as an improved global efficiency estimate are established for these methods within a unifying principle and minimal assumptions on the problem s data

nonlinear augmented lagrangian and duality theory in this paper a unified framework of a nonlinear augmented lagrangian dual problem is investigated for the primal problem of minimizing an extended real valued function by virtue of a nonlinear augmenting penalty function our framework is more general than the ones in the literature in the sense that our nonlinear augmenting penalty function is defined on an open set and that our assumptions are presented in terms of a substitution of the dual variable so our scheme includes barrier penalty functions and the weak peak at zero property as special cases by assuming that the increment of the nonlinear augmenting penalty function with respect to the penalty parameter satisfies a generalized peak at zero property necessary and sufficient conditions for the zero duality gap property are established and the existence of an exact penalty representation is obtained

the zero duality gap property and lower semicontinuity of the perturbation function we examine the validity of the zero duality gap properties for two important dual schemes a generalized augmented lagrangian dual scheme and a nonlinear lagrange type dual scheme the necessary and sufficient conditions for the zero duality gap property to hold are established in terms of the lower semicontinuity of the perturbation functions

approximate inertial manifolds for the kuramoto sivashinsky equation analysis and computations abstract we evaluate several alternative methods for the approximation of inertial manifolds for the one dimensional kuramoto sivashinsky equation kse a method motivated by the dynamics originally developed for the navier stokes equation is adapted for the kse rigorous error estimates are obtained and compared to those of other methods introduced in the literature formal relationships between these other methods and the one introduced here are established numerical bifurcation diagrams of the various approximate inertial forms for the kse are presented we discuss the correspondence between the rigorous error estimates and the accuracy of the computational results these methods can be adapted to other dissipative partial differential equations

a practical general approximation criterion for methods of multipliers based on bregman distances this paper demonstrates that for generalized methods of multipliers for convex programming based on bregman distance kernels including the classical quadratic method of multipliers the minimization of the augmented lagrangian can be truncated using a simple generally implementable stopping criterion based only on the norms of the primal iterate and the gradient or a subgradient of the augmented lagrangian at that iterate previous results in this and related areas have required conditions that are much harder to verify such as e optimality with respect to the augmented lagrangian or strong conditions on the convex program to be solved here only existence of a kkt pair is required and the convergence properties of the exact form of the method are preserved the key new element in the analysis is the use of a full conjugate duality framework as opposed to mainly examining the action of the method on the standard dual function of the convex program an existence result for the iterates stronger than those possible for the exact form of the algorithm is also included

a modified quasi boundary value method for ill posed problems in this paper we study a final value problem for first order abstract differential equation with positive self adjoint unbounded operator coefficient this problem is ill posed perturbing the final condition we obtain an approximate nonlocal problem depending on a small parameter we show that the approximate problems are well posed and that their solutions converge if and only if the original problem has a classical solution we also obtain estimates of the solutions of the approximate problems and a convergence result of these solutions finally we give explicit convergence rates

lagrangian regularization approach to constrained optimization problems abstract this paper proposes a new regularization approach referred to as the lagrangian regularization approach which aims to circumvent the nondifferentiability of a positively homogeneous function r m in a conceptual unconstrained reformulation for constrained optimization problems with the appropriate choices of regularizing function we obtain a family of smooth functions that include as special cases the existing penalty and barrier functions as such our approach can be used as an instrumental tool to resolve the nondifferentiability of r m as well as a unified way to construct penalty functions for convex programming cases we present its global convergence analysis

augmented lagrangian applied to convex quadratic problems an algorithm based on the augmented lagrangian method is proposed to solve convex quadratic programming problem the quadratic penalty is considered here hence the augmented lagrangian function is quadratic when applied to quadratic programming problem for this penalty we show that if the lagrangian function associated with the original problem is strict convex or convex then the hessian matrix of augmented lagrangian function is positive definite or positive semi definite numerical experiments are presented illustrating the performance of the algorithm for the cute test set

a unified augmented lagrangian approach to duality and exact penalization in this paper the existence of an optimal path and its convergence to the optimal set of a primal problem of minimizing an extended real valued function are established via a generalized augmented lagrangian and corresponding generalized augmented lagrangian problems in which no convexity is imposed on the augmenting function these results further imply a zero duality gap property between the primal problem and the generalized augmented lagrangian dual problem a necessary and sufficient condition for the exact penalty representation in the framework of a generalized augmented lagrangian is obtained in the context of constrained programs we show that generalized augmented lagrangians present a unified approach to several classes of exact penalization results some equivalences among exact penalization results are obtained

response characteristics of a low dimensional model neuron it is shown that a low dimensional model neuron with a response time constant smaller than the membrane time constant closely reproduces the activity and excitability behavior of a detailed conductance based model of hodgkin huxley type the fast response of the activity variable also makes it possible to reduce the model to a one dimensional model in particular for typical conditions as an example the reduction to a single variable model from a multivariable conductance based model of a neocortical pyramidal cell with somatic input is demonstrated the conditions for avoiding a spurious damped oscillatory response to a constant input are derived and it is shown that a limit cycle response cannot occur the capability of the low dimensional model to approximate higher dimensional models accurately makes it useful for describing complex dynamics of nets of interconnected neurons the simplicity of the model facilitates analytic studies elucidation of neurocomputational mechanisms and applications to large scale systems

kuramoto sivashinsky dynamics on the center unstable manifold this paper studies the dynamical behavior of solutions of the kuramoto sivashinsky partial differential equation with periodic boundary conditions on a spatial interval 0 h the length h is the bifurcation parameter and reduction is made to a two complex dimensional system on a local center unstable manifold near the second bifurcation point h 2 from the trivial solution the resulting o 2 equivariant system displays all the behavior found in high precision simulations of the partial differential equation near this bifurcation point in particular bifurcation sequences to stable traveling waves unstable modulated traveling waves and attracting heteroclinic cycles are reproduced qualitatively and quantitatively within 1 in the parameter range h 2 pm 20 a clear understanding of the global dynamical behavior in this region is thus obtained

some recent advances in theory and simulation of fractional diffusion processes to offer an insight into the rapidly developing theory of fractional diffusion processes we describe in some detail three topics of current interest i the well scaled passage to the limit from continuous time random walk under power law assumptions to space time fractional diffusion ii the asymptotic universality of the mittag leffler waiting time law in time fractional processes iii our method of parametric subordination for generating particle trajectories

nonlinear rescaling vs smoothing technique in convex optimization we introduce an alternative to the smoothing technique approach for constrained optimization as it turns out for any given smoothing function there exists a modification with particular properties we use the modification for nonlinear rescaling nr the constraints of a given constrained optimization problem into an equivalent set of constraints the constraints transformation is scaled by a vector of positive parameters the lagrangian for the equivalent problems is to the correspondent smoothing penalty functions as augmented lagrangian to the classical penalty function or mbfs to the barrier functions moreover the lagrangians for the equivalent problems combine the best properties of quadratic and nonquadratic augmented lagrangians and at the same time are free from their main drawbacks sequential unconstrained minimization of the lagrangian for the equivalent problem in primal space followed by both lagrange multipliers and scaling parameters update leads to a new class of nr multipliers methods which are equivalent to the interior quadratic prox methods for the dual problem we proved convergence and estimate the rate of convergence of the nr multipliers method under very mild assumptions on the input data we also estimate the rate of convergence under various assumptions on the input data in particular under the standard second order optimality conditions the nr method converges with q linear rate without unbounded increase of the scaling parameters which correspond to the active constraints we also established global quadratic convergence of the nr methods for linear programming with unique dual solution we provide numerical results which strongly support the theory

final value problem for nonlinear time fractional reaction diffusion equation with discrete data abstract in this paper we study the problem of finding the solution of a multi dimensional time fractional reaction diffusion equation with nonlinear source from the final value data we prove that the present problem is not well posed then regularized problems are constructed using the truncated expansion method in the case of two dimensional and the quasi boundary value method in the case of multi dimensional finally convergence rates of the regularized solutions are given and investigated numerically

recovery of multiple coefficients in a reaction diffusion equation abstract this paper considers the inverse problem of recovering both the unknown spatially dependent conductivity a x and the potential q x in a parabolic equation from overposed data consisting of the value of solution profiles taken at a later time t we show both uniqueness results and the convergence of an iteration scheme designed to recover these coefficients we also allow a more general setting in particular when the usual time derivative is replaced by one of fractional order and when the potential term is coupled with a known nonlinearity f of the form q x f u

convergence analysis of a proximal like minimization algorithm using bregman functions an alternative convergence proof of a proximal like minimization algorithm using bregman functions recently proposed by censor and zenios is presented the analysis allows the establishment of a global convergence rate of the algorithm expressed in terms of function values

an interior proximal algorithm and the exponential multiplier method for semidefinite programming we introduce an interior proximal algorithm for semidefinite optimization problems and establish its convergence properties we also study the corresponding dual algorithm leading to an exponential multiplier method for semidefinite programs potential applications and extensions are also discussed

a generalization of the proximal point algorithm the problem considered in this paper is to find a solution to the generalized equation 0 in t x y where t is a maximal monotone operator on the product h 1 times h 2 of two hilbert spaces h 1 and h 2 we give a generalization of the proximal map and the proximal point algorithm in which the proposed iterative procedure is based on just one variable applying to convex programming problems instead of adding a quadratic term for all variables as in the proximal point algorithm a quadratic term for a subset of variables is added this paper proves that under a mild assumption our algorithm has the same convergence properties as the regular proximal point algorithm

the global convergence of augmented lagrangian methods based on ncp function in constrained nonconvex optimization abstract in this paper we present the global convergence properties of the primal dual method using a class of augmented lagrangian functions based on ncp function for inequality constrained nonconvex optimization problems we construct four modified augmented lagrangian methods based on different algorithmic strategies we show that the convergence to a kkt point or a degenerate point of the original problem can be ensured without requiring the boundedness condition of the multiplier sequence

lagrange multipliers and optimality lagrange multipliers used to be viewed as auxiliary variables introduced in a problem of constrained minimization in order to write first order optimality conditions formally as a system of equations modern applications with their emphasis on numerical methods and more complicated side conditions than equations have demanded deeper understanding of the concept and how it fits into a larger theoretical picture a major line of research has been the nonsmooth geometry of one sided tangent and normal vectors to the set of points satisfying the given constraints another has been the game theoretic role of multiplier vectors as solutions to a dual problem interpretations as generalized derivatives of the optimal value with respect to problem parameters have also been explored lagrange multipliers are now being seen as arising from a general rule for the subdifferentiation of a nonsmooth objective function which allows black and white constraints to be replaced by penalty expressions this paper traces su

a survey on fuzzy fractional differential and optimal control nonlocal evolution equations abstract we survey some representative results on fuzzy fractional differential equations controllability approximate controllability optimal control and optimal feedback control for several different kinds of fractional evolution equations optimality and relaxation of multiple control problems described by nonlinear fractional differential equations with nonlocal control conditions in banach spaces are considered

existence of augmented lagrange multipliers reduction to exact penalty functions and localization principle in this article we present new general results on existence of augmented lagrange multipliers we define a penalty function associated with an augmented lagrangian and prove that under a certain growth assumption on the augmenting function an augmented lagrange multiplier exists if and only if this penalty function is exact we also develop a new general approach to the study of augmented lagrange multipliers called the localization principle the localization principle allows one to study the local behaviour of the augmented lagrangian near global optimal solutions of the initial optimization problem in order to prove the existence of augmented lagrange multipliers

augmented lagrangians in semi infinite programming we consider the class of semi infinite programming problems which became in recent years a powerful tool for the mathematical modeling of many real life problems in this paper we study an augmented lagrangian approach to semi infinite problems and present necessary and sufficient conditions for the existence of corresponding augmented lagrange multipliers furthermore we discuss two particular cases for the augmenting function the proximal lagrangian and the sharp lagrangian

an approximate solution for nonlinear backward parabolic equations we consider the backward parabolic equation ut au f t u t 0 t t u t g where a is a positive unbounded operator and f is a nonlinear function satisfying a lipschitz condition with an approximate datum g the problem is severely ill posed using the truncation method we propose a regularized solution which is the solution of a system of differential equations in finite dimensional subspaces according to some a priori assumptions on the regularity of the exact solution we obtain several explicit error estimates including an error estimate of holder type for all t 0 t an example on heat equations and numerical experiments are given

modeling multivariate time series on manifolds with skew radial basis functions we present an approach for constructing nonlinear empirical mappings from high dimensional domains to multivariate ranges we employ radial basis functions and skew radial basis functions for constructing a model using data that are potentially scattered or sparse the algorithm progresses iteratively adding a new function at each step to refine the model the placement of the functions is driven by a statistical hypothesis test that accounts for correlation in the multivariate range variables the test is applied on training and validation data and reveals nonstatistical or geometric structure when it fails at each step the added function is fit to data contained in a spatiotemporally defined local region to determine the parameters in particular the scale of the local model the scale of the function is determined by the zero crossings of the autocorrelation function of the residuals the model parameters and the number of basis functions are determined automatically from the given data and there is no need to initialize any ad hoc parameters save for the selection of the skew radial basis functions compactly supported skew radial basis functions are employed to improve model accuracy order and convergence properties the extension of the algorithm to higher dimensional ranges produces reduced order models by exploiting the existence of correlation in the range variable data structure is tested not just in a single time series but between all pairs of time series we illustrate the new methodologies using several illustrative problems including modeling data on manifolds and the prediction of chaotic time series

interior proximal and multiplier methods based on second order homogeneous kernels we study a class of interior proximal algorithms and nonquadratic multiplier methods for solving convex programs where the usual proximal quadratic term is replaced by an homogeneous functional of order two defined in terms of a convex function we prove under mild assumptions several new convergence results in both the primal and dual framework allowing also for approximate minimization in particular we introduce a new class of interior proximal methods which is globally convergent and allows for generating c multiplier methods with bounded hessians which exhibit strong convergence properties we also consider linearly constrained convex problems and establish global quadratic convergence rates results for linear programs we then study in detail a particular realization of these algorithms leading to a new class of logarithmic quadratic interior point algorithms which are shown to enjoy several attractive properties for solving constrained convex optimization problems

nonlinear rescaling and proximal like methods in convex optimization the nonlinear rescaling principle nrp consists of transforming the objective function and or the constraints of a given constrained optimization problem into another problem which is equivalent to the original one in the sense that their optimal set of solutions coincides a nonlinear transformation parameterized by a positive scalar parameter and based on a smooth sealing function is used to transform the constraints the methods based on nrp consist of sequential unconstrained minimization of the classical lagrangian for the equivalent problem followed by an explicit formula updating the lagrange multipliers we first show that the nrp leads naturally to proximal methods with an entropy like kernel which is defined by the conjugate of the scaling function and establish that the two methods are dually equivalent for convex constrained minimization problems we then study the convergence properties of the nonlinear rescaling algorithm and the corresponding entropy like proximal methods for convex constrained optimization problems special cases of the nonlinear rescaling algorithm are presented in particular a new class of exponential penalty modified barrier functions methods is introduced

dimension reduction of biological neuron models by artificial neural networks an artificial neural network approach to dimension reduction of dynamical systems is proposed and applied to conductance based neuron models networks with bottleneck layers of continuous time dynamical units could make a two dimensional model from the trajectories of the hodgkin huxley model and a three dimensional model from the trajectories of a six dimensional bursting neuron model nullcline analysis of these reduced models revealed the bifurcations of the dynamical system underlying firing and bursting behaviors

on saddle points of augmented lagrangians for constrained nonconvex optimization we present in this paper new results on the existence of saddle points of augmented lagrangian functions for constrained nonconvex optimization four classes of augmented lagrangian functions are considered the essentially quadratic augmented lagrangian the exponential type augmented lagrangian the modified barrier augmented lagrangian and the penalized exponential type augmented lagrangian we first show that under second order sufficiency conditions all these augmented lagrangian functions possess local saddle points we then prove that global saddle points of these augmented lagrangian functions exist under certain mild additional conditions the results obtained in this paper provide a theoretical foundation for the use of augmented lagrangians in constrained global optimization our findings also give new insights to the role played by augmented lagrangians in local duality theory of constrained nonconvex optimization

existence uniqueness and regularity for a class of semilinear stochastic volterra equations with multiplicative noise abstract we consider a class of semilinear volterra type stochastic evolution equation driven by multiplicative gaussian noise the memory kernel not necessarily analytic is such that the deterministic linear equation exhibits a parabolic character under appropriate lipschitz type and linear growth assumptions on the nonlinear terms we show that the unique mild solution is mean p holder continuous with values in an appropriate sobolev space depending on the kernel and the data in particular we obtain pathwise space time sobolev holder regularity of the solution together with a maximal type bound on the spatial sobolev norm as one of the main technical tools we establish a smoothing property of the derivative of the deterministic evolution operator family

message passing for graph structured linear programs proximal methods and rounding schemes the problem of computing a maximum a posteriori map configuration is a central computational challenge associated with markov random fields there has been some focus on tree based linear programming lp relaxations for the map problem this paper develops a family of super linearly convergent algorithms for solving these lps based on proximal minimization schemes using bregman divergences as with standard message passing on graphs the algorithms are distributed and exploit the underlying graphical structure and so scale well to large problems our algorithms have a double loop character with the outer loop corresponding to the proximal sequence and an inner loop of cyclic bregman projections used to compute each proximal update we establish convergence guarantees for our algorithms and illustrate their performance via some simulations we also develop two classes of rounding schemes deterministic and randomized for obtaining integral configurations from the lp solutions our deterministic rounding schemes use a re parameterization property of our algorithms so that when the lp solution is integral the map solution can be obtained even before the lp solver converges to the optimum we also propose graph structured randomized rounding schemes applicable to iterative lp solving algorithms in general we analyze the performance of and report simulations comparing these rounding schemes

circular nodes in neural networks in the usual construction of a neural network the individual nodes store and transmit real numbers that lie in an interval on the real line the values are often envisioned as amplitudes in this article we present a design for a circular node which is capable of storing and transmitting angular information we develop the forward and backward propagation formulas for a network containing circular nodes we show how the use of circular nodes may facilitate the characterization and parameterization of periodic phenomena in general we describe applications to constructing circular self maps periodic compression and one dimensional manifold decomposition we show that a circular node may be used to construct a homeomorphism between a trefoil knot in 3 and a unit circle we give an application with a network that encodes the dynamic system on the limit cycle of the kuramoto sivashinsky equation this is achieved by incorporating a circular node in the bottleneck layer of a three hidden layer bottleneck network architecture exploiting circular nodes systematically offers a neural network alternative to fourier series decomposition in approximating periodic or almost periodic functions

on dual convergence of the generalized proximal point method with bregman distances the use of generalized distances e g bregman distances instead of the euclidean one in the proximal point method for convex optimization allows for elimination of the inequality constraints from the subproblems in this paper we consider the proximal point method with bregman distances applied to linearly constrained convex optimization problems and study the behavior of the dual sequence obtained from the optimal multipliers of the linear constraints of each subproblem under rather general assumptions which cover most bregman distances of interest we obtain an ergodic convergence result namely that a sequence of weighted averages of the dual sequence converges to a specific point of the dual optimal set as an intermediate result we prove under the same assumptions that the dual central path generated by a large class of barriers including the generalized bregman distances converges to the same point

on the convergence of the exponential multiplier method for convex programming in this paper we analyze the exponential method of multipliers for convex constrained minimization problems which operates like the usual augmented lagrangian method except that it uses an exponential penalty function in place of the usual quadratic we also analyze a dual counterpart the entropy minimization algorithm which operates like the proximal minimization algorithm except that it uses a logarithmic entropy proximal term in place of a quadratic we strengthen substantially the available convergence results for these methods and we derive the convergence rate of these methods when applied to linear programs