rates of convergence for a class of global stochastic optimization algorithms inspired and motivated by the recent advances in simulated annealing algorithms this paper analyzes the convergence rates of a class of recursive algorithms for global optimization via monte carlo methods by using perturbed liapunov function methods stability results of the algorithms are established then the rates of convergence are ascertained by examining the asymptotic properties of suitably scaled estimation error sequences

global output feedback stabilisation of a class of stochastic systems with unknown growth rate this paper is concerned with the global output feedback stabilisation problem for a class of stochastic systems with unknown growth rate the output of the system under consideration contains unkno

global finite time stabilisation for a class of stochastic high order time varying nonlinear systems abstractin this paper the problem of global finite time stabilisation by state feedback is investigated for a class of stochastic high order nonlinear systems with time varying nonlinearities based on the generalised stochastic lyapunov theorem on finite time stability and by skillfully using the method of adding a power integrator a continuous state feedback controller is successfully constructed to guarantee that the closed loop system is globally finite time stable in probability finally several simulation examples are provided to illustrate the effectiveness of the proposed method

analyzing convergence and rates of convergence of particle swarm optimization algorithms using stochastic approximation methods recently much progress has been made on particle swarm optimization pso a number of works have been devoted to analyzing the convergence of the underlying algorithms nevertheless in most cases rather simplified hypotheses are used for example it often assumes that the swarm has only one particle in addition more often than not the variables and the points of attraction are assumed to remain constant throughout the optimization process in reality such assumptions are often violated moreover there are no rigorous rates of convergence results available to date for the particle swarm to the best of our knowledge in this paper we consider a general form of pso algorithms and analyze asymptotic properties of the algorithms using stochastic approximation methods we introduce four coefficients and rewrite the pso procedure as a stochastic approximation type iterative algorithm then we analyze its convergence using weak convergence method it is proved that a suitably scaled sequence of swarms converge to the solution of an ordinary differential equation we also establish certain stability results moreover convergence rates are ascertained by using weak convergence method a centered and scaled sequence of the estimation errors is shown to have a diffusion limit

finite time output feedback control for a class of stochastic low order nonlinear systems abstractthis paper discusses the problem of finite time stabilisation for a class of stochastic low order nonlinear systems via output feedback by generalising the adding a power integrator technique constructing an implementable reduced order observer and using the stochastic finite time stability criterion a finite time output feedback controller is presented to guarantee that the closed loop system is finite time stable in probability a simulation example is provided to verify the effectiveness of the proposed design method

lumping based equivalences in markovian automata algorithms and applications to product form analyses abstract in this paper we consider two relations over stochastic automata named lumpable bisimulation and exact equivalence that induce a strong and an exact lumping respectively on the underlying markov chains we show that an exact equivalence over the states of a non synchronising automaton is indeed a lumpable bisimulation for the corresponding reversed automaton and then it induces a strong lumping on the time reversed markov chain underlying the model this property allows us to prove that the class of quasi reversible models is closed under exact equivalence quasi reversibility is a pivotal property to study product form models hence exact equivalence turns out to be a theoretical tool to prove the product form of models by showing that they are exactly equivalent to models which are known to be quasi reversible algorithms for computing both lumpable bisimulation and exact equivalence are introduced case studies as well as performance tests are also presented

further results on global adaptive stabilisation for a class of uncertain stochastic nonlinear systems this paper considers the global adaptive stabilisation for a class of uncertain stochastic nonlinear systems comparing with the existing relevant literature the systems under investigation allow more serious unknowns uncertainties more importantly by remarkably reducing the dimension of dynamic compensator for system unknowns from n to 2 and considerably diminishing the overparametrisation in the existing literature this paper proposes an improved design scheme for adaptive state feedback controllers it is turned out that with the designed controller in loop all the closed loop system states are globally bounded and the original system states converge to the origin both in the sense of probability one a simulation example is also given to demonstrate the effectiveness of the theoretical results

brief paper inverse optimal stabilization for stochastic nonlinear systems whose linearizations are not stabilizable after considering the stabilization of a class of high order stochastic nonlinear systems which are neither necessarily feedback linearizable nor affine in the control input in this brief paper we further address the problem of state feedback inverse optimal stabilization in probability i e our redesigned stabilizing backstepping controller is also optimal with respect to meaningful cost functionals

turning back time in markovian process algebra product form solutions in markovian process algebra mpa are constructed using properties of reversed processes the compositionality of mpas is directly exploited allowing a large class of hierarchically constructed systems to be solved for their state probabilities at equilibrium the paper contains new results on both reversed stationary markov processes as well as mpa itself and includes a mechanisable proof in mpa notation of jackson s theorem for product form queueing networks several examples are used to illustrate the approach

a homogeneous domination approach to state feedback of stochastic high order nonlinear systems with time varying delay the homogeneous domination approach is introduced to solve the state feedback stabilization problem for stochastic high order nonlinear systems with time varying delay under the weaker conditions on the drift and diffusion terms by using the homogeneous domination approach and solving several troublesome obstacles in the design and analysis procedure a state feedback controller is constructed to render the closed loop system globally asymptotically stable in probability

a combined homogeneous domination and sign function approach to output feedback stabilization of stochastic high order nonlinear systems for a class of stochastic high order nonlinear systems by introducing the combined homogeneous domination and sign function design approach this technical note solves the problem of output feedback control under the weaker high order and nonlinear growth conditions

exact performance equivalence an equivalence relation for stochastic automata stochastic automata are an established formalism to describe and analyse systems according to their qualitative and quantitative behaviour equivalence is a basic concept for the analysis comparison and reduction of untimed automata whereas equivalence of stochastic automata is less established this paper introduces a new equivalence relation for stochastic automata denoted as exact performance equivalence it is shown that this equivalence relation preserves several important qualitative properties and also quantitative results exact performance equivalence is a congruence according to the synchronised product of stochastic automata the smallest exactly equivalent automaton exists for a stochastic automaton and can be generated by a partition refinement algorithm

a tutorial on empa a theory of concurrent processes with nondeterminism priorities probabilities and time we give an overview of empa a process algebra for modeling and analyzing concurrent systems where nondeterminism priorities probabilities and time are present aiming at a reasonable trade off between the expressive power of the calculus and the complexity of its underlying theory

global stabilization of stochastic nonlinear systems via c 1 and c infty controllers the problem of designing c 1 or c infty controllers for a class of stochastic nonlinear systems snss in lower triangular form is studied in this note by using the well known backstepping method the concept of homogeneity with monotone degrees hwmd and the sign function approach we construct a c 1 state feedback controller recursively meanwhile by employing a polynomial lyapunov function with sign functions we prove that the solution to snss is globally asymptotically stable in probability furthermore based on the concept of hwmd it shows that c 1 controllers for a class of three dimensional snss can be modified to c infty controllers under certain conditions finally two simulation examples are given to demonstrate the effectiveness of the proposed controllers

cooperative control of multiple stochastic high order nonlinear systems abstract distributed cooperative control of multiple stochastic high order nonlinear systems has not been addressed in literature this paper presents an approach to design of distributed cooperative controllers for multiple stochastic high order nonlinear systems under directed leader followers type network topology via the so called distributed integrator backstepping method by using the algebraic graph theory and stochastic analysis method it is shown that the output tracking errors between the followers and the leader can be tuned arbitrarily small while all the states of the closed loop system remain bounded in probability finally the effectiveness of the proposed control approach is illustrated on a stochastic underactuated mechanical system

output tracking of high order stochastic nonlinear systems with application to benchmark mechanical system this note considers output tracking of high order stochastic nonlinear systems without imposing any restriction on the high order and the drift and diffusion terms by using the backstepping design technique a smooth state feedback controller is given to guarantee that the solution process is bounded in probability and the error signal between the output and the reference signal can be regulated into a small neighborhood of the origin in probability a practical example of stochastic benchmark mechanical system and simulation are provided to demonstrate the effectiveness of the control scheme

reprint of on convergence analysis of particle swarm optimization algorithm abstract particle swarm optimization pso a population based stochastic optimization algorithm has been successfully used to solve many complicated optimization problems however analysis on algorithm convergence is still inadequate till now in this paper the martingale theory is applied to analyze the convergence of the standard pso spso firstly the swarm state sequence is defined and its markov properties are examined according to the theory of spso two closed sets the optimal particle state set and optimal swarm state set are then obtained afterwards a supermartingale is derived as the evolutionary sequence of particle swarm with the best fitness value finally the spso convergence analysis is carried out in terms of the supermartingale convergence theorem our results show that spso reaches the global optimum in probability moreover the analysis on spso proves that the quantum behaved particle swarm optimization qpso is also a global convergence algorithm the proof of the spso convergence in this work is new simple and more effective without specific implementation

process algebra for performance evaluation this paper surveys the theoretical developments in the field of stochastic process algebras process algebras where action occurrences may be subject to a delay that is determined by a random variable a huge class of resource sharing systems like large scale computers client server architectures networks can accurately be described using such stochastic specification formalisms the main emphasis of this paper is the treatment of operational semantics notions of equivalence and sound and complete axiomatisations of these equivalences for different types of markovian process algebras where delays are governed by exponential distributions starting from a simple actionless algebra for describing time homogeneous continuous time markov chains we consider the integration of actions and random delays both as a single entity like in known markovian process algebras like tipp pepa and empa and as separate entities like in the timed process algebras timed csp and tccs in total we consider four related calculi and investigate their relationship to existing markovian process algebras we also briefly indicate how one can profit from the separation of time and actions when incorporating more general non markovian distributions

exponential state estimation for stochastically disturbed discrete time memristive neural networks multiobjective approach the state estimation of the discrete time memristive model is studied in this article by applying the stochastic analysis technique sufficient formulas are established to ensure the exponentially mean square stability of the error model moreover the derived control gain matrix can be calculated via the linear matrix inequality lmi it should be mentioned that by extending the derived conclusion to a multiobjective optimization problem the maximum bound of the active function and the minimum bound of the disturbance attenuation are derived the corresponding simulation figures are provided in the end

computing nash equilibria through computational intelligence methods nash equilibrium constitutes a central solution concept in game theory the task of detecting the nash equilibria of a finite strategic game remains a challenging problem up to date this paper investigates the effectiveness of three computational intelligence techniques namely covariance matrix adaptation evolution strategies particle swarm optimization as well as differential evolution to compute nash equilibria of finite strategic games as global minima of a real valued nonnegative function an issue of particular interest is to detect more than one nash equilibria of a game the performance of the considered computational intelligence methods on this problem is investigated using multistart and deflection

on convergence analysis of particle swarm optimization algorithm abstract particle swarm optimization pso a population based stochastic optimization algorithm has been successfully used to solve many complicated optimization problems however analysis on algorithm convergence is still inadequate till now in this paper the martingale theory is applied to analyze the convergence of the standard pso spso firstly the swarm state sequence is defined and its markov properties are examined according to the theory of spso two closed sets the optimal particle state set and optimal swarm state set are then obtained afterwards a supermartingale is derived as the evolutionary sequence of particle swarm with the best fitness value finally the spso convergence analysis is carried out in terms of the supermartingale convergence theorem our results show that spso reaches the global optimum in probability moreover the analysis on spso proves that the quantum behaved particle swarm optimization qpso is also a global convergence algorithm the proof of the spso convergence in this work is new simple and more effective without specific implementation

comparative branching time semantics for markov chains this paper presents various semantics in the branching time spectrum of discrete time and continuous time markov chains dtmcs and ctmcs strong and weak bisimulation equivalence and simulation preorders are covered and are logically characterized in terms of the temporal logics probabilistic computation tree logic pctl and continuous stochastic logic csl apart from presenting various existing branching time relations in a uniform manner this paper presents the following new results i strong simulation for ctmcs ii weak simulation for ctmcs and dtmcs iii logical characterizations thereof including weak bisimulation for dtmcs iv a relation between weak bisimulation and weak simulation equivalence and v various connections between equivalences and pre orders in the continuous and discrete time setting the results are summarized in a branching time spectrum for dtmcs and ctmcs elucidating their semantics as well as their relationship

brief paper small gain control method for stochastic nonlinear systems with stochastic iiss inverse dynamics this paper investigates the small gain type conditions on stochastic iiss siiss systems and makes full use of these conditions in the design and analysis of the controller the contributions are as follows 1 a new proof of the stochastic lasalle theorem is provided 2 the small gain type conditions on siiss are developed and their relationship is discussed 3 based on the stochastic lasalle theorem and siiss small gain type conditions the adaptive controllers are designed to guarantee that all of the closed loop signals are bounded almost surely and the stochastic closed loop systems are globally asymptotically stable in probability

three partition refinement algorithms we present improved partition refinement algorithms for three problems lexicographic sorting relational coarsest partition and double lexical ordering our double lexical ordering algorithm uses a new efficient method for unmerging two sorted sets

state feedback stabilization for stochastic high order nonlinear systems with markovian switching abstract this paper aims to investigate the problem of state feedback stabilization for stochastic high order nonlinear systems with markovian switching by using the generalized ito formula and dynkin formula for markovian switching systems a state feedback controller is constructed to ensure that the closed loop system has a unique solution and the solution of the closed loop systems is almost surely asymptotically stable finally the effectiveness of the controller is demonstrated by a simulation example

bisimulation through probabilistic testing abstract we propose a language for testing concurrent processes and examine its strength in terms of the processes that are distinguished by a test by using probabilistic transition systems as the underlying semantic model we show how a testing algorithm can distinguish with a probability arbitrarily close to one between processes that are not bisimulation equivalent we also show a similar result in a slightly stronger form for a new process relation called 2 3 bisimulation which lies strictly between that of simulation and bisimulation finally the ultimately strength of the testing language is shown to identify a new process relation called probabilistic bisimulation which is strictly stronger than bisimulation

the inverse of any two by two nonsingular partitioned matrix and three matrix inverse completion problems a formula for the inverse of any nonsingular matrix partitioned into two by two blocks is derived through a decomposition of the matrix itself and generalized inverses of the submatrices in the matrix the formula is then applied to three matrix inverse completion problems to obtain their complete solutions

state feedback stabilization for stochastic feedforward nonlinear systems with time varying delay this paper investigates a class of stochastic feedforward nonlinear systems with time varying delay by introducing the homogeneous domination approach to stochastic systems a state feedback controller is constructed to render the closed loop system globally asymptotically stable in probability

output feedback stabilization of stochastic high order nonlinear systems under weaker conditions under the weaker conditions on the power order and the nonlinear functions this paper investigates the output feedback stabilization problem for a class of stochastic high order nonlinear systems based on the backstepping design method and homogeneous domination technique the closed loop system can be proved to be globally asymptotically stable in probability

budget dependent convergence rate of stochastic approximation convergence rate results are derived for a stochastic optimization problem where a performance measure is minimized with respect to a vector parameter t assuming that a gradient estimator is available and that both the bias and the variance of the estimator are known functions of the budget devoted to its computation the gradient estimator is employed in conjunction with a stochastic approximation sa algorithm our interest is to figure out how to allocate the total available computational budget to the successive sa iterations the effort is devoted to solving the asymptotic version of this problem by finding the convergence rate of sa toward the optimizer first as a function of the number of iterations and then as a function of the total computational effort as a result the optimal rate of increase of the computational budget per iteration can be found explicit expressions for the case where the computational budget devoted to an iteration is a polynomial in the iteration number and where the bias and variance of the gradient estimator are polynomials of the computational budget are derived applications include the optimization of steady state simulation models with likelihood ratio perturbation analysis or finite difference gradient estimators optimization of infinite horizon models with discounting optimization of functions of several expectations and so on several examples are discussed our results readily generalize to general root finding problems

backstepping controller design for nonlinear stochastic systems under a risk sensitive cost criterion this paper develops a methodology for recursive construction of optimal and near optimal controllers for strict feedback stochastic nonlinear systems under a risk sensitive cost function criterion the design procedure follows the integrator backstepping methodology and the controllers obtained guarantee any desired achievable level of long term average cost for a given risk sensitivity parameter theta furthermore they lead to closed loop system trajectories that are bounded in probability and in some cases asymptotically stable in the large these results also generalize to nonlinear systems with strongly stabilizable zero dynamics a numerical example included in the paper illustrates the analytical results

decentralized adaptive output feedback stabilization for large scale stochastic nonlinear systems in this paper the problem of decentralized adaptive output feedback stabilization is investigated for large scale stochastic nonlinear systems with three types of uncertainties including parametric uncertainties nonlinear uncertain interactions and stochastic inverse dynamics under the assumption that the inverse dynamics of the subsystems are stochastic input to state stable an adaptive output feedback controller is constructively designed by the backstepping method it is shown that under some general conditions the closed loop system trajectories are bounded in probability and the outputs can be regulated into a small neighborhood of the origin in probability in addition the equilibrium of interest is globally stable in probability and the outputs can be regulated to the origin almost surely when the drift and diffusion vector fields vanish at the origin the contributions of the work are characterized by the following novel features 1 even for centralized single input single output systems this paper presents a first result in stochastic nonlinear adaptive output feedback asymptotic stabilization 2 the methodology previously developed for deterministic large scale systems is generalized to stochastic ones at the same time novel small gain conditions for small signals are identified in the setting of stochastic systems design 3 both drift and diffusion vector fields are allowed to be dependent not only on the measurable outputs but some unmeasurable states 4 parameter update laws are used to counteract the parametric uncertainty existing in both drift and diffusion vector fields which may appear nonlinearly 5 the concept of stochastic input to state stability and the method of changing supply functions are adapted for the first time to deal with stochastic and nonlinear inverse dynamics in the context of decentralized control

reduced order observer based control design for nonlinear stochastic systems in this paper we investigate the stabilization control design problem of nonlinear stochastic siso systems in strict feedback form by introducing a novel reduced order observer an output feedback based control is constructively designed which renders the closed loop system asymptotically stable in the large when the nonlinearities and stochastic disturbance equal zero at the equilibrium point of the open loop system and bounded in probability otherwise besides the obtained controller preserves the equilibrium point of the open loop nonlinear system

further results on adaptive state feedback stabilization for stochastic high order nonlinear systems this paper aims to relax the results in xie and tian 2009 from the following two aspects completely removing the power order restriction and largely relaxing the growth conditions of nonlinear functions by using the backstepping design method and homogeneous domination technique this paper investigates the problem of adaptive state feedback stabilization for a class of stochastic high order nonlinear systems with nonlinear parameterization the closed loop system can be proved to be globally stable in probability and the states can be regulated to the origin almost surely the efficiency of the adaptive state feedback controller is demonstrated by a simulation example