variable structure control of stochastic systems abstract in this paper the problem of variable structure control of stochastic svsc systems is proposed and the corresponding variable structure control strategies with complete and incomplete state information are established the concepts of stochastic sliding mode and sliding motion band are introduced which may be regarded as the basic characteristics of svsc systems robustness of svsc systems is considered and a general design procedure for svsc systems is presented the effectiveness of the proposed control method is shown by simulation results

thompson sampling for stochastic control the continuous parameter case recently thompson sampling has been shown to achieve good theoretical performance guarantees for stochastic control problems with parameter uncertainty when the state control and parameter spaces are all finite much less is known however about the performance of thompson sampling when applied to continuous or more general spaces which constitutes an important class of problems in practice in this paper we study thompson sampling when applied to a broad class of average cost stochastic control problems where the state control and parameter spaces are all general measurable spaces the main contributions of our paper are establishing theoretical performance guarantees for thompson sampling as measured by first expected posterior sampling error and second average per period regret

convergence rate for rls based direct self tuning minimum variance regulation of armax minimum phase plants an upper bound for the error convergence rate of recursive least squares rls based direct self tuning minimum variance dstmv regulation of minimum phase multiple time delay autoregressive moving average with exogenous input armax plant is derived the rls algorithm used is k interlaced with k being the plant time delay the bound is derived for known fixed b sub 0 by extending a previously proposed methodology the bound provides a joint explanation of dstmv regulation stability and parameter estimate convergence the paper demonstrates that self tuning is based on convergence properties of rls as well as on the excitation quality of plant white noise which generates via controller feedback the plant input

convergence rates in stochastic adaptive tracking abstract for stochastic control systems described by the armax model with unknown matrix coefficients the stochastic adaptive control is designed so that the parameter estimates converge to the true values with a rate of convergence o log n log log n c n a with a 0 c 1 and the tracking error tends to its minimum value at a speed of o n 1 2e with e 0

adaptive treatment allocation and the multi armed bandit problem a class of simple adaptive allocation rules is proposed for the problem often called the multi armed bandit problem of sampling x 1 cdots x n sequentially from k populations with densities belonging to an exponential family in order to maximize the expected value of the sum s n x 1 cdots x n these allocation rules are based on certain upper confidence bounds which are developed from boundary crossing theory for the k population parameters the rules are shown to be asymptotically optimal as n rightarrow infty from both bayesian and frequentist points of view monte carlo studies show that they also perform very well for moderate values of the horizon n

a survey of some results in stochastic adaptive control some results in discrete time stochastic adaptive control are surveyed the survey divides itself into two parts bayesian and non bayesian adaptive control in the former area the problems of converting an incompletely observed system into a completely observed one multi armed bandit processes bayesian adaptive control of markov chains and bayesian adaptive control of linear systems are exposed and surveyed in the latter area non bayesian adaptive control of markov chains and the self tuning regulator are dealt with proofs are given where appropriate to illustrate the methods involved

high performance parallel numerical methods for volterra equations with weakly singular kernels non stationary discrete time waveform relaxation methods for abel systems of volterra integral equations using fractional linear multistep formulae are introduced fully parallel discrete waveform relaxation methods having an optimal convergence rate are constructed a significant expression of the error is proved which allows us to estimate the number of iterations needed to satisfy a prescribed tolerance and allows us to identify the problems where the optimal methods offer the best performance the numerical experiments confirm the theoretical expectations

discrete time stochastic adaptive control this paper establishes global convergence of a stochastic adaptive control algorithm for discrete time linear systems it is shown that with probability one the algorithm will ensure the system inputs and outputs are sample mean square bounded and the conditional mean square output tracking error achieves its global minimum possible value for linear feedback control thus asymptotically the adaptive control algorithm achieves the same performance as could be achieved if the system parameters were known

sample complexity for markov chain self tuner for self tuning control of a finite state markov chain whose parametrized transition probabilities satisfy an identifiability condition we establish a bound on the number of samples required to attain a prescribed measure of near optimality with a prescribed probability

adaptive controller design for tracking and disturbance attenuation in parametric strict feedback nonlinear systems the authors develop a systematic procedure for obtaining robust adaptive controllers that achieve asymptotic tracking and disturbance attenuation for a class of nonlinear systems which are described in the parametric strict feedback form and are subject to additional exogenous disturbance inputs their approach to adaptive control is performance based where the objective for the controller design is not only to find an adaptive controller but also to construct an appropriate cost functional compatible with desired asymptotic tracking and disturbance attenuation specifications with respect to which the adaptive controller is worst case optimal three main issues of the paper are the backstepping methodology worst case identification schemes and singular perturbations analysis closed form expressions have been obtained for an adaptive controller and the corresponding value function a numerical example involving a third order system is given

will the self tuning approach work for general cost criteria abstract we expose a fundamental limitation of the self tuning procedure for adaptive control of linear stochastic systems specifically if the cost criterion measuring the performance of the unknown system is anything other than a minimum output variance criterion then the procedure of estimating the unknown parameters of the linear system and then applying a control input which is optimal for the given cost criterion and the parameter estimates will not self tune to the optimal control law since our arguments are based on the analysis of an ordinary differential equation of ljung and since we do not in our general context have a proof that this o d e represents the asymptotic behaviour of the system further work is needed an exception to the above situation is the class of systems with a large enough delay for which self tuning to the optimal and indeed convergence to the true parameters cannot be ruled out

estimation of surface wave group velocity using slant stack in the generalized s transform domain abstractwe have developed the slant stacking approach in the generalized s transform gst domain of seismic data for the estimation of the group velocity of the multimodal surface wave we used two versions of the gst the constant scale gst uses a constant scaling factor in the gaussian window to control the spectral time frequency localization we found that a smaller scaling factor should be chosen for the low frequency surface wave whereas for higher frequencies a larger scaling factor should be chosen therefore the transform exhibits a trade off in the group velocity resolution of low frequencies with small values of the scaling factor and the group velocity resolution of high frequencies with large values of the scaling factor the modified s transform mst another version of the gst used in this study enhanced the time frequency resolution by projecting the frequency into a linear frequency function in the gaussian window this property allowed us to estimate the group velocity for a broad

on the necessity of identifying the true parameter in adaptive lq control in adaptive control problems one may drop the requirement of identifying the true system in order to simplify the problem of control it will be shown that in the adaptive lq control problem this does not at all lead to an easier problem

product integration methods for solving a system of nonlinear volterra integral equations in this paper the technique of subtracting out singularities is used to derive explicit and implicit product euler schemes with order one convergence and a product trapezoidal scheme with order two convergence for a system of volterra integral equations with a weakly singular kernel the convergence proofs of the numerical schemes are presented these are nonstandard since the nonlinear function involved in the integral equation system does not satisfy a global lipschitz condition

a robust adaptive minimum variance controller this paper addresses the twin questions of performance and robustness of an adaptive controller for single input single output linear stochastic systems the authors present an adaptive controller that has the following properties 1 attaining optimal regulation and tracking in the ideal minimum phase known upper bound on system order known sign and lower bound for the leading coefficient b 0 positive real condition on noise case and self tuning in a cesaro sense to a minimum variance regulator in the case of pure regulation 2 providing mean square stability when the system is of minimum phase with known upper bound on order but not necessarily satisfying a positive real condition on the noise 3 providing mean square stability when the system is in a graph topological neighborhood of computable size of an ideal plant as in 1 and the statistical properties of the disturbance are violated 4 continuing to stabilize the system when the adaptation gain is prevented from vanishing

nonlinear experiments optimal design and inference based on likelihood abstract nonlinear experiments involve response and regressors that are connected through a nonlinear regression type structure examples of nonlinear models include standard nonlinear regression logistic regression probit regression poisson regression gamma regression inverse gaussian regression and so on the fisher information associated with a nonlinear experiment is typically a complex nonlinear function of the unknown parameter of interest as a result we face an awkward situation designing an efficient experiment will require knowledge of the parameter but the purpose of the experiment is to generate data to yield parameter estimates our principal objective here is to investigate proper designing of nonlinear experiments that will let us construct efficient estimates of parameters we focus our attention on a very general nonlinear setup that includes many models commonly encountered in practice the experiments considered have two fundamental stages a static design in the initial stage

asymptotically efficient adaptive choice of control laws incontrolled markov chains we consider a controlled markov chain on a general state space whose transition probabilities are parameterized by an unknown parameter belonging to a compact metric space there is a one step reward associated with each pair of control and the following state of the process given a finite set of stationary control laws under each of which the markov chain is uniformly recurrent an optimal control law is this set is one that maximizes the long run average reward in ignorance of the parameter value we construct an adaptive control rule which uses the optimal control law s at a relative frequency of 1 o n 1 log n and show that this relative frequency gives an asymptotically optimal balance between the control objective and the amount of information needed to learn about the unknown parameter the basic idea underlying this construction is to introduce suitable uncertainty adjustments via sequential testing theory into the certainty equivalence rule thus resolving the apparent dilemma between control and information

certainty equivalence control with forcing revisited abstract certainty equivalence control with forcing has been shown to be optimal for several stochastic adaptive control problems with the average cost per unit time criterion recently researchers have started looking at stochastic adaptive control problems with a view to minimizing the rate of increase of the learning loss this criterion is stronger than the average cost per unit time criterion certainty equivalence control with forcing does not usually suffice for the learning loss criterion and one has to develop fairly complicated schemes in order to achieve optimality the objective of this paper is to see how well one might be able to do with a certainty equivalence control with forcing type of scheme in particular we construct a class of such schemes whose learning loss is o log n 1 for 0 whereas optimal schemes typically have a o log n learning loss

nonparametric adaptive control of discrete time partially observable stochastic systems abstract we consider discrete time partially observable stochastic control systems x t 1 f x t u t v t t 0 1 y t g u t 1 x t w t t 1 2 y 0 g 0 x 0 w 0 where x t u t and y t are respectively the state control and observation at time t and the state and observation disturbance processes v t and w t are sequences of i i d random elements with distributions and respectively we study the non parametric adaptive control problem of maximizing the infinite horizon discounted reward criterion when and are unknown and determine conditions for several adaptive policies to be asymptotically optimal and for almost surely uniform approximations of the optimal reward function our approach combines convergence results for empirical processes and recent results on parameter adaptive stochastic control problems

numerical solutions of integral and integro differential equations using chebyshev polynomials of the third kind abstract our purpose in this study is to construct an algorithm based on the use of a finite expansion in chebyshev polynomials of the third kind to solve singularly perturbed volterra integral equations first order integro differential equations of volterra type arising in fluid dynamics and volterra delay integro differential equations the convergence of the method is investigated finally some numerical experiments which confirm the theoretical results are shown and comparisons with other methods in literature are given

adaptive control of constrained markov chains adaptive control of finite state markov chains is discussed the optimal performance is characterized through the minimization of a long run average cost functional subject to constraints on several other such functionals under mild structural and feasibility conditions two explicit adaptive control policies are exhibited for the case where the transition probabilities are unknown the policies are optimal under the constrained optimization criterion they rely on a powerful estimation scheme which provides consistent estimators for the transition probabilities this scheme is of independent interest as it provides strong consistency under a large number of adaptive schemes and is independent of any identifiability conditions as an application an optimal adaptive policy is derived for a system of k competing queues with countable state space for which the constrained criteria arise naturally in the context of communication networks

mathematical programming methods in the numerical solution of volterra integral and integro differential equations with weakly singular kernel abstract nonlinear volterra integral and integro differential equations with weakly singular kernel are considered and solved numerically using nonlinear mathematical programming methods based on minimax approximations in both cases polynomial and multiquadric approximation are used

brief paper stochastic adaptive control using a modified least squares algorithm recent papers on stochastic adaptive control have established global convergence for algorithms using a stochastic approximation iteration however to date global convergence has not been established for algorithms incorporating a least squares iteration this paper establishes global convergence for a slightly modified least squares stochastic adaptive control algorithm it is shown that with probability one the algorithm will ensure that the system inputs and outputs are sample mean square bounded and the mean square output tracking error achieves its global minimum possible value for linear feedback control

optimal transmission rate control policies in a wireless link under partial state information we consider the problem of phy layer transmission rate control for maximum throughput on a wireless link over a finite time horizon for each link state there exists a most advantageous transmission rate for the controller the controller needs to rely on incomplete link state information in the form of positive acknowledgements acks and negative acknowledgements nacks for transmitted packets we cast the problem as a partially observed markov decision processes pomdp we focus on the special case of time invariant or slowly time varying link quality with two states and two control options in that case the optimal control problem is a state estimation one and the optimal strategy is of threshold structure where the thresholds are certain values of the belief of being in one of the two states the policy admits a simple interpretation increase rate when the number of successive acks exceeds a threshold and decrease rate when the number of successive nacks exceeds a threshold

adaptive control of markov chains with average cost we present an adaptive control scheme for the control of markov chains to minimize long run average cost when the system transition and reward structures are unknown q factors are estimated along a single sample path of the system and control actions are applied based on the latest estimates we prove that an optimal policy is obtained asymptotically with probability one more importantly we prove that optimal system performance is achieved as well which means that the performance of the system can not be bettered even if the system transition and reward structures are known an example is given to illustrate our adaptive control scheme

brief paper consistency in least squares estimation in a previous paper the convenience of using martingale theory in the analysis of bayesian least squares estimation was demonstrated however certain restrictions had to be imposed on either the feedback structure or on the initial values for the estimation in the present paper these restrictions are removed and necessary and sufficient conditions for strong consistency in a bayesian sense are given for the gaussian white noise case without any assumptions on closed loop stability or on the feedback structure in the open loop case the poles are shown to be consistently estimated almost everywhere and in the closed loop case certain choices of control law are shown to assure consistency finally adaptive control laws are treated and implicit self tuning regulators are shown to converge to the desired control laws

the aastrom wittenmark self tuning regulator revisited and els based adaptive trackers although considerable progress has been made in stochastic adaptive control the problem concerning the convergence of the original self tuning regulator proposed by k j aastrom and b wittenmark 1973 is still open since it is attractive in theory and important in applications this problem is examined and a rigorous proof of its stability and optimality is given related problems such as convergence of the extended least squares els based adaptive tracker are also considered

chaotic neural control we consider the problem of stabilizing unstable equilibria by discrete controls the controls take discrete values at discrete moments of time we prove that discrete control typically creates a chaotic attractor in the vicinity of an equilibrium artificial neural networks with reinforcement learning are known to be able to learn such a control scheme we consider examples of such systems discuss some details of implementing the reinforcement learning to controlling unstable equilibria and show that the arising dynamics is characterized by positive lyapunov exponents and hence is chaotic this chaos can be observed both in the controlled system and in the activity patterns of the controller

the zero divisor problem of multivariable stochastic adaptive control abstract stochastic adaptive minimum variance control algorithms require a division by a function of a recursively computed parameter estimate at each instant of time in order that the analysis of these algorithms is valid zero divisions must be events of probability zero this property is established for the stochastic gradient adaptive control algorithm under the condition that the initial state of the system and all finite segments of its random disturbance process have a joint distribution which is absolutely continuous with respect to lebesgue measure this result is deduced from the following general result established in this paper a non constant rational function of a finite set of random variables x 1 x n is absolutely continuous with respect to lebesgue measure if the joint distribution function of x 1 x n has this property

asymptotically efficient adaptive strategies in repeated games part i certainty equivalence strategies this paper addresses the problem of dynamic decision making in an uncertain and competitive environment a decision maker player 1 faces a system about which he has some parametric uncertainty and which is affected also by the actions of other agents we focus on a worst case analysts from the viewpoint of player 1 using the simplified model of a repeated matrix game with lack of information on one side where single stage rewards are random but announced and perfect observations are assumed certain ideas from the field of stochastic adaptive control are used to formulate performance criteria in a non bayesian setting and to devise appropriate control strategies the basic performance measure is the total reward accumulated by player 1 over all stages played the purpose of player 1 is to guarantee that his expected total reward will be close to what he could guarantee under complete information the present paper considers adaptive decision strategies of the certainty equivalence type based on a modified maximum likelihood estimator and studies their asymptotic long term performance a sequel paper will be devoted to asymptotically optimal strategies

polynomial spline collocation methods for the nonlinear basset equation abstract we discuss the application of a class of spline collocation methods to a first order volterra integro differential equation arising in fluid dynamics since this equation possesses a weakly singular kernel its solutions even for analytic data are not smooth on the entire interval of integration it is shown that while the use of uniform grids leads to poor convergence rates high order convergence is restored on suitably graded grids

a mathematical treatment of the fluorescence capillary fill device a mathematical model in the form of two coupled diffusion equations is provided for a competitive chemical reaction between an antigen and a labeled antigen for antibody sites on a cell wall boundary conditions are such that the problem is both nonlinear and nonlocal this is then recharacterized first as a pair of coupled singular integro differential equations and then as a system of four volterra integral equations the latter permits a proof of existence and uniqueness of the solution of the original problem small and large time asymptotic solutions are derived and from the first characterization a regular perturbation solution is obtained numerical schemes are briefly discussed and graphical results are presented for human immunoglobulin

a new family of optimal adaptive controllers for markov chains we consider the problem of adaptively controlling a markov chain with unknown transition probabilities a new family of adaptive controllers is exhibited which achieves a performance precisely equalling the optimal performance achievable if the transition probabilities i e the model or dynamics of the system were known instead hence the adaptive controllers presented here are truly optimal the performance of the system to be controlled is measured by the average of the costs incurred over an infinite operating time period these adaptive controllers can potentially be implemented on digital computers and used in the on line control of unknown systems

a nonlinear weakly singular volterra integro differential equation arising from reaction diffusion study in a small cell a simple product integration scheme is proposed to solve a nonlinear volterra integro differential equation with a weakly singular kernel and a non smooth solution a discrete gronwall inequality is derived and then employed to prove convergence of the numerical method numerical results are given which support the theoretical results

self tuning trackers we examine the problem of obtaining adaptive control laws which tune themselves to control laws minimizing the variance of the tracking error between the output of the linear armax system and a specified reference trajectory if the reference trajectory is sufficiently rich of order greater than or equal to the sum of the degrees of the control and noise polynomials in the armax system then an adaptive controller is exhibited for which the parameter estimates are strongly consistent for the linear model following problem where the trajectory to be tracked is generated as the output of a linear system it is enough for the order of sufficient richness to be greater than the degree of the noise polynomial alone further if the order of sufficient richness is even smaller as is often the case then a lower dimensional adaptive controller which does not attempt to estimate all the coefficients of the noise polynomial is self tuning

on the stability and convergence of self tuning control virtual equivalent system approach this article presents a unified understanding and judgement of the stability and convergence of a general self tuning control stc system which consists of arbitrary control strategy arbitrary parameter estimation algorithm and a deterministic stochastic linear time invariant lti plant the necessary conditions required for the global stability and convergence of a general stc system are relaxed i e the convergence of parameter estimates is removed for both deterministic and stochastic stc schemes to reach this goal virtual equivalent system ves concept and methodology is adopted with the help of ves the original nonlinear dominant nonlinear in structure problem is converted to a linear dominant linear in structure problem the results developed in this article show that stc systems are stable and convergent for the abundance of control strategies and parameter estimation algorithms which will provide great flexibility in the applications of stc

the bi gaussian s transform the s transform kernel is derived from the kernel of the fourier transform through the introduction of a scalable translating window the width of the window is a function of inverse frequency in effect the s transform is a method of spectral localization with some similarities to wavelet transforms but using the concept of frequency rather than the concept of scale an important property of the s transform is that it collapses into the fourier transform when integrated over the time axis this property requires the window to satisfy a normalizing condition the window which has been used in the majority of previous s transform research is the symmetric gaussian window introduced by stockwell mansinha and lowe ieee trans signal process 44 1996 pp 998 1001 one problem with the use of a gaussian however is degradation of time resolution in the time frequency spectrum due to the long front taper in this paper a bi gaussian window is introduced constructed through the welding of two half gaussian windows the asymmetry of the bi gaussian introduces asymmetry in the resultant time frequency spectrum with time resolution better in the front direction as compared with the back direction the bi gaussian s transform is better at resolving the sharp onset of events in a time series

convergence analysis for reduced order adaptive controller design of uncertain siso linear systems with noisy output measurements we consider in this article a class of uncertain siso linear systems that are subject to system and measurement noises reduced order adaptive controller designs have been proposed before for such systems by the authors and stability analysis of the closed loop systems has been established here we analyse further the robustness properties for these reduced order adaptive control systems by providing detailed convergence analysis results for the key closed loop signals and parameter estimates we rigorously prove that whenever the exogenous disturbance input is of finite energy and bounded and the reference trajectory and its derivatives up to rth order are bounded r being the relative degree of the transfer function of the true system a set of signals including the tracking error the estimation error between the system output and its estimate the projection signal are of finite energy and converge to zero and the system states and their estimates exhibit asymptotic behaviours with certain form

adaptive control of markov chains with local updates abstract an iterative algorithm is proposed for adaptive expected average cost control of finite state markov chains the algorithm estimates the unknown parameter using a strongly consistent parameter estimator and uses this estimate to update the control policy each update may apply to as few as one or as many as all states as long as all states are updated infinitely often as the number of iterations goes to infinity in this sense the algorithm does local updates no prior knowledge of the optimal control policy is assumed the algorithm requires only a small amount of computation at each step is easily implementable and converges to the optimal policy in finite time its complexity for queueing systems governed by birth death processes is comparable to that of a learning automaton but is based on the optimality equation of dynamic programming

thompson sampling for stochastic control the finite parameter case in this paper we apply thompson sampling to a class of average reward stochastic control problems with parameter uncertainty specifically we study an average reward stochastic control problem over an infinite horizon in which both the reward and state transition distributions are parameterized by an unknown parameter taking values in a finite space the main result of this paper is a proof showing that thompson sampling achieves a worst case average per period regret of o t 1 which is asymptotically optimal

closed loop parameter identifiability and adaptive control of a linear stochastic system abstract we show that if the parameters of a linear stochastic control system are identifiable using a persistently existing input then the same systems remains parameter identifiable if the system operates under closed loop in a certain way we assume that the controller itself depends on the test value of the system parameters and as usual the control signal is dithered the estimation problem is formulated as the problem of minimizing an appropriate asymptotic cost function it is shown that a suitable modification of the gradient of the cost function converts our problem into another problem for which ljung s scheme can be applied thus the theorem provides a general method for the solution of the local adaptive control problem

on the value of learning for bernoulli bandits with unknown parameters investigates the multiarmed bandit problem where each arm generates an infinite sequence of bernoulli distributed rewards the parameters of these bernoulli distributions are unknown and initially assumed to be beta distributed every time a bandit is selected its beta distribution is updated to new information in a bayesian way the objective is to maximize the long term discounted rewards we study the relationship between the necessity of acquiring additional information and the reward this is done by considering two extreme situations which occur when a bandit has been played n times the situation where the decision maker stops learning and the situation where the decision maker acquires full information about that bandit we show that the difference in reward between this lower and upper bound goes to zero as n grows large

a lyapunov approach to exponential stabilization of nonholonomic systems in power form in this paper a continuous feedback control law with time periodic terms is derived for the control of nonholonomic systems in power form the control law is derived by lyapunov design from a homogeneous lyapunov function global asymptotic stability is shown by applying the principle of invariance for time periodic systems exponential convergence follows since the vector fields are homogeneous of degree zero

tuning of gaussian stochastic control systems a closed loop system consisting of a control system and an adaptive controller is called tuning for a specified control objective if the real system and the ideal system defined below achieve the same value for the control objective the real system is the system consisting of the unknown control system in closed loop with the adaptive controller in which the parameters of the adaptive controller have been determined by identification under feedback or in closed loop the ideal system is the system consisting of the unknown control system in closed loop with a controller in which the controller has been synthesized with knowledge of the unknown control system and such that the closed loop system satisfies the control objective both the gaussian stochastic control system with full observations and with partial observations are considered the approach to the problem is based on stochastic realization theory for gaussian systems the control objectives of minimum variance control and pole placement are also given necessary conditions for tuning are discussed

on the convergence of self tuning stochastic servo algorithms based on stochastic approximation the problem of the adaptive tracking of stochastic reference signals is considered supposing that the process can be represented by an armax model with arbitrary time delay and the reference signal by an arma model two algorithms of the stochastic approximation type providing adaptation to both process and reference characteristics are proposed they differ by the incorporated a priori knowledge about the optimal regulator parameters global stability asymptotic optimally convergence of the adaptive control law in a cesaro sense and the strong consistency of the parameter estimates are proved the persistence of excitation condition is analyzed

an efficient and fast parallel method for volterra integral equations of abel type in this paper we present an efficient and fast parallel waveform relaxation method for volterra integral equations of abel type obtained by reformulating a nonstationary waveform relaxation method for systems of equations with linear coefficient constant kernel to this aim we consider the laplace transform of the equation and here we apply the recurrence relation given by the chebyshev polynomial acceleration for algebraic linear systems back in the time domain we obtain a three term recursion which requires at each iteration the evaluation of convolution integrals where only the laplace transform of the kernel is known for this calculation we can use a fast convolution algorithm numerical experiments have been done also on problems where it is not possible to use the original nonstationary method obtaining good results in terms of improvement of the rate of convergence with respect the stationary method

dispersion and the dissipative characteristics of surface waves in the generalized s transform domain wavenumber group velocity phase velocity and frequency dependent attenuation characterize the propagation of surface waves in dispersive attenuating media we use a mathematical model based on the generalized s transform to simultaneously estimate these characteristic parameters for later use in joint inversion for near surface shear wave velocity we use a scaling factor in the generalized s transform to enable the application of the method in a highly dispersive medium we introduce a cost function in the s domain to estimate an optimum value for the scaling factor we also use the cost function to generalize the application of the method for noisy data especially data with a low signal to noise ratio at low frequencies in that case the estimated wavenumber is perturbed as a solution we estimate wavenumber perturbation by minimizing the cost function using simulated annealing we use synthetic and real data to show the efficiency of the method for the estimation of the propagation parameters o

on kalman filtering for conditionally gaussian systems with random matrices abstract we consider linear stochastic systems with additive white gaussian noise with the added generality that the system matrices are random and adapted to the observation process the main result of this paper is that in order for the standard kalman filter to generate the conditional mean and conditional covariance of the conditionally gaussian distributed state it is sufficient for the random matrices to be finite with probability one at each time instant this generalizes the best previous results available to date to our knowledge which require the more stringent hypothesis that the entries of the random matrices should possess finite second moments at each time instant a significant application of the results of this paper is to the problem of recursive identification of the unknown parameters of a controlled linear stochastic system in such problems the observation matrix is typically generated by complicated nonlinear feedback as for example in adaptive control and the finiteness of the second moments is difficult if not impossible to establish while the finiteness with probability one has been established in many applications

on adaptive boundary control for stochastic parabolic systems with unknown potential coefficient an adaptive boundary control problem for a stochastic heat diffusion equation is studied the considered system contains an unknown potential coefficient which is a function of the spatial variables the estimation algorithm for the unknown potential coefficient is proposed by using the stochastic approximation technique after showing the strong consistency of the estimated parameter the cost for the adaptive control scheme presented here is shown to converge to the optimal ergodic cost finally some numerical examples are shown

strong consistency of parameter estimates in direct self tuning control algorithms based on stochastic approximation in this paper convergence of the regulator parameter estimates generated by direct self tuning control algorithms of stochastic approximation type is considered the process is supposed to be represented by an armax model with an arbitrary delay while a dither signal converging to zero in the mean square sense is superimposed on a deterministic reference the strong consistency of the regulator parameter estimates is proved under assumptions concerning the estimator complexity and the properties of the optimal regulator transfer function for a broad class of dither signals the impact of a priori information on the dimension of the estimator and the consistency of the parameter estimates is indicated it is demonstrated that the classical persistence of excitation condition is not satisfied for the analyzed algorithms

minimum variance control of discrete time multivariable armax systems we consider multivariable armax stochastic systems these systems can incorporate the following complicating features general delay structures nonminimum phase transfer functions different dimensions for input and output vectors we obtain the control laws which minimize the variance of the output process while maintaining system stability

comparing policies in markov decision processes mandl s lemma revisited a general framework is developed for comparing the long run average cost of a markov stationary policy with that of another related policy the underlying methodology constitutes an extension of some ideas of mandl to randomized policies and to polish state and action spaces sufficient conditions for the applicability of the methodology are given these conditions which are easy to verify have a natural probabilistic interpretation in terms of the stability of the chain and of the convergence of the control values the usefulness of the general framework proposed here is illustrated on several applications standard results on the convergence of adaptive policies are readily recovered under conditions which are more transparent than the ones existing in the literature and the convergence of randomized policies is handled as a special case finally a novel application to probing controls is outlined

asymptotic behavior of stochastic systems possessing markovian realizations markovian stochastic systems of the form phi k 1 f phi k w k 1 are considered where bf w is an independent and identically distributed process and bf phi is a markov chain both evolving in euclidean space a condition called weak stochastic controllability w s c is introduced which is a generalization of the concept of controllability in linear system theory two different formulations of stability boundedness in probability and boundedness in probability on average are presented and it is shown that these conditions are equivalent under the w s c condition these results are related to the ergodic behavior of bf phi by a set of results that includes the following if bf phi is bounded in probability and w s c then i sample path averages of functions of the state process converge for every initial condition and ii if the stability is uniform then state process probabilities converge to a periodic orbit of probabilities finally if the origin o

extended least squares and their applications to adaptive control and prediction in linear systems herein strong consistency of recursive extended least squares is established under considerably weaker assumptions than previously assumed in the literature the argument used to establish consistency also leads to certain basic properties of adaptive predictors based on these recursive estimators making use of these properties of the adaptive predictors simple modifications of the astrom wittenmark self tuning regulator are proposed and shown to be asymptotically optimal

analysis of dispersive waves by wave field transformation the dispersive waves in a common shot wave field can be transformed into images of the dispersion curves of each mode in the data the procedure consists of two linear transformations a slant stack of the data produces a wave field in the phase slowness time intercept p plane in which phase velocities are separated the spectral peak of the one dimensional 1 d fourier transform of the p wave field then gives the frequency associated with each phase velocity thus the data wave field is linearly transformed from the time distance domain into the slowness frequency p domain where dispersion curves are imaged all the data are present throughout the transformations dispersion curves for the mode overtones as well as the fundamental are directly observed in the transformed wave field in the p domain each mode is separated from the others even when its presence is not visually detectable in the untransformed data the resolution achieved in the result is indicated in the p wave

theory for automatic learning under partially observed markov dependent noise a vigorous branch of automatic learning is directed at the task of locating a global minimum of an unknown multimodal function f theta on the basis of noisy observations l theta i f theta i w theta i taken at sequentially chosen control points theta i in all preceding convergence deviations known to the authors the noise is postulated to depend on the past only through control selection here they allow the observation noise sequence to be stochastically dependent in particular a function of an unknown underlying markov decision process the observations being the stagewise losses in a sense in order to be made precise the algorithm offered is shown to attain asymptotically optimal performance and rates are assured a motivating example from queueing theory is offered and connections with classical problems of markov control theory and other disciplines are mentioned

computationally efficient algorithms for on line optimization of markov decision processes abstract iterative algorithms are proposed for on line optimization of finite state markov decision processes the optimization criterion is the minimization of long run average cost the transition probabilities of the underlying markov chain are assumed to depend on certain unknown parameters the algorithms estimate the unknown para meters using a strongly consistent estimator and use this estimate in place of the true parameter to generate a sequence of control policies which converge to an optimal policy the main feature of the algorithms is that the updating step does not have to be carried out in all states at each iteration at each iteration the updating step may be performed in any number of states as few as one as long as all states are updated often enough as the number of iterations increases in this sense the algorithms are asynchronous the asynchronous nature of our algorithms allows us to update the costly states more often as will be illustrated through simulation results the asynchronism of our algorithms makes them feasible for on line optimization

optimal adaptive controllers for unknown markov chains we consider the problem of adaptively controlling an unknown markov chain no prior information regarding the values of the transition probabilities is provided us except for a list of forbidden zero probability transitions which is usually obtained as a byproduct of the modeling process itself the goal is to design an adaptive controller to adequately control the unknown system when its performance is measured by the average cost incurred over a long operating time period our main result is the exhibition of a family of adaptive controllers which when applied to the unknown system will result in a performance precisely equal to the optimal performance attainable if the system i e the transition probabilities were known hence the adaptive controllers proposed here are truly optimal even when operating on an unknown system the results presented here extend similar results in 1 where we assume to be initially provided with a finite set of possible models one of which is guaranteed to be the true one this paper directly addresses those practical situations where a finite set of possible models with such a guarantee is hard to come by

adaptive control with the stochastic approximation algorithm geometry and convergence new geometric properties possessed by the sequence of parameter estimates are exhibited which yield valuable insight into the behavior of the stochastic approximation based algorithm as it is used in minimum variance adaptive control in particular these geometric properties together with certain probabilistic arguments prove that if the system does not have a reduced order minimum variance controller then the parameter estimates converge to a random multiple of the true parameter an explicit expression for the limiting parameter estimate is also available with strictly positive probability the limiting parameter estimate is not the true parameter and in some cases differs from the true parameter with probability one if the system possesses reduced order minimum variance controllers then convergence to a minimum variance controller in a cesaro sense is shown the geometry of the limit set is described sufficient conditions are also given for some of these results to hold for parameter estimation schemes other than stochastic approximation

parallel recursive algorithms in asymptotically efficient adaptive control of linear stochastic systems first a review of some recent developments in stochastic adaptive control of linear stochastic systems is given by integrating and refining several basic ideas in these developments a relatively complete asymptotic solution to the adaptive control problem is then provided for such systems the solution involves parallel implementation of a few basic recursive identification algorithms that serve to complement each other it also involves occasional use of a dither signal to probe the system for information

an operational matrix based scheme for numerical solutions of nonlinear weakly singular partial integro differential equations abstract in this article we introduce an operational matrix scheme based on two dimensional wavelets for the volterra weakly singular nonlinear partial integro differential equations by implementing two dimensional wavelets approximations and its operational matrices of integration and differentiation along with collocation points the weakly singular partial integro differential equations are reduced into the system of nonlinear algebraic equations moreover bernoulli wavelet approximation and legendre wavelet approximation have been used for inspecting the errors and convergence analysis of the given problems some numerical examples are included to establish the accuracy of the proposed scheme via bernoulli wavelet approximation and legendre wavelet approximation respectively additionally comparisons of error values between the two wavelets have been presented

adaptive lqg control of input output systems a cost biased approach in this paper we consider linear systems in input output form and introduce a new adaptive linear quadratic gaussian lqg control scheme which is shown to be self optimizing the identification algorithm incorporates a cost biasing term which favors the parameters with smaller lqg optimal cost and a second term that aims at moderating the time variability of the estimate the corresponding closed loop scheme is proven to be stable and to achieve an asymptotic lqg cost equal to the one obtained under complete knowledge of the true system self optimization the results of this paper extend in a nontrivial way previous results established along the cost biased approach in other settings

stochastic dynamic programming caution and probing the purpose of this paper is to unify the concepts of caution and probing put forth by feldbaum 14 with the mathematical technique of stochastic dynamic programming originated by bellman 5 the decomposition of the expected cost in a stochastic control problem recently developed in 8 is used to assess quantitatively the caution and probing effects of the system uncertainties on the control it is shown how in some problems because of the uncertainties the control becomes cautious less aggressive while in other problems it will probe by becoming more aggressive in order to enhance the estimation identification while controlling the system following this a classification of stochastic control problems according to the dominant effect is discussed this is then used to point out which are the stochastic control problems where substantial improvements can be expected from using a sophisticated algorithm versus a simple one

convergence of adaptive control schemes using least squares parameter estimates the stability convergence asymptotic optimality and self tuning properties of stochastic adaptive control schemes based on least squares estimates of the unknown parameters are examined it is assumed that the additive noise is i i d and gaussian and that the true system is of minimum phase the bayesian embedding technique is used to show that the recursive least squares parameter estimates converge in general the normal equations of least squares are used to establish that all stable control law designs used in a certainty equivalent i e indirect procedure generally yield a stable adaptive control system four results are given to characterize the limiting behavior precisely a certainty equivalent self tuning regulator is shown to yield strongly consistent parameter estimates when the delay is strictly greater than one even without any excitation in the reference trajectory

the numerical analysis of implicit runge kutta methods for a certain nonlinear integro differential equation the mathematical theory and numerical methods for the nonlinear integro differential equation u t p t u t 0 t k t s u t s u s ds q t 0 t t u 0 u 0 it is proved that the implicit runge kutta methods with m stages are of optimal approximation order p 2m

deconvolutive radon transform abstractthe radon transform rt plays an important role in seismic data processing for its ability to focus seismic events in the transform domain the band limited nature of seismic events due to the blurring effects of the source wavelet however causes a decrease in the temporal resolution of the transform we have developed the deconvolutive rt decrt as a generalization of conventional rt and to increase the temporal resolution unlike the conventional counterpart the new basis functions can take an arbitrary shape in the time direction this method is thus proposed to adaptively infer the temporal wave shape from the input data while finding a sparse representation of it the new transform significantly improves the sparsity and thus the temporal resolution of the resulting seismic data the applicability of the hyperbolic decrt in seismic data processing is demonstrated for random noise attenuation primary and multiple separation high quality stacking and automatic velocity model building t

localization of the complex spectrum the s transform the s transform which is introduced in the present correspondence is an extension of the ideas of the continuous wavelet transform cwt and is based on a moving and scalable localizing gaussian window it is shown to have some desirable characteristics that are absent in the continuous wavelet transform the s transform is unique in that it provides frequency dependent resolution while maintaining a direct relationship with the fourier spectrum these advantages of the s transform are due to the fact that the modulating sinusoids are fixed with respect to the time axis whereas the localizing scalable gaussian window dilates and translates

a note on the structure of two subsets of the parameter space in adaptive control problems we study the geometric structure of two subsets of the parameter space that are of interest in the context of adaptive lq control the first set can be considered as the set of possible limit points of an adaptive control algorithm whereas the second can be seen as the set of desirable limit points our main result is that these sets are c omega manifolds