the minimum rank of symmetric matrices described by a graph a survey abstract the minimum rank of a simple graph g is defined to be the smallest possible rank over all symmetric real matrices whose ijth entry for i j is nonzero whenever i j is an edge in g and is zero otherwise this paper surveys the current state of knowledge on the problem of determining the minimum rank of a graph and related issues

variable structure control of stochastic systems abstract in this paper the problem of variable structure control of stochastic svsc systems is proposed and the corresponding variable structure control strategies with complete and incomplete state information are established the concepts of stochastic sliding mode and sliding motion band are introduced which may be regarded as the basic characteristics of svsc systems robustness of svsc systems is considered and a general design procedure for svsc systems is presented the effectiveness of the proposed control method is shown by simulation results

thompson sampling for stochastic control the continuous parameter case recently thompson sampling has been shown to achieve good theoretical performance guarantees for stochastic control problems with parameter uncertainty when the state control and parameter spaces are all finite much less is known however about the performance of thompson sampling when applied to continuous or more general spaces which constitutes an important class of problems in practice in this paper we study thompson sampling when applied to a broad class of average cost stochastic control problems where the state control and parameter spaces are all general measurable spaces the main contributions of our paper are establishing theoretical performance guarantees for thompson sampling as measured by first expected posterior sampling error and second average per period regret

on the controllability properties of circulant networks this paper examines the controllability of a group of first order agents adopting a weighted consensus type coordination protocol over a circulant network specifically it is shown that a circulant network with laplacian eigenvalues of maximum algebraic multiplicity q is controllable from q nodes our approach leverages on the cauchy binet formula which in conjunction with the popov belevitch hautus test leads to new insights on structural aspects of network controllability

convergence rates in stochastic adaptive tracking abstract for stochastic control systems described by the armax model with unknown matrix coefficients the stochastic adaptive control is designed so that the parameter estimates converge to the true values with a rate of convergence o log n log log n c n a with a 0 c 1 and the tracking error tends to its minimum value at a speed of o n 1 2e with e 0

adaptive treatment allocation and the multi armed bandit problem a class of simple adaptive allocation rules is proposed for the problem often called the multi armed bandit problem of sampling x 1 cdots x n sequentially from k populations with densities belonging to an exponential family in order to maximize the expected value of the sum s n x 1 cdots x n these allocation rules are based on certain upper confidence bounds which are developed from boundary crossing theory for the k population parameters the rules are shown to be asymptotically optimal as n rightarrow infty from both bayesian and frequentist points of view monte carlo studies show that they also perform very well for moderate values of the horizon n

a survey of some results in stochastic adaptive control some results in discrete time stochastic adaptive control are surveyed the survey divides itself into two parts bayesian and non bayesian adaptive control in the former area the problems of converting an incompletely observed system into a completely observed one multi armed bandit processes bayesian adaptive control of markov chains and bayesian adaptive control of linear systems are exposed and surveyed in the latter area non bayesian adaptive control of markov chains and the self tuning regulator are dealt with proofs are given where appropriate to illustrate the methods involved

controlled markov chains and stochastic networks controlled markov chains with average cost criterion and with special cost and transition structures are studied existence of optimal stationary strategies is established for the average cost criterion corresponding dynamic programming equations are derived a stochastic network problem that includes interconnected queues as a special case is described and studied within this framework

sample complexity for markov chain self tuner for self tuning control of a finite state markov chain whose parametrized transition probabilities satisfy an identifiability condition we establish a bound on the number of samples required to attain a prescribed measure of near optimality with a prescribed probability

adaptive controller design for tracking and disturbance attenuation in parametric strict feedback nonlinear systems the authors develop a systematic procedure for obtaining robust adaptive controllers that achieve asymptotic tracking and disturbance attenuation for a class of nonlinear systems which are described in the parametric strict feedback form and are subject to additional exogenous disturbance inputs their approach to adaptive control is performance based where the objective for the controller design is not only to find an adaptive controller but also to construct an appropriate cost functional compatible with desired asymptotic tracking and disturbance attenuation specifications with respect to which the adaptive controller is worst case optimal three main issues of the paper are the backstepping methodology worst case identification schemes and singular perturbations analysis closed form expressions have been obtained for an adaptive controller and the corresponding value function a numerical example involving a third order system is given

zero forcing sets and controllability of dynamical systems defined on graphs in this technical note controllability of systems defined on graphs is discussed we consider the problem of controllability of the network for a family of matrices carrying the structure of an underlying directed graph a one to one correspondence between the set of leaders rendering the network controllable and zero forcing sets is established to illustrate the proposed results special cases including path cycle and complete graphs are discussed moreover as shown for graphs with a tree structure the proposed results of the present technical note together with the existing results on the zero forcing sets lead to a minimal leader selection scheme in particular cases

nonlinear experiments optimal design and inference based on likelihood abstract nonlinear experiments involve response and regressors that are connected through a nonlinear regression type structure examples of nonlinear models include standard nonlinear regression logistic regression probit regression poisson regression gamma regression inverse gaussian regression and so on the fisher information associated with a nonlinear experiment is typically a complex nonlinear function of the unknown parameter of interest as a result we face an awkward situation designing an efficient experiment will require knowledge of the parameter but the purpose of the experiment is to generate data to yield parameter estimates our principal objective here is to investigate proper designing of nonlinear experiments that will let us construct efficient estimates of parameters we focus our attention on a very general nonlinear setup that includes many models commonly encountered in practice the experiments considered have two fundamental stages a static design in the initial stage

a robust adaptive minimum variance controller this paper addresses the twin questions of performance and robustness of an adaptive controller for single input single output linear stochastic systems the authors present an adaptive controller that has the following properties 1 attaining optimal regulation and tracking in the ideal minimum phase known upper bound on system order known sign and lower bound for the leading coefficient b 0 positive real condition on noise case and self tuning in a cesaro sense to a minimum variance regulator in the case of pure regulation 2 providing mean square stability when the system is of minimum phase with known upper bound on order but not necessarily satisfying a positive real condition on the noise 3 providing mean square stability when the system is in a graph topological neighborhood of computable size of an ideal plant as in 1 and the statistical properties of the disturbance are violated 4 continuing to stabilize the system when the adaptation gain is prevented from vanishing

asymptotically efficient adaptive choice of control laws incontrolled markov chains we consider a controlled markov chain on a general state space whose transition probabilities are parameterized by an unknown parameter belonging to a compact metric space there is a one step reward associated with each pair of control and the following state of the process given a finite set of stationary control laws under each of which the markov chain is uniformly recurrent an optimal control law is this set is one that maximizes the long run average reward in ignorance of the parameter value we construct an adaptive control rule which uses the optimal control law s at a relative frequency of 1 o n 1 log n and show that this relative frequency gives an asymptotically optimal balance between the control objective and the amount of information needed to learn about the unknown parameter the basic idea underlying this construction is to introduce suitable uncertainty adjustments via sequential testing theory into the certainty equivalence rule thus resolving the apparent dilemma between control and information

certainty equivalence control with forcing revisited abstract certainty equivalence control with forcing has been shown to be optimal for several stochastic adaptive control problems with the average cost per unit time criterion recently researchers have started looking at stochastic adaptive control problems with a view to minimizing the rate of increase of the learning loss this criterion is stronger than the average cost per unit time criterion certainty equivalence control with forcing does not usually suffice for the learning loss criterion and one has to develop fairly complicated schemes in order to achieve optimality the objective of this paper is to see how well one might be able to do with a certainty equivalence control with forcing type of scheme in particular we construct a class of such schemes whose learning loss is o log n 1 for 0 whereas optimal schemes typically have a o log n learning loss

nonparametric adaptive control of discrete time partially observable stochastic systems abstract we consider discrete time partially observable stochastic control systems x t 1 f x t u t v t t 0 1 y t g u t 1 x t w t t 1 2 y 0 g 0 x 0 w 0 where x t u t and y t are respectively the state control and observation at time t and the state and observation disturbance processes v t and w t are sequences of i i d random elements with distributions and respectively we study the non parametric adaptive control problem of maximizing the infinite horizon discounted reward criterion when and are unknown and determine conditions for several adaptive policies to be asymptotically optimal and for almost surely uniform approximations of the optimal reward function our approach combines convergence results for empirical processes and recent results on parameter adaptive stochastic control problems

strong structural controllability of systems on colored graphs this article deals with strong structural controllability of leader follower networks the system matrix defining the network dynamics is a pattern matrix in which a priori given entries are equal to zero while the remaining entries take nonzero values these nonzero entries correspond to edges in the network graph the network is called strongly structurally controllable if for all choices of real values for the nonzero entries in the pattern matrix the system is controllable in the classical sense the novelty of this article is that we consider the situation that prespecified nonzero entries in the system s pattern matrix are constrained to take identical nonzero values these constraints can be caused by symmetry properties or physical constraints on the network restricting the system matrices to those satisfying these constraints yields a new notion of strong structural controllability the aim of this article is to establish graph theoretic conditions for this more general property of strong structural controllability

adaptive control of constrained markov chains adaptive control of finite state markov chains is discussed the optimal performance is characterized through the minimization of a long run average cost functional subject to constraints on several other such functionals under mild structural and feasibility conditions two explicit adaptive control policies are exhibited for the case where the transition probabilities are unknown the policies are optimal under the constrained optimization criterion they rely on a powerful estimation scheme which provides consistent estimators for the transition probabilities this scheme is of independent interest as it provides strong consistency under a large number of adaptive schemes and is independent of any identifiability conditions as an application an optimal adaptive policy is derived for a system of k competing queues with countable state space for which the constrained criteria arise naturally in the context of communication networks

constructing a controllable graph under edge constraints abstract this paper is concerned with the problem of constructing a controllable graph subject to some practical edge constraints specifically suppose the total amount of vertices and the upper bounds on the graph diameter or on the vertex degree are given we consider the problem of exploring a class of feasible graphs that satisfy the constraints using the hybrid of a path graph and an antiregular graph we propose a simple and systematic method to generate a class of controllable graphs whose diameters or degrees cover the full possible ranges the method to select the control vector to ensure the controllability of the combined graph is also proposed numerical examples are provided to demonstrate our results

zero forcing sets and the minimum rank of graphs the minimum rank of a simple graph g is defined to be the smallest possible rank over all symmetric real matrices whose ijth entry for i 6 j is nonzero whenever i j is an edge in g and is zero otherwise this paper introduces a new graph parameter z g that is the minimum size of a zero forcing set of vertices and uses it to bound the minimum rank for numerous families of graphs often enabling computation of the minimum rank

optimal transmission rate control policies in a wireless link under partial state information we consider the problem of phy layer transmission rate control for maximum throughput on a wireless link over a finite time horizon for each link state there exists a most advantageous transmission rate for the controller the controller needs to rely on incomplete link state information in the form of positive acknowledgements acks and negative acknowledgements nacks for transmitted packets we cast the problem as a partially observed markov decision processes pomdp we focus on the special case of time invariant or slowly time varying link quality with two states and two control options in that case the optimal control problem is a state estimation one and the optimal strategy is of threshold structure where the thresholds are certain values of the belief of being in one of the two states the policy admits a simple interpretation increase rate when the number of successive acks exceeds a threshold and decrease rate when the number of successive nacks exceeds a threshold

adaptive control of markov chains with average cost we present an adaptive control scheme for the control of markov chains to minimize long run average cost when the system transition and reward structures are unknown q factors are estimated along a single sample path of the system and control actions are applied based on the latest estimates we prove that an optimal policy is obtained asymptotically with probability one more importantly we prove that optimal system performance is achieved as well which means that the performance of the system can not be bettered even if the system transition and reward structures are known an example is given to illustrate our adaptive control scheme

zero forcing linear and quantum controllability for systems evolving on networks we study the dynamics of systems on networks from a linear algebraic perspective the control theoretic concept of controllability describes the set of states that can be reached for these systems our main result says that controllability in the quantum sense expressed by the lie algebra rank condition and controllability in the sense of linear systems expressed by the controllability matrix rank condition are equivalent conditions we also investigate how the graph theoretic concept of a zero forcing set impacts the controllability property if a set of vertices is a zero forcing set the associated dynamical system is controllable these results open up the possibility of further exploiting the analogy between networks linear control systems theory and quantum systems lie algebraic theory this study is motivated by several quantum systems currently under study including continuous quantum walks modeling transport phenomena

chaotic neural control we consider the problem of stabilizing unstable equilibria by discrete controls the controls take discrete values at discrete moments of time we prove that discrete control typically creates a chaotic attractor in the vicinity of an equilibrium artificial neural networks with reinforcement learning are known to be able to learn such a control scheme we consider examples of such systems discuss some details of implementing the reinforcement learning to controlling unstable equilibria and show that the arising dynamics is characterized by positive lyapunov exponents and hence is chaotic this chaos can be observed both in the controlled system and in the activity patterns of the controller

total forcing versus total domination in cubic graphs abstract a set s of vertices in a graph g is a total dominating set of g if every vertex has a neighbor in s the total domination number t g is the minimum cardinality of a total dominating set of g a total forcing set in a graph g is a forcing set zero forcing set in g which induces a subgraph without isolated vertices the total forcing number of g denoted ft g is the minimum cardinality of a total forcing set in g our main contribution is to show that the total forcing number and the total domination number of a cubic graph are related more precisely we prove that if g is a connected cubic graph different from k3 3 then f t g 3 2 t g

asymptotically efficient adaptive strategies in repeated games part i certainty equivalence strategies this paper addresses the problem of dynamic decision making in an uncertain and competitive environment a decision maker player 1 faces a system about which he has some parametric uncertainty and which is affected also by the actions of other agents we focus on a worst case analysts from the viewpoint of player 1 using the simplified model of a repeated matrix game with lack of information on one side where single stage rewards are random but announced and perfect observations are assumed certain ideas from the field of stochastic adaptive control are used to formulate performance criteria in a non bayesian setting and to devise appropriate control strategies the basic performance measure is the total reward accumulated by player 1 over all stages played the purpose of player 1 is to guarantee that his expected total reward will be close to what he could guarantee under complete information the present paper considers adaptive decision strategies of the certainty equivalence type based on a modified maximum likelihood estimator and studies their asymptotic long term performance a sequel paper will be devoted to asymptotically optimal strategies

optical orthogonal codes design analysis and applications an optical orthogonal code ooc is a family of 0 1 sequences with good auto and cross correlation properties i e the autocorrelation of each sequence exhibits the thumbtack shape and the cross correlation between any two sequences remains low throughout the use of optical orthogonal codes enables a large number of asynchronous users to transmit information efficiently and reliably the thumbtack shaped autocorrelation facilitates the detection of the desired signal and low profile cross correlation reduces interference from unwanted signals theoretical upper and lower bounds on the maximum possible size of oocs are derived methods for the design and analysis of oocs using tools from projective geometry the greedy algorithm iterative constructions algebraic coding theory block design and various other combinational disciplines are discussed

convergence analysis for reduced order adaptive controller design of uncertain siso linear systems with noisy output measurements we consider in this article a class of uncertain siso linear systems that are subject to system and measurement noises reduced order adaptive controller designs have been proposed before for such systems by the authors and stability analysis of the closed loop systems has been established here we analyse further the robustness properties for these reduced order adaptive control systems by providing detailed convergence analysis results for the key closed loop signals and parameter estimates we rigorously prove that whenever the exogenous disturbance input is of finite energy and bounded and the reference trajectory and its derivatives up to rth order are bounded r being the relative degree of the transfer function of the true system a set of signals including the tracking error the estimation error between the system output and its estimate the projection signal are of finite energy and converge to zero and the system states and their estimates exhibit asymptotic behaviours with certain form

a distance based approach to strong target control of dynamical networks this paper deals with controllability of dynamical networks it is often unfeasible or unnecessary to fully control large scale networks which motivates the control of a prescribed subset of agents of the network this specific form of output control is known under the name target control we consider target control of a family of linear control systems associated with a network and provide both a necessary and a sufficient topological condition under which the network is strongly targeted controllable furthermore a leader selection algorithm is presented to compute leader sets achieving target control

performance of multiuser detection for optical cdma i error probabilities characterization of optimum multiuser performance in intensity modulation direct detection optical cdma channels is addressed analysis is based on some simple bounds on optimum performance provided by receivers with side information or by a linear decorrelating receiver numerical comparisons of the performance bounds are obtained via application of saddlepoint approximations to the multiuser photocount statistics general analytical relations comparing these approximations to similar asymptotic approximations based on the chernoff bound are also derived the numerical results illustrate the fundamental importance of two of the performance bounds in addition the numerical error probability results verify the relevance of the asymptotically based performance analysis for optical cdma in part ii

adaptive control of markov chains with local updates abstract an iterative algorithm is proposed for adaptive expected average cost control of finite state markov chains the algorithm estimates the unknown parameter using a strongly consistent parameter estimator and uses this estimate to update the control policy each update may apply to as few as one or as many as all states as long as all states are updated infinitely often as the number of iterations goes to infinity in this sense the algorithm does local updates no prior knowledge of the optimal control policy is assumed the algorithm requires only a small amount of computation at each step is easily implementable and converges to the optimal policy in finite time its complexity for queueing systems governed by birth death processes is comparable to that of a learning automaton but is based on the optimality equation of dynamic programming

thompson sampling for stochastic control the finite parameter case in this paper we apply thompson sampling to a class of average reward stochastic control problems with parameter uncertainty specifically we study an average reward stochastic control problem over an infinite horizon in which both the reward and state transition distributions are parameterized by an unknown parameter taking values in a finite space the main result of this paper is a proof showing that thompson sampling achieves a worst case average per period regret of o t 1 which is asymptotically optimal

closed loop parameter identifiability and adaptive control of a linear stochastic system abstract we show that if the parameters of a linear stochastic control system are identifiable using a persistently existing input then the same systems remains parameter identifiable if the system operates under closed loop in a certain way we assume that the controller itself depends on the test value of the system parameters and as usual the control signal is dithered the estimation problem is formulated as the problem of minimizing an appropriate asymptotic cost function it is shown that a suitable modification of the gradient of the cost function converts our problem into another problem for which ljung s scheme can be applied thus the theorem provides a general method for the solution of the local adaptive control problem

tuning of gaussian stochastic control systems a closed loop system consisting of a control system and an adaptive controller is called tuning for a specified control objective if the real system and the ideal system defined below achieve the same value for the control objective the real system is the system consisting of the unknown control system in closed loop with the adaptive controller in which the parameters of the adaptive controller have been determined by identification under feedback or in closed loop the ideal system is the system consisting of the unknown control system in closed loop with a controller in which the controller has been synthesized with knowledge of the unknown control system and such that the closed loop system satisfies the control objective both the gaussian stochastic control system with full observations and with partial observations are considered the approach to the problem is based on stochastic realization theory for gaussian systems the control objectives of minimum variance control and pole placement are also given necessary conditions for tuning are discussed

on the value of learning for bernoulli bandits with unknown parameters investigates the multiarmed bandit problem where each arm generates an infinite sequence of bernoulli distributed rewards the parameters of these bernoulli distributions are unknown and initially assumed to be beta distributed every time a bandit is selected its beta distribution is updated to new information in a bayesian way the objective is to maximize the long term discounted rewards we study the relationship between the necessity of acquiring additional information and the reward this is done by considering two extreme situations which occur when a bandit has been played n times the situation where the decision maker stops learning and the situation where the decision maker acquires full information about that bandit we show that the difference in reward between this lower and upper bound goes to zero as n grows large

on the convergence of self tuning stochastic servo algorithms based on stochastic approximation the problem of the adaptive tracking of stochastic reference signals is considered supposing that the process can be represented by an armax model with arbitrary time delay and the reference signal by an arma model two algorithms of the stochastic approximation type providing adaptation to both process and reference characteristics are proposed they differ by the incorporated a priori knowledge about the optimal regulator parameters global stability asymptotic optimally convergence of the adaptive control law in a cesaro sense and the strong consistency of the parameter estimates are proved the persistence of excitation condition is analyzed

on the reachability and observability of path and cycle graphs in this technical note we investigate the reachability and observability properties of a network system running a laplacian based average consensus algorithm when the communication graph is a path or a cycle specifically we provide necessary and sufficient conditions based on simple rules from number theory to characterize all and only the nodes from which the network system is reachable respectively observable interesting immediate corollaries of our results are i a path graph is reachable observable from any single node if and only if the number of nodes of the graph is a power of two n 2i i n and ii a cycle is reachable observable from any pair of nodes if and only if n is a prime number for any set of control observation nodes we provide a closed form expression for the unreachable unobservable eigenvalues and for the eigenvectors of the unreachable unobservable subsystem

strong consistency of parameter estimates in direct self tuning control algorithms based on stochastic approximation in this paper convergence of the regulator parameter estimates generated by direct self tuning control algorithms of stochastic approximation type is considered the process is supposed to be represented by an armax model with an arbitrary delay while a dither signal converging to zero in the mean square sense is superimposed on a deterministic reference the strong consistency of the regulator parameter estimates is proved under assumptions concerning the estimator complexity and the properties of the optimal regulator transfer function for a broad class of dither signals the impact of a priori information on the dimension of the estimator and the consistency of the parameter estimates is indicated it is demonstrated that the classical persistence of excitation condition is not satisfied for the analyzed algorithms

on adaptive boundary control for stochastic parabolic systems with unknown potential coefficient an adaptive boundary control problem for a stochastic heat diffusion equation is studied the considered system contains an unknown potential coefficient which is a function of the spatial variables the estimation algorithm for the unknown potential coefficient is proposed by using the stochastic approximation technique after showing the strong consistency of the estimated parameter the cost for the adaptive control scheme presented here is shown to converge to the optimal ergodic cost finally some numerical examples are shown

comparing policies in markov decision processes mandl s lemma revisited a general framework is developed for comparing the long run average cost of a markov stationary policy with that of another related policy the underlying methodology constitutes an extension of some ideas of mandl to randomized policies and to polish state and action spaces sufficient conditions for the applicability of the methodology are given these conditions which are easy to verify have a natural probabilistic interpretation in terms of the stability of the chain and of the convergence of the control values the usefulness of the general framework proposed here is illustrated on several applications standard results on the convergence of adaptive policies are readily recovered under conditions which are more transparent than the ones existing in the literature and the convergence of randomized policies is handled as a special case finally a novel application to probing controls is outlined

asymptotic behavior of stochastic systems possessing markovian realizations markovian stochastic systems of the form phi k 1 f phi k w k 1 are considered where bf w is an independent and identically distributed process and bf phi is a markov chain both evolving in euclidean space a condition called weak stochastic controllability w s c is introduced which is a generalization of the concept of controllability in linear system theory two different formulations of stability boundedness in probability and boundedness in probability on average are presented and it is shown that these conditions are equivalent under the w s c condition these results are related to the ergodic behavior of bf phi by a set of results that includes the following if bf phi is bounded in probability and w s c then i sample path averages of functions of the state process converge for every initial condition and ii if the stability is uniform then state process probabilities converge to a periodic orbit of probabilities finally if the origin o

minimum rank problems abstract a graph describes the zero nonzero pattern of a family of matrices with the type of graph undirected or directed simple or allowing loops determining what type of matrices symmetric or not necessarily symmetric diagonal entries free or constrained are described by the graph the minimum rank problem of the graph is to determine the minimum among the ranks of the matrices in this family the determination of maximum nullity is equivalent this problem has been solved for simple trees p m nylen minimum rank matrices with prescribed graph linear algebra appl 248 1996 303 316 c r johnson a leal duarte the maximum multiplicity of an eigenvalue in a matrix whose graph is a tree linear and multilinear algebra 46 1999 139 144 trees allowing loops l m dealba t l hardy i r hentzel l hogben a wangsness minimum rank and maximum eigenvalue multiplicity of symmetric tree sign patterns linear algebra appl 418 2006 389 415 and directed trees allowing loops f barioli s fallat d hershkowitz h t hall l hogben h van der holst b shader on the minimum rank of not necessarily symmetric matrices a preliminary study electron j linear algebra 18 2000 126 145 we survey these results from a unified perspective and solve the minimum rank problem for simple directed trees

computationally efficient algorithms for on line optimization of markov decision processes abstract iterative algorithms are proposed for on line optimization of finite state markov decision processes the optimization criterion is the minimization of long run average cost the transition probabilities of the underlying markov chain are assumed to depend on certain unknown parameters the algorithms estimate the unknown para meters using a strongly consistent estimator and use this estimate in place of the true parameter to generate a sequence of control policies which converge to an optimal policy the main feature of the algorithms is that the updating step does not have to be carried out in all states at each iteration at each iteration the updating step may be performed in any number of states as few as one as long as all states are updated often enough as the number of iterations increases in this sense the algorithms are asynchronous the asynchronous nature of our algorithms allows us to update the costly states more often as will be illustrated through simulation results the asynchronism of our algorithms makes them feasible for on line optimization

zero forcing number constrained matchings and strong structural controllability abstract the zero forcing number is a graph invariant introduced to study the minimum rank of the graph in 2008 aazami proved the np hardness of computing the zero forcing number of a simple undirected graph we complete this np hardness result by showing that the non equivalent problem of computing the zero forcing number of a directed graph allowing loops is also np hard the rest of the paper is devoted to the strong controllability of a networked system this kind of controllability takes into account only the structure of the interconnection graph but not the interconnection strengths along the edges we provide a necessary and sufficient condition in terms of zero forcing sets for the strong controllability of a system whose underlying graph is a directed graph allowing loops moreover we explain how our result differs from a recent related result discovered by monshizadeh et al finally we show how to solve the problem of finding efficiently a minimum size input set for the strong controllability of a self damped system with a tree structure

theory for automatic learning under partially observed markov dependent noise a vigorous branch of automatic learning is directed at the task of locating a global minimum of an unknown multimodal function f theta on the basis of noisy observations l theta i f theta i w theta i taken at sequentially chosen control points theta i in all preceding convergence deviations known to the authors the noise is postulated to depend on the past only through control selection here they allow the observation noise sequence to be stochastically dependent in particular a function of an unknown underlying markov decision process the observations being the stagewise losses in a sense in order to be made precise the algorithm offered is shown to attain asymptotically optimal performance and rates are assured a motivating example from queueing theory is offered and connections with classical problems of markov control theory and other disciplines are mentioned

parallel recursive algorithms in asymptotically efficient adaptive control of linear stochastic systems first a review of some recent developments in stochastic adaptive control of linear stochastic systems is given by integrating and refining several basic ideas in these developments a relatively complete asymptotic solution to the adaptive control problem is then provided for such systems the solution involves parallel implementation of a few basic recursive identification algorithms that serve to complement each other it also involves occasional use of a dither signal to probe the system for information

stochastic dynamic programming caution and probing the purpose of this paper is to unify the concepts of caution and probing put forth by feldbaum 14 with the mathematical technique of stochastic dynamic programming originated by bellman 5 the decomposition of the expected cost in a stochastic control problem recently developed in 8 is used to assess quantitatively the caution and probing effects of the system uncertainties on the control it is shown how in some problems because of the uncertainties the control becomes cautious less aggressive while in other problems it will probe by becoming more aggressive in order to enhance the estimation identification while controlling the system following this a classification of stochastic control problems according to the dominant effect is discussed this is then used to point out which are the stochastic control problems where substantial improvements can be expected from using a sophisticated algorithm versus a simple one

monotonicity of likelihood ratio of map correlation detection for ocdma communication in this paper the monotonicity of the likelihood ratio used in the optimal correlation detection is established for optical code division multiple access ocdma the monotonicity property not only guarantees the optimality of the detection it also enables us to calculate the bit error rate bounds for ocdma in the presence of uncertain a priori probabilities