supervisory control of a class of discrete event processes this paper studies the control of a class of discrete event processes i e processes that are discrete asynchronous and possibly nondeter ministic the controlled process is described as the generator of a formal language while the controller or supervisor is constructed from the grammar of a specified target language that incorporates the desired closed loop system behavior the existence problem for a supervisor is reduced to finding the largest controllable language contained in a given legal language two examples are provided

metrics for labelled markov processes the notion of process equivalence of probabilistic processes is sensitive to the exact probabilities of transitions thus a slight change in the transition probabilities will result in two equivalent processes being deemed no longer equivalent this instability is due to the quantitative nature of probabilistic processes in a situation where the process behavior has a quantitative aspect there should be a more robust approach to process equivalence this paper studies a metric between labelled markov processes this metric has the property that processes are at zero distance if and only if they are bisimilar the metric is inspired by earlier work on logics for characterizing bisimulation and is related in spirit to the hutchinson metric

bisimulation for probabilistic transition systems a coalgebraic approach the notion of bisimulation as proposed by larsen and skou for discrete probabilistic transition systems is shown to coincide with a coalgebraic definition in the sense of aczel and mendier in terms of a set functor this coalgebraic formulation makes it possible to generalize the concepts to a continuous setting involving borel probability measures under reasonable conditions generalized probabilistic bisimilarity can be characterized categorically application of the final coalgebra paradigm then yields an internally fully abstract semantical domain with respect to probabilistic bisimulation

reactive generative and stratified models of probabilistic processes abstract we introduce three models of probabilistic processes namely reactive generative and stratified these models are investigated within the context of pccs an extension of milner s sccs in which each summand of a process summation expression is guarded by a probability and the sum of these probabilities is 1 for each model we present a structural operational semantics of pccs and a notion of bisimulation equivalence which we prove to be a congruence we also show that the models form a hierarchy the reactive model is derivable from the generative model by abstraction from the relative probabilities of different actions and the generative model is derivable from the stratified model by abstraction from the purely probabilistic branching structure moreover the classical nonprobabilistic model is derivable from each of these models by abstraction from all probabilities

a note on deciding the controllability of a language k with respect to a language l it has been shown by p j ramadge and w m wonham 1987 if g is a plant automaton and k contained in l g is a prefix closed language there exists a supervisor theta that is complete with respect to g so that l theta g k intersection l g k if and only if k is controllable with respect to the plant language l d it is shown that if l g and k are represented as free labeled petri nets then the controllability of k with respect to l g is detectable this result is a direct consequence of the decidability of the petri net and reachability problem in effect a modeling framework capable of finitely representing a class of infinite state systems is identified and controllability is decidable within this framework

signal reconstruction from noisy partial information of its transform the usual assumption that the available partial information is noiseless is replaced by a more realistic statistical model which compensates for the presence of noise the signal reconstruction is thus viewed as a parameter estimation problem for which the em iterative algorithm of a p dempster n m laird and d b rubin j roy stat soc b vol 39 p 1 37 1977 is especially suitable the posterior probability of the signal increases from iteration to iteration till the signal converges to a stationary point of the posterior distribution each iteration involves one transformation followed by an inverse transformation usually discrete fourier transformation dft and inverse dft algorithms for reconstruction of both one and two dimensional signals from their spectral magnitude spectral phase or modified short time fourier transform are typical examples of the proposed scheme

identification and restoration of noisy blurred images using the expectation maximization algorithm a maximum likelihood approach to the blur identification problem is presented the expectation maximization algorithm is proposed to optimize the nonlinear likelihood function in an efficient way in order to improve the performance of the identification algorithm low order parametric image and blur models are incorporated into the identification method the resulting iterative technique simultaneously identifies and restores noisy blurred images

observation equivalence as a testing equivalence a notion of testing is developed for transition systems with divergence the forms of testing include traces refusals copying and global testing both denotational and operational formulations of testing are given the equivalence based on this notion of testing is shown to coincide with observation equivalence

approximating and computing behavioural distances in probabilistic transition systems in an earlier paper we presented a pseudometric on the states of a probabilistic transition system yielding a quantitative notion of behavioural equivalence the behavioural pseudometric was defined via the terminal coalgebra of a functor based on a metric on borel probability measures in the present paper we give a polynomial time algorithm based on linear programming to calculate the distances between states up to a prescribed degree of accuracy

a note on the properties of the supremal controllable sublanguage in pushdown systems consider an event alphabet sigma the supervisory control theory of ramadge and wonham asks the question given a plant model g with language lm g sube sigma and another language k sube lm g is there a supervisor phi such that lm phi g k ramadge and wonham showed that a necessary condition for this to be true is the so called controllability of k with respect to lm g they showed that when g is a finite state automaton and k is a regular language also generated by a finite state automaton then there is a regular supremal controllable sublanguage supc k sube k that is effectively constructable from generators of k and g in this paper we show 1 there is an algorithm to compute the supremal controllable sublanguage of a prefix closed k accepted by a deterministic pushdown automaton dpda when the plant language is also prefix closed and accepted by a finite state automaton and 2 in this case we show that this supremal controllable sublanguage is also accepted by a dpda

testing probabilistic equivalence through reinforcement learning checking if a given system implementation respects its specification is often done by proving that the two are equivalent the equivalence is chosen in particular for its computability and of course for its meaning that is for its adequacy with what is observable from the two systems implementation and specification trace equivalence is easily testable decidable from interaction but often considered too weak in contrast bisimulation is accepted as the canonical equivalence for interaction but it is not testable richer than an equivalence is a form of distance it is zero between equivalent systems and it provides an estimation of their difference if the systems are not equivalent our main contribution is to define such a distance in a context where 1 the two systems to be compared have a stochastic behavior 2 the model of one of them e g the implementation is unknown hence our only knowledge is obtained by interacting with it 3 consequently the target equivalence observed when distance is zero must be testable to overcome the problem that the model is unknown we use a reinforcement learning approach that provides powerful stochastic algorithms that only need to interact with the model our second main contribution is a new family of testable equivalences called k moment the weakest of them 1 moment equivalence is trace equivalence as k grows k moment equivalences become finer all remaining as well as their limit weaker than bisimulation we propose a framework to define and test a bigger class of testable equivalences test observation equivalences toes and we show how they can be made coarser or not by tuning some parameters

testing preorders for probabilistic processes we present a testing preorder for probabilistic processes based on a quantification of the probability with which processes pass tests the theory enjoys close connections with the classical testing theory of de nicola and hennessy in that whenever a process passes a test with probability 1 respectively some nonzero probability in our setting then the process must respectively may pass the test in the classical theory we also develop an alternative characterization of the probabilistic testing preorders that takes the form of a mapping from probabilistic traces to the interval 0 1 where a probabilistic trace is an alternating sequence of actions and probability distributions over actions finally we give proof techniques derived from the alternative characterizations for establishing preorder relationships between probabilistic processes the utility of these techniques is demonstrated by means of some simple examples

planning and acting in partially observable stochastic domains in this paper we bring techniques from operations research to bear on the problem of choosing optimal actions in partially observable stochastic domains we begin by introducing the theory of markov decision processes mdps and partially observable mdps pomdps we then outline a novel algorithm for solving pomdps off line and show how in some cases a finite memory controller can be extracted from the solution to a pomdp we conclude with a discussion of how our approach relates to previous work the complexity of finding exact solutions to pomdps and of some possibilities for finding approximate solutions

bisimulation through probabilistic testing abstract we propose a language for testing concurrent processes and examine its strength in terms of the processes that are distinguished by a test by using probabilistic transition systems as the underlying semantic model we show how a testing algorithm can distinguish with a probability arbitrarily close to one between processes that are not bisimulation equivalent we also show a similar result in a slightly stronger form for a new process relation called 2 3 bisimulation which lies strictly between that of simulation and bisimulation finally the ultimately strength of the testing language is shown to identify a new process relation called probabilistic bisimulation which is strictly stronger than bisimulation

kalman filtering in two dimensions further results the two dimensional reduced update kalman filter was recently introduced the corresponding scalar filtering equations were derived for the case of estimating a gaussian signal in white gaussian noise and were shown to constitute a general nonsymmetric half plane recursive filter this paper extends the method to the deconvolution problem of image restoration this paper also provides a more thorough treatment of the uniquely two dimensional boundary condition problems numerical and subjective examples are presented

algebraic laws for nondeterminism and concurrency since a nondeterministic and concurrent program may in general communicate repeatedly with its environment its meaning cannot be presented naturally as an input output function as is often done in the denotational approach to semantics in this paper an alternative is put forth first a definition is given of what it is for two programs or program parts to be equivalent for all observers then two program parts are said to be observation congruent if they are in all program contexts equivalent the behavior of a program part that is its meaning is defined to be its observation congruence class the paper demonstrates for a sequence of simple languages expressing finite terminating behaviors that in each case observation congruence can be axiomatized algebraically moreover with the addition of recursion and another simple extension the algebraic language described here becomes a calculus for writing and specifying concurrent programs and for proving their properties

on the supermal controllable sublanguage of a given language the concept of controllable language has been shown to play a basic role in the existence theory of supervisory controls for discrete event processes in this paper the supremal controllable sublanguage s of a given language l is characterized as the largest fixpoint of a monotone operator omega in the case where the languages involved are regular it is shown that the fixpoint s can be computed as the limit of the finite sequence k j given by k j 1 omega k j k 0 l an effective computational algorithm is developed and three examples are provided for illustration

on controllability and normality of discrete event dynamical systems abstract this paper studies the supervisor synthesis problem of discrete event dynamical systems deds s through the use of synchronous composition of the plant and the supervisor and discusses some of the simplifications achieved by using the synchronous composition to model supervisory control an alternate definition of controllability is presented this definition of controllability is then used to derive an algorithm that is computationally more efficient than previously existing ones for the construction of the supremal controllable sublanguage the algorithm is also shown to be optimal the observability and normality issues arising due to the partial observation of the system dynamics under an arbitrary mask are then investigated closed form representations of the supremal normal and supremal closd and normal sublanguages are derived in the more general setting of arbitrary mask and nonclosed languages thus extending earlier results of the authors and others

a note on deciding controllability in pushdown systems consider an event alphabet spl sigma the supervisory control theory of ramadge and wonham asks the question given a plant model g with language k l sub m g spl sube spl sigma sup and another language k spl sube l sub m g is there a supervisor spl psi such that l sub m spl psi g k ramadge and wonham showed that a necessary condition for this to be true is the so called controllability of k with respect to l sub m g they showed that when g is a finite state automaton and k is a regular language also generated by a finite state automaton then the controllability property was decidable for k the class of languages generated by pushdown automata properly includes the regular languages they are accepted by finite state machines coupled with pushdown stack memory this makes them interesting candidates as supervisory languages since the supervisor will have nonfinite memory in this note we show the following i if s is a specification given by a deterministic pushdown automaton and l is generated by a finite state machine then there is an algorithm to decide whether k s spl cap l is controllable with respect to l ii it is undecidable for an arbitrary specification s generated by a nondeterministic pushdown automaton and plant language l generated by a finite state machine whether k s spl cap l is controllable with respect to l

formulas for calculating supremal controllable and normal sublanguages abstract supremal controllable and normal sublanguages have been shown to play an important role in supervisor synthesis in this paper we discuss the computation of supremal controllable and normal sublanguages we derive formulas for both supremal controllable sublanguages and supremal normal sublanguages when the languages involved are closed as a result those languages can be computed without applying recursive algorithms we also discuss those aspects of these formulas