asymptotic optimality theory for decentralized sequential hypothesis testing in sensor networks the decentralized sequential hypothesis testing problem is studied in sensor networks where a set of sensors receive independent observations and send summary messages to the fusion center which makes a final decision in the scenario where the sensors have full access to their past observations the first asymptotically bayes sequential test is developed having the same asymptotic performance as the optimal centralized test that has access to all sensor observations next in the scenario where the sensors do not have full access to their past observations a simple but asymptotically bayes sequential tests is developed in which sensor message functions are what we call tandem quantizer where each sensor only uses two different sensor quantizers with at most one switch between these two possibilities moreover a new minimax formulation of optimal stationary sensor quantizers is proposed and is studied in detail in the case of additive gaussian sensor noise finally our results show that in the simplest models feedback from the fusion center does not improve asymptotic performance in the scenario with full local memory however even a one shot one bit feedback can significantly improve performance in the case of limited local memory

graph identification by simulated annealing a new theoretical approach to graph processing based on the transformation of a topological graph into a time dependent periodic melody is proposed by using a self tuning oscillatory dynamical network graphs can be uniquely reconstructed from their melodies this opens a perspective for a content addressable memory invariant under large topology preserving distortions

learning in boltzmann trees we introduce a large family of boltzmann machines that can be trained by standard gradient descent the networks can have one or more layers of hidden units with tree like connectivity we show how to implement the supervised learning algorithm for these boltzmann machines exactly without resort to simulated or mean field annealing the stochastic averages that yield the gradients in weight space are computed by the technique of decimation we present results on the problems of n bit parity and the detection of hidden symmetries

quantization effect on the log likelihood ratio and its application to decentralized sequential detection it is well known that quantization cannot increase the kullback leibler divergence which can be thought of as the expected value or first moment of the log likelihood ratio in this paper we investigate the quantization effects on the second moment of the log likelihood ratio it is shown via the convex domination technique that quantization may result in an increase in the case of the second moment but the increase is bounded above by 2 e the result is then applied to decentralized sequential detection problems not only to provide simpler sufficient conditions for asymptotic optimality theories in the simplest models but also to shed new light on more complicated models in addition some brief remarks on other higher order moments of the log likelihood ratio are also provided

constructive neural networks as estimators of bayesian discriminant functions abstract of crucial importance to the successful use of artificial neural networks for pattern classification problems is how the appropriate network size can be automatically determined this issue is addressed by formulating the process as an automatic search in the space of functions that corresponds to a subclass of multilayer feedforward networks learning is thus a dynamic network construction process which involves adjusting both the network weights and the topology adding new hidden units corresponds to extracting higher level features from the original input features for reducing the residual classification errors it can be shown that the resultant network approximates a bayesian classifier that implements the bayesian decision rule for classification the empirical results of several pattern classification experiments are also reported

information bounds and quick detection of parameter changes in stochastic systems by using information theoretic bounds and sequential hypothesis testing theory this paper provides a new approach to optimal detection of abrupt changes in stochastic systems this approach not only generalizes previous work in the literature on optimal detection far beyond the relatively simple models treated but also suggests alternative performance criteria which are more tractable and more appropriate for general stochastic systems in addition it leads to detection rules which have manageable computational complexity for on line implementation and yet are nearly optimal under the different performance criteria considered

information bounds and quickest change detection in decentralized decision systems the quickest change detection problem is studied in decentralized decision systems where a set of sensors receive independent observations and send summary messages to the fusion center which makes a final decision in the system where the sensors do not have access to their past observations the previously conjectured asymptotic optimality of a procedure with a monotone likelihood ratio quantizer mlrq is proved in the case of additive gaussian sensor noise if the signal to noise ratios snr at some sensors are sufficiently high this procedure can perform as well as the optimal centralized procedure that has access to all the sensor observations even if all snrs are low its detection delay will be at most pi 2 1ap 57 larger than that of the optimal centralized procedure next in the system where the sensors have full access to their past observations the first asymptotically optimal procedure in the literature is developed surprisingly the procedure has the same asymptotic performance as the optimal centralized procedure although it may perform poorly in some practical situations because of slow asymptotic convergence finally it is shown that neither past message information nor the feedback from the fusion center improves the asymptotic performance in the simplest model

asymptotic optimality theory for decentralized sequential multihypothesis testing problems the bayesian formulation of sequentially testing m 3 hypotheses is studied in the context of a decentralized sensor network system in such a system local sensors observe raw observations and send quantized sensor messages to a fusion center which makes a final decision when stopping taking observations asymptotically optimal decentralized sequential tests are developed from a class of two stage tests that allows the sensor network system to make a preliminary decision in the first stage and then optimize each local sensor quantizer accordingly in the second stage it is shown that the optimal local quantizer at each local sensor in the second stage can be defined as a maximin quantizer which turns out to be a randomization of at most m 1 unambiguous likelihood quantizers ulq we first present in detail our results for the system with a single sensor and binary sensor messages and then extend to more general cases involving any finite alphabet sensor messages multiple sensors or composite hypotheses

digital image restoration using spatial interaction models this paper is concerned with developing fast nonrecursive algorithms for the minimum mean squared error restoration of degraded images the degradation is assumed to be due to a space invariant periodic nonseparable known point spread function and additive white noise our basic approach is to represent the images by a class of spatial interaction models namely the simultaneous autoregressve models and the conditional markov models defined on toroidal lattices and develop minimum mean squared error restoration algorithms using these models the restoration algorithms are optimal if the parameters characterizing the interaction models are exactly known however in practice the parameters are estimated from the images by using spatial interaction models we develop restoration algorithms that do not require the availability of the original image or its prototype the specific structure of the underlying lattice enables the implementation of the filters using fast fourier transform fft computations several restoration examples are given

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

reward rate maximization in sequential identification under a stochastic deadline any intelligent system performing evidence based decision making under time pressure must negotiate a speed accuracy trade off in computer science and engineering this is typically modeled as minimizing a bayes risk functional that is a linear combination of expected decision delay and expected terminal decision loss in neuroscience and psychology however it is often modeled as maximizing the long term reward rate or the ratio of expected terminal reward and expected decision delay the two approaches have opposing advantages and disadvantages while bayes risk minimization can be solved with powerful dynamic programming techniques unlike reward rate maximization it also requires the explicit specification of the relative costs of decision delay and error which is obviated by reward rate maximization here we demonstrate that for a large class of sequential multihypothesis identification problems under a stochastic deadline the reward rate maximization is equivalent to a special case of bayes r

a repeated significance test with applications to sequential detection in sensor networks in this paper we introduce a randomly truncated sequential hypothesis test using the framework of a repeated significance test rst we study a sequential test with truncation time based on a random stopping time using the functional central limit theorem fclt for a sequence of statistics we derive a general result that can be employed in developing a repeated significance test with random sample size we present effective methods for evaluating accurate approximations for the probability of type i error and the power function numerical results are presented to evaluate the accuracy of these approximations we apply the proposed test to a decentralized sequential detection problem in sensor networks sns with communication constraints finally a sequential detection problem with measurements at random times is investigated

decentralized estimation in an inhomogeneous sensing environment we consider decentralized estimation of a noise corrupted deterministic parameter by a bandwidth constrained sensor network with a fusion center the sensor noises are assumed to be additive zero mean spatially uncorrelated but otherwise unknown and possibly different across sensors due to varying sensor quality and inhomogeneous sensing environment the classical best linear unbiased estimator blue linearly combines the real valued sensor observations to minimize the mean square error mse unfortunately such a scheme cannot be implemented in a practical bandwidth constrained sensor network due to its requirement to transmit real valued messages in this paper we construct a decentralized estimation scheme des where each sensor compresses its observation to a small number of bits with length proportional to the logarithm of its local signal to noise ratio snr the resulting compressed bits from different sensors are then collected and combined by the fusion center to estimate the unknown parameter the proposed des is universal in the sense that each sensor compression scheme requires only the knowledge of local snr rather than the noise probability distribution functions pdf while the final fusion step is also independent of the local noise pdfs we show that the mse of the proposed des is within a constant factor of 25 8 of that achieved by the classical centralized blue estimator

markov random field texture models we consider a texture to be a stochastic possibly periodic two dimensional image field a texture model is a mathematical procedure capable of producing and describing a textured image we explore the use of markov random fields as texture models the binomial model where each point in the texture has a binomial distribution with parameter controlled by its neighbors and number of tries equal to the number of gray levels was taken to be the basic model for the analysis a method of generating samples from the binomial model is given followed by a theoretical and practical analysis of the method s convergence examples show how the parameters of the markov random field control the strength and direction of the clustering in the image the power of the binomial model to produce blurry sharp line like and blob like textures is demonstrated natural texture samples were digitized and their parameters were estimated under the markov random field model a hypothesis test was used for an objective assessment of goodness of fit under the markov random field model overall microtextures fit the model well the estimated parameters of the natural textures were used as input to the generation procedure the synthetic microtextures closely resembled their real counterparts while the regular and inhomogeneous textures did not

lyapunov exponents for two nonlinear systems driven by real noises for a van der pol duffing oscillator and a codimension two bifurcation system excited parametrically by a small real noise a model of enhanced generality is developed by assuming that the real noise is the first component of an output vector of a linear filter system which conforms with the detailed balance condition the strong mixing condition and the hypothesis i e the infinitesimal generator of the noise has only one isolated simple zero eigenvalue employed by arnold and co workers in 1986 are not considered to tackle the complexity encountered in the present work the asymptotic analysis approach and the eigenfunction expansion for the solution to the fokker planck equations are applied to calculate the asymptotic expansions of the invariant measures and the maximal lyapunov exponents for the relevant systems

learning symmetry groups with hidden units beyond the perceptron learning to recognize mirror rotational and translational symmetries is a difficult problem for massively parallel network models these symmetries cannot be learned by first order perceptrons or hopfield networks which have no means for incorporating additional adaptive units that are hidden from the input and output layers we demonstrate that the boltzmann learning algorithm is capable of finding sets of weights which turn hidden units into useful higher order feature detectors capable of solving symmetry problems

detection of an anomalous cluster in a network we consider the problem of detecting whether or not in a given sensor network there is a cluster of sensors which exhibit an unusual behavior formally suppose we are given a set of nodes and attach a random variable to each node we observe a realization of this process and want to decide between the following two hypotheses under the null the variables are i i d standard normal under the alternative there is a cluster of variables that are i i d normal with positive mean and unit variance while the rest are i i d standard normal we also address surveillance settings where each sensor in the network collects information over time the resulting model is similar now with a time series attached to each node we again observe the process over time and want to decide between the null where all the variables are i i d standard normal and the alternative where there is an emerging cluster of i i d normal variables with positive mean and unit variance the growth models used to represent the emerging cluster are quite general and in particular include cellular automata used in modelling epidemics in both settings we consider classes of clusters that are quite general for which we obtain a lower bound on their respective minimax detection rate and show that some form of scan statistic by far the most popular method in practice achieves that same rate within a logarithmic factor our results are not limited to the normal location model but generalize to any one parameter exponential family when the anoma lous clusters are large enough

almost sure asymptotic stability of a general four dimensional system driven by real noise in the first part of this paper we construct an asymptotic expansion for the maximal lyapunov exponent the exponential growth rate of solutions to a linear stochastic system and the rotation numbers for a general four dimensional dynamical system driven by a small intensity real noise process stability boundaries are obtained provided the natural frequencies are noncommensurable and the infinitesimal generator associated with the noise process has an isolated simple zero eigenvalue this work is an extension of the work of sri namachchivaya and van roessel and is general in the sense that general stochastic perturbations of nonautonomous systems with two noncommensurable natural frequencies are considered

maximal lyapunov exponent and rotation number for stochastically perturbed co dimension two bifurcations abstract almost sure asymptotic stability of three and four dimensional co dimension two dynamical systems under small intensity stochastic excitations is investigated the method of stochastic averaging is used to derive a set of approximate ito equations these equations along with their sample properties are then examined to obtain the almost sure stability conditions the sample properties of the process are based on the boundary behavior of the associated scalar diffusion process of the amplitude ito equations the maximal lyapunov exponent is calculated using the ergodic scalar diffusive process which in turn yields the almost sure stability conditions this method is then applied to a linear gyroscopic problem of a rotating shaft with a random loading

decentralized sequential hypothesis testing using asynchronous communication an asymptotically optimum test for the problem of decentralized sequential hypothesis testing is presented the induced communication between sensors and fusion center is asynchronous and limited to 1 bit data when the sensors observe continuously stochastic processes with continuous paths the proposed test is order 2 asymptotically optimal in the sense that its inflicted performance loss is bounded when the sensors take discrete time observations the proposed test achieves order 1 asymptotic optimality i e the ratio of its performance over the optimal performance tends to 1 moreover we show theoretically and corroborate with simulations that the performance of the suggested test in discrete time can be significantly improved when the sensors sample their underlying continuous time processes more frequently a property which is not enjoyed by other centralized or decentralized tests in the literature

on the almost sure stability condition for a co dimension two bifurcation system under the parametric excitation of a real noise abstract in this paper the almost sure stability condition for a co dimension two bifurcation system on a three dimensional central manifold which is parametrically excited by a real noise is investigated a model of enhanced generality is developed by assuming the real noise as the first component of an output of a linear filter system a zero mean stationary gaussian diffusion vectoral process which conforms to the detailed balance condition the strong mixing condition which is the essential theoretic basis for the stochastic averaging method is removed in the present study to solve the complicated problem encountered in this work the asymptotic analysis approach and the eigenfunction expansion of the solutions to the relevant fokker planck equations are employed in the construction of the asymptotic expansions of the invariant measures and the maximal lyapunov exponents for the relevant system

extremal properties of likelihood ratio quantizers m hypotheses and a random variable y with a different probability distribution under each hypothesis are considered a quantizer is applied to form a quantized random variable gamma y the extreme points of the set of possible probability distributions of gamma y as gamma ranges over all quantizers is characterized optimality properties of likelihood ratio quantizers are established for a very broad class of quantization problems including problems involving the maximization of an ali silvey 1966 distance measure and the neyman pearson variant of the decentralized detection problem

learning to generalize from single examples in the dynamic link architecture a large attraction of neural systems lies in their promise of replacing programming by learning a problem with many current neural models is that with realistically large input patterns learning time explodes this is a problem inherent in a notion of learning that is based almost entirely on statistical estimation we propose here a different learning style where significant relations in the input pattern are recognized and expressed by the unsupervised self organization of dynamic links the power of this mechanism is due to the very general a priori principle of conservation of topological structure we demonstrate that style with a system that learns to classify mirror symmetric pixel patterns from single examples

multihypothesis sequential probability ratio tests i asymptotic optimality the problem of sequential testing of multiple hypotheses is considered and two candidate sequential test procedures are studied both tests are multihypothesis versions of the binary sequential probability ratio test sprt and are referred to as msprts the first test is motivated by bayesian optimality arguments while the second corresponds to a generalized likelihood ratio test it is shown that both msprts are asymptotically optimal relative not only to the expected sample size but also to any positive moment of the stopping time distribution when the error probabilities or more generally risks associated with incorrect decisions are small the results are first derived for the discrete time case of independent and identically distributed i i d observations and simple hypotheses they are then extended to general possibly continuous time statistical models that may include correlated and nonhomogeneous observation processes it also demonstrated that the results can be extended to hypothesis testing problems with nuisance parameters where the composite hypotheses due to nuisance parameters can be reduced to simple ones by using the principle of invariance these results provide a complete generalization of the results given by veeravalli and baum see ibid vol 41 p 1994 97 1995 where it was shown that the quasi bayesian msprt is asymptotically efficient with respect to the expected sample size for i i d observations