a discussion of various approaches to the identification approximation problem the identification approximation of a linear system by poles and residues from a measured finite length input output record of the system is discussed the objective of this paper is to illustrate that several different formulations for characterizing the impulse response of a system yield the same set of poles

combined sequence of markov parameters and moments in linear systems the combined sequence of markov parameters and moments in linear systems is discussed this sequence is shown to be generated by a linear feedback shift register application to the minimal degree partial realization from a mixture of any number of markov parameters and moments is considered central in this particular realization problem is the extension of a toeplitz partial behavior matrix this is accomplished on the basis of the algebraic theory of iohvidov indexes for toeplitz matrices

a new discrete impulse response gramian and its application to model reduction some fundamental properties of an impulse response gramian for linear time invariant asymptotically stable discrete single input single output siso systems are derived this gramian is system invariant and can be found by solving a lyapunov equation the connection with standard controllability observability and cross gramians is proven the significance of these results in model order reduction is highlighted with an efficient procedure

on the feedback capacity of power constrained gaussian noise channels with memory for a stationary additive gaussian noise channel with a rational noise power spectrum of a finite order l we derive two new results for the feedback capacity under an average channel input power constraint first we show that a very simple feedback dependent gauss markov source achieves the feedback capacity and that kalman bucy filtering is optimal for processing the feedback based on these results we develop a new method for optimizing the channel inputs for achieving the cover pombra block length n feedback capacity by using a dynamic programming approach that decomposes the computation into n sequentially identical optimization problems where each stage involves optimizing o l 2 variables second we derive the explicit maximal information rate for stationary feedback dependent sources in general evaluating the maximal information rate for stationary sources requires solving only a few equations by simple nonlinear programming for first order autoregressive and or moving average arma noise channels this optimization admits a closed form maximal information rate formula the maximal information rate for stationary sources is a lower bound on the feedback capacity and it equals the feedback capacity if the long standing conjecture that stationary sources achieve the feedback capacity holds

representation of electromagnetic responses in time domain using state space system identification method this paper presents a control system methodology based on system identification si to derive state matrices and system dynamics for the extrapolation of simulated time domain electromagnetic em responses using both input and output data the method is applied to model transient responses for a dielectric resonator and a wideband slot antenna computed by the finite integration method using non gaussian excitation pulses typically employed in digital signals using state space si formulation a compact representation of the output time domain signal is derived including the early time transient responses and its numerical implementation depicts no instability in late time responses plotted well beyond the steady state furthermore all the system poles are found to lie within the unit circle in the complex plane excellent model corroboration is achieved with independently computed frequency domain data or measured results using modest computer resources

efficient representation of induced currents on large scatterers using the generalized pencil of function method it is shown that the currents induced over large and smooth scatterers can often be represented by a series of complex exponentials with only a few terms the generalized pencil of function method is employed to extract these exponentials from the numerical solution of the scattering problem derived by using the method of moments illustrative examples include a partially coated two dimensional scatterer modeling a helicopter blade and the three dimensional problem of scattering by a thin plate the ultimate objective of this effort is to solve a class of large body scattering problems for arbitrary angles of incidence including the grazing angle by utilizing the numerically derived complex exponential type of basis functions extrapolated to higher frequencies it is well known that the physical optics approximation for the induced current becomes inaccurate when the angle of incidence is close to grazing and the asymptotic methods e g the gtd can become unwieldy for scatterers with edge treatments and those with complex geometrical shapes the approach based upon the use of numerically derived entire domain basis functions on a portion of the body is proposed as a means to circumventing this difficulty

paper approximation of multivariable linear systems with impulse response and autocorrelation sequences this paper considers the construction of approximants of multi input multi output discrete time linear systems from the finite data of the impulse response and autocorrelation sequences in the approximation of a multivariable linear system it is common practice to use a finite portion of its impulse response sequence this is formally equivalent to the pade approximation technique which may produce unstable approximants even though the original system is stable mullis and roberts proposed a new method which yields stable approximants in connection with approximation of digital filters this is however restricted to the single input single output case this paper extends their method to the multi input multi output case and shows a fast recursive algorithm to construct stable approximants of linear systems

loss surface of xor artificial neural networks training an artificial neural network involves an optimization process over the landscape defined by the cost loss as a function of the network parameters we explore these landscapes using optimization tools developed for potential energy landscapes in molecular science the number of local minima and transition states saddle points of index one as well as the ratio of transition states to minima grow rapidly with the number of nodes in the network there is also a strong dependence on the regularization parameter with the landscape becoming more convex fewer minima as the regularization term increases we demonstrate that in our formulation stationary points for networks with n h hidden nodes including the minimal network required to fit the xor data are also stationary points for networks with n h 1 hidden nodes when all the weights involving the additional node are zero hence smaller networks trained on xor data are embedded in the landscapes of larger networks our results clarify certain aspects of the classification and sensitivity to perturbations in the input data of minima and saddle points for this system and may provide insight into dropout and network compression

a recursive algorithm for linear system identification this paper deals with the pole zero identification of a linear system from a measured input output record it is shown that the minimization of a modified version of the squared kalman equation error can be implemented by an order recursive algorithm in the time domain the algorithm is based on the gram schmidt orthogonalization of intertwined krylov sequences involving a skew self adjoint linear operator

a single letter upper bound on the feedback capacity of unifilar finite state channels an upper bound on the feedback capacity of unifilar finite state channels fscs is derived a new technique called the q context mapping is based on a construction of a directed graph that is used for a sequential quantization of the receiver s output sequences to a finite set of contexts for any choice of q graph the feedback capacity is bounded by a single letter expression c mathrm fb leq sup i x s y q where the supremum is over p x s q and the distribution of s q is their stationary distribution it is shown that the bound is tight for all unifilar fscs where feedback capacity is known channels where the state is a function of the outputs the trapdoor channel ising channels the no consecutive ones input constrained erasure channel and the memoryless channel its efficiency is also demonstrated by deriving a new capacity result for the dicode erasure channel the upper bound is obtained directly from the above mentioned expression and its tightness is concluded with a general sufficient condition on the optimality of the upper bound this sufficient condition is based on a fixed point principle of the bcjr equation and indeed formulated as a simple lower bound on feedback capacity of unifilar fscs for arbitrary q graphs this upper bound indicates that a single letter expression might exist for the capacity of finite state channels with or without feedback based on a construction of auxiliary random variable with specified structure such as the q graph and not with i i d distribution the upper bound also serves as a non trivial bound on the capacity of channels without feedback a problem that is still open

on the solution of the minimal rational interpolation problem it is shown that the reachability and the observability indices of appropriately defined pairs of matrices provide the key for solving the minimal rational bitangen tial interpolation problem

linear system reduction by impulse energy approximation a method is given for linear system reduction which is a more general form of the routh method the reduced models retain stability and a fuller contribution is made to the impulse response energy of the reduced system by appropriate selection of retained alpha and beta parameters it requires no more computation than the routh method and can be presented in a compact tabular form examples are given to illustrate the method

numerical approximation of electromagnetic signals arising in the evaluation of geological formations the integration of complicated oscillatory functions arises in computational electromagnetics when evaluating signals produced by the propagation of electromagnetic radiation through the anisotropic layers of a geological formation the computation of exact integrals involves the evaluation of sommerfeld integrals the matrix pencil method is used for the numerical approximation of such signals numerical results show accuracy and robustness of the method for the approximation of these signals and efficiency in their numerical integration sampling frequency is discussed and numerical efficiency is improved by down sampling

a projection approach to covariance equivalent realizations of discrete systems covariance equivalent realization theory has been used recently in continuous systems for model reduction 1 4 and controller reduction 2 5 in model reduction this technique produces a reduced order model that matches q 1 output covariances and q markov parameters of the full order model in controller reduction it produces a reduced controller that is close to matching q 1 output covariances of the full order controller and q markov parameters of the closed loop system for discrete systems a method was devised to produce a reduced order model that matches the q 1 covariances 6 but not any markov parameters this method requires a factorization to obtain the input matrix and since the dimension of this matrix factor depends upon rank properties not known a priori this method may not maintain the original dimension of the input vector hence this method 6 is obviously not suitable for controller reduction a new projection method is described that matches q covariances and q markov parameters of the original system while maintaining the correct dimension of the input vector

on the computation of the gram matrix in time domain and its application the gram matrix of the system involves evaluation of the scalar product of repeated integrals and has formerly been computed in frequency domain a time domain method for evaluation of the gram matrix is presented it is based on new properties of the gram matrix derived here a model reduction algorithm based on these properties is presented the method is illustrated by a numerical example

weighted q markov covariance equivalent realizations abstract reduced order models which preserve the first q markov parameters and the first q covariances are called q markov covariance equivalent realizations q markov covers this paper extends these notions to the case where the input is coloured instead of white noise we call this the weighted q markov cover

capacity of the trapdoor channel with feedback we establish that the feedback capacity of the trapdoor channel is the logarithm of the golden ratio and provide a simple communication scheme that achieves capacity as part of the analysis we formulate a class of dynamic programs that characterize capacities of unifilar finite state channels the trapdoor channel is an instance that admits a simple closed form solution

distributed channel synthesis two familiar notions of correlation are rediscovered as the extreme operating points for distributed synthesis of a discrete memoryless channel in which a stochastic channel output is generated based on a compressed description of the channel input wyner s common information is the minimum description rate needed however when common randomness independent of the input is available the necessary description rate reduces to shannon s mutual information this paper characterizes the optimal tradeoff between the amount of common randomness used and the required rate of description we also include a number of related derivations including the effect of limited local randomness rate requirements for secrecy applications to game theory and new insights into common information duality our proof makes use of a soft covering lemma known in the literature for its role in quantifying the resolvability of a channel the direct proof achievability constructs a feasible joint distribution over all parts of the system using a soft covering from which the behavior of the encoder and decoder is inferred with no explicit reference to joint typicality or binning of auxiliary interest this paper also generalizes and strengthens this soft covering tool

a discussion of various approaches to the linear system identification problem this paper deals with the pole zero identification of a linear system from a measured input output record one objective is to show that the pencil of function method minimizes a weighted version of the kalman equation error it follows that the pencil of function method is capable of yielding robust estimates for poles located in a given region of the complex s plane the second objective of this paper is to illustrate that identical sets of equations arise in three supposedly different analytical techniques for obtaining the impulse response of a system the techniques investigated are 1 the least squares technique based on the discrete wiener hopf equation 2 pisarenko s eigenvalue method and 3 jain s pencil of function method the proof of equivalence is valid only for the noise free case when the system order is known instead of using the conventional differential equation formulation equivalence is shown with the integral form utilized in the pencil of function method

novel interpretation of the pencil of functions approximation identification method jain s pencil of functions method based on the linear dependence independence of a set of functions is revisited it is shown that no matter what model order is chosen every estimated denominator coefficient may be regarded as the geometric mean of two values obtained in a least squares sense

interpolatory weighted h2 model reduction this paper introduces an interpolation framework for the weighted h 2 model reduction problem we obtain a new representation of the weighted h 2 norm of siso systems that provides new interpolatory first order necessary conditions for an optimal reduced order model the h 2 norm representation also provides an error expression that motivates a new weighted h 2 model reduction algorithm several numerical examples illustrate the effectiveness of the proposed approach

model reduction by moment matching for linear and nonlinear systems the model reduction problem for single input single output linear and nonlinear systems is addressed using the notion of moment a re visitation of the linear theory allows to obtain novel results for linear systems and to develop a nonlinear enhancement of the notion of moment this in turn is used to pose and solve the model reduction problem by moment matching for nonlinear systems to develop a notion of frequency response for nonlinear systems and to solve model reduction problems in the presence of constraints on the reduced model connections between the proposed results projection methods the covariance extension problem and interpolation theory are presented finally the theory is illustrated by means of simple worked out examples and case studies

discrete time controlled markov processes with average cost criterion a survey this work is a survey of the average cost control problem for discrete time markov processes the authors have attempted to put together a comprehensive account of the considerable research on this problem over the past three decades the exposition ranges from finite to borel state and action spaces and includes a variety of methodologies to find and characterize optimal policies the authors have included a brief historical perspective of the research efforts in this area and have compiled a substantial yet not exhaustive bibliography the authors have also identified several important questions that are still open to investigation

discrete system reduction via impulse response gramians and its relation to q markov covers a method for model reduction of linear discrete systems is proposed it is based on the impulse response gramian proposed by the authors 1989 for discrete systems this gramian is an extension of the one proposed for linear continuous systems and contains information on the input output behavior of the system the rth order reduced order models are made to retain the first r markov parameters and the first r r elements of the impulse response gramian of the original system the relation between this method and the q markov cover method is also discussed the method is illustrated by a numerical example

principal component analysis in linear systems controllability observability and model reduction kalman s minimal realization theory involves geometric objects controllable unobservable subspaces which are subject to structural instability specifically arbitrarily small perturbations in a model may cause a change in the dimensions of the associated subspaces this situation is manifested in computational difficulties which arise in attempts to apply textbook algorithms for computing a minimal realization structural instability associated with geometric theories is not unique to control it arises in the theory of linear equations as well in this setting the computational problems have been studied for decades and excellent tools have been developed for coping with the situation one of the main goals of this paper is to call attention to principal component analysis hotelling 1933 and an algorithm golub and reinsch 1970 for computing the singular value decompositon of a matrix together they form a powerful tool for coping with structural instability in dynamic systems as developed in this paper principal component analysis is a technique for analyzing signals singular value decomposition provides the computational machinery for this reason kalman s minimal realization theory is recast in terms of responses to injected signals application of the signal analysis to controllability and observability leads to a coordinate system in which the internally balanced model has special properties for asymptotically stable systems this yields working approximations of x c x bar o the controllable and unobservable subspaces it is proposed that a natural first step in model reduction is to apply the mechanics of minimal realization using these working subspaces

relationship between matrix pencil and state space based harmonic retrieval methods the author studies the relationship between a state space approach and the matrix pencil method for the problem of estimating the parameters of damped and undamped sinusoids it is shown that the methods are very similar in the exact data case and have minor difference in implementation in the noisy data case

training neural nets with the reactive tabu search in this paper the task of training subsymbolic systems is considered as a combinatorial optimization problem and solved with the heuristic scheme of the reactive tabu search rts an iterative optimization process based on a modified local search component is complemented with a meta strategy to realize a discrete dynamical system that discourages limit cycles and the confinement of the search trajectory in a limited portion of the search space the possible cycles are discouraged by prohibiting i e making tabu the execution of moves that reverse the ones applied in the most recent part of the search the prohibition period is adapted in an automated way the confinement is avoided and a proper exploration is obtained by activating a diversification strategy when too many configurations are repeated excessively often the rts method is applicable to nondifferentiable functions is robust with respect to the random initialization and effective in continuing the search after local minima three tests of the technique on feedforward and feedback systems are presented

model reduction using balanced realizations with improved low frequency behaviour abstract model reduction using balanced realization techniques is known to give a better approximation of the high frequency behaviour than the low frequency one in this paper a simple modification to the technique is proposed to get a better approximation of the low frequency behaviour of the original system the proposed technique is illustrated by numerical examples

entropy bounds for discrete random variables via maximal coupling this paper derives new bounds on the difference of the entropies of two discrete random variables in terms of the local and total variation distances between their probability mass functions the derivation of the bounds relies on maximal coupling and they apply to discrete random variables which are defined over finite or countably infinite alphabets loosened versions of these bounds are demonstrated to reproduce some previously reported results the use of the new bounds is exemplified for the poisson approximation where bounds on the local and total variation distances follow from stein s method

approximation of boolean functions by sigmoidal networks part i xor and other two variable functions we prove the existence of a manifold of exact solutions mean square error e 0 of weights and thresholds for sigmoidal networks for xor and other 2 variable boolean functions we also prove the existence of a manifold of local minima of e where e 0

improved sign based learning algorithm derived by the composite nonlinear jacobi process in this paper a globally convergent first order training algorithm is proposed that uses sign based information of the batch error measure in the framework of the nonlinear jacobi process this approach allows us to equip the recently proposed jacobi rprop method with the global convergence property i e convergence to a local minimiser from any initial starting point we also propose a strategy that ensures the search direction of the globally convergent jacobi rprop is a descent one the behaviour of the algorithm is empirically investigated in eight benchmark problems simulation results verify that there are indeed improvements on the convergence success of the algorithm

the feedback capacity of the binary erasure channel with a no consecutive ones input constraint the input constrained erasure channel with feedback is considered where the binary input sequence contains no consecutive ones i e it satisfies the 1 infty rll constraint we derive the capacity for this setting which can be expressed as c epsilon max 0 leq p leq 0 5 frac 1 epsilon h b p 1 1 epsilon p where epsilon is the erasure probability and h b cdot is the binary entropy function moreover we prove that a priori knowledge of the erasure at the encoder does not increase the feedback capacity the feedback capacity was calculated using an equivalent dynamic programming dp formulation with an optimal average reward that is equal to the capacity furthermore we obtained an optimal encoding procedure from the solution of the dp leading to a capacity achieving zero error coding scheme for our setting dp is thus shown to be a tool not only for solving optimization problems such as capacity calculation but also for constructing optimal coding schemes the derived capacity expression also serves as the only non trivial upper bound known on the capacity of the input constrained erasure channel without feedback a problem that is still open

technical communique improvement of system order reduction via balancing using the method of singular perturbations in this paper we first present several alternative techniques for system order reduction based on system balancing by employing the method of singular perturbations the techniques obtained have the same robustness accuracy evaluated with respect to the h norm of the reduced order system as the two techniques existing in the control literature known as the direct truncation and balancing residualization methods the newly proposed techniques preserve the exact dc gain as the original system and produce very good to excellent accuracy at low and medium frequencies we also present a simple formula that fixes the dc gain for the direct truncation method in the second part of the paper we propose how to perform order reduction of a balanced system using theory of singular perturbations that can produce very good accuracy at high frequencies particularly for systems that have lightly damped highly oscillatory modes the result obtained for such kind of systems requires that the modes with higher hankel singular values should be neglected and those with smaller hankel singular values should be retained this is a counter intuitive result in the spirit of h theory but an intuitive result in the spirit of theory of singular perturbations the second part of the paper raises an important question about validity of the system robustness estimate based on the h norm for linear systems with lightly damped highly oscillatory modes

clarification of issues on the closed form green s functions in stratified media the closed form green s functions cfgf derived for the vector and scalar potentials in planar multilayer media have been revisited to clarify some issues and misunderstandings on the derivation of these green s functions in addition the range of validity of these green s functions is assessed with and without explicit evaluation of the surface wave contributions as it is well known the derivation of the cfgf begins with the approximation of the spectral domain green s functions by complex exponentials and continues with applying the sommerfeld identity to cast these approximated spectral domain green s functions into the space domain in closed forms questions and misunderstandings of this derivation which have mainly originated from the approximation process of the spectral domain green s functions in terms of complex exponentials can be categorized and discussed under the topics of 1 branch point contributions 2 surface wave pole contributions and 3 the accuracy of the obtained cfgf when these issues are clarified the region of validity of the cfgf so obtained may be defined better therefore in this paper these issues will be addressed first and then their origins and possible remedies will be provided with solid analysis and numerical demonstrations

rational modeling by pencil of functions method pole zero modeling of signals such as an electromagnetic scatterer response is considered in this paper it is shown by use of the pencil of functions theorem that a the true parameters can be recovered in the ideal case where the signal is the impulse reponse of a rational function h z and b the parameters are optimal in the functional dependence sense when the observed data are corrupted by additive noise or by systematic error although the computations are more involved than in all pole modeling they are considerably less than those required in iterative schemes of pole zero modeling the advantages of the method are demonstrated by a simulation example and through application to the electromagnetic response of a scatterer the paper also includes very recent and promising results on a new approach to noise correction in contradistinction with spectral subtraction techniques where only amplitude information is emphasized and phase is ignored we propose a method that a estimates the noise spectral density for the data frame and then b performs the subtraction of the noise correlation matrix from the gram matrix of the signal

linear system approximation via covariance equivalent realizations abstract a truncation technique for model reduction to simultaneously match a specified number of output covariance derivatives is described these covariance equivalent realizations which match q 1 covariance derivatives are called q covers in general q covers are not unique the additional freedom is used herein so that the q cover obtained also matches q markov parameters the truncation technique uses a form of the observability matrices of the full order system to determine a priori the order required of the reduced order model to match a specified number of output covariance derivatives and markov parameters the resulting realization is shown to be independent of the basis of the complete model to within a unitary transformation stability conditions for the reduced order model are also described and the relationships are established between stochastically equivalent realizations and the q covers

generalized pencil of function method for extracting poles of an em system from its transient response a generalized pencil of function gpof method is developed for extracting the poles of an electromagnetic system from its transient response the gpof method needs the solution of a generalized eigenvalue problem to find the poles subspace decomposition is also used to optimize the performance of the gpof method the gpof method has advantages over the prony method in both computation and noise sensitivity and approaches the cramer rao bound when the signal to noise ratio snr is above threshold an application of the gpof method to a thin wire target is presented

an improved pencil of functions method and comparisons with traditional methods of pole extraction prony s method is usually used for the determination of poles from experimental data in the singularity expansion method sem literature it is the aim here to show that this is not the best method and that jain s pencil of functions pof method can be generalized to provide a better method an example of such a method is given

arma based time signature estimator for analyzing resonant structures by the fdtd method we propose an algorithm for estimation of the optimal system parameters of time sequences tss computed by the finite difference time domain fdtd method with the goal of accurate representation of the time signature using low order models the fdtd method requires computation of very long time sequences to accurately characterize the slowly decaying transient behavior of resonant structures therefore it becomes critical to investigate methods of reducing the computational time for such objects several researchers have argued that the fdtd ts can be modeled as the impulse response ir of an autoregressive moving average arma transfer function however it is known that determination of arma parameters by ir matching is a complex nonlinear optimization problem hence many existing methods in em literature tend to use prony based linear predictor type spectrum estimation algorithms which minimize a linearized equation error criterion that approximates the true nonlinear model fitting error criterion as a result significantly high model orders are needed by these methods to achieve good corroboration in the frequency domain especially when a magnitude spectrum has deep nulls or notches we propose to use a deterministic arma approach which minimizes the true nonlinear criterion iteratively and attains significantly improved ir fit over prony s 1795 method using fewer arma model parameters for a given time sequence of an analyzed circuit the issues of model order selection and choice of decimation factor are also addressed systematically the improved performance of the proposed algorithm is demonstrated with transient simulation and signal analysis of microstrip structures which manifest deep nulls in the frequency domain

geometric methods for determining system poles from transient response the problem treated is that of identifying the poles of a finite order system by observing its transient decay after cessation of input for a limited time using possibly multiple observation points and experimental repetition various approaches are studied having the common characteristic that a homogeneous matrix equation must be solved several techniques that have been given scant attention in the literature are consolidated by a geometric treatment together with new results including an analytical treatment of the consequences of assuming an excessively high system order derivation of a statistically unbiased estimate for an intermediate parameter in the solution new theorems on error effects a recipe for effective use of the singular value decomposition and a new method for suppression of extraneous poles

two sided time domain moment matching for linear systems in this technical note we study the problem of computing reduced order models of order nu that match 2 nu moments of the given to be reduced system at two sets of nu interpolation points called two sided moment matching in the time domain framework we also study the time domain moment matching problem of the state space representations of the first order derivative of a transfer function based on this we show that the solution of the two sided time domain moment matching problem at non distinct nu interpolation points yields a unique nu order model that satisfies the following properties the model matches the moments of the given system at the interpolation points and the same property holds for the first order derivatives of the corresponding transfer functions

nonlinear moment matching based model order reduction in this paper we present a time domain notion of moments for a class of single input single output nonlinear systems in terms of the evolution of the output of a generalized signal generator driven by the nonlinear system we also define a new notion of moment matching and present a family of nonlinear parametrized reduced order models that achieve moment matching we establish relations with existing notions of moment for nonlinear systems showing that the newly derived and the existing families of reduced order models that achieve nonlinear moment matching respectively are equivalent furthermore we compute the reduced order model that matches the moments at two chosen signal generators one exciting the input of the system and another driven by the system simultaneously we also present a family of models computed on the basis of a nonlinear extension of the petrov galerkin projection that achieve moment matching finally we specialize the results to the case of nonlinear input affine systems

feedback capacity of finite state machine channels we consider a finite state machine channel with a finite memory length e g finite length intersymbol interference channels with finite input alphabets also known as partial response channels for such a finite state machine channel we show that feedback dependent markov sources achieve the feedback capacity and that the required memory length of the markov process matches the memory length of the channel further we show that the whole history of feedback is summarized by the causal posterior channel state distribution which is computed by the sum product forward recursion of the bahl cocke jelinek raviv bcjr baum welch discrete time wonham filtering algorithm these results drastically reduce the space over which the optimal feedback dependent source distribution needs to be sought further the feedback capacity computation may then be formulated as an average reward per stage stochastic control problem which is solved by dynamic programming with the knowledge of the capacity achieving source distribution the value of the capacity is easily estimated using markov chain monte carlo methods when the feedback is delayed we show that the feedback capacity can be computed by similar procedures we also show that the delayed feedback capacity is a tight upper bound on the feedforward capacity by comparing it to tight existing lower bounds we demonstrate the applicability of the method by computing the feedback capacity of partial response channels and the feedback capacity of run length limited rll sequences over binary symmetric channels bscs

model reduction of linear discrete systems via weighted impulse response gramians abstract a new method for model reduction of linear discrete systems is proposed it is based on the weighted impulse response gramians proposed here for discrete systems these gramians are extensions of the ones proposed for linear continuous systems agathoklis and sreeram 1988 a 1990 and contain information on the input output behaviour of the system the reduced order models are obtained by retaining the states corresponding to the dominant eigenvalues of these gramians and are stable if the original system is stable the method is illustrated by a numerical example and is compared with well known model reduction techniques

reduction of linear continuous time multivariable systems by matching first and second order information this paper considers the approximation of stable continuous time multivariable linear systems from a finite number of markov parameters and second order information indexes it is shown that by properly choosing these indexes it is possible to uniquely identify an input output model of given order from an equal number of first and second order data

model reduction via balanced state space representations a model reduction procedure based on balanced state space representations is studied in this paper the reduced order model is examined from the point of view of stability controllability and observability both continuous time and discrete time systems are considered

covariance sequence approximation for parametric spectrum modeling parametric methods of spectrum analysis are founded on finite dimensional models for covariance sequences rational spectrum approximants for continuous spectra are based on autoregressive ar moving average ma or autoregressive moving average arma models for covariance sequences line spectrum approximants to discrete spectra are based on cosinusoidal models for covariance sequences in this paper we make the point that a wide variety of spectrum types admit to modal analysis wherein the modes are characterized by amplitudes frequencies and damping factors the associated modal decomposition is appropriate for both continuous and discrete components of the spectrum the domain of attraction for the decomposition includes arma sequences harmonically or nonharmonically related sinusoids damped sinusoids white noise and linear combinations of these the parametric spectrum analysis problem now becomes one of identifying mode parameters this we achieve by solving two modified least squares problems numerical results are presented to illustrate the identification of mode parameters and corresponding spectra from finite records of perfect and estimated covariance sequences the results for sinusoids and sinusoids in white noise are interpreted in terms of in phase and quadrature effects attributable to the finite record length

a class of gradient unconstrained minimization algorithms with adaptive stepsize in this paper the development convergence theory and numerical testing of a class of gradient unconstrained minimization algorithms with adaptive stepsize are presented the proposed class comprises four algorithms the first two incorporate techniques for the adaptation of a common stepsize for all coordinate directions and the other two allow an individual adaptive stepsize along each coordinate direction all the algorithms are computationally efficient and possess interesting convergence properties utilizing estimates of the lipschitz constant that are obtained without additional function or gradient evaluations the algorithms have been implemented and tested on some well known test cases as well as on real life artificial neural network applications and the results have been very satisfactory

average optimality in dynamic programming with general state space a markovian decision model with general state space compact action space and the average cost as criterion is considered the existence of an optimal policy is shown via an optimality inequality in terms of the minimal average cost g and a relative value function w the existence of some w is usually shown via relative compactness in a space of real valued functions on the state space here it shall be shown that one can instead do with pointwise relative compactness in the set of real numbers if one makes use of a generalized lower limit of functions an application to an inventory model is given

finite state channels with time invariant deterministic feedback we consider capacity of discrete time channels with feedback for the general case where the feedback is a time invariant deterministic function of the output samples under the assumption that the channel states take values in a finite alphabet we find a sequence of achievable rates and a sequence of upper bounds on the capacity the achievable rates and the upper bounds are computable for any n and the limits of the sequences exist we show that when the probability of the initial state is positive for all the channel states then the capacity is the limit of the achievable rate sequence we further show that when the channel is stationary indecomposable and has no intersymbol interference isi its capacity is given by the limit of the maximum of the normalized directed information between the input xn and the output yn i e c limnrarrinfin 1 n max i xnrarryn where the maximization is taken over the causal conditioning probability q xnparzn 1 defined in this paper the main idea for obtaining the results is to add causality into gallager s results on finite state channels the capacity results are used to show that the source channel separation theorem holds for time invariant determinist feedback and if the state of the channel is known both at the encoder and the decoder then feedback does not increase capacity

matrix pencil method for estimating parameters of exponentially damped undamped sinusoids in noise a study of a matrix pencil method for estimating frequencies and damping factors of exponentially damped and or undamped sinusoids in noise is presented comparison of this method to a polynomial method svd prony method shows that the matrix pencil method and the polynomial method are two special cases of a matrix prediction approach and that the pencil method is more efficient in computation and less restrictive about signal probes it is found through perturbation analysis and simulation that for signals with unknown damping factors the pencil method is less sensitive to noise than the polynomial method an expression of the cramer rao bound for the exponential signals is presented

singular perturbational model reduction of balanced systems the balanced representations of linear systems due to moore are shown to be natural and convenient mediums for singular perturbational model reduction

sylvester equations and projection based model reduction in this paper we establish a connection between krylov subspace techniques for multipoint pade interpolation and the use of sylvester equations for constructing reduced order models we also briefly point out that this connection partly extends to adi type techniques and to the smith iteration for computing approximate solutions of lyapunov equations

the capacity of channels with feedback in this paper we introduce a general framework for treating channels with memory and feedback first we prove a general feedback channel coding theorem based on massey s concept of directed information second we present coding results for markov channels this requires determining appropriate sufficient statistics at the encoder and decoder we give a recursive characterization of these sufficient statistics third a dynamic programming framework for computing the capacity of markov channels is presented fourth it is shown that the average cost optimality equation acoe can be viewed as an implicit single letter characterization of the capacity fifth scenarios with simple sufficient statistics are described sixth error exponents for channels with feedback are presented

balanced gains and their significance for l 2 model reduction the singular values do not give enough information for model reduction in the l2sense other invariants are introduced which help describe the contribution of each state to the l2magnitude of the impulse response this leads to another suboptimal balanced truncation criterion

families of moment matching based low order approximations for linear systems the problem of finding a framework for linear moment matching based approximation of linear systems is discussed it is shown that it is possible to define five equivalent notions of moments i e one from a complex s domain point of view two using krylov projections and two from a time domain perspective based on these notions classes of parameterized reduced order models that achieve moment matching are obtained we analyze the controllability and observability properties of the models belonging to each of the classes of reduced order models we find the subclasses of models of orders lower than the number of matched moments i e we find the sets of parameters that allow for pole zero cancellations to occur furthermore we compute the subclass of minimal order model s the minimal order is equal to half of the number of matched moments i e generically the largest number of possible pole zero cancellations is half of the number of matched moments finally we present classes of reduced order models that match larger numbers of moments based on interconnections between reduced order models

the capacity of finite state markov channels with feedback we consider a class of finite state markov channels with feedback we first introduce a simplified equivalent channel model and then construct the optimal stationary and nonstationary input processes that maximize the long term directed mutual information furthermore we give a sufficient condition under which the channel s shannon capacity can be achieved by a stationary input process the corresponding converse coding theorem and direct coding theorem are proved

the error surface of the simplest xor network has only global minima the artificial neural network with one hidden unit and the input units connected to the output unit is considered it is proven that the error surface of this network for the patterns of the xor problem has minimum values with zero error and that all other stationary points of the error surface are saddlepoints also the volume of the regions in weight space with saddlepoints is zero hence training this network on the four patterns of the xor problem using e g backpropagation with momentum the correct solution with error zero will be reached in the limit with probability one

the discrete time q markov cover models with improved low frequency approximation a new method of obtaining q markov cover models for linear discrete systems based on singular perturbation technique is proposed the reduced order models obtained not only match initial markov parameters and high frequency power moments like conventional q markov covers but also match the initial time moments and low frequency power moments of the original system

model reduction by matching markov parameters time moments and impulse response energies this paper presents a simple procedure for constructing reduced order models that match some markov parameters and time moments of an original system as well as the energies of its impulse response and of other combinations of the system modes in this way the stability of the reduced model of a stable system is ensured

controllability observability and the solution of ax xb c a closed form finite series representation of the unique solution x of the matrix equation ax xb c is developed using this representation the image kernel and rank of x are related to the controllable and unobservable subspaces of the a c and c b pairs respectively bounds on the rank of x are obtainedin terms of the dimensions of these subspaces in the case that c has unitary rank an exact calculation of rank x is made the generic rank of x with a b fixed and c generic is evaluated

important moments in systems and control the moment problem matured from its various special forms in the late 19th and early 20th centuries to a general class of problems that continues to exert profound influence on the development of analysis and its applications to a wide variety of fields in particular the theory of systems and control is no exception where the applications have historically been to circuit theory optimal control robust control signal processing spectral estimation stochastic realization theory and the use of the moments of a probability density many of these applications are also still works in progress in this paper we consider the generalized moment problem expressed in terms of a basis of a finite dimensional subspace p of the banach space c a b and a positive sequence c but with a new wrinkle inspired by the applications to systems and control we seek to parameterize solutions which are positive rational measures in a suitably generalized sense our parameterization is given in terms of smooth objects in particular the desired solution space arises naturally as a manifold which can be shown to be diffeomorphic to a euclidean space and which is the domain of some canonically defined functions the analysis of these functions and related maps yields interesting corollaries for the moment problem and its applications which we compare to those in the recent literature and which play a crucial role in part of our proof

nonextensive statistical mechanics for hybrid learning of neural networks abstract this paper introduces a new hybrid approach for learning systems that builds on the theory of nonextensive statistical mechanics the proposed learning scheme uses only the sign of the gradient and combines adaptive stepsize local searches with global search steps that make use of an annealing schedule inspired from nonextensive statistics as proposed by tsallis the performance of the hybrid approach is empirically investigated through simulation in benchmark problems from the uci repository of machine learning databases preliminary results provide evidence that the synergy of techniques from nonextensive statistics provide neural learning schemes with significant benefits in terms of learning speed and convergence success

on the generalized q markov cover models for discrete time systems a new method for obtaining generalized q markov cover models for discrete time siso systems is proposed and is based on the inverse solution of the lyapunov equation the reduced order models not only match the markov parameters and high frequency power moments like the conventional q markov cover techniques but also match time moments and low frequency power moments of the original system

identification reduction to a balanced realization via the extended impulse response gramian a new identification reduction algorithm for linear discrete time invariant ldti systems is proposed which is based on the extended impulse response gramian first defined here for ldti systems the reduction algorithm employs singular value decomposition to retain states corresponding to dominant singular values of these gramians the proven properties of the reduced order models include convergence to a balanced realization with conditional controllability observability and asymptotic stability a suboptimal property of the model in minimizing an impulse response error norm is also demonstrated the proposed technique can handle impulse response data of deterministic or stochastic nature for system identification application