a family of lower and higher order transversal linearization techniques in non linear stochastic engineering dynamics sample pathwise numerical integration of noise driven engineering dynamical systems cannot generally be performed beyond a limited level of accuracy especially when the noise processes are modelled using filtered white noises recently a locally transversal linearization ltl strategy has been proposed by the author proc roy soc london a 2001 457 539 566 for direct integration of deterministic and stochastic non linear dynamical systems the present effort is focussed on a host of extensions along with detailed theoretical error analyses of the linearization approach especially as applied to problems in non linear stochastic engineering dynamics thus to begin with estimates of local and global error orders in the basic ltl scheme are obtained separately for the displacement and velocity vectors when the system is driven either by a set of additive noises or by an arbitrary combination of independently evolving additive and multiplicative noises following this a new family of higher order ltl schemes is proposed in order to improve upon the basic ltl method and the associated error orders are established a stepwise implementation of the lower and higher order versions of the ltl method along with certain computational aspects is also outlined the proposed schemes are numerically illustrated to a limited extent for a single degree of freedom sdof and a two degree of freedom tdof non linear engineering systems under additive and or multiplicative white noise excitations

direct method for the steady state response of structures a new method for finding the steady state response of structures is presented it is an alternative to the ritz and lanczos vectors methods for a wide class of excitations and the exact steady state response can be obtained in a finite number of matrix operations the duhamel integral and natural modes are completely eliminated a simple subroutine is written for all possible combinations of polynomial exponential and sinusoidal excitations piecewise continuous excitations are also studied heavy damping can be analyzed without modifications the method is very suitable for microcomputers

steady state response of undamped systems to excitations expressed as polynomials in time it is pointed out that an excitation which expressed as a polynomial in time a particular integral of the governing partial differential equation of an undamped continuous system may be assumed to be polynomial as well this provides an alternative to the traditional use of the duhamel integral the particular integral in polynomial form is the steady state response corresponding to the polynomial excitation acting on the system since the infinite past frequency independent stiffness and mass matrices only are required to solve for the steady state exactly transients can be included by modal analysis recurrent formulae for piecewise linear and cubic forcing functions are given explicitly

precise time integration method with dimensional expanding for structural dynamic equations the precise time integration method proposed for a linear time invariant dynamic system can give precise numerical results approaching the exact solution at the integration points however dife culties arise when the algorithm is used for nonhomogeneous dynamic systems due to the inverse matrix calculation required a new algorithm isproposedto convertnonhomogeneousdynamicequationsinto homogeneousequationsbymeansofthe dimensional expanding method with this conversion the inverse matrix calculation is not required in the precise time integration method the new algorithm has enhanced the precise time integration method by benee ting the programming implementation and the numerical stability it has improved the computational efe ciency as well numerical examples are given to demonstrate the validity and efe ciency of the algorithm nomenclature ai coefe cient of the solution of fourier series ai coefe cient vector bi coefe cient of the solution of fourier series bi coefe cient vector c magnitude of the load on the nodes of the structure d coefe cient matrix of ordinary differential equations satise ed by the nonhomogeneous vector d number of terms of one series f load vector g damping matrix h coefe cient matrix of structural dynamic system h coefe cient matrix of expanding structural dynamic system i identity matrix k number of time steps k stiffness matrix k order of time step l number of dynamic modes m mass matrix m number of 2 n n algorithm parameter of precise time integration n dimension of the structural dynamic system p transformed system state variable q displacement vector r nonhomogeneous vector of structural dynamic system ta small part of the matrix exponential tc part of matrix exponential according to matrix c td part of matrix exponential according to matrix d t time v state variable of structure dynamic system v state variable vector of expanding structural

typical performance of regular low density parity check codes over general symmetric channels typical performance of low density parity check ldpc codes over a general binary input output symmetric memoryless channel is investigated using methods of statistical mechanics relationship between the free energy in statistical mechanics approach and the mutual information used in the information theory literature is established within a general framework gallager and mackay neal codes are studied as specific examples of ldpc codes it is shown that basic properties of these codes known for particular channels including their potential to saturate shannon s bound hold for general symmetric channels the binary input additive white gaussian noise channel and the binary input laplace channel are considered as specific channel models

statistical mechanics of lossy compression using multilayer perceptrons statistical mechanics is applied to lossy compression using multilayer perceptrons for unbiased boolean messages we utilize a treelike committee machine committee tree and treelike parity machine parity tree whose transfer functions are monotonic for compression using a committee tree a lower bound of achievable distortion becomes small as the number of hidden units k increases however it cannot reach the shannon bound even where k infinity for a compression using a parity tree with k or 2 hidden units the rate distortion function which is known as the theoretical limit for compression is derived where the code length becomes infinity

a precise time step integration method by step response and impulsive response matrices for dynamic problems in this paper a precise time step integration method for dynamic problems is presented the second order differential equations for dynamic problems are manipulated directly a general damping matrix is considered the transient responses are expressed in terms of the steady state responses the given initial conditions and the step response and impulsive response matrices the steady state responses for various types of excitations are readily obtainable the computation of the step response and impulsive response matrices and their time derivatives are studied in this paper a direct computation of these matrices using the taylor series solutions is not efficient when the time step size t is not small in this paper the recurrence formulae relating the response matrices at t t to those at t t 2 are constructed a recursive procedure is proposed to evaluate these matrices at t t from the matrices at t t 2m the matrices at t t 2m are obtained from the taylor series solutions to improve the computational efficiency the relations between the response matrices and their time derivatives are investigated in addition these matrices are expressed in terms of two symmetric matrices that can also be evaluated recursively besides from the physical point of view these matrices should be banded for small t both the stability and accuracy characteristics of the present algorithm are studied three numerical examples are used to illustrate the highly precise and stable algorithm 1997 john wiley sons ltd

inference in alpha rhythm phase and amplitude modeled on markov random field using belief propagation from electroencephalograms alpha rhythm is a major component of spontaneous electroencephalographic eeg data we develop a novel method that can be used to estimate the instantaneous phases and amplitudes of the alpha rhythm with high accuracy by modeling the alpha rhythm phase and amplitude as markov random field mrf models by using a belief propagation technique we construct an exact inference algorithm that can be used to estimate instantaneous phases and amplitudes and calculate the marginal likelihood maximizing the marginal likelihood enables us to estimate the hyperparameters on the basis of type ii maximum likelihood estimation we prove that the instantaneous phase and amplitude estimation by our method is consistent with that by the hilbert transform which has been commonly used to estimate instantaneous phases and amplitudes of a signal filtered from observed data in the limited case that the observed data consist of only one frequency signal whose amplitude is constant and a gaussian noise comparison of the performances of observation noise reduction by our method and by a gaussian mrf model of alpha rhythm signal indicates that our method reduces observation noise more efficiently moreover the instantaneous phase and amplitude estimates obtained using our method are more accurate than those obtained by the hilbert transform application of our method to experimental eeg data also demonstrates that the relationship between the alpha rhythm phase and the reaction time emerges more clearly by using our method than the hilbert transform this indicates our method s practical usefulness therefore applying our method to experimental eeg data will enable us to estimate the instantaneous phases and amplitudes of the alpha rhythm more precisely

approximate method to compute the eigenvalues and eigensensitivities of mechanical systems abstract the paper presents a new approximate method to compute the eigenvalues and eigensensitivities of first and second order to obtain greater economy of computer time in large mechanical systems design with good accuracy we consider the first sensitivity eigenvalue formula obtained from taylor s expansion as a differential equation of first order with constant coefficients the procedure is extended to the eigenvectors and displacements an illustrative example concerning a discrete mechanical system with three degrees of freedom with some masses and springs parametrized is presented this method may be easily extended to other systems used in industrial applications space structures shape optimization etc

statistical mechanics of the bayesian image restoration under spatially correlated noise we investigated the use of the bayesian inference to restore noise degraded images under conditions of spatially correlated noise the generative statistical models used for the original image and the noise were assumed to obey multidimensional gaussian distributions whose covariance matrices are translational invariant we derived an exact description to be used as the expectation for the restored image by the fourier transformation and restored an image distorted by spatially correlated noise by using a spatially uncorrelated noise model we found that the resulting hyperparameter estimations for the minimum error and maximal posterior marginal criteria did not coincide when the generative probabilistic model and the model used for restoration were in different classes while they did coincide when they were in the same class

precise integration methods based on lagrange piecewise interpolation polynomials this paper introduces two new types of precise integration methods for dynamic response analysis of structures namely the integral formula method and the homogenized initial system method the applied loading vectors in the two algorithms are simulated by the lagrange piecewise interpolation polynomials based on the zeros of the first chebyshev polynomial developed on the basis of the integral formula and the lagrange piecewise interpolation polynomial and combined with the recurrence relationship of some key parameters in the integral computation suggested in this paper with the solving process of linear algebraic equations the integral formula method has been set up on the basis of the lagrange piecewise interpolation polynomial and transforming the non homogenous initial system into the homogeneous dynamic system the homogenized initial system method without dimensional expanding is presented this homogenized initial system method avoids the matrix inversion operation and is a general homogenized high precision direct integration scheme the accuracy of the presented time integration schemes is studied and is compared with those of other commonly used schemes the presented time integration schemes have arbitrary order of accuracy wider application and are less time consuming two numerical examples are also presented to demonstrate the applicability of these new methods copyright 2008 john wiley sons ltd

accuracy of the bethe approximation for hyperparameter estimation in probabilistic image processing we investigate the accuracy of statistical mechanical approximations for the estimation of hyperparameters from observable data in probabilistic image processing which is based on bayesian statistics and maximum likelihood estimation hyperparameters in statistical science correspond to interactions or external fields in the statistical mechanics context in this paper hyperparameters in the probabilistic model are determined so as to maximize a marginal likelihood a practical algorithm is described for grey level image restoration based on a gaussian graphical model and the bethe approximation the algorithm corresponds to loopy belief propagation in artificial intelligence we examine the accuracy of hyperparameter estimation when we use the bethe approximation it is well known that a practical algorithm for probabilistic image processing can be prescribed analytically when a gaussian graphical model is adopted as a prior probabilistic model in bayes formula we are therefore able to compare in a numerical study results obtained through mean field type approximations with those based on exact calculation

physical consideration of an image in image restoration using bayes formula abstract we consider image restoration by bayes formula and investigate the relationship between an image and a prior probability from the following two viewpoints hyperparameter estimation and the accuracy of a restored image the q ising model is adopted as a prior probability in bayes formula not the q ising energy but the potts energy plays an important role in the hyperparameter estimation from the viewpoint of the hyperparameter estimation the relationship between a natural image and a prior probability is characterized through the potts energy and magnetization of an image the potts energy and magnetization of an image are defined by a set of pixels state of an image the closer to the average potts energy and magnetization over a prior probability the potts energy and magnetization of a natural image is the closer to the true value of a hyperparameter the estimated value of a hyperparameter from a degraded image is for the accuracy of a restored image the image which has a smaller q ising energy is better restored by bayes formula composed of the q ising prior the consideration for the relationship between an image and a prior probability is expected to be valid for a more complicated prior probability

second order design sensitivity analysis of mechanical system dynamics dependence of dynamic response of nonlinear mechanical systems on design variables is analysed an adjoint variable method is used to derive first and second order derivatives of measures of dynamic response with respect to design variables a computational algorithm is presented for numerical calculation of first and second design derivatives a simple oscillator example is solved analytically and by the adjoint variable method with identical results a burst fire automatic weapon mechanism with linear and nonlinear damping is treated numerically it is shown that quadratic appriximations of dynamic response using results of second order design sensitivity analysis can be substantially better than conventional linear approximations

first and second order sensitivity analysis of linear and nonlinear structures this paper employs the principle of virtual work to derive sensitivity derivatives of structural response with respect to stiffness parameters using both direct and adjoint approaches the computations required are based on additional load conditions characterized by imposed initial strains body forces or surface tractions as such they are equally applicable to numerical or analytical solution techniques the relative efficiency of various approaches for calculating first and second derivatives is assessed it is shown that for the evaluation of second derivatives the most efficient approach is one that makes use of both the first order sensitivities and adjoint vectors two example problems are used for demonstrating the various approaches

statistical trajectory of an approximate em algorithm for probabilistic image processing we calculate analytically a statistical average of trajectories of an approximate expectation maximization em algorithm with generalized belief propagation gbp and a gaussian graphical model for the estimation of hyperparameters from observable data in probabilistic image processing a statistical average with respect to observed data corresponds to a configuration average for the random field ising model in spin glass theory in the present paper hyperparameters which correspond to interactions and external fields of spin systems are estimated by an approximate em algorithm a practical algorithm is described for gray level image restoration based on a gaussian graphical model and gbp the gbp approach corresponds to the cluster variation method in statistical mechanics our main result in the present paper is to obtain the statistical average of the trajectory in the approximate em algorithm by using loopy belief propagation and gbp with respect to degraded images generated from a probability density function with true values of hyperparameters the statistical average of the trajectory can be expressed in terms of recursion formulas derived from some analytical calculations

hyperparameter estimation in image restoration the hyperparameter in image restoration by the bayes formula is an important quantity this communication shows a physical method for the estimation of the hyperparameter without approximation for artificially generated images by prior probability the hyperparameter is computed accurately for practical images accuracy of the estimated hyperparameter depends on the magnetization and energy of the images we discuss the validity of prior probability for an original image

second order adjoint sensitivity analysis methodology 2nd asam for computing exactly and efficiently first and second order sensitivities in large scale linear systems i computational methodology abstract this work presents the second order forward and adjoint sensitivity analysis methodologies 2nd fsam and 2nd asam for computing exactly and efficiently the second order functional derivatives of physical engineering biological etc system responses i e system performance parameters to the system s model parameters the definition of system parameters used in this work includes all computational input data correlations initial and or boundary conditions etc for a physical system comprising n parameters and n r responses we note that the 2nd fsam requires a total of n 2 2 3 n 2 large scale computations for obtaining all of the first and second order sensitivities for all n r system responses on the other hand for one functional type system response the 2nd asam requires one large scale computation using the first level adjoint sensitivity system for obtaining all of the first order sensitivities followed by at most n large scale computations using the second level adjoint sensitivity systems for obtaining exactly all of the second order sensitivities therefore the 2nd fsam should be used when n r n while the 2nd asam should be used when n n r the original 2nd asam presented in this work should enable the hitherto very difficult if not intractable exact computation of all of the second order response sensitivities i e functional gateaux derivatives for large systems involving many parameters as usually encountered in practice very importantly the implementation of the 2nd asam requires very little additional effort beyond the construction of the adjoint sensitivity system needed for computing the first order sensitivities

precise time step integration algorithms using response matrices with expanded dimension in this paper the precise time step integration method by step response and impulsive response matrices is further developed by expanding the dimension of the matrices so as to avoid computing the particular solutions separately two new precise time step integration algorithms with excitations described by second order and first order differential equations are proposed the first method is a direct extension of the existing algorithms however the extended system matrices are not symmetrical in the second method the duhamel integrals are used as the particular solutions as a result the responses can be expressed in terms of the given initial conditions and the step response matrix the impulsive response matrix and a newly derived duhamel response matrix the symmetry property of the system matrices can be used in the computation however it will first require a calculation of the duhamel response matrix and its derivative to reduce the computational effort the relation between the duhamel response matrix and its derivative is established a special computational procedure for periodic excitation is also discussed numerical examples are given to illustrate the present highly efficient algorithms

an efficient time frequency domain algorithm for modal analysis of non linear models discretized by the fem abstract the present paper describes an efficient time frequency domain approach for modal analysis of non linear models discretized by the fem an efficient recurrence relationship for displacement and velocity based on the green s function of the model and its time derivative is presented in both a time and a frequency domain context in the time domain procedure the green s function related terms are analytical in the frequency domain procedure the fundamental terms are indirectly evaluated taking into account steady state responses of the system the pseudo force method is adopted in order to deal with the model non linearity results for the time frequency domain approaches discussed here showed the same level of accuracy as those obtained by the newmark modified newton algorithm at a lower computational cost

design sensitivity analysis and optimization of dynamic response abstract the paper presents methods of design sensitivity analysis and optimization of dynamic response of mechanical and structural systems a key feature of the paper is the development of procedures to handle point wise state variable constraints difficulties with a previous treatment where such constraints were transformed to equivalent integral constraints are noted and explained from theoretical as well as engineering standpoints an alternate treatment of such constraints is proposed developed and evaluated in this treatment each point wise state variable constraint is replaced by several constraints that are imposed at all the local max points for the original constraint function the differential equations of motion are formulated in the first order form so as to handle more general problems the direct differentiation and adjoint variable methods of design sensitivity analysis to deal with the point wise constraints are presented with the adjoint variable methods there are two ways of calculating design sensitivity coefficients the first approach uses an impulse load and the second approach uses a step load for the corresponding adjoint equation since the adjoint variable methods are better for a large class of problems an efficient computational algorithm with these methods is presented in detail optimum results for several problems are obtained and compared with those available in the literature the new formulation works extremely well as precise optimum designs are obtained

an adjoint variable method for wideband second order sensitivity analysis through fdtd we propose an efficient adjoint variable method avm for estimating wideband second order response sensitivities using the finite difference time domain fdtd approach our avm approach reduces the number of required simulations for computing the hessian matrix of the desired objective function from o n 2 to only n 1 where n is the number of designable parameters all first order sensitivities are also estimated as a by product our sensitivity technique is illustrated through 1 d 2 d and 3 d electromagnetic examples involving both dispersive and nondispersive materials the accuracy of our avm approach is verified through comparison with analytical sensitivity results or sensitivities estimated using the accurate but computationally intensive central finite difference cfd approximations

on the free vibration analysis of spinning structures by using discrete or distributed mass models abstract the analysis of the natural frequencies and modes of vibration of a spinning structure is in general complicated by the presence of gyroscopic or coriolis forces leading to a complex hermitian dynamic stiffness or impedance matrix it is shown that if the analysis is performed by using a discrete model with n degrees of freedom the leading principal minors of the n th order dynamic stiffness matrix exhibit a type of sturm sequence property this leads to a theorem which can be used for the systematic calculation of the natural frequencies of either a discrete system which is assembled from sub structures or an assembly of distributed mass members in both of these cases the order n of the matrix is less than the number of degrees of freedom and its determinant possesses poles as well as zeros the theorem is identical with a corresponding one for non spinning structures previously derived by the authors the application of the theorem to a spinning two dimensional frame with distributed mass members is discussed in some detail

a time domain fem approach based on implicit green s functions for non linear dynamic analysis the present paper describes an unconditionally stable algorithm to integrate the equations of motion in time the standard fem displacement model is employed to perform space discretization and the time marching process is carried out through an algorithm based on the green s function of the mechanical system in nodal co ordinates in the present implicit green s function approach imga mechanical system green s functions are not explicitly computed rather they are implicitly considered through an iterative pseudo forces process under certain simplifying hypothesis iterations are not necessary and the imga becomes cheaper than standard newmark newton raphson algorithm at the end of the paper numerical examples are presented in order to illustrate the accuracy of the present approach

probabilistic image processing by means of the bethe approximation for the q ising model the framework of bayesian image restoration for multi valued images by means of the q ising model with nearest neighbour interactions is presented hyperparameters in the probabilistic model are determined so as to maximize the marginal likelihood a practical algorithm is described for multi valued image restoration based on the bethe approximation the algorithm corresponds to loopy belief propagation in artificial intelligence we conclude that in real world grey level images the q ising model can give us good results

statistical mechanics of lossy data compression using a nonmonotonic perceptron the performance of a lossy data compression scheme for uniformly biased boolean messages is investigated via methods of statistical mechanics inspired by a formal similarity to the storage capacity problem in neural network research we utilize a perceptron of which the transfer function is appropriately designed in order to compress and decode the messages employing the replica method we analytically show that our scheme can achieve the optimal performance known in the framework of lossy compression in most cases when the code length becomes infinite the validity of the obtained results is numerically confirmed

statistical mechanics of typical set decoding the performance of typical set pairs decoding for ensembles of gallager s linear code is investigated using statistical physics in this decoding method errors occur either when the information transmission is corrupted by atypical noise or when multiple typical sequences satisfy the parity check equation as provided by the received corrupted codeword we show that the average error rate for the second type of error over a given code ensemble can be accurately evaluated using the replica method including the sensitivity to message length our approach generally improves the existing analysis known in the information theory community which was recently reintroduced in ieee trans inf theory 45 399 1999 and is believed to be the most accurate to date 2002 the american physical society

cheap second order directional derivatives of stiff ode embedded functionals a second order adjoint method is described for calculating directional derivatives of stiff ode embedded functionals the derivation of the general directional second order adjoint equations for point and integral form functionals is presented a numerical procedure for calculating these directional derivatives that is relatively insensitive to the number of parameters is described and showcased by combining automatic differentiation ad to obtain the adjoint and sensitivity equations with the staggered corrector method to solve the sensitivity systems we achieve computational costs noticeably lower than directional finite differences based on a first order adjoint code

statistical mechanics of an error correcting code using monotonic and nonmonotonic treelike multilayer perceptrons an error correcting code using a treelike multilayer perceptron is proposed an original message mathbit s 0 is encoded into a codeword mathbit y 0 using a treelike committee machine committee tree or a treelike parity machine parity tree based on these architectures several schemes featuring monotonic or nonmonotonic units are introduced the codeword mathbit y 0 is then transmitted via a binary asymmetric channel where it is corrupted by noise the analytical performance of these schemes is investigated using the replica method of statistical mechanics under some specific conditions some of the proposed schemes are shown to saturate the shannon bound at the infinite codeword length limit the influence of the monotonicity of the units on the performance is also discussed

a hybrid time laplace integration method based on numerical green s functions in conduction heat transfer abstract the present paper describes an efficient time laplace domain approach to analyze numerically heat conduction problems an efficient recurrence relationship for the temperature in the time domain based on the green s functions of the model is presented primarily green s functions in nodal coordinates are explicitly calculated by the finite element method in the laplace domain and subsequently the stehfest and the zakian laplace inversion schemes are employed to compute numerically green s functions that transfer solution at time 0 to time t as a result a new family of highly accurate time integration methods called exga stehfest and exga zakian is obtained finally numerical examples are presented in order to illustrate the high accuracy and potentialities of these novel approaches

numerical methods for second order shape sensitivity analysis with applications to heat conduction problems in this paper two finite element based schemes for second order shape sensitivity analysis are presented in the first formulation the av dd method the first order shape sensitivity equation is derived and expressed in terms of state and adjoint variables the resultant equation is then directly differentiated to obtain the second order shape sensitivity equation in the second formulation the dd av method the functional of concern is differentiated twice to yield the second order sensitivity equation in which the second order shape derivatives can be eliminated by introducing a proper adjoint equation a thermal fin problem and a thermal insulation layer problem have been studied to validate the proposed schemes it is shown that both methods yield identical results though the dd av method is computationally more efficient

second order adjoint sensitivity analysis methodology 2nd asam for computing exactly and efficiently first and second order sensitivities in large scale linear systems this work presents an illustrative application of the second order adjoint sensitivity analysis methodology 2nd asam to a paradigm neutron diffusion problem which is sufficiently simple to admit an exact solution thereby making transparent the underlying mathematical derivations the general theory underlying 2nd asam indicates that for a physical system comprising n parameters the computation of all of the first and second order response sensitivities requires per response at most 2 n 1 large scale computations using the first level and respectively second level adjoint sensitivity systems 1st lass and 2nd lass very importantly however the illustrative application presented in this work shows that the actual number of adjoint computations needed for computing all of the first and second order response sensitivities may be significantly less than 2 n 1 per response for this illustrative problem four large scale adjoint computations sufficed for the complete and exact computations of all 4 first and 10 distinct second order derivatives furthermore the construction and solution of the 2nd lass requires very little additional effort beyond the construction of the adjoint sensitivity system needed for computing the first order sensitivities very significantly only the sources on the right sides of the diffusion differential operator needed to be modified the left side of the differential equations and hence the solver in large scale practical applications remained unchanged all of the first order relative response sensitivities to the model parameters have significantly large values of order unity also importantly most of the second order relative sensitivities are just as large and some even up to twice as large as the first order sensitivities in the illustrative example presented in this work the second order sensitivities contribute little to the response variances and covariances however they have the following major impacts on the computed moments of the response distribution a they cause the expected value of the response to differ from the computed nominal value of the response and b they contribute decisively to causing asymmetries in the response distribution indeed neglecting the second order sensitivities would nullify the third order response correlations and hence would nullify the skewness of the response consequently any events occurring in a response s long and or short tails which are characteristic of rare but decisive events e g major accidents catastrophes would likely be missed the 2nd asam is expected to affect significantly other fields that need efficiently computed second order response sensitivities e g optimization data assimilation adjustment model calibration and predictive modeling we apply the second order adjoint sensitivity analysis methodology 2nd asam to a paradigm diffusion problem four 2nd asam adjoint computations exactly determine all fourteen 1st and 2nd order sensitivities non zero second order response sensitivities decisively skew response distributions

statistical mechanical approach to image processing the basic frameworks and techniques of the bayesian approach to image restoration are reviewed from the statistical mechanical point of view first a few basic notions in digital image processing are explained to convince the reader that statistical mechanics has a close formal similarity to this problem second the basic formulation of the statistical estimation from the observed degraded image by using the bayes formula is demonstrated the relationship between bayesian statistics and statistical mechanics is also listed particularly it is explained that some correlation inequalities on the nishimori line of the random spin model also play an important role in bayesian image restoration third the framework of bayesian image restoration for binary images by means of the ising model is reviewed some practical algorithms for binary image restoration are given by employing the mean field and the bethe approximations finally bayesian image restoration for a grey level image using the gaussian model is reviewed and the gaussian model is extended to a more practical probabilistic model by introducing the line state to treat the effects of edges the line state is also extended to quantized values

statistical mechanical evaluation of a spread spectrum watermarking model with image restoration in cases in which an original image is blind a decoding method where both the image and the messages can be estimated simultaneously is desirable we propose a spread spectrum watermarking model with image restoration based on bayes estimation we therefore need to assume some prior probabilities the probability for estimating the messages is given by the uniform distribution and the ones for the image are given by the infinite range model and two dimensional 2d ising model any attacks from unauthorized users can be represented by channel models we can obtain the estimated messages and image by maximizing the posterior probability we analyzed the performance of the proposed method by the replica method in the case of the infinite range model we first calculated the theoretical values of the bit error rate from obtained saddle point equations and then verified them by computer simulations for this purpose we assumed that the image is binary and is generated from a given prior probability we also assume that attacks can be represented by the gaussian channel the computer simulation retults agreed with the theoretical values in the case of prior probability given by the 2d ising model in which each pixel is statically connected with four neighbors we evaluated the decoding performance by computer simulations since the replica theory could not be applied results using the 2d ising model showed that the proposed method with image restoration is as effective as the infinite range model for decoding messages we compared the performances in a case in which the image was blind and one in which it was informed the difference between these cases was small as long as the embedding and attack rates were small this demonstrates that the proposed method with simultaneous estimation is effective as a watermarking decoder

non monotonic behaviour in relaxation dynamics of image restoration we have investigated the relaxation dynamics of image restoration through a bayesian approach the relaxation dynamics is much faster at zero temperature than at the nishimori temperature where the pixel wise error rate is minimized in equilibrium at low temperature we observed non monotonic development of the overlap we suggest that the optimal performance is realized through premature termination in the relaxation processes in the case of the infinite range model we also performed markov chain monte carlo simulations to clarify the underlying mechanism of non trivial behaviour at low temperature by checking the local field distributions of each pixel

inverse iteration for the quadratic eigenvalue problem the inverse iteration method for the linear eigenvalue problem is extended to the quadratic eigenvalue problem based on the famous newtonian iteration for algebraic equations the process is shown to converge to the desired modes by means of a newly introduced continuation parameter for highly non conservative systems only physical co ordinates are involved and the sparsity of the original matrices is preserved to achieve high computational efficiency the required orthonormalization condition and the generalized rayleigh s quotient are given the method can handle complex eigensolution without difficulties

finite horizon h filter and its 2n algorithm a 2n algorithm will double a time or space step in each evaluation for initial value problem the 2n algorithm for the integration of filtering differential equation of the finite horizon h filter is presented in this paper since it is a boundary value problem within a time range a new 2n algorithm is introduced by merging two intervals each time so that the time interval is doubled in each evaluation if one divides the original time range into one million intervals 20 evaluations will complete the whole process owing to the extremely small initial time interval the first few terms of the taylor expansion of the interval matrices are sufficient for very accurate results since the filter gain matrices are the solution of the riccati differential equation and the existence of the solution depends on the induced norm the computation of critical value is reviewed first then according to the result and the prespecified performance index the suitable parameter 2 can be selected and the precise numerical solution of the riccati differential equation and the filtering differential equation can be obtained by using the 2n algorithm although the filtering equation is time varying the 2n algorithm for interval merging is given explicitly copyright 2001 john wiley sons ltd

statistical mechanics of low density parity check codes we review recent theoretical progress on the statistical mechanics of error correcting codes focusing on low density parity check ldpc codes in general and on gallager and mackay neal codes in particular by exploiting the relation between ldpc codes and ising spin systems with multispin interactions one can carry out a statistical mechanics based analysis that determines the practical and theoretical limitations of various code constructions corresponding to dynamical and thermodynamical transitions respectively as well as the behaviour of error exponents averaged over the corresponding code ensemble as a function of channel noise we also contrast the results obtained using methods of statistical mechanics with those derived in the information theory literature and show how these methods can be generalized to include other channel types and related communication problems

a frequency domain fem approach based on implicit green s functions for non linear dynamic analysis abstract the present paper describes an efficient algorithm to integrate the equations of motion implicitly in the frequency domain the standard fem displacement model galerkin formulation is employed to perform space discretization and the time marching process is carried out through an algorithm based on the green s function of the mechanical system in nodal coordinates in the present formulation mechanical system green s functions are implicitly calculated in the frequency domain once the green s functions related matrices are computed a time integration procedure which demands low computational effort when applied to non linear mechanical systems becomes available at the end of the paper numerical examples are presented in order to illustrate the accuracy of the present approach

krylov precise time step integration method an efficient precise time step integration pti algorithm to solve large scale transient problems is presented in this paper the krylov subspace method and the pade approximations are applied to modify the original pti algorithm in order to improve the computational efficiency both the stability and accuracy characteristics of the resultant algorithms are investigated the efficiency can be further improved by expanding the dimension to avoid the computation of the particular solutions the present algorithm can also be extended to tackle nonlinear problems without difficulty two numerical examples are given to illustrate the highly accurate and efficient algorithm

a statistical mechanics approach to large system analysis of cdma multiuser detectors we present a theory based on statistical mechanics to evaluate analytically the performance of uncoded fully synchronous randomly spread code division multiple access cdma multiuser detectors with additive white gaussian noise awgn channel under perfect power control and in the large system limit application of the replica method a tool developed in the literature of statistical mechanics allows us to derive analytical expressions for the bit error rate as well as the multiuser efficiency of the individually optimum io and jointly optimum jo multiuser detectors over the whole range of noise levels the information theoretic capacity of the randomly spread cdma channel and the performance of decorrelating and linear minimum mean square error mmse detectors are also derived in the same replica formulation thereby demonstrating validity of the statistical mechanical approach

an algorithm for matrix polynomial eigenproblems abstract when a system is described by higher order differential equations with time as the independent variable the solution for the homogeneous system is defined by a matrix polynomial eigenproblem a method alternative to the classical companion matrix method is introduced to expand the determinant algebraically to result in a scalar polynomial equation for the eigenvalues the eigenvectors are obtained by inverse iteration it is shown that the new method is computationally more advantageous than the conventional companion matrix method a computer program is given for general matrix polynomials as well as hermitian matrix polynomials defective eigenproblems can be handled without special attention

time dependent variance and covariance of responses of structures to non stationary random excitations abstract in this paper techniques for the analysis of non stationary random responses of linear structures discretized by the finite element method so that they can be analyzed as multi degree of freedom systems subjected to non stationary random excitation are developed the non stationary random excitation is represented as a product of a an exponentially decaying function and a white noise process and b a modulating function in the form of an exponential envelope and a white noise process closed form expressions for the time dependent variance and covariance of response of structures are presented application of these expressions is made for the analysis of non stationary random responses of a physical model of a class of mast antenna structures subjected to base excitation it is concluded that a the coupling terms do have a definite influence on the response the magnitude of the influence is proportional to the amount of damping in the structure and proximity of the modes excited b the non stationary random excitations considered are general in that the modulating functions are not necessarily identical and therefore the influence of various modulating functions of the excitations applied to different locations of the structure on responses can be examined quantivatively and c for a given damping parameter the magnitudes of the modulating function parameters cannot be chosen arbitrarily though the shapes of normalized modulating functions can be selected to best fit the excitation realizations

an efficient hybrid time laplace domain method for elastodynamic analysis based on the explicit green s approach abstract this work aims at presenting a novel hybrid method applied to the numerical solution of elastodynamic problems the formulation is based on the so called explicit green s approach exga recently developed by mansur et al mansur w j loureiro f s soares jr d dors c 2007 explicit time domain approaches based on numerical green s functions computed by finite differences the exga family journal of computational physics 227 851 870 in which a recursive time integration procedure based on green s function is adopted here the finite element method is employed to calculate the green s function in the laplace domain afterwards the zakian s laplace inversion algorithm is employed to compute numerically the green s function in the time domain only at time t t time integration is then carried out by employing the standard exga procedure and another where the convolution integral is replaced by a particular solution that is directly related to the applied load finally some numerical problems are solved to demonstrate the computational efficiency and reliability of the proposed hybrid method

mean field theory of em algorithm for bayesian grey scale image restoration the em algorithm for the bayesian grey scale image restoration is investigated in the framework of the mean field theory our model system is identical to the infinite range random field ising model the maximum marginal likelihood method is applied to the determination of hyper parameters we calculate both the data averaged mean square error between the original image and its maximizer of posterior marginal estimate and the data averaged marginal likelihood function exactly after evaluating the hyper parameter dependence of the data averaged marginal likelihood function we derive the em algorithm which updates the hyper parameters to obtain the maximum likelihood estimate analytically the time evolutions of the hyper parameters and so called q function are obtained the relation between the speed of convergence of the hyper parameters and the shape of the q function is explained from the viewpoint of dynamics

explicit time domain approaches based on numerical green s functions computed by finite differences the exga family the present paper describes a new family of time stepping methods to integrate dynamic equations of motion the scalar wave equation is considered here however the method can be applied to time domain analyses of other hyperbolic e g elastodynamics or parabolic e g transient diffusion problems the algorithms presented require the knowledge of the green s function of mechanical systems in nodal coordinates the finite difference method is used here to compute numerically the problem green s function however any other numerical method can be employed e g finite elements finite volumes etc the green s matrix and its time derivative are computed explicitly through the range 0 dt with either the fourth order runge kutta algorithm or the central difference scheme in order to improve the stability of the algorithm based on central differences an additional matrix called step response is also calculated the new methods become more stable and accurate when a sub stepping procedure is adopted to obtain the green s and step response matrices and their time derivatives at the end of the time step three numerical examples are presented to illustrate the high precision of the present approach

efficient computation of sensitivities for ordinary differential equation boundary value problems for models described by ordinary differential equation boundary value problems ode bvps we derive adjoint equations for sensitivity analysis giving explicit forms for the boundary conditions of the adjoint boundary value problem the solutions of the adjoint equations are used to efficiently compute gradients of both integral form and pointwise constraints existence and stability results are given for the adjoint system and its numerical solution the use of the method is demonstrated for a simple example where it is seen that the method is particularly advantageous for problems with more than a few parameters

transient dynamics for sequence processing neural networks an exact solution of the transient dynamics for a sequential associative memory model is discussed through both the path integral method and the statistical neurodynamics although the path integral method has the ability to give an exact solution of the transient dynamics only stationary properties have been discussed for the sequential associative memory we have succeeded in deriving an exact macroscopic description of the transient dynamics by analysing the correlation of crosstalk noise surprisingly the order parameter equations of this exact solution are completely equivalent to those of the statistical neurodynamics which is an approximation theory that assumes crosstalk noise to obey the gaussian distribution in order to examine our theoretical findings we numerically obtain cumulants of the crosstalk noise we verify that the third and fourth order cumulants are equal to zero and that the crosstalk noise is normally distributed even in the non retrieval case we show that the results obtained by our theory agree with those obtained by computer simulations we have also found that the macroscopic unstable state completely coincides with the separatrix

dynamic stiffness for structures with distributed deterministic or random loads abstract the dynamic stiffness method applies mainly to excitations of harmonic nodal forces for distributed loads modal analysis is generally required in the case of a clamped clamped beam the modal decomposition of a uniformly distributed load by the eigenbeam functions inherits slow convergence because the finite loads at the beam ends cannot be represented efficiently by the zero deflection and zero slope of the clamped clamped beam functions the computed reactions at the supports do not converge at all the problem is eliminated in this paper by using the finite element interpolation functions for the distributed load if the distributed load is adequately represented explicit exact solutions are found otherwise the residual load is expanded in the modal space as the residual modal force is much smaller and agrees well with the clamped clamped conditions fast convergence is achieved by means of the principle of superposition a structure with members having distributed loads can be analyzed by two systems one is associated with the individual members having distributed loads and the other is associated with resulting equivalent nodal forces the required frequency functions are given for all possible cases the results presented are exact if the load is interpolated adequately by finite element shape functions both deterministic and random loads are considered closed form solutions are obtained for the first time

sensitivity analysis of discrete structural systems methods for calculating sensitivity derivatives for discrete structural systems are surveyed primarily covering literature published during the past two decades methods are described for calculating derivatives of static displacements and stresses eigenvalues and eigenvectors transient structural response and derivatives of optimum structural designs with respect to problem parameters the survey is focused on publications developed in nonstructural fields such as electronics controls and physical chemistry which are directly applicable to structural problems most notable among the nonstructural based methods are the adjoint variable technique from control theory and the green s function and fast methods from physical chemistry

stability of an explicit high order spectral element method for acoustics in heterogeneous media based on local element stability criteria this paper considers the stability of an explicit leapfrog time marching scheme for the simulation of acoustic wave propagation in heterogeneous media with high order spectral elements the global stability criterion is taken as a minimum over local element stability criteria obtained through the solution of element borne eigenvalue problems first an explicit stability criterion is obtained for the particular case of a strongly heterogeneous and or rapidly fluctuating medium using asymptotic analysis this criterion is only dependent upon the maximum velocity at the vertices of the mesh elements and not on the velocity at the interior nodes of the high order elements second in a more general setting bounds are derived using statistics of the coefficients of the elemental dispersion matrices different bounds are presented discussed and compared several numerical experiments show the accuracy of the proposed criteria in one dimensional test cases as well as in more realistic large scale 3d problems

dynamics of the maximum marginal likelihood hyperparameter estimation in image restoration gradient descent versus expectation and maximization algorithm dynamical properties of image restoration and hyperparameter estimation are investigated by means of statistical mechanics we introduce an exactly solvable model for image restoration and derive differential equations with respect to macroscopic quantities from these equations we evaluate relaxation processes of the system to the equilibrium state our statistical mechanical approach also enables us to investigate the hyperparameter estimation by means of maximization of the marginal likelihood by using gradient descent and the expectation and maximization algorithm from the dynamical point of view

error correcting codes and image restoration with multiple stages of dynamics we consider the problems of error correcting codes and image restoration with multiple stages of dynamics information extracted from the former stage can be used selectively to improve the performance of the latter one analytic results were derived for the mean field systems using the cavity method we find that it has the advantage of being tolerant to uncertainties in hyperparameter estimation as confirmed by simulations

statistical mechanics of image restoration and error correcting codes we develop a statistical mechanical formulation for image restoration and error correcting codes these problems are shown to be equivalent to the ising spin glass with ferromagnetic bias under random external fields we prove that the quality of restoration decoding is maximized at a specific set of parameter values determined by the source and channel properties for image restoration in a mean field system a line of optimal performance is shown to exist in the parameter space these results are illustrated by solving exactly the infinite range model the solutions enable us to determine how precisely one should estimate unknown parameters monte carlo simulations are carried out to see how far the conclusions from the infinite range model are applicable to the more realistic two dimensional case in image restoration

non stationary random responses of a multi degree of freedom system by the theory of evolutionary spectra general expressions for the closed form transient receptance evolutionary spectral and cross spectral densities required in the non stationary random analysis of structures discretized by the finite element method to non stationary random excitations are presented application of the derived results to the non stationary random analysis of a mast antenna structure subjected to base excitations treated as a uniformly modulated random process is made the computed results of evolutionary spectral and cross spectral densities are presented graphically