surface integral equation method for the analysis of an obliquely stratified half space the radiation of a source in the presence of an obliquely stratified half space is studied using the surface integral equation method a generalized extinction theorem for the obliquely stratified half space problem is derived which gives eight surface integral equations for the fields the green s functions for the stratified media are found by a semianalytical method these surface integral equations can be used to solve the rather complicated two dimensional problem by the one dimensional finite element method fem which saves computer memory and computation time compared to the conventional two dimensional fem for some special cases the results obtained by this method are compared with those obtained by the numerical mode matching method and the fourier integral technique excellent agreement has been observed between them this method is general for any obliquely stratified half space with arbitrarily many layers in each region

mean field message passing equations in the hopfield model and its generalizations motivated by recent progress in using restricted boltzmann machines as preprocessing algorithms for deep neural network we revisit the mean field equations belief propagation and thouless anderson palmer tap equations in the best understood of such machines namely the hopfield model of neural networks and we explicit how they can be used as iterative message passing algorithms providing a fast method to compute the local polarizations of neurons in the retrieval phase where neurons polarize in the direction of one memorized pattern we point out a major difference between the belief propagation and tap equations the set of belief propagation equations depends on the pattern which is retrieved while one can use a unique set of tap equations this makes the latter method much better suited for applications in the learning process of restricted boltzmann machines in the case where the patterns memorized in the hopfield model are not independent but are correlated through a combinatorial structure we show that the tap equations have to be modified this modification can be seen either as an alteration of the reaction term in tap equations or more interestingly as the consequence of message passing on a graphical model with several hidden layers where the number of hidden layers depends on the depth of the correlations in the memorized patterns this layered structure is actually necessary when one deals with more general restricted boltzmann machines

expectation consistent approximate inference we propose a novel framework for approximations to intractable probabilistic models which is based on a free energy formulation the approximation can be understood as replacing an average over the original intractable distribution with a tractable one it requires two tractable probability distributions which are made consistent on a set of moments and encode different features of the original intractable distribution in this way we are able to use gaussian approximations for models with discrete or bounded variables which allow us to include non trivial correlations these are neglected in many other methods we test the framework on toy benchmark problems for binary variables on fully connected graphs and 2d grids and compare with other methods such as loopy belief propagation good performance is already achieved by using single nodes as tractable substructures significant improvements are obtained when a spanning tree is used instead

properties of the geometry of solutions and capacity of multilayer neural networks with rectified linear unit activations rectified linear units relus have become the main model for the neural units in current deep learning systems this choice was originally suggested as a way to compensate for the so called vanishing gradient problem which can undercut stochastic gradient descent learning in networks composed of multiple layers here we provide analytical results on the effects of relus on the capacity and on the geometrical landscape of the solution space in two layer neural networks with either binary or real valued weights we study the problem of storing an extensive number of random patterns and find that quite unexpectedly the capacity of the network remains finite as the number of neurons in the hidden layer increases at odds with the case of threshold units in which the capacity diverges possibly more important a large deviation approach allows us to find that the geometrical landscape of the solution space has a peculiar structure while the majority of solutions are close in distance but still isolated there exist rare regions of solutions which are much more dense than the similar ones in the case of threshold units these solutions are robust to perturbations of the weights and can tolerate large perturbations of the inputs the analytical results are corroborated by numerical findings

supervised learning with restricted training sets a generating functional analysis we study the dynamics of supervised on line learning of realizable tasks in feed forward neural networks we focus on the regime where the number of examples used for training is proportional to the number of input channels n using generating functional techniques from spin glass theory we are able to average over the composition of the training set and transform the problem for n to an effective single pattern system described completely by the student autocovariance the student teacher overlap and the student response function with exact closed equations our method applies to arbitrary learning rules i e not necessarily of a gradient descent type the resulting exact macroscopic dynamical equations can be integrated without finite size effects up to any degree of accuracy but their main value is in providing an exact and simple starting point for analytical approximation schemes finally we show how in the region of absent anomalous response and using the hypothesis that as in detailed balance systems the short time part of the various operators can be transformed away one can describe the stationary state of the network succesfully by a set of coupled equations involving only four scalar order parameters

green s function for the harmonic potential of the three dimensional wedge transmission problem the point source static potential in a wedge geometry consisting of two homogeneous media is solved via the kontorovich lebedev and fourier transforms inverse transforms enable the solution of laplace s equation to be expressed in terms of image contributions plus residue sums fourier series of toroidal functions as in previous wave equation solutions for isovelocity wedges explicit expressions for the poles that are the site of the residues are exploited when the wedge angle is a rational multiple of spl pi

3 d numerical mode matching nmm method for resistivity well logging tools a three dimensional 3 d numerical mode matching nmm method is presented for poisson s equation in general inhomogeneons media it reduces the original 3 d problem into a series of two dimensional 2 d eigenvalue problems plus a one dimensional 1 d layered medium problem which can be modeled efficiently by a recursion procedure the algorithm is tested for several 3 d inhomogeneous media and an excellent agreement between the nmm and analytical solutions is obtained for all test cases we demonstrate some typical applications of the 3 d nmm algorithm in resistivity well logging including invasion zones of noncircular shape vertical and horizontal fractures and horizontal wells the solution procedure proposed is directly applicable to wave propagation in 3 d inhomogeneous media

the diaphanous wedge complete solutions for the scattering by a diaphanous wedge meaning a wedge with identical wavenumbers inside and outside the wedge are presented the results are obtained from an integral equation for the fields on the wedge which is solved by the mellin and kantorovich lebedev transforms in the static and dynamic cases respectively pertinent formulations of gegenbauer s addition theorems play an important part in the derivation of the results which are presented in closed form

image solution for poisson s equation in wedge geometry the poisson s equation for a dielectric wedge is solved by deriving a static image that corresponds to the potential contribution of the wedge the problem is studied mainly in three dimensions but the solution for the analog two dimensional problem is also provided it appears that the use of image sources provides numerically a simple and efficient method to compute the potential inside and outside the wedge image sources for the exterior and interior regions consist of a line charge that decays exponentially as a function of complex angle and a set of point charges that can be interpreted as reflection or transmission images of a dielectric plane all known closed form solutions in terms of elementary functions are derived for an electrically and magnetically conducting half plane

solution of the fundamental problem in resistivity logging with a hybrid method the fundamental resistivity logging problem of a resistivity tool in the presence of both vertical and horizontal boundaries is solved with a hybrid method the hybrid method combines the mode concept in waveguide theory together with the finite element method in the mathematical formulation the horizontal boundaries are used to separate the geometry of the problem into different regions in each region the waveguide modes are obtained through the solution of an equivalent variational problem the solutions are calculated by a one dimensional finite element method the vertical boundaries are taken into account in these calculations the orthonormality of modes in each region allows a series representation of the potential in the regions boundary conditions at horizontal bed boundaries then couple the modes between different regions and enable the solutions for the potential to be expressed in terms of reflection and transmission matrices of modes the source excitation determines the amplitudes of th

transient dynamics of on line learning in two layered neural networks the dynamics of on line learning in neural networks with continuous units is dominated by plateaux in the time dependence of the generalization error using tools from statistical mechanics we show for a soft committee machine the existence of several fixed points of the dynamics of learning that give rise to complicated behaviour such as cascade like runs through different plateaux with a decreasing value of the corresponding generalization error we find learning rate dependent phenomena such as splitting and disappearing of fixed points of the equations of motion the dependence of plateau lengths on the initial conditions is described analytically and simulations confirm the results

generalization learning in a perceptron with binary synapses we consider the generalization problem for a perceptron with binary synapses implementing the stochastic belief propagation inspired sbpi learning algorithm which we proposed earlier and perform a mean field calculation to obtain a differential equation which describes the behaviour of the device in the limit of a large number of synapses n we show that the solving time of sbpi is of order n sqrt log n while the similar well known clipped perceptron cp algorithm does not converge to a solution at all in the time frame we considered the analysis gives some insight into the ongoing process and shows that in this context the sbpi algorithm is equivalent to a new simpler algorithm which only differs from the cp algorithm by the addition of a stochastic unsupervised meta plastic reinforcement process whose rate of application must be less than sqrt 2 pi n for the learning to be achieved effectively the analytical results are confirmed by simulations

optimal storage capacity of neural networks at finite temperatures gardner s analysis 1989 of the optimal storage capacity of neural networks is extended to study finite temperature effects the typical volume of the space of interactions is calculated for strongly diluted networks as a function of the storage ratio alpha temperature t and the tolerance parameter m from which the optimal storage capacity alpha c is obtained as a function of t and m at zero temperature it is found that alpha c 2 regardless of m while alpha c in general increases with the tolerance at finite temperatures the authors show how the best performance for given alpha and t is obtained which reveals a first order transition from high quality performance to a low quality one at low temperatures an approximate criterion for recalling which is valid near m 1 is also discussed

the simplest model of jamming we study a well known neural network model the perceptron as a simple statistical physics model of jamming of hard objects we exhibit two regimes 1 a convex optimization regime where jamming is hypostatic and non critical 2 a non convex optimization regime where jamming is isostatic and critical we characterize the critical jamming phase through exponents describing the distribution laws of forces and gaps surprisingly we find that these exponents coincide with the corresponding ones recently computed in high dimensional hard spheres in addition modifying the perceptron to a random linear programming problem we show that isostaticity is not a sufficient condition for singular force and gap distributions for that fragmentation of the space of solutions replica symmetry breaking appears to be a crucial ingredient we hypothesize universality for a large class of non convex constrained satisfaction problems with continuous variables

the complete extension of the biot tolstoy solution to the density contrast wedge with sample calculations the biot tolstoy exact time domain solution for the three dimensional impulse response of an impenetrable wedge is extended to accommodate the isovelocity or density contrast wedge fourier transformation of the time and axial variables along with a kantorovich lebedev transform applied to the cylindrical radial coordinate leads to a solution in terms of residue series when the wedge angle is a rational fraction of the residue series can be reduced to a finite sum which is evaluated for some special cases the total pressure field consists of geometrical acoustics contributions as predicted by snell s laws plus a modified version of the biot tolstoy diffraction field

content addressability and learning in neural networks the content addressability of patterns stored in ising spin neural network models with symmetric interactions is studied numerical results from simulations on the icl distributed array processor dap involving systems with up to 2048 neurons are presented behaviour consistent with finite size scaling characteristic of a first order phase transition is shown to be exhibited by the basins of attraction of the stored patterns both in the case of the hopfield model and for systems using a local iterative learning algorithm designed to optimise the basins of attraction estimates are obtained for the critical minimum overlaps which an input pattern must have with a stored pattern in order to successfully retrieve it

general solutions of the narrow strip and slot integral equations analytic solutions of the integral equations for the current induced on a narrow conducting strip are obtained by two methods one method follows from an expansion of the strip current in a series of chebyshev polynomials while the other is based on a special power series expansion in the former fourier type coefficients depending on integrals of the excitation are derived while in the latter the coefficients depend upon derivatives of the excitation from knowledge of the strip current simple expressions for the far zone scattered field are derived explicit results are given for plane wave excitation the solution methods apply to the equations for the narrow slot in a planar conducting screen

performance of the bayesian online algorithm for the perceptron in this letter we derive continuum equations for the generalization error of the bayesian online algorithm bona for the one layer perceptron with a spherical covariance matrix using the rosenblatt potential and show by numerical calculations that the asymptotic performance of the algorithm is the same as the one for the optimal algorithm found by means of variational methods with the added advantage that the bona does not use any inaccessible information during learning

accurate calculation of green s function of the schrodinger equation in a block layered potential in this paper a new algorithm is presented for calculating the green s function of the schrodinger equation in the presence of block layered potentials such green s functions have various and practical applications in quantum modelling of electron transport within nano mosfet transistors the proposed method is based on expansions of the eigenfunctions of the subordinate sturm liouville problems and a collocation matching procedure along possibly curved interfaces of the potential blocks accurate numerical results are provided to validate the proposed algorithm

the roles of stability and symmetry in the dynamics of neural networks the authors study the retrieval phase of spin glass like neural networks considering that the dynamics should depend only on gauge invariant quantities they propose that two such parameters characterising the symmetry of the neural net s connections and the stabilities of the patterns are responsible for most of the dynamical effects this is supported by a numerical study of the shape of the basins of attraction for a one pattern neural network model the effects of stability and symmetry on the short time dynamics of this model are studied analytically and the full dynamics for vanishing symmetry is shown to be exactly solvable

scattering by polygonal cross section dielectric cylinders at oblique incidence we analyze the scattering by polygonal cross section lossless dielectric cylinders illuminated by an obliquely incident plane wave the problem is formulated in terms of a system of surface integral equations opportunely devised so as to be valid for objects with edges and incident angles including the total reflection limit angle by means of galerkin s method in the spectral domain with analytically fourier transformable expansion functions factorizing the correct edge behaviors and continuity conditions of the unknowns convergence of exponential type is achieved and the coefficients of the scattering matrix are reduced to single integrals that can be efficiently evaluated numerical results for both near field and far field parameters are presented showing the quick convergence of the method even when applied to composite cylindrical objects

diffraction of axisymmetric waves in a borehole by bed boundary discontinuities this paper presents the calculation of the diffraction of axisymmetric borehole waves by bed boundary discontinuities the bed boundary is assumed to be horizontal and the inhomogeneities to be axially symmetric in such a geometry an axially symmetric source will produce only axially symmetric waves since the borehole is an open structure the mode spectrum consists of a discrete part as well as a continuum the scattering of a continuum of waves by bed boundaries is difficult to treat the approach used in the past in treating this class of problem has been approximate in nature or highly numerical such as the finite element method we present here a systematic way to approximate the continuum of modes by discrete modes after discretization the scattering problem can be treated simply since the approach is systematic it allows derivation of the solution to any desired degree of accuracy in theory but in practice it is limited by the computational resources available we also show that our appr

perturbation corrections in approximate inference mixture modelling applications bayesian inference is intractable for many interesting models making deterministic algorithms for approximate inference highly desirable unlike stochastic methods which are exact in the limit the accuracy of these approaches cannot be reasonably judged in this paper we show how low order perturbation corrections to an expectation consistent ec approximation can provide the necessary tools to ameliorate inference accuracy and to give an indication of the quality of approximation without having to resort to monte carlo methods further comparisons are given with variational bayes and parallel tempering pt combined with thermodynamic integration on a gaussian mixture model to obtain practical results we further generalize pt to temper from arbitrary distributions rather than a prior in bayesian inference

storage capacity and optimal learning of potts model perceptrons by a cavity method by means of a general formulation for the optimal learning capacity of perceptrons with multi state neurons and real valued couplings with spherical constraints which we derive by a cavity method we calculate the optimal learning capacity alpha c q k pmax n q 1 for perceptrons with a q resp q state potts model input resp output neurons as a function of q and the stability parameter k among other results the asymptote for q to infinity is found and it is shown that for k 0 the information gain per coupling delta i alpha c in q q 1 converges slowly to 1 2 in this limit moreover for q to infinity the same asymptotics also apply for the simple case of hebbian learning

optimal pruning in neural networks we study pruning strategies in simple perceptrons subjected to supervised learning our analytical results obtained through the statistical mechanics approach to learning theory are independent of the learning algorithm used in the training process we calculate the post training distribution mathcal p j of synaptic weights which depends only on the overlap ensuremath rho 0 achieved by the learning algorithm before pruning and the fraction ensuremath kappa of relevant weights in the teacher network from this distribution we calculate the optimal pruning strategy for deleting small weights the optimal pruning threshold grows from zero as ensuremath theta mathrm opt ensuremath rho 0 ensuremath kappa ensuremath propto ensuremath rho 0 ensuremath ensuremath rho c ensuremath kappa 1 2 above some critical value ensuremath rho c ensuremath kappa thus the elimination of weak synapses enhances the network performance only after a critical learning period possible implications for biological pruning phenomena are discussed

a new technique for the derivation of closed form electromagnetic green s functions for unbounded planar layered media a closed form electromagnetic green s function for unbounded planar layered media is derived in terms of a finite sum of hankel functions the derivation is based on the direct inverse hankel transform of a pole residue representation of the spectral domain form of the green s function such a pole residue form is obtained through the solution of the spectral domain form of the governing green s function equation numerically through a finite difference approximation rather than analytically the proposed methodology can handle any number of layers including the general case where the planar media exhibit arbitrary variation in their electrical properties in the vertical direction the numerical implementation of the proposed methodology is straightforward and robust and does not require any preprocessing of the spectrum of the green s function for the extraction of surface wave poles or its quasi static part the number of terms in the derived closed form expression is chosen adaptively with the distance between source and observation point as parameter the development of the closed form green s function is presented for both vertical and horizontal dipoles its accuracy is verified through a series of numerical examples and comparisons with results from other established methods

response of a source on top of a vertically stratified half space the solution of the response of a source on top of a horizontally stratified half space is well known however when the half space is vertically stratified the problem can only be solved with numerical methods like the finite element method here a semi analytic approach to solve such a problem is presented the three dimensional variation of the problem is reduced to two dimensional variation by fourier transform in one coordinate variable the remaining two dimensional problem is solved by finding the eigensolution in each of the half spaces the eigensolutions of each region are found from the partial differential equation directly using the same basis set of expansion functions this makes the calculation of the reflection and transmission operators very efficient the reflection and transmission operators account for the mode conversion reflection and transmission of the waves with the reflection and transmission operators the field everywhere can be calculated the solution reduces to that of the sommerfeld s half space problem when the two half spaces are homogeneous

learning dynamics on different timescales the special character of certain degrees of freedom in two layered neural networks is investigated for on line learning of realizable rules our analysis shows that the dynamics of these degrees of freedom can be put on a faster timescale than those remaining with the profit of speeding up the overall adaptation process this is shown for two groups of degrees of freedom second layer weights and bias weights for the former case our analysis provides a theoretical explanation of phenomenological findings the resulting learning algorithm is compared with natural gradient descent in order to check whether the proposed scaling can be naturally derived from that type of learning rule

the failure distribution in percolation models of breakdown the probability of finding a largest defect cluster of size n in a percolation network is calculated analytically using a new distribution function scaling equation from this result and the stress voltage enhancement at the tip of the most critical defect in the network the probability of failure in percolation models of breakdown is calculated for defect fractions less than the percolation point this distribution is found to be of the form exponential of an exponential numerical simulations on the two dimensional random fuse network confirm the new distribution function and convincingly distinguish between it and the weibull 1951 form most often used in the fitting of breakdown data

mellin transform solution for the static line source excitation of a dielectric wedge an integral transform analysis of the static scattering of the two dimensional potential radiated by a line source in the vicinity of a penetrable wedge is presented the mellin transform is used to derive the exact static solution to laplace s equation for the dielectric wedge in the form of a modal series the important dielectric edge condition behavior is explicitly contained in this analytic solution

electrostatic fractional image methods for perfectly conducting wedges and cones engheta 1996 introduced a definition for the electric charge fractional order multipoles using the concept of fractional derivatives and integrals here we utilize that definition to introduce a detailed image theory for the two dimensional 2 d electrostatic potential distributions in front of a perfectly conducting wedge with arbitrary wedge angles and for the three dimensional potential in front of a perfectly conducting cone with arbitrary cone angles we show that the potentials in the presence of these structures can be described equivalently as the electrostatic potentials of sets of equivalent image charge distributions that effectively behave as fractional order multipoles hence the name fractional image methods the fractional orders of these so called fractional images depend on the wedge angle for the wedge problem and on the cone angle for the cone problem special cases where these fractional images behave like the discrete images are discussed and physical justification and insights into these results are given

adaptive and self averaging thouless anderson palmer mean field theory for probabilistic modeling we develop a generalization of the thouless anderson palmer tap mean field approach of disorder physics which makes the method applicable to the computation of approximate averages in probabilistic models for real data in contrast to the conventional tap approach where the knowledge of the distribution of couplings between the random variables is required our method adapts to the concrete set of couplings we show the significance of the approach in two ways our approach reproduces replica symmetric results for a wide class of toy models assuming a nonglassy phase with given disorder distributions in the thermodynamic limit on the other hand simulations on a real data model demonstrate that the method achieves more accurate predictions as compared to conventional tap approaches

fast analytical methods for macroscopic electrostatic models in biomolecular simulations we review recent developments of fast analytical methods for macroscopic electrostatic calculations in biological applications including the poisson boltzmann pb and the generalized born models for electrostatic solvation energy the focus is on analytical approaches for hybrid solvation models especially the image charge method for a spherical cavity and also the generalized born theory as an approximation to the pb model this review places much emphasis on the mathematical details behind these methods

estimates of optimal storage conditions in neural network memories based on random matrix theory the authors formulate a method for estimating the critical conditions for storage of sets of data in neural network memory this variational method is based on random matrix theory and calculating the average spectrum of a matrix whose elements are given by overlaps of the stored patterns several generic cases of random overlap matrices are considered they investigate the cases of simply uncorrelated random patterns and spatially and semantically correlated ones they obtain bounds of the critical curve in the control parameters space which determine the stability of the stored data sets

three dimensional elastic problems of three dimensional quasicrystals in this paper the three dimensional initial value problem for three dimensional quasicrystals is considered for special polynomial form of initial conditions and nonhomogeneous terms it is proved that the solution can be presented in a similar form an analytic method for finding the exact solution of the problem with this polynomial form of initial conditions and nonhomogeneous terms is presented

lower bounds on generalization errors for drifting rules the problem of generalization by single layer perceptrons is studied in the case of time dependent rules lower bounds for the generalization errors within the single presentation of examples case are obtained for randomly drifting rules these bounds suggest a learning algorithm which uses knowledge of the error itself since this error is not readily available it has to be estimated through a mechanism of self evaluation the capacity of incorporating recency information into the error estimate is highly desirable the mechanism proposed has the advantage beyond good performance of being self adaptive in the sense that it adjusts to changes in the unknown drift rate of the rule the performance of the rule is also studied for sudden changes in an attempt to mimic the so called wisconsin test

on line learning in the ising perceptron on line learning of both binary and continuous rules in an ising space is studied learning is achieved by using an artificial parameter a weight vector which is constrained to the surface of a hypersphere spherical constraint in the case of a binary rule the generalization error decays to zero super exponentially as exp c 2 where is the number of examples divided by n the size of the input vector and c 0 much faster learning is obtained in the case of continuous activation functions where the generalization error decays as exp e the number of steps required for perfect learning is estimated for both scenarios and compared with simulations

the transmission problem on a three dimensional wedge we consider the transmission problem for the laplace equation on an infinite three dimensional wedge determining the complex parameters for which the problem is well posed and characterizing the infinite multiplicity nature of the spectrum this is carried out in two formulations leading to rather different spectral pictures one formulation is in terms of square integrable boundary data the other is in terms of finite energy solutions we use the layer potential method which requires the harmonic analysis of a non commutative non unimodular group associated with the wedge

the space of interactions in neural networks gardner s computation with the cavity method gardner s 1987 1988 computation of the number of n bit patterns which can be stored in an optical neural network used as an associative memory is derived without replicas using the cavity method this allows for a unified presentation whatever the basic measure in the space of coupling constants but above all it gives the clear physical content of the assumption of replica symmetry tap equations are also derived

on line learning in the committee machine the dynamics of learning from examples in the k 3 non overlapping committee machine with single presentation of examples is studied the optimal algorithm in the sense of mean generalization is obtained from a variational analysis of the differential equations which describe the dynamics the agreement of the theoretical predictions and the results of numerical simulations is excellent the optimized dynamics has the extra advantage with respect to the non optimized cases in that it uncouples the differential equations which describe the evolution of the relevant parameters i e the student teacher overlap and the norm of the student synaptic vector this in turn translates into the possibility of constructing useful practical optimized on line algorithms for the optimal algorithm the generalization error decays as approximately 0 88 alpha 1 the same nominal error as for the simple perceptron with optimized dynamics

gaussian windowed frame based method of moments formulation of surface integral equation for extended apertures scattering or coupling of electromagnetic beam field at a surface discontinuity separating two homogeneous or inhomogeneous media with different propagation characteristics is formulated using surface integral equation which are solved by the method of moments with the aid of the gabor based gaussian window frame set of basis and testing functions the application of the gaussian window frame provides i a mathematically exact and robust tool for spatial spectral phase space formulation and analysis of the problem ii a system of linear equations in a transmission line like form relating mode like wave objects of one medium with mode like wave objects of the second medium iii furthermore an appropriate setting of the frame parameters yields mode like wave objects that blend plane wave properties as if solving in the spectral domain with green s function properties as if solving in the spatial domain and iv a representation of the scattered field with gaussian beam propagators that may be used in many large in terms of wavelengths systems

slowly evolving connectivity in recurrent neural networks i the extreme dilution regime we study extremely diluted spin models of neural networks in which the connectivity evolves in time although adiabatically slowly compared to the neurons according to stochastic equations which on average aim to reduce frustration the fast neurons and slow connectivity variables equilibrate separately but at different temperatures our model is exactly solvable in equilibrium we obtain phase diagrams upon making the condensed ansatz i e recall of one pattern these show that as the connectivity temperature is lowered the volume of the retrieval phase diverges and the fraction of mis aligned spins is reduced still one always retains a region in the retrieval phase where recall states other than the one corresponding to the condensed pattern are locally stable so the associative memory character of our model is preserved

mellin transform and image charge method for dielectric sphere in an electrolyte we revisit the image charge method for the green s function problem of the poisson boltzmann equation for a dielectric sphere immersed in ionic solutions using finite mellin transformation we represent the reaction potential due to a source charge inside the sphere in terms of a one dimensional distribution of image charges the image charges are generically composed of a point image at the kelvin point and a line image extending from the kelvin point to infinity with an oscillatory line charge strength we further develop an efficient and accurate algorithm for discretization of the line image using pade approximation and finite fraction expansion finally we illustrate the power of our method by applying it in a multiscale reaction field monte carlo simulation of monovalent electrolytes

image solution for the 3 d static green s function of a vertically stratified two layer half space the general form of 3 d static green s function for a general structure consisting of n arbitrary adjacent dielectric wedges is derived using the kontorovich lebedev and fourier transforms it consists of several point sources in physical space and a line image source in complex space for each region of the problem geometry while the point sources can be treated conventionally the line image distributions are obtained by extracting a proper asymptotic behavior of the solution which can be numerically computed as an example the case of a vertically stratified two layer half space is investigated and the respective point and line image sources are obtained the efficiency of the proposed solution method is demonstrated by comparing the computational time for a given structure with that required by a commercially available numerical code using the finite integration technique

parallel strategy for optimal learning in perceptrons we developed a parallel strategy for learning optimally specific realizable rules by perceptrons in an online learning scenario our result is a generalization of the caticha kinouchi ck algorithm developed for learning a perceptron with a synaptic vector drawn from a uniform distribution over the n dimensional sphere so called the typical case our method outperforms the ck algorithm in almost all possible situations failing only in a denumerable set of cases the algorithm is optimal in the sense that it saturates bayesian bounds when it succeeds

selecting examples for perceptrons a technique is introduced for analysing the learning process of perceptrons which continually select their own examples the most efficient training algorithm yet devised the authors predict a 42 reduction in the number of examples wasted in training an ising perceptron compared to the case in which examples are random optimally stable spherical perceptrons may also be taught significantly more efficiently and their results compare well with an existing numerical simulation

on line learning in parity machines the optimal algorithm for on line learning in the tree k parity machine is studied we introduce a set of recursion relations for the relevant probability distributions which permit study of the general k case the generalization error curve is determined and shown to decay to zero for large as even in the presence of noise there is no critical noise level the dynamics of on line learning is studied analytically near the origin in the absence of previous knowledge the learning dynamics has a fixed point at previous knowledge is needed in at least k 1 branches for the learning to take place

transient image theory for 2 d and 3 d conducting wedge problems in the present paper two and three dimensional transient scattering from a perfectly conducting wedge is studied the exciting sources analyzed are an infinite impulsive line current parallel to edge of the conductor and an arbitrarily oriented electric dipole the analysis presented is based on the time harmonic image solution found for the scattered field the frequency domain image current corresponding to the field contribution from the wedge is an exact function expressed in terms of simple trigonometric functions it is dependent only on the coordinates and the amplitude of the source but it is independent of the observation point it is shown that the transient scattering can be interpreted as arising from an image source by generalizing the time domain green s function to complex source points

equations of anisotropic elastodynamics as a symmetric hyperbolic system deriving the time dependent fundamental solution the dynamic system of anisotropic elasticity from three second order partial differential equations is written in the form of the time dependent first order symmetric hyperbolic system with respect to displacement velocity and stress components a new method of deriving the time dependent fundamental solution of the obtained system is suggested in this paper this method consists of the following the fourier transform image of the fundamental solution with respect to a space variable is presented as a power series expansion relative to the fourier parameters then explicit formulae for the coefficients of these power series are derived successively using these formulae the computer calculation of fundamental solution components displacement velocity and stress components arising from pulse point forces has been made for general anisotropic media orthorhombic and monoclinic and the simulation of elastic waves has been obtained these computational examples confirm the robustness of the suggested method

initial value problem for the dynamic system of anisotropic elasticity the main object of this paper is the cauchy problem for the dynamic system of anisotropic elasticity existence and uniqueness theorems of weak and smooth solutions of this problem are established by the reduction of the original elasticity system into a symmetric hyperbolic system of the first order the numerical method of the cauchy problem solving for anisotropic elastic system with polynomial data is obtained and its correctness is established the simulations of the numerical solutions are presented

the dynamics of on line principal component analysis the learning dynamics of an on line algorithm for principal component analysis is described exactly in the thermodynamic limit by means of coupled ordinary differential equations for a set of order parameters it is demonstrated that learning is delayed significantly because existing symmetries among student vectors have to be broken a closely related effect is the perfect or partial loss of initial knowledge in the course of learning the analysis shows that different learning rates for the student vectors improve the performance of the algorithm drastically

the space of interactions in neural network models the typical fraction of the space of interactions between each pair of n ising spins which solve the problem of storing a given set of p random patterns as n bit spin configurations is considered the volume is calculated explicitly as a function of the storage ratio alpha p n of the value kappa 0 of the product of the spin and the magnetic field at each site and of the magnetisation m here m may vary between 0 no correlation and 1 completely correlated the capacity increases with the correlation between patterns from alpha 2 for correlated patterns with kappa 0 and tends to infinity as m tends to 1 the calculations use a saddle point method and the order parameters at the saddle point are assumed to be replica symmetric this solution is shown to be locally stable a local iterative learning algorithm for updating the interactions is given which will converge to a solution of given kappa provided such solutions exist

optimal storage properties of neural network models the authors calculate the number p alpha n of random n bit patterns that an optimal neural network can store allowing a given fraction f of bit errors and with the condition that each right bit is stabilised by a local field at least equal to a parameter k for each value of alpha and k there is a minimum fraction fmin of wrong bits they find a critical line alpha c k with alpha c 0 2 the minimum fraction of wrong bits vanishes for alpha alpha c k the calculations are done using a saddle point method and the order parameters at the saddle point are assumed to be replica symmetric this solution is locally stable in a finite region of the k alpha plane including the line alpha c k but there is a line above which the solution becomes unstable and replica symmetry must be broken

on line learning in a discrete state space on line learning of a rule given by an n dimensional ising perceptron is considered for the case when the student is constrained to take values in a discrete state space of size for l 2 no on line algorithm can achieve a finite overlap with the teacher in the thermodynamic limit however if l is on the order of hebbian learning does achieve a finite overlap

learning drifting concepts with neural networks the learning of time dependent concepts with a neural network is studied analytically and numerically the linearly separable target rule is represented by an n vector whose time dependence is modelled by a random or deterministic drift process a single layer network is trained online using different hebb like algorithms training is based on examples which are chosen randomly and according to a query strategy the evolution of the generalization error can be calculated exactly in the thermodynamic limit n to infinity the rule is never learnt perfectly but can be tracked within a certain error margin the generalization performance of various learning rules is compared and simulations confirm the analytic results

learning algorithms with optimal stability in neural networks to ensure large basins of attraction in spin glass like neural networks of two state elements xi imu or 1 the authors propose to study learning rules with optimal stability delta where delta is the largest number satisfying delta or sigma j jij xi jmu xi imu mu 1 p i 1 n where n is the number of neurons and p is the number of patterns they motivate this proposal and provide optimal stability learning rules for two different choices of normalisation for the synaptic matrix jij in addition numerical work is presented which gives the value of the optimal stability for random uncorrelated patterns

on the ability of the optimal perceptron to generalise a linearly separable boolean function is derived from a set of examples by a perceptron with optimal stability the probability to reconstruct a pattern which is not learnt is calculated analytically using the replica method

biased learning in boolean perceptrons abstract the generalization ability of hebbian boolean perceptrons can be improved by a kind of feedback mechanism in which the student net judges the difficulty of a new example from its previous experience it is shown that by giving a higher weight to the hard examples both generalization and learning abilities can be increased analytical as well as numerical results are presented for both cases where the examples are drawn at random or selected in an intelligent form

the rosenblatt bayesian algorithm learning in a nonstationary environment in this letter we study online learning in neural networks nns obtained by approximating bayesian learning the approach is applied to gibbs learning with the rosenblatt potential in a nonstationary environment the online scheme is obtained by the minimization maximization of the kullback leibler divergence cross entropy between the true posterior distribution and the parameterized one the complexity of the learning algorithm is further decreased by projecting the posterior onto a gaussian distribution and imposing a spherical covariance matrix we study in detail the particular case of learning linearly separable rules in the case of a fixed rule we observe an asymptotic generalization error egpropalpha 1 for both the spherical and the full covariance matrix approximations however in the case of drifting rule only the full covariance matrix algorithm shows a good performance this good performance is indeed a surprise since the algorithm is obtained by projecting without the benefit of the extra information on drifting

analysis of the effects of lesions on a perceptron gardner s 1988 analysis is used to study the evolution of a perceptron trained on a rule controlled mapping with exceptions after it has gone through an irreversible random process of deterioration or lesion it is shown that entropy considerations lead to useful statistical inferences based on partial information about the consequences of the lesions in particular it is shown that patterns that follow the rule are more robust than patterns that have an exceptional response

entropy landscape of solutions in the binary perceptron problem the statistical picture of the solution space for a binary perceptron is studied the binary perceptron learns a random classification of input random patterns by a set of binary synaptic weights the learning of this network is difficult especially when the pattern constraint density is close to the capacity which is supposed to be intimately related to the structure of the solution space the geometrical organization is elucidated by the entropy landscape from a reference configuration and of solution pairs separated by a given hamming distance in the solution space we evaluate the entropy at the annealed level as well as replica symmetric level and the mean field result is confirmed by the numerical simulations on single instances using the proposed message passing algorithms from the first landscape a random configuration as a reference we see clearly how the solution space shrinks as more constraints are added from the second landscape of solution pairs we deduce the coexistence of clustering and freezing in the solution space

optimal generalization in perceptions a new learning algorithm for the one layer perceptron is presented it aims to maximize the generalization gain per example analytical results are obtained for the case of single presentation of each example the weight attached to a hebbian term is a function of the expected stability of the example in the teacher perceptron this leads to the obtention of upper bounds for the generalization ability this scheme can be iterated and the results of numerical simulations show that it converges within errors to the theoretical optimal generalization ability of the bayes algorithm analytical and numerical results for an algorithm with maximized generalization in the learning strategy with selection of examples are obtained and it is proved that as expected orthogonal selection is optimal exponential decay of the generalization error is obtained for the single presentation of selected examples

numerical mode matching method for the multiregion vertically stratified media em wave propagation the response of a source in the presence of an n region vertically stratified media is solved using the numerical mode matching method by treating the fields propagating in the direction parallel to the subboundaries of the stratified media in terms of the propagators and by introducing the concepts of reflection operators transmission operators and generalized reflection operators the two dimensional problem is reduced to several one dimensional problems which are solved by the one dimensional finite element method fem the one dimensional method saves computer storage and computation time compared to the two dimensional version a formulation valid for a general n region vertically stratified medium is derived when there are only three regions the results compare well with those in the literature some typical numerical results for n 3 are also shown

non deterministic learning dynamics in large neural networks due to structural data bias we study the dynamics of on line learning in large n perceptrons for the case of training sets with a structural n0 bias of the input vectors by deriving exact and closed macroscopic dynamical laws using non equilibrium statistical mechanical tools in sharp contrast to the more conventional theories developed for homogeneously distributed or only weakly biased data these laws are found to describe a non trivial and persistently non deterministic macroscopic evolution and a generalization error which retains both stochastic and sample to sample fluctuations even for infinitely large networks furthermore for the standard error correcting microscopic algorithms such as the perceptron learning rule one obtains learning curves with distinct bias induced phases our theoretical predictions find excellent confirmation in numerical simulations

dynamics of batch training in a perceptron early stopping and weight decay are studied in a linear perceptron using a new simplified approach to the dynamics in the thermodynamical limit the approach is directly deduced from the gradient descent weight update it allows an exact description of the dynamics of the batch training process the results are compared with a recent study of early stopping and weight decay based on the equilibrium statistical mechanics approach it is shown that the equilibrium results for early stopping are good approximations and are exact for weight decay furthermore in the dynamical approach it is possible to determine the necessary number of training steps to fulfil certain termination conditions it can be shown that asymptotically i e if the number of examples is large only two batch steps are required to reach the optimal convergence if the learning rate is optimally chosen

optimal storage of invariant sets of patterns in neural network memories the authors investigate optimal conditions for the storage of invariant sets of patterns in neural network memories the sets of patterns that they consider are highly correlated non random and invariant with respect to some symmetry transformations they generalize gardner s method to study the fractional volume in the space of neural interactions that allow for storage of invariant patterns they demonstrate that optimal storage conditions correspond to non trivial relations between the number of stored patterns their stability and other symmetry specific characteristics of patterns

expectation propagation in the large data limit summary expectation propagation ep is a widely successful algorithm for variational inference ep is an iterative algorithm used to approximate complicated distributions typically to find a gaussian approximation of posterior distributions in many applications of this type ep performs extremely well surprisingly despite its widespread use there are very few theoretical guarantees on gaussian ep and it is quite poorly understood to analyse ep we first introduce a variant of ep averaged ep which operates on a smaller parameter space we then consider averaged ep and ep in the limit of infinite data where the overall contribution of each likelihood term is small and where posteriors are almost gaussian in this limit we prove that the iterations of both averaged ep and ep are simple they behave like iterations of newton s algorithm for finding the mode of a function we use this limit behaviour to prove that ep is asymptotically exact and to obtain other insights into the dynamic behaviour of ep e g that it may diverge under poor initialization exactly like newton s method ep is a simple algorithm to state but a difficult one to study our results should facilitate further research into the theoretical properties of this important method

generalization performance of bayes optimal classification algorithm for learning a perceptron the generalization error of the bayes optimal classification algorithm when learning a perceptron from noise free random training examples is calculated exactly using methods of statistical mechanics it is shown that if an assumption of replica symmetry is made then in the thermodynamic limit the error of the bayes optimal algorithm is less than the error of a canonical stochastic learning algorithm by a factor approaching ensuremath surd 2 as the ratio of the number of training examples to perceptron weights grows in addition it is shown that approximations to the generalization error of the bayes optimal algorithm can be achieved by learning algorithms that use a two layer neutral net to learn a perceptron

electromagnetic scattering from a dielectric wedge and the single hypersingular integral equation electromagnetic fields scattered by a finite dielectric wedge are computed using a hypersingular integral equation hie the results are compared with those obtained previously using a singular integral equation sie and with the theory that predicts that the fields near the edge of the wedge behave like static fields the hie produces more consistent results than the sie probably because the unknown boundary function tends to a constant near the edge instead of diverging the two numerical methods agree reasonably well and these results agree only in part with the static field behavior

a truncated em approach for spike and slab sparse coding we study inference and learning based on a sparse coding model with spike and slab prior as in standard sparse coding the model used assumes independent latent sources that linearly combine to generate data points however instead of using a standard sparse prior such as a laplace distribution we study the application of a more flexible spike and slab distribution which models the absence or presence of a source s contribution independently of its strength if it contributes we investigate two approaches to optimize the parameters of spike and slab sparse coding a novel truncated em approach and for comparison an approach based on standard factored variational distributions the truncated approach can be regarded as a variational approach with truncated posteriors as variational distributions in applications to source separation we find that both approaches improve the state of the art in a number of standard benchmarks which argues for the use of spike and slab priors for the corresponding data domains furthermore we find that the truncated em approach improves on the standard factored approach in source separation tasks which hints to biases introduced by assuming posterior independence in the factored variational approach likewise on a standard benchmark for image denoising we find that the truncated em approach improves on the factored variational approach while the performance of the factored approach saturates with increasing numbers of hidden dimensions the performance of the truncated approach improves the state of the art for higher noise levels

anisotropic elastodynamics in a half space an analytic method for polynomial data an initial boundary value problem for the dynamic system of anisotropic elasticity in a half space is studied in the paper a novel method of finding an exact solution of this problem for a special polynomial form of initial data and inhomogeneous term of the system is described on the base of this method the simulation of elastic waves in different homogeneous anisotropic half spaces is implemented

two numerical solutions for the parallel plate fed slot antenna an integrodifferential equation for the electric field in the aperture of a parallel plate waveguide fed slot antenna is numerically solved using galerkin s method with a basis of edge condition weighted chebyshev polynomials and pulse expansion with point matching efficient algorithms are obtained for the elements of the reduced matrix equations for both moment methods the numerical convergence of the techniques is demonstrated by the resultant aperture field distributions and input reflection coefficients for the case of dominant transverse electromagnetic mode excitation

expectation propagation with factorizing distributions a gaussian approximation and performance results for simple models we discuss the expectation propagation ep algorithm for approximate bayesian inference using a factorizing posterior approximation for neural network models we use a central limit theorem argument to make ep tractable when the number of parameters is large for two types of models we show that ep can achieve optimal generalization performance when data are drawn from a simple distribution