association schemes and coding theory this paper contains a survey of association scheme theory with its algebraic and analytical aspects and of its applications to coding theory in a wide sense it is mainly concerned with a class of subjects that involve the central notion of the distance distribution of a code special emphasis is put on the linear programming method inspired by the macwilliams transform this produces upper bounds for the size of a code with a given minimum distance and lower bounds for the size of a design with a given strength the most specific results are obtained in the case where the underlying association scheme satisfies certain well defined polynomial properties this leads one into the realm of orthogonal polynomial theory in particular some universal bounds are derived for codes and designs in polynomial type association schemes throughout the paper the main concepts methods and results are illustrated by two examples that are of major significance in classical coding theory namely the hamming scheme and the johnson scheme other topics that receive special attention are spherical codes and designs and additive codes in translation schemes including z sub 4 additive binary codes

the linear programming bound for binary linear codes combining delsarte s 1973 linear programming bound with the information that certain weights cannot occur new upper bounds for d sub min n k the maximum possible minimum distance of a binary linear code with given word length n and dimension k are derived

three best binary linear block codes of minimum distance fifteen three new best codes 37 9 15 42 12 15 and 48 16 15 their weight distributions and a brief description of the used search algorithms are presented

a distance spectrum interpretation of turbo codes the performance of turbo codes is addressed by examining the code s distance spectrum the error floor that occurs at moderate signal to noise ratios is shown to be a consequence of the relatively low free distance of the code it is also shown that the error floor can be lowered by increasing the size of the interleaver without changing the free distance of the code alternatively the free distance of the code may be increased by using primitive feedback polynomials the excellent performance of turbo codes at low signal to noise ratios is explained in terms of the distance spectrum the interleaver in the turbo encoder is shown to reduce the number of low weight codewords through a process called spectral thinning this thinned distance spectrum results in the free distance asymptote being the dominant performance parameter for low and moderate signal to noise ratios

mls based variable node elements compatible with quadratic interpolation part i formulation and application for non matching meshes two dimensional variable node elements compatible with quadratic interpolation are developed using the moving least square mls approximation the mapping from the parental domain to the physical element domain is implicitly obtained from mls approximation with the shape functions and their derivatives calculated and saved only at the numerical integration points it is shown that the present mls based variable node elements meet the patch test if a sufficiently large number of integration points are employed for numerical integration the cantilever problem with non matching meshes is chosen to check the feasibility of the present mls based variable node elements and the result is compared with that from the lower order case compatible with linear interpolation copyright 2005 john wiley sons ltd

bump formation in a binary attractor neural network the conditions for the formation of local bumps in the activity of binary attractor neural networks with spatially dependent connectivity are investigated we show that these formations are observed when asymmetry between the activity during the retrieval and learning is imposed an analytical approximation for the order parameters is derived the corresponding phase diagram shows a relatively large and stable region where this effect is observed although critical storage and information capacities drastically decrease inside that region we demonstrate that the stability of the network when starting from the bump formation is larger than the stability when starting even from the whole pattern finally we show a very good agreement between the analytical results and the simulations performed for different topologies of the network

content addressability and learning in neural networks the content addressability of patterns stored in ising spin neural network models with symmetric interactions is studied numerical results from simulations on the icl distributed array processor dap involving systems with up to 2048 neurons are presented behaviour consistent with finite size scaling characteristic of a first order phase transition is shown to be exhibited by the basins of attraction of the stored patterns both in the case of the hopfield model and for systems using a local iterative learning algorithm designed to optimise the basins of attraction estimates are obtained for the critical minimum overlaps which an input pattern must have with a stored pattern in order to successfully retrieve it

mls based variable node elements compatible with quadratic interpolation part ii application for finite crack element two dimensional finite crack elements for simulation of propagating cracks are developed using the moving least square mls approximation the mapping from the parental domain to the physical element domain is implicitly obtained from mls approximation with the shape functions and their derivatives calculated and saved only at the numerical integration points the mls based variable node elements are extended to construct the crack elements which allow the discontinuity of crack faces and the crack tip singularity the accuracy of the crack elements is checked by calculating the stress intensity factor under mode i loading the crack elements turn out to be very efficient and accurate for simulating crack propagations only with the minimal amount of element adjustment and node addition as the crack tip moves numerical results and comparison to the results from other works demonstrate the effectiveness and accuracy of the present scheme for the crack elements copyright 2005 john wiley sons ltd

bounds on the decoding error probability of binary linear codes via their spectra bounds on the error probability of maximum likelihood decoding of a binary linear code are considered the bounds derived use the weight spectrum of the code and they are tighter than the conventional union bound in the case of large noise in the channel the bounds derived are applied to a code with an average spectrum and the result is compared to the random coding exponent the author shows that the bound considered for the binary symmetrical channel case coincides asymptotically with the random coding bound for the case of awgn channel the author shows that berlekamp s 1980 tangential bound can be improved but even this improved bound does not coincide with the random coding bound although it can be very close to it

learning of higher order perceptrons with tunable complexities we study learning from examples by higher order perceptrons which realize polynomially separable rules the model complexities of the networks are made tunable by varying the relative orders of different monomial terms we analyse the learning curves of higher order perceptrons when the gibbs algorithm is used for training it is found that learning occurs in a stepwise manner this is because the number of examples needed to constrain the corresponding phase space component scales differently

the mean field theory of a q state neural network model the mean field equations of a q state clock neural network model are derived in the replica symmetric approximation using the replica method these equations are studied for the cases q 2 3 4 and q to infinity it is shown that an infinite number of patterns can be stored in the network even in the limit q to infinity the phase diagram and storage capacity of the network are calculated and the information content considered although the overlap achieved with a nominated pattern is shown to decrease as q increases the information stored in the network is shown to increase

on the generalization ability of diluted perceptrons a linearly separable boolean function is teamed by a diluted perceptron with optimal stability a different level of dilution is allowed for teacher and student perceptron the learning algorithms used were the optimal annealed dilution and hebbian dilution the generalization ability i e the probability to recognize a pattern which has not been learned before is calculated in replica symmetry

dynamics of a neural network model with finite connectivity and cycle stored patterns the spatiotemporal evolution and memory retrieval properties of a hopfield like neural network with cycle stored patterns and finite connectivity are studied the analytical studies on a mean field version show that given the number of stored patterns p there is a critical connectivity kc such that the retrieval states are stable fixed points if and only if k kc the dependence of kc on the number of stored patterns is also present the numerical simulations are applied to the short ranged model with local interaction it is revealed that given p the memory retrieval function is kept if the connectivity is high enough while the dynamics of the system is in the frozen phase however when the connectivity k is less than a critical value kc the system is in the chaotic phase and loses its memory retrieval ability the critical points of both the dynamical phase transition and memory loss phase transition are obtained by simulation data

linear programming based optimization of the distance spectrum of linear block codes we describe an approach for the identification of good distance spectra for possibly existing binary linear block codes based on linear programming and the macwilliams delsarte 1977 1972 identities specifically the linear program is defined by an expression characterizing the performance of a potential code in terms of its distance spectrum and constraints imposed by the macwilliams delsarte identities using the union bound to characterize performance our results suggest that the best distance spectrum is not a function of signal to noise ratio snr above the cutoff rate snr and also suggest the existence of several unknown good codes characterizing the performance using the maximum spectral error component of the union bound suggests spectral thinning with decreasing snr

information processing in three state neural networks we introduce networks of three state neurons where the additional state embodies the absence of information their dynamical behavior is studied from the standpoint of information processing these networks display strong pattern completion capabilities moreover inference naturally occurs between coherent patterns

information and topology in attractor neural networks a wide range of networks including those with small world topology can be modeled by the connectivity ratio and randomness of the links both learning and attractor abilities of a neural network can be measured by the mutual information mi as a function of the load and the overlap between patterns and retrieval states in this letter we use mi to search for the optimal topology with regard to the storage and attractor properties of the network in an amari hopfield model we find that while an optimal storage implies an extremely diluted topology a large basin of attraction leads to moderate levels of connectivity this optimal topology is related to the clustering and path length of the network we also build a diagram for the dynamical phases with random or local initial overlap and show that very diluted networks lose their attractor ability

the nonexistence of certain binary linear codes some of the upper bounds given by t verhoeff 1987 are improved by proving the nonexistence of codes with certain parameters necessary preliminary results are stated the results give rise to upper bounds on d n k for many further values of n k by using simple standard techniques the further bounds would automatically follow from theorems by using verhoeff s computerized updating program but the authors state them here because several of them are required in the proofs of subsequent theorems

learning algorithms with optimal stability in neural networks to ensure large basins of attraction in spin glass like neural networks of two state elements xi imu or 1 the authors propose to study learning rules with optimal stability delta where delta is the largest number satisfying delta or sigma j jij xi jmu xi imu mu 1 p i 1 n where n is the number of neurons and p is the number of patterns they motivate this proposal and provide optimal stability learning rules for two different choices of normalisation for the synaptic matrix jij in addition numerical work is presented which gives the value of the optimal stability for random uncorrelated patterns

the space of interactions in neural network models the typical fraction of the space of interactions between each pair of n ising spins which solve the problem of storing a given set of p random patterns as n bit spin configurations is considered the volume is calculated explicitly as a function of the storage ratio alpha p n of the value kappa 0 of the product of the spin and the magnetic field at each site and of the magnetisation m here m may vary between 0 no correlation and 1 completely correlated the capacity increases with the correlation between patterns from alpha 2 for correlated patterns with kappa 0 and tends to infinity as m tends to 1 the calculations use a saddle point method and the order parameters at the saddle point are assumed to be replica symmetric this solution is shown to be locally stable a local iterative learning algorithm for updating the interactions is given which will converge to a solution of given kappa provided such solutions exist

selective pattern recall in neural networks by chemical modulation the authors present a simple neural network model in which the outputs of groups of neurons are chemically modulated this feature takes into account the existence of neuromodulators which are known to have a strong effect on the behaviour of real biological networks the effective non symmetric connection matrix thereby becomes a function of the modulator concentrations by deriving evolution equations for the pattern overlaps they show that during recall the system can select distinct subclasses from the stored patterns by varying these concentrations this model may increase the understanding of the function of neuromodulators

chaotic dynamics of high order neural networks the dynamics of an extremely diluted neural network with high order synapses acting as corrections to the hopfield model is investigated the learning rules for the high order connections contain mixing of memories different from all the previous generalizations of the hopfield model the dynamics may display fixed points or periodic and chaotic orbits depending on the weight of the high order connections e the noise levelt and the network load defined as the ratio between the number of stored patterns and the mean connectivity per neuron p c as in the related fully connected case there is an optimal value of the weight e that improves the storage capacity of the system the capacity diverges

learning by dilution in a neural network a perceptron with n random weights can store of the order of n patterns by removing a fraction of the weights without changing their strengths the critical storage capacity as a function of the concentration of the remaining bonds for random outputs and for outputs given by a teacher perceptron is calculated a simple hebb like dilution algorithm is presented which in the teacher case reaches the optimal generalization ability

partially connected models of neural networks a partially connected hopfield neural network model is studied under the restriction that w the ratio of connections per site to the size of the system remains finite as the size n to infinity with the connection structure at each site being the same the replica symmetric mean field theory equations for the order parameters are derived the zero temperature forms of these equations are then solved numerically for a few different local connectivity architectures showing phase transitions at different critical storage ratios alpha c where the states which the authors are trying to store in the network become discontinuously unstable they show that the information capacity per connection improves for partially connected systems

1 dimensional attractor neural networks we solve a class of attractor neural network models with a mixture of 1d nearest neighbour interactions and infinite range interactions which are both of a hebbian type form our solution is based on a combination of mean field methods transfer matrices and 1d random field techniques and is obtained both for boltzmann type equilibrium following sequential glauber dynamics and peretto type equilibrium following parallel dynamics competition between the alignment forces mediated via short range interactions and those mediated via infinite range ones is found to generate novel phenomena such as multiple locally stable pure states first order transitions between recall states 2 cycles and non recall states and domain formation leading to extremely long relaxation times we test our results against numerical simulations and simple benchmark cases and find excellent agreement