finite state automata and simple recurrent networks we explore a network architecture introduced by elman 1988 for predicting successive elements of a sequence the network uses the pattern of activation over a set of hidden units from time step t1 together with element t to predict element t 1 when the network is trained with strings from a particular finite state grammar it can learn to be a perfect finite state recognizer for the grammar when the network has a minimal number of hidden units patterns on the hidden units come to correspond to the nodes of the grammar although this correspondence is not necessary for the network to act as a perfect finite state recognizer we explore the conditions under which the network can carry information about distant sequential contingencies across intervening elements such information is maintained with relative ease if it is relevant at each intermediate step it tends to be lost when intervening elements do not depend on it at first glance this may suggest that such networks are not relevant to natural language in which dependencies may span indefinite distances however embeddings in natural language are not completely independent of earlier information the final simulation shows that long distance sequential contingencies can be encoded by the network even if only subtle statistical properties of embedded strings depend on the early information

self dual codes with an automorphism of order 11 using a method for constructing self dual codes having an automorphism of odd prime order we classify up to equivalence all binary self dual codes with an automorphism of order 11 with 6 cycles and minimum distance 12 this classification gives new 72 36 12 codes with weight enumerator that was previously not obtained as well as many 66 33 12 68 34 12 and 70 35 12 codes with new values of the parameters in their respective weight enumerators

control of stochastic discrete event systems modeled by probabilistic languages in garg et al 1999 and garg 1992 the formalism of probabilistic languages for modeling the stochastic qualitative behavior of discrete event systems dess was introduced in this paper we study their supervisory control where the control is exercised by dynamically disabling certain controllable events thereby nulling the occurrence probabilities of disabled events and increasing the occurrence probabilities of enabled events proportionately this is a special case of probabilistic supervision introduced in lawford and wonham 1993 the control objective is to design a supervisor such that the controlled system never executes any illegal traces their occurrence probability is zero and legal traces occur with minimum prespecified occurrence probabilities in other words the probabilistic language of the controlled system lies within a prespecified range where the upper bound is a nonprobabilistic language representing a legality constraint we provide a condition for the existence of a supervisor we also present an algorithm to test this existence condition when the probabilistic languages are regular so that they admit probabilistic automata representation with finitely many states next we give a technique to compute a maximally permissive supervisor online

a machine learning method for extracting symbolic knowledge from recurrent neural networks neural networks do not readily provide an explanation of the knowledge stored in their weights as part of their information processing until recently neural networks were considered to be black boxes with the knowledge stored in their weights not readily accessible since then research has resulted in a number of algorithms for extracting knowledge in symbolic form from trained neural networks this article addresses the extraction of knowledge in symbolic form from recurrent neural networks trained to behave like deterministic finite state automata dfas to date methods used to extract knowledge from such networks have relied on the hypothesis that networks states tend to cluster and that clusters of network states correspond to dfa states the computational complexity of such a cluster analysis has led to heuristics that either limit the number of clusters that may form during training or limit the exploration of the space of hidden recurrent state neurons these limitations while necessary may lead to decreased fidelity in which the extracted knowledge may not model the true behavior of a trained network perhaps not even for the training set the method proposed here uses a polynomial time symbolic learning algorithm to infer dfas solely from the observation of a trained network s input output behavior thus this method has the potential to increase the fidelity of the extracted knowledge

model uncertainty in discrete event systems earlier work concerning control of discrete event systems usually assumed that a correct model of the system to be controlled was available a goal of this work is to provide an algorithm for determining the correct model from a set of models the result of the algorithm is a finite language that can be used to test for the correct model or notification that the remaining models cannot be controllably distinguished we use the finite state machine model with controllable and uncontrollable events presented by ramadge and wonham

packings of a charged line on a sphere we find equilibrium configurations of open and closed lines of charge on a sphere and track them with respect to varying sphere radius closed lines transition from a circle to a spiral like shape through two low wave number bifurcations char22 baseball seam and twist char22 which minimize coulomb energy the spiral shape is the unique stable equilibrium of the closed line other unstable equilibria arise through tip splitting events an open line transitions smoothly from an arc of a great circle to a spiral as the sphere radius decreases under repulsive potentials with faster than coulomb power law decay the spiral is tighter in initial stages of sphere shrinkage but at later stages of shrinkage the equilibria for all repulsive potentials converge on a spiral with uniform spacing between turns multiple stable equilibria of the open line are observed

the n vortex problem on a rotating sphere ii heterogeneous platonic solid equilibria we describe a new method of constructing point vortex equilibria on a sphere made up of n vortices with different strengths such equilibria called heterogeneous equilibria are obtained for the five platonic solid configurations hence for graphic 1 the method is based on calculating a basis set for the nullspace of a matrix obtained by enforcing the necessary and sufficient condition that the mutual distances between each pair of vortices remain constant by symmetries inherent in the platonic solid configurations this matrix is reduced for each case and we call the dimension of the nullspace the degree of heterogeneity of the structure for the tetrahedron n 4 and octahedron n 6 the degree of heterogeneity is 4 and 6 respectively hence we are free to choose each of the vortex strengths independently for the cube n 8 the degree of heterogeneity is 5 for the icosahedron n 12 it is 7 while for the dodecahedron n 20 it is 4 thus the entire set of equilibria based on the platonic solid configurations is obtained including substructures associated with each configuration constructed by taking different linear combinations of the basis elements 1 embed inline graphic 1 gif

construction of point vortex equilibria via brownian ratchets a theory capable of producing equilibrium configurations of point vortices in the plane along with a numerical scheme to compute them is described the theory is formulated as a problem in linear algebra where one must find solutions to the matrix equation agz 0 where a is the 1 2 n nk1 n non normal configuration matrix obtained by requiring that all intervortical distances remain fixed and g2r n are the n vortex strengths for existence of an equilibrium a must have a non trivial nullspace we consider the singular values of a when this has one or more zero singular values the nullspace of a is non empty and an equilibrium exists for some choice of g new equilibrium configurations are found numerically by randomly depositing n points in the plane which generically gives rise to a configuration matrix a with empty nullspace using the sum of squares of the k smallest singular values of a as a ratchet we thermally fluctuate the configuration allowing each point to execute a random walk in the plane retaining only those configurations which reduce this quantity at the next step the configuration is thus driven to one with nullspace a zko0 these converged states are not necessarily nearby their initial configurations typically they are asymmetric and often we can drive the same initial state to several different equilibria a reverse ratchet method is also described which can produce initial conditions that would evolve to a specified equilibrium state once a converged final state is achieved the full singular value decomposition of a is used to calculate an optimal basis set for the nullspace of a and thus all allowable g the distribution of the singular values gives important information on the size of each equilibrium state as measured by frobenius norm their distance from each other spacing and density and how far a randomly chosen system of n points in the plane is from the nearest equilibrium configuration with a specified rank as well as its shannon entropy

vortex lattice theory a particle interaction perspective recent experiments on the formation of vortex lattices in bose einstein condensates has produced the need for a mathematical theory that is capable of predicting a broader class of lattice patterns ones that are free of discrete symmetries and can form in a random environment we give an overview of an n particle based hamiltonian theory which if formulated in terms of the o n 2 interparticle distances leads to the analysis of a nonnormal configuration matrix whose nullspace structure determines the existence or nonexistence of a lattice the singular value decomposition of this matrix leads to a method in which all lattice patterns and the associated particle strengths in principle can be classified and calculated by a random walk scheme which systematically uses the m smallest singular values as a ratchet mechanism to home in on lattices with m dimensional nullspaces where 0 m le n the resulting distribution of singular values encodes detailed geometric properties of the lattice and allows us to identify and calculate important quantitative measures associated with each lattice including its size as measured by the frobenius or 2 norms distance between the lattices hence lattice density robustness and shannon entropy as a quantitative measure of its level of disorder this article gives an overview of vortex lattice theory from 1957 to the present highlighting recent experiments in bose einstein condensate systems and formulating questions that can be addressed by understanding the singular value decomposition of the configuration matrix we discuss some of the computational challenges associated with producing large n lattices the subtleties associated with understanding and exploiting complicated hamiltonian energy surfaces in high dimensions and we highlight ten important directions for future research in this area

point vortex equilibria on the sphere via brownian ratchets we describe a brownian ratchet scheme that we use to calculate relative equilibrium configurations of n point vortices of mixed strength on the surface of a unit sphere we formulate it as a proble

green s function for the laplace beltrami operator on a toroidal surface green s function for the laplacebeltrami operator on the surface of a three dimensional ring torus is constructed an integral ingredient of our approach is the stereographic projection of the toru

extremal binary self dual codes in this correspondence we investigate binary extremal self dual codes numerous extremal self dual codes and interesting self dual codes with minimum weight d 14 and 16 are constructed in particular the first extremal type i 86 43 16 code and new extremal self dual codes with weight enumerators which were not previously known to exist for lengths 40 50 52 and 54 are constructed we also determine the possible weight enumerators for extremal type i codes of lengths 66 100

the calculi of emergence computation dynamics and induction abstract defining structure and detecting the emergence of complexity in nature are inherently subjective though essential scientific activities despite the difficulties these problems can be analyzed in terms of how model building observers infer from measurements the computational capabilities embedded in nonlinear processes an observer s notion of what is ordered what is random and what is complex in its environment depends directly on its computational resources the amount of raw measurement data of memory and of time available for estimation and inference the discovery of structure in an environment depends more critically and subtlely though on how those resources are organized the descriptive power of the observer s chosen or implicit computational model class for example can be an overwhelming determinant in finding regularity in data this paper presents an overview of an inductive framework hierarchical machine reconstruction in which the emergence of complexity is associated with the innovation of new computational model classes complexity metrics for detecting structure and quantifying emergence along with an analysis of the constraints on the dynamics of innovation are outlined illustrative examples are drawn from the onset of unpredictability in nonlinear systems finitary nondeterministic processes and cellular automata pattern recognition they demonstrate how finite inference resources drive the innovation of new structures and so lead to the emergence of complexity

some new extremal codes of lengths 76 and 78 we find new extremal 76 38 14 and 78 39 14 binary self dual codes the code of length 76 is the first known self dual code with these parameters it is shadow extremal and the shortest known self dual code with minimal distance 14 it has an automorphism of order 19 without fixed coordinates the code of length 78 has a new weight enumerator and an automorphism of order 19 with two fixed coordinates

point vortex equilibria and optimal packings of circles on a sphere we answer the question of whether optimal packings of circles on a sphere are equilibrium solutions to the logarithmic particle interaction problem for values of n 3 12 and 24 the only values of n for which the optimal packing problem also known as the tammes problem has rigorously known solutions we also address the cases n 13 23 where optimal packing solutions have been computed but not proven analytically as in jamaloodeen newton jamaloodeen newton 2006 proc r soc lond ser a 462 3277 3299 doi 10 1098 rspa 2006 1731 1 a logarithmic or point vortex equilibrium is determined by formulating the problem as the one in linear algebra graphic 2 where a is a n n 1 2 n non normal configuration matrix obtained by requiring that all interparticle distances remain constant if a has a kernel the strength vector graphic 3 is then determined as a right singular vector associated with the zero singular value or a vector that lies in the nullspace of a where the kernel is multi dimensional first we determine if the known optimal packing solution for a given value of n has a configuration matrix a with a non empty nullspace the answer is yes for n 3 9 11 14 16 and no for n 10 15 17 24 we then determine a basis set for the nullspace of a associated with the optimally packed state answer the question of whether n equal strength particles graphic 4 form an equilibrium for this configuration and describe what is special about the icosahedral configuration from this point of view we also find new equilibria by implementing two versions of a random walk algorithm first we cluster sub groups of particles into patterns during the packing process and grow a packed state using a version of the yin yang algorithm of longuet higgins longuet higgins 2008 proc r soc a doi 10 1098 rspa 2008 0219 we also implement a version of our brownian ratchet algorithm to find new equilibria near the optimally packed state for n 10 15 17 24 1 http dx doi org 10 1098 rspa 2006 1731 2 embed inline graphic 1 gif 3 embed inline graphic 2 gif 4 embed inline graphic 3 gif

snub polyhedra and organic growth this paper describes a new application of polyhedral theory to the growth of the outer sheath of certain viruses such structures are often modular consisting of one or two types of units arranged in a symmetric pattern in particular the polyoma virus has a structure apparently related to the snub dodecahedron here we consider the problem of how such patterns might grow in time starting from a given number n of randomly placed circles on the surface of a sphere the circles are first jostled by random perturbations then their radii are enlarged then they are jostled again and so on this yin yang method of growth can result in some very close packings when n 12 the closest packing corresponds to the snub tetrahedron and when n 24 the closest packing corresponds to the snub cube however when n 60 the closest packing does not correspond to the snub dodecahedron but to a less symmetric arrangement special attention is given to the structure of the human polyoma virus for which n 72 it is shown that the yin yang procedure successfully assembles the observed structure provided that the 72 circles are pre assembled in clusters of six each cluster consists of five circles arranged symmetrically around a sixth at the centre as in a flower with five petals this has implications for the assembly of the capsomeres in a polyoma virus

a method for constructing inequivalent self dual codes with applications to length 56 necessary and sufficient conditions are presented for two doubly even codes with an automorphism of odd prime order to be equivalent under certain assumptions for the codes as an application we obtain all extremal doubly even codes of length 56 with an automorphism of order 13 sixteen such codes are inequivalent

construction of self dual codes construction methods for self dual codes are given by using these methods some new extremal self dual 66 33 12 and 68 34 12 codes are obtained

spectral gradient flow and equilibrium configurations of point vortices we formulate the problem of finding equilibrium configurations of n point vortices in the plane in terms of a gradient flow on the smallest singular value of a skew symmetric matrix m whose nullspa

extracting finite state representations from recurrent neural networks trained on chaotic symbolic sequences concerns neural based modeling of symbolic chaotic time series we investigate the knowledge induction process associated with training recurrent mural nets rnn on single long chaotic symbolic sequences even though training rnn to predict the next symbol leaves the standard performance measures such as the mean square error on the network output virtually unchanged the nets extract a lot of knowledge we monitor the knowledge extraction process by considering the nets stochastic sources and letting them generate sequences which are then confronted with the training sequence via information theoretic entropy and cross entropy measures we also study the possibility of reformulating the knowledge gained by rnn in a compact easy to analyze form of finite state stochastic machines the experiments are performed on two sequences with different complexities measured by the size and state transition structure of the induced crutchfield s spl epsiv machines 1991 1994 the extracted machines can achieve comparable or even better entropy and cross entropy performance they reflect the training sequence complexity in their dynamical state representations that can be reformulated using finite state means the findings are confirmed by a much more detailed analysis of model generated sequences we also introduce a visual representation of allowed block structure in the studied sequences that allows for an illustrative insight into both rnn training and finite state stochastic machine extraction processes

recurrent neural networks with small weights implement definite memory machines recent experimental studies indicate that recurrent neural networks initialized with small weights are inherently biased toward definite memory machines tino cernansky benuskova 2002a 2002b this article establishes a theoretical counterpart transition function of recurrent network with small weights and squashing activation function is a contraction we prove that recurrent networks with contractive transition function can be approximated arbitrarily well on input sequences of unbounded length by a definite memory machine conversely every definite memory machine can be simulated by a recurrent network with contractive transition function hence initialization with small weights induces an architectural bias into learning with recurrent neural networks this bias might have benefits from the point of view of statistical learning theory it emphasizes one possible region of the weight space where generalization ability can be formally proved it is well known that standard recurrent neural networks are not distribution independent learnable in the probably approximately correct pac sense if arbitrary precision and inputs are considered we prove that recurrent networks with contractive transition function with a fixed contraction parameter fulfill the so called distribution independent uniform convergence of empirical distances property and hence unlike general recurrent networks are distribution independent pac learnable

energies and spacings of point charges on a sphere numerical methods are in general required for the determination of the stable configurations of n point charges on a sphere the stable configurations for n up to 50 have previously been ascertained and the authors extend the calculations for values up to 101 they report for the first time some remarkable global features of these configurations they show that the minimum energy accurately follows a simple half integral power law in 1 n over the full range they have investigated this power law is explicable in terms of the idealization of mapping a planar wigner lattice onto the surface of the unit sphere the pair distribution functions of the larger n configurations indicate predominant hexagonal coordination the coefficients of the observed power law are closely straddled by values calculated on the basis of hexagonal and square wigner lattices this highly accurate description of the energy permits them to remark on the detailed deviations of the individual structures from the general trend for n 30 they note that structures with n prime are relatively less stable while structures with n equal to 6 12 32 44 48 and 60 seem more stable

motion of vortex lines in the ginzburg landau model equations of motion for vortex lines in the ginzburg landau theory are derived we construct asymptotic approximations for the complex order parameter which are valid in the core region and in the far field using the method of matched asymptotic expansions we show that to leading order the line moves in the binormal direction with a curvature dependent velocity we also consider the contribution of remote parts of the line interaction between several vortex lines and interaction with external fields

the crystallizing substochastic sequential machine extractor cryssmex this letter presents an algorithm cryssmex for extracting minimal finite state machine descriptions of dynamic systems such as recurrent neural networks unlike previous algorithms cryssmex is parameter free and deterministic and it efficiently generates a series of increasingly refined models a novel finite stochastic model of dynamic systems and a novel vector quantization function have been developed to take into account the state space dynamics of the system the experiments show that 1 extraction from systems that can be described as regular grammars is trivial 2 extraction from high dimensional systems is feasible and 3 extraction of approximative models from chaotic systems is possible the results are promising and an analysis of shortcomings suggests some possible further improvements some largely overlooked connections of the field of rule extraction from recurrent neural networks to other fields are also identified

the closest packing of equal circles on a sphere the closest packing of x circles on the surface of a sphere is examined with the use of techniques that have been developed to determine the stereochemical arrangement of atoms packed around a central atom the technique is based on the concept that the centres of the circles repel one another and minimizing the total repulsion energy leads to an approximate structure from which exact solutions can be determined a number of improved packings have been discovered for values of x in the range 20 40 many different types of structure are found that are of lower symmetries than those previously described the packing density p defined as the fraction of the spherical surface that is enclosed by the circles is found to increase as the number of circles increases in contrast to the conclusion from previous studies however this increase in p is very slight and the values remain substantially below that for an infinite number of circles or a close packed plane

shadow bounds for self dual codes conway and sloane 1990 have previously given an upper bound on the minimum distance of a singly even self dual binary code using the concept of the shadow of a self dual code we improve their bound finding that the minimum distance of a self dual binary code of length n is at most 4 n 24 4 except when n mod 24 22 when the bound is 4 n 24 6 we also show that a code of length a multiple of 24 meeting the bound cannot be singly even the same technique gives similar results for additive codes over gf 4 relevant to quantum coding theory

learning regular sets from queries and counterexamples the problem of identifying an unknown regular set from examples of its members and nonmembers is addressed it is assumed that the regular set is presented by a minimamy adequate teacher which can answer membership queries about the set and can also test a conjecture and indicate whether it is equal to the unknown set and provide a counterexample if not a counterexample is a string in the symmetric difference of the correct set and the conjectured set a learning algorithm l is described that correctly learns any regular set from any minimally adequate teacher in time polynomial in the number of states of the minimum dfa for the set and the maximum length of any counterexample provided by the teacher it is shown that in a stochastic setting the ability of the teacher to test conjectures may be replaced by a random sampling oracle ex a polynomial time learning algorithm is shown for a particular problem of context free language identification

a 72 36 16 doubly even code does not have an automorphism of order 11 corresp we prove that there does not exist a 72 36 16 doubly even code with an automorphism of order 11 if such a code exists it can be decomposed as a direct sum of two codes one can be viewed as a self dual 12 6 4 binary code and the other a self dual 6 3 4 code over f 1024 this is shown to be impossible

automorphisms of codes with applications to extremal doubly even codes of length 48 general results on automorphisms of self dual binary codes are given these results are applied to the study of extremal self dual doubly even binary codes of length 48 the main theorem proved is that an extremal self dual doubly even code of length 48 with a nontrivial automorphism of odd order is equivalent to the extended quadratic residue code interesting constructions of the binary extended golay code as well as a conjecture about a possible connection between an extremal self dual doubly even code of length 72 and an extremal quaternary code of length 24 arc yielded by techniques used in the proof

a new upper bound on the minimal distance of self dual codes it is shown that the minimal distance d of a binary self dual code of length n or 74 is at most 2 n 6 10 this bound is a consequence of some new conditions on the weight enumerator of a self dual code obtained by considering a particular translate of the code called its shadow these conditions also enable one to find the highest possible minimal distance of a self dual code for all n or 60 to show that self dual codes with d or 22 with d or 8 exist precisely for n 24 32 and n or 26 and with d or 10 exist precisely for n or 46 and to show that there are exactly eight self dual codes of length 32 with d 8 several of the self dual codes of length 34 have trivial group this appears to be the smallest length where this can happen