permitted and forbidden sets in discrete time linear threshold recurrent neural networks the concepts of permitted and forbidden sets enable a new perspective of the memory in neural networks such concepts exhibit interesting dynamics in recurrent neural networks this paper studies the basic theories of permitted and forbidden sets of the linear threshold discrete time recurrent neural networks the linear threshold transfer function has been regarded as an adequate transfer function for recurrent neural networks networks with this transfer function form a class of hybrid analog and digital networks which are especially useful for perceptual computations networks in discrete time can directly provide algorithms for efficient implementation in digital hardware the main contribution of this paper is to establish foundations of permitted and forbidden sets necessary and sufficient conditions for the linear threshold discrete time recurrent neural networks are obtained for complete convergence existence of permitted and forbidden sets as well as conditionally multiattractivity respectively simulation studies explore some possible interesting practical applications

affine automata and related techniques for generation of complex images abstract in this paper we introduce probabilistic affine automata paa which are probabilistic finite generators having transitions labeled with affine transformations it is shown that paa are capable of generating highly complex images barnsley s 1988 ifs method to generate fractals is a special case of paa when the automaton happens to have only a single state a number of theoretical results on paa are shown the relationship between paa mutually recursive function systems mrfs and affine regular sets is investigated

finite state automata and simple recurrent networks we explore a network architecture introduced by elman 1988 for predicting successive elements of a sequence the network uses the pattern of activation over a set of hidden units from time step t1 together with element t to predict element t 1 when the network is trained with strings from a particular finite state grammar it can learn to be a perfect finite state recognizer for the grammar when the network has a minimal number of hidden units patterns on the hidden units come to correspond to the nodes of the grammar although this correspondence is not necessary for the network to act as a perfect finite state recognizer we explore the conditions under which the network can carry information about distant sequential contingencies across intervening elements such information is maintained with relative ease if it is relevant at each intermediate step it tends to be lost when intervening elements do not depend on it at first glance this may suggest that such networks are not relevant to natural language in which dependencies may span indefinite distances however embeddings in natural language are not completely independent of earlier information the final simulation shows that long distance sequential contingencies can be encoded by the network even if only subtle statistical properties of embedded strings depend on the early information

a feti dp formulation for the three dimensional stokes problem without primal pressure unknowns a feti dp dual primal finite element tearing and interconnecting algorithm for the three dimensional stokes problem is developed and analyzed this is an extension of the previous work for the two dimensional problem in h h kim c o lee and e h park siam j numer anal 47 2010 pp 4142 4162 advantages of this approach are the coarse problem without primal pressure unknowns and the use of a computationally cheap lumped preconditioner especially in three dimensions these advantages provide a more practical feti dp algorithm in three dimensions the velocity unknowns at subdomain corners and the averages of velocity unknowns over common faces are selected as the primal unknowns in the feti dp formulation its condition number bound is analyzed to be ch h where c is a positive constant which is independent of any mesh parameters and h h is the number of elements across each subdomain numerical results are included

dynamical analysis of continuous higher order hopfield networks for combinatorial optimization in this letter the ability of higher order hopfield networks to solve combinatorial optimization problems is assessed by means of a rigorous analysis of their properties the stability of the continuous network is almost completely clarified 1 hyperbolic interior equilibria which are unfeasible are unstable 2 the state cannot escape from the unitary hypercube and 3 a lyapunov function exists numerical methods used to implement the continuous equation on a computer should be designed with the aim of preserving these favorable properties the case of nonhyperbolic fixed points which occur when the hessian of the target function is the null matrix requires further study we prove that these nonhyperbolic interior fixed points are unstable in networks with three neurons and order two the conjecture that interior equilibria are unstable in the general case is left open

global exponential stability of discrete time recurrent neural networks with impulses abstract in this paper the global exponential stability of a discrete time recurrent neural network with impulses is discussed first conditions for existence and uniqueness of the equilibrium point of these neural networks are introduced second some sufficient rules for the global exponential stability of these neural networks are obtained by using the lyapunov function

overlapping schwarz methods for mixed linear elasticity and stokes problems overlapping schwarz preconditioners are introduced and studied for saddle point problems with a penalty term such as stokes equations and mixed formulations of linear elasticity these preconditioners are based on the solution of local saddle point problems on overlapping subdomains and the solution of a coarse saddle point problem numerical experiments show that these are parallel and scalable preconditioners since the rate of convergence of the preconditioned operator is independent of the mesh size h the number of subdomains n and the penalty parameter

a two level nonoverlapping schwarz algorithm for the stokes problem without primal pressure unknowns a two level nonoverlapping schwarz algorithm is developed for the stokes problem the main feature of the algorithm is that a mixed problem with both velocity and pressure unknowns is solved with a balancing domain decomposition by constraints bddc type preconditioner which consists of solving local stokes problems and one global coarse problem related to only primal velocity unknowns our preconditioner allows to use a smaller set of primal velocity unknowns than other bddc preconditioners without much concern on certain flux conditions on the subdomain boundaries and the inf sup stability of the coarse problem in the two dimensional case velocity unknowns at subdomain corners are selected as the primal unknowns in addition to them averages of each velocity component across common faces are employed as the primal unknowns for the three dimensional case by using its close connection to the dual primal finite element tearing and interconnecting feti dp algorithm siam j sci comput 2010 32 3301 3322 siam j numer anal 2010 47 4142 4162 it is shown that the resulting matrix of our algorithm has the same eigenvalues as the feti dp algorithm except zero and one the maximum eigenvalue is determined by h h the number of elements across each subdomains and the minimum eigenvalue is bounded below by a constant which does not depend on any mesh parameters convergence of the method is analyzed and numerical results are included copyright 2011 john wiley sons ltd

globally asymptotical stability of discrete time analog neural networks some globally asymptotical stability criteria for the equilibrium states of a general class of discrete time dynamic neural networks with continuous states are presented using a diagonal lyapunov function approach the neural networks are assumed to have the asymmetrical weight matrices throughout the paper the resulting criteria are described by the diagonal stability of some matrices associated with the network parameters some novel stability conditions represented by either the existence of the positive diagonal solutions of the lyapunov equations or some inequalities are given using the equivalence between the diagonal stability and the schur stability for a nonnegative matrix some simplified global stability conditions are also presented finally some examples are provided for demonstrating the effectiveness of the global stability conditions presented

a machine learning method for extracting symbolic knowledge from recurrent neural networks neural networks do not readily provide an explanation of the knowledge stored in their weights as part of their information processing until recently neural networks were considered to be black boxes with the knowledge stored in their weights not readily accessible since then research has resulted in a number of algorithms for extracting knowledge in symbolic form from trained neural networks this article addresses the extraction of knowledge in symbolic form from recurrent neural networks trained to behave like deterministic finite state automata dfas to date methods used to extract knowledge from such networks have relied on the hypothesis that networks states tend to cluster and that clusters of network states correspond to dfa states the computational complexity of such a cluster analysis has led to heuristics that either limit the number of clusters that may form during training or limit the exploration of the space of hidden recurrent state neurons these limitations while necessary may lead to decreased fidelity in which the extracted knowledge may not model the true behavior of a trained network perhaps not even for the training set the method proposed here uses a polynomial time symbolic learning algorithm to infer dfas solely from the observation of a trained network s input output behavior thus this method has the potential to increase the fidelity of the extracted knowledge

identification of fuzzy dynamic systems using max min recurrent neural networks we present a new model of a max min recurrent neural network that is able to identify fuzzy dynamic systems from a set ofexamples once the neural network is trained the f relation that describes the system is encoded in its weights c 2001 elsevier science b v all rights reserved

a preconditioner for substructuring based on constrained energy minimization a preconditioner for substructuring based on constrained energy minimization concepts is presented the preconditioner is applicable to both structured and unstructured meshes and offers a straightforward approach for the iterative solution of second and fourth order structural mechanics problems the approach involves constraints associated with disjoint sets of nodes on substructure boundaries these constraints provide the means for preconditioning at both the substructure and global levels numerical examples are presented that demonstrate the good performance of the method in terms of iterations compute time and condition numbers of the preconditioned equations

a bddc method for mortar discretizations using a transformation of basis a bddc balancing domain decomposition by constraints method is developed for elliptic equations with discontinuous coefficients discretized by mortar finite element methods for geometrically nonconforming partitions in both two and three space dimensions the coarse component of the preconditioner is defined in terms of one mortar constraint for each edge face which is the intersection of the boundaries of a pair of subdomains a condition number bound of the form c max i 1 log h i h i 2 is established under certain assumptions on the geometrically nonconforming subdomain partition in the three dimensional case here h i and h i are the subdomain diameters and the mesh sizes respectively in the geometrically conforming case and the geometrically nonconforming cases in two dimensions no assumptions on the subdomain partition are required this bddc preconditioner is also shown to be closely related to the neumann dirichlet version of the feti dp algorithm the results are illustrated by numerical experiments which confirm the theoretical results

pattern storage bifurcations and groupwise correlation structure of an exactly solvable asymmetric neural network model despite their biological plausibility neural network models with asymmetric weights are rarely solved analytically and closed form solutions are available only in some limiting cases or in some m

stability analysis of neural net controllers using fuzzy neural networks neural networks can only be trained with a crisp and finite data set therefore the approximation quality of a trained network is hard to verify so a common way in proving stability of a trained neural net controller is to demonstrate the existence of a lyapunov function in this article we propose a new method how stability of a neural net controller used as a closed loop feedback controller can be proven instead of finding a lyapunov function conditions for a fuzzy training set are developed if a fuzzy neural net is trained using this training set special properties of fuzzy neural nets can be used for estimating the generalization error after defuzzification of the fuzzy net finite stability of the process can be concluded

bddc and feti dp without matrices or vectors abstract the connection between the bddc balancing domain decomposition by constraints algorithm and the feti dp finite element tearing and interconnecting dual primal algorithm is discussed in the language of function spaces operators and linear functionals

architectural bias in recurrent neural networks fractal analysis we have recently shown that when initialized with small weights recurrent neural networks rnns with standard sigmoid type activation functions are inherently biased toward markov models even prior to any training rnn dynamics can be readily used to extract finite memory machines hammer tino 2002 tino cernansk benuskova 2002a 2002b following christiansen and chater 1999 we refer to this phenomenon as the architectural bias of rnns in this article we extend our work on the architectural bias in rnns by performing a rigorous fractal analysis of recurrent activation patterns we assume the network is driven by sequences obtained by traversing an underlying finite state transition diagram a scenario that has been frequently considered in the past for example when studying rnn based learning and implementation of regular grammars and finite state transducers we obtain lower and upper bounds on various types of fractal dimensions such as box counting and hausdorff dimensions it turns out that not only can the recurrent activations inside rnns with small initial weights be explored to build markovian predictive models but also the activations form fractal clusters the dimension of which can be bounded by the scaled entropy of the underlying driving source the scaling factors are fixed and are given by the rnn parameters

an algebraic framework to represent finite state machines in single layer recurrent neural networks in this paper we present an algebraic framework to represent finite state machines fsms in single layer recurrent neural networks slrnns which unifies and generalizes some of the previous proposals this framework is based on the formulation of both the state transition function and the output function of an fsm as a linear system of equations and it permits an analytical explanation of the representational capabilities of first order and higher order slrnns the framework can be used to insert symbolic knowledge in rnns prior to learning from examples and to keep this knowledge while training the network this approach is valid for a wide range of activation functions whenever some stability conditions are met the framework has already been used in practice in a hybrid method for grammatical inference reported elsewhere sanfeliu and alquezar 1994

online stabilization of block diagonal recurrent neural networks deals with a discrete time recurrent neural network dtrnn with a block diagonal feedback weight matrix called the block diagonal recurrent neural network bdrnn that allows a simplified approach to online training and to address network and training stability issues the structure of the bdrnn is exploited to modify the conventional backpropagation through time bptt algorithm to reduce its storage requirement by a numerically stable method of recomputing the network state variables the network and training stability is addressed by exploiting the bdrnn structure to directly monitor and maintain stability during weight updates by developing a functional measure of system stability that augments the cost function being minimized simulation results are presented to demonstrate the performance of the bdrnn architecture its training algorithm and the stabilization method

balancing neumann neumann methods for incompressible stokes equations this paper describes the unerlying mathematical model and the balancing neumann neumann methods are introduced and studied for incompressible stokes equations discretized with mixed finite or spectral elements with discontinuous pressures after decomposing the original domain of the problem into nonoverlapping subdomains the interior unknowns which are the interior velocity component and all except the constant pressure component of each subdomain problem are implicitly eliminated the resulting saddle point schur complement is solved with a krylov space method with a balancing neumann neumann preconditioner based on the solution of a coarse stokes problem with a few degrees of freedom per subdomain and on the solution of local stokes problems with natural neumann velocity and essential boundary conditions on the subdomains this preconditioner is of hybrid form in which the coarse problem is treated multiplicatively while the local problems are treated additively the condition number of the preconditioned operator is independent of the number of subdomains and is bounded from above by the product of the square of the logarithm of the local number of unknowns in each subdomain and the inverse of the inf sup constants of the discrete problem and of the coarse subproblem numerical results show that the method is quite fast they are also fully consistent with the theory

on improving the efficiency of tensor voting this paper proposes two alternative formulations to reduce the high computational complexity of tensor voting a robust perceptual grouping technique used to extract salient information from noisy data the first scheme consists of numerical approximations of the votes which have been derived from an in depth analysis of the plate and ball voting processes the second scheme simplifies the formulation while keeping the same perceptual meaning of the original tensor voting the stick tensor voting and the stick component of the plate tensor voting must reinforce surfaceness the plate components of both the plate and ball tensor voting must boost curveness whereas junctionness must be strengthened by the ball component of the ball tensor voting two new parameters have been proposed for the second formulation in order to control the potentially conflictive influence of the stick component of the plate vote and the ball component of the ball vote results show that the proposed formulations can be used in applications where efficiency is an issue since they have a complexity of order o 1 moreover the second proposed formulation has been shown to be more appropriate than the original tensor voting for estimating saliencies by appropriately setting the two new parameters

attractive periodic sets in discrete time recurrent networks with emphasis on fixed point stability and bifurcations in two neuron networks we perform a detailed fixed point analysis of two unit recurrent neural networks with sigmoid shaped transfer functions using geometrical arguments in the space of transfer function derivatives we partition the network state space into distinct regions corresponding to stability types of the fixed points unlike in the previous studies we do not assume any special form of connectivity pattern between the neurons and all free parameters are allowed to vary we also prove that when both neurons have excitatory self connections and the mutual interaction pattern is the same i e the neurons mutually inhibit or excite themselves new attractive fixed points are created through the saddle node bifurcation finally for an n neuron recurrent network we give lower bounds on the rate of convergence of attractive periodic points toward the saturation values of neuron activations as the absolute values of connection weights grow

dimensionality estimation manifold learning and function approximation using tensor voting we address instance based learning from a perceptual organization standpoint and present methods for dimensionality estimation manifold learning and function approximation under our approach manifolds in high dimensional spaces are inferred by estimating geometric relationships among the input instances unlike conventional manifold learning we do not perform dimensionality reduction but instead perform all operations in the original input space for this purpose we employ a novel formulation of tensor voting which allows an n d implementation tensor voting is a perceptual organization framework that has mostly been applied to computer vision problems analyzing the estimated local structure at the inputs we are able to obtain reliable dimensionality estimates at each instance instead of a global estimate for the entire data set moreover these local dimensionality and structure estimates enable us to measure geodesic distances and perform nonlinear interpolation for data sets with varying density outliers perturbation and intersections that cannot be handled by state of the art methods quantitative results on the estimation of local manifold structure using ground truth data are presented in addition we compare our approach with several leading methods for manifold learning at the task of measuring geodesic distances finally we show competitive function approximation results on real data

discrete time analogs for a class of continuous time recurrent neural networks this paper is concerned with the problem of local and global asymptotic stability for a class of discrete time recurrent neural networks which provide discrete time analogs to their continuous time counterparts i e continuous time recurrent neural networks with distributed delay some stability criteria which include some existing results as their special cases are derived a discussion about the dynamical consistence of discrete time neural networks versus their continuous time counterparts is provided an unconventional finite difference method is proposed and an example is also given to show the effectiveness of the method

fuzzy finite state automata can be deterministically encoded into recurrent neural networks there has been an increased interest in combining fuzzy systems with neural networks because fuzzy neural systems merge the advantages of both paradigms on the one hand parameters in fuzzy systems have clear physical meanings and rule based and linguistic information can be incorporated into adaptive fuzzy systems in a systematic way on the other hand there exist powerful algorithms for training various neural network models however most of the proposed combined architectures are only able to process static input output relationships they are not able to process temporal input sequences of arbitrary length fuzzy finite state automats ffas can model dynamical processes whose current state depends on the current input and previous states unlike in the case of deterministic finite state automats dfas ffas are not in one particular state rather each state is occupied to some degree defined by a membership function based on previous work on encoding dfas in discrete time second order recurrent neural networks we propose an algorithm that constructs an augmented recurrent neural network that encodes a ffa and recognizes a given fuzzy regular language with arbitrary accuracy we then empirically verify the encoding methodology by correct string recognition of randomly generated ffas in particular we examine how the networks performance varies as a function of synaptic weight strengths

multistability of discrete time recurrent neural networks with unsaturating piecewise linear activation functions this paper studies the multistability of a class of discrete time recurrent neural networks with unsaturating piecewise linear activation functions it addresses the nondivergence global attractivity and complete stability of the networks using the local inhibition conditions for nondivergence are derived which not only guarantee nondivergence but also allow for the existence of multiequilibrium points under these nondivergence conditions global attractive compact sets are obtained complete stability is studied via constructing novel energy functions and using the well known cauchy convergence principle examples and simulation results are used to illustrate the theory

classification of temporal sequences via prediction using the simple recurrent neural network abstract an approach to classify temporal sequences using the simple recurrent neural network srnn is developed in this paper a classification problem is formulated as a component prediction problem and two training methods are described to train a single srnn to predict the components of temporal sequences belonging to multiple classes issues related to the selection of the dimension of the context vector and the influence of the context vector on classification are identified and investigated the use of a different initial context vector for each class is proposed as a means to improve classification and a classification rule which incorporates the different initial context vectors is formulated a systematic method in which the srnn is trained with noisy exemplars is developed to enhance the classification performance of the network a 4 class localized object classification problem is selected to demonstrate that a a single srnn can be trained to classify real multi class sequences via component prediction b the classification accuracy can be improved by using a distinguishing initial context vector for each class and c the classification accuracy of the srnn can be improved significantly by using the distinguishing initial context vector in conjunction with the systematic re training method it is concluded that through the approach developed in this paper the srnn can robustly classify temporal sequences which may have an unequal number of components

on the selection of primal unknowns for a feti dp formulation of the stokes problem in two dimensions selection of primal unknowns is important in convergence of feti dp dual primal finite element tearing and interconnecting methods which are known to be the most scalable dual iterative substructuring methods a feti dp algorithm for the stokes problem without primal pressure unknowns was developed and analyzed by kim et al 2010 1 only the velocity unknowns at the subdomain vertices are selected to be the primal unknowns and convergence of the algorithm with a lumped preconditioner is determined by the condition number bound c h h 1 log h h where h h is the number of elements across subdomains in this work primal unknowns corresponding to the averages on edges are introduced and a better condition number bound c h h is proved for such a selection of primal unknowns numerical results are included

stability and synchronization of discrete time markovian jumping neural networks with mixed mode dependent time delays in this paper we introduce a new class of discrete time neural networks dnns with markovian jumping parameters as well as mode dependent mixed time delays both discrete and distributed time delays specifically the parameters of the dnns are subject to the switching from one to another at different times according to a markov chain and the mixed time delays consist of both discrete and distributed delays that are dependent on the markovian jumping mode we first deal with the stability analysis problem of the addressed neural networks a special inequality is developed to account for the mixed time delays in the discrete time setting and a novel lyapunov krasovskii functional is put forward to reflect the mode dependent time delays sufficient conditions are established in terms of linear matrix inequalities lmis that guarantee the stochastic stability we then turn to the synchronization problem among an array of identical coupled markovian jumping neural networks with mixed mode dependent time delays by utilizing the lyapunov stability theory and the kronecker product it is shown that the addressed synchronization problem is solvable if several lmis are feasible hence different from the commonly used matrix norm theories such as the m matrix method a unified lmi approach is developed to solve the stability analysis and synchronization problems of the class of neural networks under investigation where the lmis can be easily solved by using the available matlab lmi toolbox two numerical examples are presented to illustrate the usefulness and effectiveness of the main results obtained

extracting finite state representations from recurrent neural networks trained on chaotic symbolic sequences concerns neural based modeling of symbolic chaotic time series we investigate the knowledge induction process associated with training recurrent mural nets rnn on single long chaotic symbolic sequences even though training rnn to predict the next symbol leaves the standard performance measures such as the mean square error on the network output virtually unchanged the nets extract a lot of knowledge we monitor the knowledge extraction process by considering the nets stochastic sources and letting them generate sequences which are then confronted with the training sequence via information theoretic entropy and cross entropy measures we also study the possibility of reformulating the knowledge gained by rnn in a compact easy to analyze form of finite state stochastic machines the experiments are performed on two sequences with different complexities measured by the size and state transition structure of the induced crutchfield s spl epsiv machines 1991 1994 the extracted machines can achieve comparable or even better entropy and cross entropy performance they reflect the training sequence complexity in their dynamical state representations that can be reformulated using finite state means the findings are confirmed by a much more detailed analysis of model generated sequences we also introduce a visual representation of allowed block structure in the studied sequences that allows for an illustrative insight into both rnn training and finite state stochastic machine extraction processes

inference of surfaces 3d curves and junctions from sparse noisy 3d data we address the problem of obtaining dense surface information from a sparse set of 3d data in the presence of spurious noise samples the input can be in the form of points or points with an associated tangent or normal allowing both position and direction to be corrupted by noise most approaches treat the problem as an interpolation problem which is solved by fitting a surface such as a membrane or thin plate to minimize some function we argue that these physical constraints are not sufficient and propose to impose additional perceptual constraints such as good continuity and cosurfacity these constraints allow us to not only infer surfaces but also to detect surface orientation discontinuities as well as junctions all at the same time the approach imposes no restriction on genus number of discontinuities number of objects and is noniterative the result is in the form of three dense saliency maps for surfaces intersections between surfaces i e 3d curves and 3d junctions respectively these saliency maps are then used to guide a marching process to generate a description e g a triangulated mesh making information about surfaces space curves and 3d junctions explicit the traditional marching process needs to be refined as the polarity of the surface orientation is not necessarily locally consistent these three maps are currently not integrated and this is the topic of our ongoing research we present results on a variety of computer generated and real data having varying curvature of different genus and multiple objects

iterated function systems and control languages valuations morphisms from e to 0 1 are a generalization of bernoulli morphisms introduced by eilenberg automata languages and machines academic press new york 1974 here we show how to generalize the notion of entropy of a language in order to obtain new formulas to determine the hausdorff dimension of fractal sets also in euclidean spaces especially defined via regular languages by doing this we can sharpen and generalize earlier results in two ways first we treat the case where the underlying basic iterated function system contains noncontractive mappings and second we obtain results valid for nonregular languages as well

equilibrium characterization of dynamical neural networks and a systematic synthesis procedure for associative memories several novel results concerning the characterization of the equilibrium conditions of a continuous time dynamical neural network model and a systematic procedure for synthesizing associative memory networks with nonsymmetrical interconnection matrices are presented the equilibrium characterization focuses on the exponential stability and instability properties of the network equilibria and on equilibrium confinement viz ensuring the uniqueness of an equilibrium in a specific region of the state space while the equilibrium confinement result involves a simple test the stability results given obtain explicit estimates of the degree of exponential stability and the regions of attraction of the stable equilibrium points using these results as valuable guidelines a systematic synthesis procedure for constructing a dynamical neural network that stores a given set of vectors as the stable equilibrium points is developed

feti dp a dual primal unified feti method part i a faster alternative to the two level feti method the feti method and its two level extension feti 2 are two numerically scalable domain decomposition methods with lagrange multipliers for the iterative solution of second order solid mechanics and fourth order beam plate and shell structural problems respectively the feti 2 method distinguishes itself from the basic or one level feti method by a second set of lagrange multipliers that are introduced at the subdomain cross points to enforce at each iteration the exact continuity of a subset of the displacement field at these specific locations in this paper we present a dual primal formulation of the feti 2 concept that eliminates the need for that second set of lagrange multipliers and unifies all previously developed one level and two level feti algorithms into a single dual primal feti dp method we show that this new feti dp method is numerically scalable for both second order and fourth order problems we also show that it is more robust and more computationally efficient than existing feti solvers particularly when the number of subdomains and or processors is very large copyright 2001 john wiley sons ltd

a dual primal feti method for incompressible stokes equations in this paper a dual primal feti method is developed for incompressible stokes equations approximated by mixed finite elements with discontinuous pressures the domain of the problem is decomposed into nonoverlapping subdomains and the continuity of the velocity across the subdomain interface is enforced by introducing lagrange multipliers by a schur complement procedure the solution of an indefinite stokes problem is reduced to solving a symmetric positive definite problem for the dual variables i e the lagrange multipliers this dual problem is solved by the conjugate gradient method with a dirichlet preconditioner in each iteration step both subdomain problems and a coarse level problem are solved by a direct method it is proved that the condition number of this preconditioned dual problem is independent of the number of subdomains and bounded from above by the square of the product of the inverse of the inf sup constant of the discrete problem and the logarithm of the number of unknowns in the individual subdomains numerical experiments demonstrate the scalability of this new method

a feti dp type domain decomposition algorithm for three dimensional incompressible stokes equations the feti dp dual primal finite element tearing and interconnecting algorithms proposed by the authors in siam j numer anal 51 2013 pp 1235 1253 and internat j numer methods engrg 94 2013 pp 128 149 for solving incompressible stokes equations are extended to three dimensional problems a new analysis of the condition number bound for using the dirichlet preconditioner is given the algorithm and analysis are valid for mixed finite elements with both continuous and discontinuous pressures an advantage of this new analysis is that the numerous coarse level velocity components required in the previous analysis to enforce the divergence free subdomain boundary velocity conditions are no longer needed this greatly reduces the size of the coarse level problem in the algorithm especially for three dimensional problems the coarse level velocity space can be chosen as simple as those coarse spaces for solving scalar elliptic problems corresponding to each velocity component both th

induction of finite state languages using second order recurrent networks second order recurrent networks that recognize simple finite state languages over 0 1 are induced from positive and negative examples using the complete gradient of the recurrent network and sufficient training examples to constrain the definition of the language to be induced solutions are obtained that correctly recognize strings of arbitrary length

a two level nonoverlapping schwarz algorithm for the stokes problem numerical study abstract a general framework of a two level nonoverlapping schwarz algorithm for the stokes problem is developed by relaxing average zero condition on pressure unknowns this framework allows both discontinuous and continuous pressure finite element spaces the coarse problem is built by algebraic manipulation after selecting appropriate primal unknowns just like in bddc algorithms performance of the suggested algorithm is presented depending on the selection of finite elements and primal unknowns under the same set of primal unknowns the algorithm for the case with discontinuous pressure functions outperforms one with continuous pressure functions for the two dimensional stokes problem the algorithm with a set of primal unknowns consisting of velocity unknowns at corners averages of velocity components over common edges and pressure unknowns at corners presents good scalability when continuous pressure test functions are used in both two and three dimensional stokes problems an improvement can be made for the case with continuous pressure test functions by applying the suggested algorithm to the interface problem which is obtained by eliminating velocity unknowns and pressure unknowns interior to each subdomains

the crystallizing substochastic sequential machine extractor cryssmex this letter presents an algorithm cryssmex for extracting minimal finite state machine descriptions of dynamic systems such as recurrent neural networks unlike previous algorithms cryssmex is parameter free and deterministic and it efficiently generates a series of increasingly refined models a novel finite stochastic model of dynamic systems and a novel vector quantization function have been developed to take into account the state space dynamics of the system the experiments show that 1 extraction from systems that can be described as regular grammars is trivial 2 extraction from high dimensional systems is feasible and 3 extraction of approximative models from chaotic systems is possible the results are promising and an analysis of shortcomings suggests some possible further improvements some largely overlooked connections of the field of rule extraction from recurrent neural networks to other fields are also identified

absolute stability conditions for discrete time recurrent neural networks an analysis of the absolute stability for a general class of discrete time recurrent neural networks rnn s is presented a discrete time model of rnn s is represented by a set of nonlinear difference equations some sufficient conditions for the absolute stability are derived using ostrowski s theorem and the similarity transformation approach for a given rnn model these conditions are determined by the synaptic weight matrix of the network the results reported in this paper need fewer constraints on the weight matrix and the model than in previously published studies

inferring global perceptual contours from local features we address the problem of contour inference from partial data as obtained from state of the art edge detectors we argue that in order to obtain more pereeptually salient contours it is necessary to impose generic constraints such as continuity and co curvilinearity the implementation is in the form of a convolution with a mask which encodes both the orientation and the strength of the possible continuations we first show how the mask called the extension field is derived then how the contributions from different sites are collected to produce a saliency map we show that the scheme can handle a variety of input data from dot patterns to oriented edgels in a unified manner and demonstrate results on a variety of input stimuli we also present a similar approach to the problem of inferring contours formed by end points in both cases the scheme is non linear non iterative and unified in the sense that all types of input tokens are handled in the same manner

comments on constructive learning of recurrent neural networks limitations of recurrent cascade correlation and a simple solution giles et al 1995 have proven that fahlman s recurrent cascade correlation rcc architecture is not capable of realizing finite state automata that have state cycles of length more than two under a constant input signal this paper extends the conclusions of giles et al by showing that there exists a corollary to their original proof which identifies a large second class of automata that is also unrepresentable by rcc

a neumann dirichlet preconditioner for a feti dp formulation of the two dimensional stokes problem with mortar methods a feti dp dual primal finite element tearing and interconnecting formulation for the two dimensional stokes problem with mortar methods is considered separate sets of unknowns are used for velocity on interfaces and the mortar constraints are enforced on the velocity unknowns by lagrange multipliers average constraints on edges are further introduced as primal constraints to solve the stokes problem correctly and to obtain a scalable feti dp algorithm a neumann dirichlet preconditioner is shown to give a condition number bound c maxi 1 n 1 log hi hi 2 where hi and hi are the subdomain size and the mesh size respectively and the constant c is independent of the mesh parameters hi and hi

constructive learning of recurrent neural networks limitations of recurrent cascade correlation and a simple solution it is often difficult to predict the optimal neural network size for a particular application constructive or destructive methods that add or subtract neurons layers connections etc might offer a solution to this problem we prove that one method recurrent cascade correlation due to its topology has fundamental limitations in representation and thus in its learning capabilities it cannot represent with monotone i e sigmoid and hard threshold activation functions certain finite state automata we give a preliminary approach on how to get around these limitations by devising a simple constructive training method that adds neurons during training while still preserving the powerful fully recurrent structure we illustrate this approach by simulations which learn many examples of regular grammars that the recurrent cascade correlation method is unable to learn

a feti dp formulation for the stokes problem without primal pressure components a scalable feti dp dual primal finite element tearing and interconnecting algorithm for the stokes problem that employs a lumped preconditioner is developed and analyzed a pair of inf sup stable velocity and pressure finite element spaces is used to obtain a discrete problem differently from previous approaches no primal pressure unknowns are selected and only velocity primal unknowns at subdomain corners are selected this leads to a symmetric and positive definite coarse problem matrix in the feti dp operator while a larger and indefinite coarse problem appears in the previous approaches in addition its condition number bound is proved to be the same as the feti dp algorithm with a lumped preconditioner for elliptic problems numerical results are included

an overlapping schwarz algorithm for almost incompressible elasticity overlapping schwarz methods are extended to mixed finite element approximations of linear elasticity which use discontinuous pressure spaces the coarse component of the preconditioner is based on a low dimensional space previously developed for scalar elliptic problems and a domain decomposition method of iterative substructuring type i e a method based on nonoverlapping decompositions of the domain while the local components of the preconditioner are based on solvers on a set of overlapping subdomains a bound is established for the condition number of the algorithm which grows in proportion to the logarithm of the number of degrees of freedom in individual subdomains and essentially to the third power of the relative overlap between the overlapping subdomains and which is independent of the poisson ratio as well as jumps in the lame parameters across the interface between the subdomains a positive definite reformulation of the discrete problem makes the use of the standard preconditioned conjugate gradient method straightforward numerical results which include a comparison with problems of compressible elasticity illustrate the findings

stability analysis of linear systems under state and rate saturations in this paper we provide a stability analysis for linear systems which have simultaneous saturations in the states and their rates dynamics the stability analysis is conducted via a formulation based on the polytopic representations of two saturation functions state and rate saturation functions sufficient conditions to guarantee local and global asymptotical stability of such systems are derived furthermore using the invariance analysis we estimate the domain of attraction of the system when it is locally asymptotically stable the stability conditions are formulated as an iterative lmi ilmi algorithm finally numerical examples are presented to validate the proposed method

parameter space structure of continuous time recurrent neural networks a fundamental challenge for any general theory of neural circuits is how to characterize the structure of the space of all possible circuits over a given model neuron as a first step in this direction this letter begins a systematic study of the global parameter space structure of continuous time recurrent neural networks ctrnns a class of neural models that is simple but dynamically universal first we explicitly compute the local bifurcation manifolds of ctrnns we then visualize the structure of these manifolds in net input space for small circuits these visualizations reveal a set of extremal saddle node bifurcation manifolds that divide ctrnn parameter space into regions of dynamics with different effective dimensionality next we completely characterize the combinatorics and geometry of an asymptotically exact approximation to these regions for circuits of arbitrary size finally we show how these regions can be used to calculate estimates of the probability of encountering different kinds of dynamics in ctrnn parameter space

on the synthesis of brain state in a box neural models with application to associative memory in this article we present techniques for designing associative memories to be implemented by a class of synchronous discrete time neural networks based on a generalization of the brain state in a box neural model first we address the local qualitative properties and global qualitative aspects of the class of neural networks considered our approach to the stability analysis of the equilibrium points of the network gives insight into the extent of the domain of attraction for the patterns to be stored as asymptotically stable equilibrium points and is useful in the analysis of the retrieval performance of the network and also for design purposes by making use of the analysis results as constraints the design for associative memory is performed by solving a constraint optimization problem whereby each of the stored patterns is guaranteed a substantial domain of attraction the performance of the designed network is illustrated by means of three specific examples

feti dp bddc and block cholesky methods the feti dp and bddc algorithms are reformulated using block cholesky factorizations an approach which can provide a useful framework for the design of domain decomposition algorithms for solving symmetric positive definite linear system of equations instead of introducing lagrange multipliers to enforce the coarse level primal continuity constraints in these algorithms a change of variables is used such that each primal constraint corresponds to an explicit degree of freedom with the new formulation of these algorithms a simplified proof is provided that the spectra of a pair of feti dp and bddc algorithms with the same set of primal constraints are essentially the same numerical experiments for a two dimensional laplace s equation also confirm this result copyright 2005 john wiley sons ltd