finite state automata and simple recurrent networks we explore a network architecture introduced by elman 1988 for predicting successive elements of a sequence the network uses the pattern of activation over a set of hidden units from time step t1 together with element t to predict element t 1 when the network is trained with strings from a particular finite state grammar it can learn to be a perfect finite state recognizer for the grammar when the network has a minimal number of hidden units patterns on the hidden units come to correspond to the nodes of the grammar although this correspondence is not necessary for the network to act as a perfect finite state recognizer we explore the conditions under which the network can carry information about distant sequential contingencies across intervening elements such information is maintained with relative ease if it is relevant at each intermediate step it tends to be lost when intervening elements do not depend on it at first glance this may suggest that such networks are not relevant to natural language in which dependencies may span indefinite distances however embeddings in natural language are not completely independent of earlier information the final simulation shows that long distance sequential contingencies can be encoded by the network even if only subtle statistical properties of embedded strings depend on the early information

control of stochastic discrete event systems modeled by probabilistic languages in garg et al 1999 and garg 1992 the formalism of probabilistic languages for modeling the stochastic qualitative behavior of discrete event systems dess was introduced in this paper we study their supervisory control where the control is exercised by dynamically disabling certain controllable events thereby nulling the occurrence probabilities of disabled events and increasing the occurrence probabilities of enabled events proportionately this is a special case of probabilistic supervision introduced in lawford and wonham 1993 the control objective is to design a supervisor such that the controlled system never executes any illegal traces their occurrence probability is zero and legal traces occur with minimum prespecified occurrence probabilities in other words the probabilistic language of the controlled system lies within a prespecified range where the upper bound is a nonprobabilistic language representing a legality constraint we provide a condition for the existence of a supervisor we also present an algorithm to test this existence condition when the probabilistic languages are regular so that they admit probabilistic automata representation with finitely many states next we give a technique to compute a maximally permissive supervisor online

upward planar drawing of single source acyclicdigraphs an upward plane drawing of a directed acyclic graph is a plane drawing of the digraph in which each directed edge is represented as a curve monotone increasing in the vertical direction thomassen has given a nonalgorithmic graph theoretic characterization of those directed graphs with a single source that admit an upward plane drawing this paper presents an efficient algorithm to test whether a given single source acyclic digraph has an upward plane drawing and if so to find a representation of one such drawing this result is made more significant in light of the recent proof by garg and tamassia that the problem is np complete for general digraphs the algorithm decomposes the digraph into biconnected and triconnected components and defines conditions for merging the components into an upward plane drawing of the original digraph to handle the triconnected components we provide a linear algorithm to test whether a given plane drawing of a single source digraph admits an upward plane drawing with the same faces and outer face which also gives a simpler algorithmic proof of thomassen s result the entire testing algorithm for general single source directed acyclic graphs operates in o n 2 time and o n space n being the number of vertices in the input digraph and represents the first polynomial time solution to the problem

a machine learning method for extracting symbolic knowledge from recurrent neural networks neural networks do not readily provide an explanation of the knowledge stored in their weights as part of their information processing until recently neural networks were considered to be black boxes with the knowledge stored in their weights not readily accessible since then research has resulted in a number of algorithms for extracting knowledge in symbolic form from trained neural networks this article addresses the extraction of knowledge in symbolic form from recurrent neural networks trained to behave like deterministic finite state automata dfas to date methods used to extract knowledge from such networks have relied on the hypothesis that networks states tend to cluster and that clusters of network states correspond to dfa states the computational complexity of such a cluster analysis has led to heuristics that either limit the number of clusters that may form during training or limit the exploration of the space of hidden recurrent state neurons these limitations while necessary may lead to decreased fidelity in which the extracted knowledge may not model the true behavior of a trained network perhaps not even for the training set the method proposed here uses a polynomial time symbolic learning algorithm to infer dfas solely from the observation of a trained network s input output behavior thus this method has the potential to increase the fidelity of the extracted knowledge

region oriented compression of color images using fuzzy inference and fast merging a new region based lossy compression scheme for color images is proposed the segmentation method belongs to the split and merge category splitting is carried out using the watershed transform in the merging stage a fuzzy color preserving rule based system and a novel one dimensional graph structure are introduced to provide accurate results with reduced computational complexity the compression part is based on the shape adaptive dct with dc correction method the quantization matrices used have been designed according to the properties of the employed transform promising perceptual results for the low bit rate range compared to previously reported compression methods have also been reported

model uncertainty in discrete event systems earlier work concerning control of discrete event systems usually assumed that a correct model of the system to be controlled was available a goal of this work is to provide an algorithm for determining the correct model from a set of models the result of the algorithm is a finite language that can be used to test for the correct model or notification that the remaining models cannot be controllably distinguished we use the finite state machine model with controllable and uncontrollable events presented by ramadge and wonham

extraction of tumor region in color images using wavelets abstract in this paper a wavelet based method is proposed to extract the tumor region from a color medical image in the past the diagnosis of tumors usually relied upon the tissue section stained by methylene blue however traditional approaches have relied primarily on manual processes and erroneous assessment may arise from subjective judgement and eye measurement instead of the previous method we propose an improved radial search technique to increase the accuracy of the extraction of the tumor border experimental results show that the proposed method is fast and can extract the tumor region successfully moreover the results of this research will be helpful in the development of a computer aided image analysis system that will automate visual feature extraction for tumor evaluation

boundary detection by constrained optimization a statistical framework is used for finding boundaries and for partitioning scenes into homogeneous regions the model is a joint probability distribution for the array of pixel gray levels and an array of labels in boundary finding the labels are binary zero or one representing the absence or presence of boundary elements in partitioning the label values are generic two labels are the same when the corresponding scene locations are considered to belong to the same region the distribution incorporates a measure of disparity between certain spatial features of block pairs of pixel gray levels using the kolmogorov smirnov nonparametric measures of difference between the distributions of these features the number of model parameters is minimized by forbidding label configurations which are assigned probability zero the maximum a posteriori estimator of boundary placements and partitionings is examined the forbidden states introduce constraints into the calculation of these configurations stochastic relaxation methods are extended to accommodate constrained optimization

color image segmentation using adaptive mean shift and statistical model based methods in this paper we propose an unsupervised segmentation algorithm for color images based on gaussian mixture models gmms the number of mixture components is determined automatically by adaptive mean shift in which local clusters are estimated by repeatedly searching for higher density points in feature vector space for the estimation of parameters of gmms the mean field annealing expectation maximization em is employed the mean field annealing em provides a global optimal solution to overcome the local maxima problem in a mixture model by combining the adaptive mean shift and the mean field annealing em natural color images are segmented automatically without over segmentation or isolated regions the experiments show that the proposed algorithm can produce satisfactory segmentation without any a priori information

multiscale roughness measure for color image segmentation color image segmentation is always an important technique in image processing system highly precise segmentation with low computation complexity can be achieved through roughness measurement which approximate the color histogram based on rough set theory however due to the imprecise description of neighborhood similarity the existing roughness measure tends to over focus on the trivial homogeneous regions but is not accurate enough to measure the color homogeneity this paper aims to construct a multiscale roughness measure through simulating the human vision we apply the theories of linear scale space and rough sets to generate the hierarchical roughness of color distribution under multiple scales this multiscale roughness can tolerate the disturbance of trivial regions and also can provide the multilevel homogeneity representation in vision which therefore produces precise and intuitive segmentation results furthermore we propose roughness entropy for scale selection the optimal scale for segmentation is decided by the entropy variation the proposed method shows the encouraging performance in the experiments based on berkeley segmentation database

the effect of number of hamiltonian paths on the complexity of a vertex coloring problem a generalization of dobkin and lipton s element uniqueness problem is introduced for any fixed undirected graph g on vertex set v 1 v 2 cdots v n the problem is to determine given n real numbers x 1 x 2 cdots x n whether x i ne x j for every edge upsilon i upsilon j in g this problem is shown to have upper and lower bounds of theta n log n linear comparisons if g is any dense graph the proof of the lower bound involves showing that any dense graph must contain a subgraph with many hamiltonian paths and demonstrating the relevance of these hamiltonian paths to a geometric argument in addition we exhibit relatively sparse graphs for which the same lower bound holds and relatively dense graphs for which a linear upper bound holds

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

quantization based clustering algorithm in this paper a quantization based clustering algorithm qbca is proposed to cluster a large number of data points efficiently unlike previous clustering algorithms qbca places more emphasis on the computation time of the algorithm specifically qbca first assigns the data points to a set of histogram bins by a quantization function then it determines the initial centers of the clusters according to this point distribution finally qbca performs clustering at the histogram bin level rather than the data point level we also propose two approaches to improve the performance of qbca further i a shrinking process is performed on the histogram bins to reduce the number of distance computations and ii a hierarchical structure is constructed to perform efficient indexing on the histogram bins finally we analyze the performance of qbca theoretically and experimentally and show that the approach 1 can be easily implemented 2 identifies the clusters effectively and 3 outperforms most of the current state of the art clustering approaches in terms of efficiency

a fuzzy region growing approach for segmentation of color images segmentation is one of the most important preprocessing steps towards pattern recognition and image understanding and a significant step towards image compression and coding with detecting edges most of the large segments can be found and separated from others by edge pixels it is however the pixels on edge locations or those in high detailed areas whose association to adjacent segments must be found a pixel can be a part of the closest segment or in association with the neighboring pixels from a new smaller segment in this paper two segmentation algorithms are presented one is used for fine segmentation towards compression and coding of images and the other for coarse segmentation towards other applications like object recognition and image understanding edge detection and region growing approaches are combined to find large and crisp segments for coarse segmentation segments can grow or expand based on two fuzzy criteria the fuzzy region growing and expanding approaches presented here use histogram tables for fine segmentation the procedures introduced here can be used in any order or combination to yield the best result for any particular application or image type

color image segmentation advances and prospects abstract image segmentation is very essential and critical to image processing and pattern recognition this survey provides a summary of color image segmentation techniques available now basically color segmentation approaches are based on monochrome segmentation approaches operating in different color spaces therefore we first discuss the major segmentation approaches for segmenting monochrome images histogram thresholding characteristic feature clustering edge detection region based methods fuzzy techniques neural networks etc then review some major color representation methods and their advantages disadvantages finally summarize the color image segmentation techniques using different color representations the usage of color models for image segmentation is also discussed some novel approaches such as fuzzy method and physics based method are investigated as well

the calculi of emergence computation dynamics and induction abstract defining structure and detecting the emergence of complexity in nature are inherently subjective though essential scientific activities despite the difficulties these problems can be analyzed in terms of how model building observers infer from measurements the computational capabilities embedded in nonlinear processes an observer s notion of what is ordered what is random and what is complex in its environment depends directly on its computational resources the amount of raw measurement data of memory and of time available for estimation and inference the discovery of structure in an environment depends more critically and subtlely though on how those resources are organized the descriptive power of the observer s chosen or implicit computational model class for example can be an overwhelming determinant in finding regularity in data this paper presents an overview of an inductive framework hierarchical machine reconstruction in which the emergence of complexity is associated with the innovation of new computational model classes complexity metrics for detecting structure and quantifying emergence along with an analysis of the constraints on the dynamics of innovation are outlined illustrative examples are drawn from the onset of unpredictability in nonlinear systems finitary nondeterministic processes and cellular automata pattern recognition they demonstrate how finite inference resources drive the innovation of new structures and so lead to the emergence of complexity

on the statistical analysis of dirty pictures may 7th 1986 professor a f m smith in the chair summary a continuous two dimensional region is partitioned into a fine rectangular array of sites or pixels each pixel having a particular colour belonging to a prescribed finite set the true colouring of the region is unknown but associated with each pixel there is a possibly multivariate record which conveys imperfect information about its colour according to a known statistical model the aim is to reconstruct the true scene with the additional knowledge that pixels close together tend to have the same or similar colours in this paper it is assumed that the local characteristics of the true scene can be represented by a nondegenerate markov random field such information can be combined with the records by bayes theorem and the true scene can be estimated according to standard criteria however the computational burden is enormous and the reconstruction may reflect undesirable largescale properties of the random field thus a simple iterative method of reconstruction is proposed which does not depend on these large scale characteristics the method is illustrated by computer simulations in which the original scene is not directly related to the assumed random field some complications including parameter estimation are discussed potential applications are mentioned briefly

extracting finite state representations from recurrent neural networks trained on chaotic symbolic sequences concerns neural based modeling of symbolic chaotic time series we investigate the knowledge induction process associated with training recurrent mural nets rnn on single long chaotic symbolic sequences even though training rnn to predict the next symbol leaves the standard performance measures such as the mean square error on the network output virtually unchanged the nets extract a lot of knowledge we monitor the knowledge extraction process by considering the nets stochastic sources and letting them generate sequences which are then confronted with the training sequence via information theoretic entropy and cross entropy measures we also study the possibility of reformulating the knowledge gained by rnn in a compact easy to analyze form of finite state stochastic machines the experiments are performed on two sequences with different complexities measured by the size and state transition structure of the induced crutchfield s spl epsiv machines 1991 1994 the extracted machines can achieve comparable or even better entropy and cross entropy performance they reflect the training sequence complexity in their dynamical state representations that can be reformulated using finite state means the findings are confirmed by a much more detailed analysis of model generated sequences we also introduce a visual representation of allowed block structure in the studied sequences that allows for an illustrative insight into both rnn training and finite state stochastic machine extraction processes

efficient multilevel image segmentation through fuzzy entropy maximization and graph cut optimization the fuzzy c partition entropy has been widely adopted as a global optimization technique for finding the optimized thresholds for multilevel image segmentation however it involves expensive computation as the number of thresholds increases and often yields noisy segmentation results since spatial coherence is not enforced in this paper an iterative calculation scheme is presented for reducing redundant computations in entropy evaluation the efficiency of threshold selection is further improved through utilizing the artificial bee colony algorithm as the optimization technique finally instead of performing thresholding for each pixel independently the presented algorithm oversegments the input image into small regions and uses the probabilities of fuzzy events to define the costs of different label assignments for each region the final segmentation results is computed using graph cut which produces smooth segmentation results the experimental results demonstrate the presented iterative calculation scheme can greatly reduce the running time and keep it stable as the number of required thresholds increases quantitative evaluations over 20 classic images also show that the presented algorithm outperforms existing multilevel segmentation approaches

multiresolution color image segmentation image segmentation is the process by which an original image is partitioned into some homogeneous regions in this paper a novel multiresolution color image segmentation mcis algorithm which uses markov random fields mrf s is proposed the proposed approach is a relaxation process that converges to the map maximum a posteriori estimate of the segmentation the quadtree structure is used to implement the multiresolution framework and the simulated annealing technique is employed to control the splitting and merging of nodes so as to minimize an energy function and therefore maximize the map estimate the multiresolution scheme enables the use of different dissimilarity measures at different resolution levels consequently the proposed algorithm is noise resistant since the global clustering information of the image is required in the proposed approach the scale space filter ssf is employed as the first step the multiresolution approach is used to refine the segmentation experimental results of both the synthesized and real images are very encouraging in order to evaluate experimental results of both synthesized images and real images quantitatively a new evaluation criterion is proposed and developed

on the number of upward planar orientations of maximal planar graphs we consider the problem of determining the maximum and the minimum number of upward planar orientations a maximal planar graph can have we show that every n vertex maximal planar graph has at least w 1 189 n and at most o 4 n upward planar orientations moreover we show that there exist n vertex maximal planar graphs having o 2 n upward planar orientations and n vertex maximal planar graphs having w 2 599 n upward planar orientations further we present bounds for the maximum and the minimum number of acyclic orientations a maximal planar graph can have

recurrent neural networks with small weights implement definite memory machines recent experimental studies indicate that recurrent neural networks initialized with small weights are inherently biased toward definite memory machines tino cernansky benuskova 2002a 2002b this article establishes a theoretical counterpart transition function of recurrent network with small weights and squashing activation function is a contraction we prove that recurrent networks with contractive transition function can be approximated arbitrarily well on input sequences of unbounded length by a definite memory machine conversely every definite memory machine can be simulated by a recurrent network with contractive transition function hence initialization with small weights induces an architectural bias into learning with recurrent neural networks this bias might have benefits from the point of view of statistical learning theory it emphasizes one possible region of the weight space where generalization ability can be formally proved it is well known that standard recurrent neural networks are not distribution independent learnable in the probably approximately correct pac sense if arbitrary precision and inputs are considered we prove that recurrent networks with contractive transition function with a fixed contraction parameter fulfill the so called distribution independent uniform convergence of empirical distances property and hence unlike general recurrent networks are distribution independent pac learnable

further improvements in feature weighted fuzzy c means in cluster analysis certain features of a given data set may exhibit higher relevance than others to address this issue feature weighted fuzzy c means fwfcm approaches have emerged in recent years however there are certain deficiencies in the existing fwfcms e g the elements in a feature weight vector cannot be adaptively adjusted during the training phase and the update formulas of a feature weight vector cannot be derived analytically in this study an improved fwfcm ifwfcm is proposed to overcome these shortcomings the ifwfcm kd based on the kernelized distance is also proposed experimental results reported for five numerical data sets and the color images show that ifwfcm is superior to the existing fwfcms an interesting conclusion that ifwfcm kd might not improve the performance of ifwfcm is also obtained by applying ifwfcm kd to tackle the above mentioned numerical data sets and color images

the crystallizing substochastic sequential machine extractor cryssmex this letter presents an algorithm cryssmex for extracting minimal finite state machine descriptions of dynamic systems such as recurrent neural networks unlike previous algorithms cryssmex is parameter free and deterministic and it efficiently generates a series of increasingly refined models a novel finite stochastic model of dynamic systems and a novel vector quantization function have been developed to take into account the state space dynamics of the system the experiments show that 1 extraction from systems that can be described as regular grammars is trivial 2 extraction from high dimensional systems is feasible and 3 extraction of approximative models from chaotic systems is possible the results are promising and an analysis of shortcomings suggests some possible further improvements some largely overlooked connections of the field of rule extraction from recurrent neural networks to other fields are also identified

an adaptive unsupervised approach toward pixel clustering and color image segmentation this paper proposes an adaptive unsupervised scheme that could find diverse applications in pattern recognition as well as in computer vision particularly in color image segmentation the algorithm named ant colony fuzzy c means hybrid algorithm afha adaptively clusters image pixels viewed as three dimensional data pieces in the rgb color space the ant system as algorithm is applied for intelligent initialization of cluster centroids which endows clustering with adaptivity considering algorithmic efficiency an ant subsampling step is performed to reduce computational complexity while keeping the clustering performance close to original one experimental results have demonstrated afha clustering s advantage of smaller distortion and more balanced cluster centroid distribution over fcm with random and uniform initialization quantitative comparisons with the x means algorithm also show that afha makes a better pre segmentation scheme over x means we further extend its application to natural image segmentation taking into account the spatial information and conducting merging steps in the image space extensive tests were taken to examine the performance of the proposed scheme results indicate that compared with classical segmentation algorithms such as mean shift and normalized cut our method could generate reasonably good or better image partitioning which illustrates the method s practical value

on the computational complexity of upward and rectilinear planarity testing a directed graph is upward planar if it can be drawn in the plane such that every edge is a monotonically increasing curve in the vertical direction and no two edges cross an undirected graph is rectilinear planar if it can be drawn in the plane such that every edge is a horizontal or vertical segment and no two edges cross testing upward planarity and rectilinear planarity are fundamental problems in the effective visualization of various graph and network structures for example upward planarity is useful for the display of order diagrams and subroutine call graphs while rectilinear planarity is useful for the display of circuit schematics and entity relationship diagrams we show that upward planarity testing and rectilinear planarity testing are np complete problems we also show that it is np hard to approximate the minimum number of bends in a planar orthogonal drawing of an n vertex graph with an o n 1 epsilon error for any epsilon 0

learning regular sets from queries and counterexamples the problem of identifying an unknown regular set from examples of its members and nonmembers is addressed it is assumed that the regular set is presented by a minimamy adequate teacher which can answer membership queries about the set and can also test a conjecture and indicate whether it is equal to the unknown set and provide a counterexample if not a counterexample is a string in the symmetric difference of the correct set and the conjectured set a learning algorithm l is described that correctly learns any regular set from any minimally adequate teacher in time polynomial in the number of states of the minimum dfa for the set and the maximum length of any counterexample provided by the teacher it is shown that in a stochastic setting the ability of the teacher to test conjectures may be replaced by a random sampling oracle ex a polynomial time learning algorithm is shown for a particular problem of context free language identification

on the color image segmentation algorithm based on the thresholding and the fuzzy c means techniques abstract in this paper a segmentation algorithm for color images based on the thresholding and the fuzzy c means fcm techniques is presented the scale space filter is used as a tool for analyzing the histograms of three color components the methodology uses a coarse fine concept to reduce the computational burden required for the fcm the coarse segmentation attempts to segment coarsely using the thresholding technique while the fine segmentation assigns the pixels which remain unclassified after the coarse segmentation to the closest class using the fcm attempts also have been made to compare the performance of the proposed algorithm with other existing algorithms ohlander s rosenfeld s and bezdek s intensive computer simulation has been performed and the results are discussed in this paper the simulation results indicate that the proposed algorithm yields the most accurate segmented image on the color coordinate proposed by ohta et al while requiring a reasonable amount of computational effort

color images segmentation using scale space filter and markov random field a new hybrid method is presented that combines the scale space filter ssf and markov random field mrf for color image segmentation the fundamental idea of the ssf is to use the convolution of gaussian functions and image histogram to generate a scale space image and then find the proper interval bounded by the local extrema of the derivatives the gaussian function is with zero mean and varied standard deviation using the ssf the different scaled histogram is separated into intervals corresponding to peaks and valleys the mrf makes use of the property that each pixel in an image has some relationship with other pixels the basic construction of an mrf is a joint probability given the original data the original data is the image that is obtained from the source and the result is called the label image because the mrf needs a number of segments before it converges to the global minimum the ssf is exploited to do coarse segmentation cs and then mrf is used to do fine segmentation fs of the images basically the former is histogram based segmentation whereas the latter is neighborhood based segmentation finally experimental results obtained from using ssf alone mrf using iterated conditional mode icm and mrf using gibbs sampling are compared

algorithms for plane representations of acyclic digraphs abstract acyclic digraphs are widely used for representing hierarchical structures examples include pert networks subroutine call graphs family trees organization charts hasse diagrams and isa hierarchies in knowledge representation diagrams we investigate the problem of representing acyclic digraphs in the plane in such a way that all edges flow in the same direction e g from the left to the right or from the bottom to the top three plane representations are considered straight drawings visibility representations and grid drawings we provide efficient algorithms that construct these representations with all edges flowing in the same direction the time complexity is o n for visibility representations and grid drawings and o n log n for straight drawings where n is the number of vertices of the digraph for covering digraphs of lattices the complexity of constructing straight drawings is o n we also show that the planar digraphs that admit any one of these representations are exactly the subgraphs of planar st graphs

relaxation using models from quantum mechanics abstract we make an analogy between a single pixel and its first order neighbours in an image and single particle models from quantum physics the analogy is created by identifying the observed grey levels and the noise variance with the potential function and the energy of the physical system the approach is used to set up novel and strongly data dependent local weights in relaxation schemes both image restoration and segmentation are considered the new methods competed favourably with the besag icm and rosenfeld relaxation on several simulated and real images