Anticipation Model for Sequential Learning of Complex Sequences
One of the fundamental aspects of human intelligence is the ability to process temporal information (Lashley, 1951). Learning and reproducing temporal sequences are closely associated with our ability to perceive and generate body movements, speech and language, music, etc. A considerable body of neural network literature is devoted to temporal pattern generation (see Wang, 2001, for a recent review). These models generally treat a temporal pattern as a sequence of discrete patterns, called a temporal sequence. Most of the models are based on either multilayer perceptrons with backpropagation training or the Hopfield model of associative recall. The basic idea for the former class of models is to view a temporal sequence as a set of associations between consecutive components, and learn these associations as input-output transformations (Jordan, 1986; Elman, 1990; Mozer, 1993). To deal with temporal dependencies beyond consecutive components, part of the input layer is used to keep a trace of history, behaving as short-term memory (STM). Similarly, for temporal recall based on the Hopfield associative memory, a temporal sequence is viewed as associations between consecutive components. These associations are stored in extended versions of the Hopfield model that includes some time delays (Sompolinsky & Kanter, 1986; Buhmann & Schulten, 1987; Heskes & Gielen, 1992). To deal with longer temporal dependencies, high-order networks have been proposed (Guyon et al., 1988).

title: Image Segmentation Based on Oscillatory Correlation
abstract: We study image segmentation on the basis of locally excitatory, globally inhibitory oscillator networks (LEGION), whereby the phases of oscillators encode the binding of pixels. We introduce a lateral potential for each oscillator so that only oscillators with strong connections from their neighborhood can develop high potentials. Based on the concept of the lateral potential, a solution to remove noisy regions in an image is proposed for LEGION, so that it suppresses the oscillators corresponding to noisy regions but without affecting those corresponding to major regions. We show that the resulting oscillator network separates an image into several major regions, plus a background consisting of all noisy regions, and we illustrate network properties by computer simulation. The network exhibits a natural capacity in segmenting images. The oscillatory dynamics leads to a computer algorithm, which is applied successfully to segmenting real gray-level images. A number of issues regarding biological plausibility and perceptual organization are discussed. We argue that LEGION provides a novel and effective framework for image segmentation and figure-ground segregation.

title: A schema-based model for phonemic restoration
abstract: Phonemic restoration is the perceptual synthesis of phonemes when masked by appropriate replacement sounds by utilizing linguistic context. Current models attempting to accomplish acoustic restoration of phonemes, however, use only temporal continuity and produce poor restoration of unvoiced phonemes, and are also limited in their ability to restore voiced phonemes. We present a schema-based model for phonemic restoration. The model employs a missing data speech recognition system to decode speech based on intact portions and activates word templates corresponding to the words containing the masked phonemes. An activated template is dynamically time warped to the noisy word and is then used to restore the speech frames corresponding to the masked phoneme, thereby synthesizing it. The model is able to restore both voiced and unvoiced phonemes with a high degree of naturalness. Systematic testing shows that this model outperforms a Kalman-filter based model.

title: Intrinsic generalization analysis of low dimensional representations
abstract: Low dimensional representations of images impose equivalence relations in the image space; the induced equivalence class of an image is named as its intrinsic generalization. The intrinsic generalization of a representation provides a novel way to measure its generalization and leads to more fundamental insights than the commonly used recognition performance, which is heavily influenced by the choice of training and test data. We demonstrate the limitations of linear subspace representations by sampling their intrinsic generalization, and propose a nonlinear representation that overcomes these limitations. The proposed representation projects images nonlinearly into the marginal densities of their filter responses, followed by linear projections of the marginals. We use experiments on large datasets to show that the representations that have better intrinsic generalization also lead to better recognition performance.

title: Texture classification using spectral histograms
abstract: Based on a local spatial/frequency representation,we employ a spectral histogram as a feature statistic for texture classification. The spectral histogram consists of marginal distributions of responses of a bank of filters and encodes implicitly the local structure of images through the filtering stage and the global appearance through the histogram stage. The distance between two spectral histograms is measured using chi(2)-statistic. The spectral histogram with the associated distance measure exhibits several properties that are necessary for texture classification. A filter selection algorithm is proposed to maximize classification performance of a given dataset. Our classification experiments using natural texture images reveal that the spectral histogram representation provides a robust feature statistic for textures and generalizes well. Comparisons show that our method produces a marked improvement in classification performance. Finally we point out the relationships between existing texture features and the spectral histogram, suggesting that the latter may provide a unified texture feature.

title: On Connectedness: A Solution Based on Oscillatory Correlation
abstract: A long-standing problem in neural computation has been the problem of connectedness, first identified by Minsky and Papert (1969). This problem served as the cornerstone for them to establish analytically that perceptrons are fundamentally limited in computing geometrical (topological) properties. A solution to this problem is offered by a different class of neural networks: oscillator networks. To solve the problem, the representation of oscillatory correlation is employed, whereby one pattern is represented as a synchronized block of oscillators and different patterns are represented by distinct blocks that desynchronize from each other. Oscillatory correlation emerges from LEGION (locally excitatory globally inhibitory oscillator network), whose architecture consists of local excitation and global inhibition among neural oscillators. It is further shown that these oscillator networks exhibit sensitivity to topological structure, which may lay a neurocomputational foundation for explaining the psychophysical phenomenon of topological perception.

title: Protein Sequences Classification Using Modular RBF Neural Networks
abstract: A protein super-family consists of proteins which share amino acid sequence homology and which may therefore be functionally and structurally related. One of the benefits from this category grouping is that some hint of function may be deduced for individual members from information on other members of the family. Traditionally, two protein sequences are classified into the same class if they have high homology in terms of feature patterns extracted through sequence alignment algorithms. These algorithms compare an unseen protein sequence with all the identified protein sequences and returned the higher scored protein sequences. As the sizes of the protein sequence databases are very large, it is a very time consuming job to perform exhaustive comparison of existing protein sequence. Therefore, there is a need to build an improved classification system for effectively identifying protein sequences. This paper presents a modular neural classifier for protein sequences with improved classification criteria. The intelligent classification techniques described in this paper aims to enhance the performance of single neural classifiers based on a centralized information structure in terms of recognition rate, generalization and reliability. The architecture of the proposed model is a modular RBF neural network with a compensational combination at the transition output layer. The connection weights between the final output layer and the transition output layer are optimized by delta rule, which serve as an integrator of the local neural classifiers. To enhance the classification reliability, we present two heuristic rules to apply to decision-making. Two sets of protein sequences with ten classes of superfamilies downloaded from a public domain database, Protein Information Resources (PIR), are used in our simulation study. Experimental results with performance comparisons are carried out between single neural classifiers and the proposed modular neural classifier.

title: Automated Abstraction Refinement for Model Checking Large State Spaces Using SAT Based Conflict Analysis
abstract: We introduce a SAT based automatic abstraction refinement framework for model checking systems with several thousand state variables in the cone of influence of the specification. The abstract model is constructed by designating a large number of state variables as invisible. In contrast to previous work where invisible variables were treated as free inputs we describe a computationally more advantageous approach in which the abstract transition relation is approximated by pre-quantifying invisible variables during image computation. The abstract counterexamples obtained from model-checking the abstract model are symbolically simulated on the concrete system using a state-of-the-art SAT checker. If no concrete counterexample is found, a subset of the invisible variables is reintroduced into the system and the process is repeated. The main contribution of this paper are two new algorithms for identifying the relevant variables to be reintroduced. These algorithms monitor the SAT checking phase in order to analyze the impact of individual variables. Our method is complete for safety properties (AGp) in the sense that - performance permitting - a property is either verified or disproved by a concrete counterexample. Experimental results are given to demonstrate the power of our method on real-world designs.

title: Theoretical Foundation for Nonlinear Edge-Preserving Regularized Learning Image Restoration
abstract: Image restoration is an important issue in image processing, which helps recovering of degraded images caused by various factors in different circumstances. Neural networks models have been successfully applied in handling image restoration problems and some progress and promising results have been reported in the past. A popular neural networks model for image restoration is the Hopfield network due to its ability on dealing with optimization problems. A degraded image may have multiple corresponding solutions, i.e., the restored images, and the obtained solution can be sub-optimal which is related to a local minimum point in the weight space of the Hopfield network. This paper gives an algebraic characterization on images and shows that images with lower complexity can be restored uniquely from their degraded images and edge information. The obtained result in this paper establishes a mathematical basis for employing the mapping neural networks to realize a learning based image restoration scheme. The linear image restoration model is firstly generalized to a nonlinear one to broaden the scope of application. Secondly, we view the image restoration task as a set of approximation problems in a high dimensional space, and a mapping relationship between the degraded images with edge information and the source images is then built using feed-forward neural networks and the well trained neural network can be used to restore the degraded images in realtime. Computer simulations demonstrate the effectiveness of the learning image restoration techniques proposed in this paper.

title: On avoidable and unavoidable claws
abstract: A claw of degree k is a directed tree consisting of k paths emerging from a common root. We prove that every claw of order n with degree less than 19 50 n appears in every n -vertex tournament. We also construct avoidable claws with degree approaching 11 23 n . Thus for large n , the maximum λ such that every claw with degree λn appears in every n -vertex tournament satisfies λ ⩽ 11 23 . This improves earlier bounds.

title: Computer Algebra and Neurodynamics
abstract: We review the capabilities of computer algebra and the characteristics of neurodynamics. Possible applications of computer algebra to the study of neurodynamics are then discussed. We have investigated some of the applications, in particular, to the computer aided analysis and derivation of neurodynamics by using and developing tools in computer algebra systems. Part of our own work in this direction is summarized.