bluuebunny/crossref_metadata_2025 huggingface dataset:
Letter to the Editor: A DISCRETE-TIME LAGRANGIAN NETWORK FOR SOLVING CONSTRAINED QUADRATIC PROGRAMS
Authors: TANG WAI SUM , WANG JUN
Published: April 2003

A discrete-time recurrent neural network which is called the discrete-time Lagrangian network is proposed in this letter for solving convex quadratic programs. It is developed based on the classical Lagrange optimization method and solves quadratic programs without using any penalty parameter. The condition for the neural network to globally converge to the optimal solution of the quadratic program is given. Simulation results are presented to illustrate its performance.
INSTABILITIES AND OSCILLATION IN THE DETERMINISTIC BOLTZMANN MACHINE
Authors: SCHNEIDER ROLAND , CARD HOWARD C.
Published: April 2003

Simulations indicate that the deterministic Boltzmann machine, unlike the stochastic Boltzmann machine from which it is derived, exhibits unstable behavior during contrastive Hebbian learning of nonlinear problems, including oscillation in the learning algorithm and extreme sensitivity to small weight perturbations. Although careful choice of the initial weight magnitudes, the learning rate, and the annealing schedule will produce convergence in most cases, the stability of the resulting solution depends on the parameters in a complex and generally indiscernible way. We show that this unstable behavior is the result of over parameterization (excessive freedom in the weights), which leads to continuous rather than isolated optimal weight solution sets. This allows the weights to drift without correction by the learning algorithm until the free energy landscape changes in such a way that the settling procedure employed finds a different minimum of the free energy function than it did previously and a gross output error occurs. Because all the weight sets in a continuous optimal solution set produce exactly the same network outputs, we define reliability, a measure of the robustness of the network, as a new performance criterion.
A GENETIC ALGORITHM WITH ADAPTIVE MUTATIONS AND FAMILY COMPETITION FOR TRAINING NEURAL NETWORKS
Authors: YANG JINN-MOON , HORNG JORNG-TZONG , KAO CHENG-YAN
Published: April 2003

In this paper, we present a new evolutionary technique to train three general neural networks. Based on family competition principles and adaptive rules, the proposed approach integrates decreasing-based mutations and self-adaptive mutations to collaborate with each other. Different mutations act as global and local strategies respectively to balance the trade-off between solution quality and convergence speed. Our algorithm is then applied to three different task domains: Boolean functions, regular language recognition, and artificial ant problems. Experimental results indicate that the proposed algorithm is very competitive with comparable evolutionary algorithms. We also discuss the search power of our proposed approach.
FEATURES SELECTION AND ARCHITECTURE OPTIMIZATION IN CONNECTIONIST SYSTEMS
Authors: YACOUB MÉZIANE , BENNANI YOUNÈS
Published: April 2003

In this paper, we propose a features selection measure and an architecture optimization procedure for Multi-Layer Perceptrons (MLP). The algorithm presented in this contribution employs a heuristic measure named HVS (Heuristic for Variable Selection). This new measure allows us to identify and select important variables in the features space. This can be achieved by eliminating redundant features and those which do not contain enough relevant information. The proposed measure is used in a new procedure aimed at selecting the "best" MLP architecture given an initial structure. Application results for two generic problems: regression and discrimination, demonstrates the proposed selection algorithm's effectiveness in identifying optimized connectionist models with higher accuracy. Finally, an extension of HVS, named εHVS, is proposed for discriminative features detection and architecture optimization for Time Delay Neural Networks models (TDNN).
KERNEL AND NONLINEAR CANONICAL CORRELATION ANALYSIS
Authors: LAI P. L. , FYFE C.
Published: April 2003

We review a neural implementation of the statistical technique of Canonical Correlation Analysis (CCA) and extend it to nonlinear CCA. We then derive the method of kernel-based CCA and compare these two methods on real and artificial data sets before using both on the Blind Separation of Sources.
SHORT-TERM PREDICTION OF CHAOTIC TIME SERIES BY USING RBF NETWORK WITH REGRESSION WEIGHTS
Authors: ROJAS I. , GONZALEZ J. , CAÑAS A. , DIAZ A. F. , ROJAS F. J. , RODRIGUEZ M.
Published: April 2003

We propose a framework for constructing and training a radial basis function (RBF) neural network. The structure of the gaussian functions is modified using a pseudo-gaussian function (PG) in which two scaling parameters σ are introduced, which eliminates the symmetry restriction and provides the neurons in the hidden layer with greater flexibility with respect to function approximation. We propose a modified PG-BF (pseudo-gaussian basis function) network in which the regression weights are used to replace the constant weights in the output layer. For this purpose, a sequential learning algorithm is presented to adapt the structure of the network, in which it is possible to create a new hidden unit and also to detect and remove inactive units. A salient feature of the network systems is that the method used for calculating the overall output is the weighted average of the output associated with each receptive field. The superior performance of the proposed PG-BF system over the standard RBF are illustrated using the problem of short-term prediction of chaotic time series.
COMBINING REGRESSION TREES AND RADIAL BASIS FUNCTION NETWORKS
Authors: ORR MARK , HALLAM JOHN , TAKEZAWA KUNIO , MURRAY ALAN , NINOMIYA SEISHI , OIDE MARI , LEONARD TOM
Published: April 2003

We describe a method for non-parametric regression which combines regression trees with radial basis function networks. The method is similar to that of Kubat,1 who was first to suggest such a combination, but has some significant improvements. We demonstrate the features of the new method, compare its performance with other methods on DELVE data sets and apply it to a real world problem involving the classification of soybean plants from digital images.
Letter to the Editor: NEUROCOMPUTERS: A DEAD END?
Authors: OMONDI AMOS R.
Published: April 2003

The last decade saw a proliferation of research into the design of neurocomputers. Although such work still continues, much of it is never beyond the prototype-machine stage. In this paper, we argue that, on the whole, neurocomputers are no longer viable; like, say, database computers before them, their time has passed before they became a common reality. We consider the implementation of hardware neural networks, from the level of arithmetic to complete individual processors and parallel processors and show that currents trends in computer architecture and implementation are not supportive of a case for custom neurocomputers. We argue that in the future, neural-network processing ought to be mostly restricted to general-purpose processors or to processors that have been designed for other widely-used applications. There are just one or two, rather narrow, exceptions to this.
Letter to the Editor: RIVAL PENALIZED COMPETITIVE LEARNING BASED APPROACH FOR DISCRETE-VALUED SOURCE SEPARATION
Authors: CHEUNG YIU-MING , XU LEI
Published: April 2003

This paper presents an approach based on Rival Penalized Competitive Learning (RPCL) rules for discrete-valued source separation. In this approach, we first build a connection between the source number and the cluster number of observations. Then, we use the RPCL rule to automatically find out the correct number of clusters such that the source number is determined. Moreover, we tune the de-mixing matrix based on the cluster centers instead of the observation themselves, whereby the noise interference is considerably reduced. The experiments have shown that this new approach not only quickly and automatically determines the number of sources, but also is insensitive to the noise in performing blind source separation.
Improved long-term temperature prediction by chaining of neural networks
Authors: DUHOUX M. , SUYKENS J. , DE MOOR B. , VANDEWALLE J.
Published: April 2003

When an artificial neural network (ANN) is trained to predict signals p steps ahead, the quality of the prediction typically decreases for large values of p. In this paper, we compare two methods for prediction with ANNs: the classical recursion of one-step ahead predictors and a new kind of chain structure. When applying both techniques to the prediction of the temperature at the end of a blast furnace, we conclude that the chaining approach leads to an improved prediction of the temperature and avoidance of instabilities, since the chained networks gradually take the prediction of their predecessors in the chain as an extra input. It is observed that instabilities might occur in the iterative case, which does not happen with the chaining approach. To select relevant inputs and decrease the number of weights in this approach, Automatic Relevance Determination (ARD) for multilayer perceptrons is applied.
UNSUPERVISED LEARNING AND TEMPORAL CONTEXT TO RECALL COMPLEX ROBOT TRAJECTORIES
Authors: DE A. BARRETO GUILHERME , ARAÚJO ALUIZIO F. R.
Published: April 2003

An unsupervised neural network is proposed to learn and recall complex robot trajectories. Two cases are considered: (i) A single trajectory in which a particular arm configuration (state) may occur more than once, and (ii) trajectories sharing states with each other. Ambiguities occur in both cases during recall of such trajectories. The proposed model consists of two groups of synaptic weights trained by competitive and Hebbian learning laws. They are responsible for encoding spatial and temporal features of the input sequences, respectively. Three mechanisms allow the network to deal with repeated or shared states: local and global context units, neurons disabled from learning, and redundancy. The network reproduces the current and the next state of the learned sequences and is able to resolve ambiguities. The model was simulated over various sets of robot trajectories in order to evaluate learning and recall, trajectory sampling effects and robustness.
ANALOGUE VLSI FOR PROBABILISTIC NETWORKS AND SPIKE-TIME COMPUTATION
Authors: MURRAY ALAN
Published: April 2003

The history and some of the methods of analogue neural VLSI are described. The strengths of analogue techniques are described, along with residual problems to be solved. The nature of hardware-friendly and hardware-appropriate algorithms is reviewed and suggestions are offered as to where analogue neural VLSI's future lies.
A neural network architecture for data classification
Authors: Lezoray Olivier , Cardot Hubert
Published: April 2003

This article aims at showing an architecture of neural networks designed for the classification of data distributed among a high number of classes. A significant gain in the global classification rate can be obtained by using our architecture. This latter is based on a set of several little neural networks, each one discriminating only two classes. The specialization of each neural network simplifies their structure and improves the classification. Moreover, the learning step automatically determines the number of hidden neurons. The discussion is illustrated by tests on databases from the UCI machine learning database repository. The experimental results show that this architecture can achieve a faster learning, simpler neural networks and an improved performance in classification.
Best Harmony, Unified RPCL and Automated Model Selection for Unsupervised and Supervised Learning on Gaussian Mixtures, Three-Layer Nets and ME-RBF-SVM Models
Authors: Xu Lei
Published: April 2003

After introducing the fundamentals of BYY system and harmony learning, which has been developed in past several years as a unified statistical framework for parameter learning, regularization and model selection, we systematically discuss this BYY harmony learning on systems with discrete inner-representations. First, we shown that one special case leads to unsupervised learning on Gaussian mixture. We show how harmony learning not only leads us to the EM algorithm for maximum likelihood (ML) learning and the corresponding extended KMEAN algorithms for Mahalanobis clustering with criteria for selecting the number of Gaussians or clusters, but also provides us two new regularization techniques and a unified scheme that includes the previous rival penalized competitive learning (RPCL) as well as its various variants and extensions that performs model selection automatically during parameter learning. Moreover, as a by-product, we also get a new approach for determining a set of 'supporting vectors' for Parzen window density estimation. Second, we shown that other special cases lead to three typical supervised learning models with several new results. On three layer net, we get (i) a new regularized ML learning, (ii) a new criterion for selecting the number of hidden units, and (iii) a family of EM-like algorithms that combines harmony learning with new techniques of regularization. On the original and alternative models of mixture-of-expert (ME) as well as radial basis function (RBF) nets, we get not only a new type of criteria for selecting the number of experts or basis functions but also a new type of the EM-like algorithms that combines regularization techniques and RPCL learning for parameter learning with either least complexity nature on the original ME model or automated model selection on the alternative ME model and RBF nets. Moreover, all the results for the alternative ME model are also applied to other two popular nonparametric statistical approaches, namely kernel regression and supporting vector machine. Particularly, not only we get an easily implemented approach for determining the smoothing parameter in kernel regression, but also we get an alternative approach for deciding the set of supporting vectors in supporting vector machine.
A SHORT-TERM TEMPERATURE FORECASTER BASED ON A NOVEL RADIAL BASIS FUNCTIONS NEURAL NETWORK
Authors: LANZA PEDRO A. GONZALEZ , COSME JESUS M. ZAMARREÑO
Published: April 2003

Many applications dealing with electric load forecasting in buildings require temperature prediction. A new method for short-term temperature forecasting based on a Radial Basis Functions Neural Network, initialized by a Regression Tree, is presented. In this method, each terminal node of the tree contributes one hidden unit to the RBF network. The forecaster uses the current coded hour and the temperature as inputs, and predicts the next hour temperature. The results demonstrate this predictor can be used for load forecasting.
NEURAL NETWORK CONTROL SYSTEMS THAT LEARN TO PERFORM APPROPRIATELY
Authors: BULLINARIA JOHN A. , RIDDELL PATRICIA M.
Published: April 2003

Setting up a neural network with a learning algorithm that determines how it can best operate is an efficient way to formulate control systems for many engineering applications, and is often much more feasible than direct programming. This paper examines three important aspects of this approach: the details of the cost function that is used with the gradient descent learning algorithm, how the resulting system depends on the initial pre-learning connection weights, and how the resulting system depends on the pattern of learning rates chosen for the different components of the system. We explore these issues by explicit simulations of a toy model that is a simplified abstraction of part of the human oculomotor control system. This allows us to compare our system with that produced by human evolution and development. We can then go on to consider how we might improve on the human system and apply what we have learnt to control systems that have no human analogue.
MODELLING AND PREDICTION FOR CHAOTIC FIR LASER ATTRACTOR USING RATIONAL FUNCTION NEURAL NETWORK
Authors: Cho Seongyun
Published: April 2003

Many real-world systems such as irregular ECG signal, volatility of currency exchange rate and heated fluid reaction exhibit higly complex nonlinear characteristic known as chaos. These chaotic systems cannot be retreated satisfactorily using linear system theory due to its high dimensionality and irregularity. This research focuses on prediction and modelling of chaotic FIR (Far InfraRed) laser system for which the underlying equations are not given. This paper proposed a method for prediction and modelling a chaotic FIR laser time series using rational function neural network. Three network architectures, TDNN (Time Delayed Neural Network), RBF (radial basis function) network and the RF (rational function) network, are also presented. Comparisons between these networks performance show the improvements introduced by the RF network in terms of a decrement in network complexity and better ability of predictability.
NEURAL NETWORKS AND LOGICAL REASONING SYSTEMS: A TRANSLATION TABLE
Authors: MARTINS JOÃO , MENDES R. VILELA
Published: April 2003

A correspondence is established between the basic elements of logic reasoning systems (knowledge bases, rules, inference and queries) and the structure and dynamical evolution laws of neural networks. The correspondence is pictured as a translation dictionary which might allow to go back and forth between symbolic and network formulations, a desirable step in learning-oriented systems and multicomputer networks. In the framework of Horn clause logics, it is found that atomic propositions with n arguments correspond to nodes with nth order synapses, rules to synaptic intensity constraints, forward chaining to synaptic dynamics and queries either to simple node activation or to a query tensor dynamics.
STOCHASTIC RADIAL BASIS FUNCTIONS
Authors: CARD HOWARD C.
Published: April 2003

Stochastic signal processing can implement gaussian activation functions for radial basis function networks, using stochastic counters. The statistics of neural inputs which control the increment and decrement operations of the counter are governed by Bernoulli distributions. The transfer functions relating the input and output pulse probabilities can closely approximate gaussian activation functions which improve with the number of states in the counter. The means and variances of these gaussian approximations can be controlled by varying the output combinational logic function of the binary counter variables.
PERCEPTION OF ODORS BY A NONLINEAR MODEL OF THE OLFACTORY BULB
Authors: BREAKSPEAR MICHAEL
Published: April 2003

The behavior of the olfactory bulb is modeled as a network of interconnected cells with nonlinear dynamics. External inputs from sensory neurons are introduced as perturbations to subsets of cells within the network. We describe the attractors of the system and show how they can be classified and ordered according to their varying degrees of symmetry. By studying networks of attractors in the system's phase space, it is shown how different perturbations may evoke specific switches between various patterns of behavior. This ensures that different odors, even if present at extremely low concentrations, are able to evoke a specific spatio-temporal behavior in the olfactory bulb, permitting their unique perception. The model incorporates many of the processes proposed to mediate perception, such as the topographic organisation of sensory systems, destabilization of cortex by sensory input and synchronisation between neurons. It is also consistent with the character of the olfactory electroencephalogram.
DMP3: A DYNAMIC MULTILAYER PERCEPTRON CONSTRUCTION ALGORITHM
Authors: ANDERSEN TIMOTHY L. , MARTINEZ TONY R.
Published: April 2003

This paper presents DMP3 (Dynamic Multilayer Perceptron 3), a multilayer perceptron (MLP) constructive training method that constructs MLPs by incrementally adding network elements of varying complexity to the network. DMP3 differs from other MLP construction techniques in several important ways, and the motivation for these differences are given. Information gain rather than error minimization is used to guide the growth of the network, which increases the utility of newly added network elements and decreases the likelihood that a premature dead end in the growth of the network will occur. The generalization performance of DMP3 is compared with that of several other well-known machine learning and neural network learning algorithms on nine real world data sets. Simulation results show that DMP3 performs better (on average) than any of the other algorithms on the data sets tested. The main reasons for this result are discussed in detail.
A SELECTIVE LEARNING METHOD TO IMPROVE THE GENERALIZATION OF MULTILAYER FEEDFORWARD NEURAL NETWORKS
Authors: GALVÁN I. M. , ISASI P. , ALER R. , VALLS J. M.
Published: April 2003

Multilayer feedforward neural networks with backpropagation algorithm have been used successfully in many applications. However, the level of generalization is heavily dependent on the quality of the training data. That is, some of the training patterns can be redundant or irrelevant. It has been shown that with careful dynamic selection of training patterns, better generalization performance may be obtained. Nevertheless, generalization is carried out independently of the novel patterns to be approximated. In this paper, we present a learning method that automatically selects the training patterns more appropriate to the new sample to be predicted. This training method follows a lazy learning strategy, in the sense that it builds approximations centered around the novel sample. The proposed method has been applied to three different domains: two artificial approximation problems and a real time series prediction problem. Results have been compared to standard backpropagation using the complete training data set and the new method shows better generalization abilities.
NEURAL COST FUNCTIONS AND SEARCH STRATEGIES FOR THE GENERATION OF BLOCK DESIGNS: AN EXPERIMENTAL EVALUATION
Authors: BOFILL PAU , TORRAS CARME
Published: April 2003

A constraint satisfaction problem, namely the generation of Balanced Incomplete Block Designs (v, b, r, k, λ)-BIBDs, is cast in terms of function optimization. A family of cost functions that both suit the problem and admit a neural implementation is defined. An experimental comparison spanning this repertoire of cost functions and three neural relaxation strategies (Down-Hill search, Simulated Annealing and a new Parallel Mean Search procedure), as applied to all BIBDs of up to 1000 entries, has been undertaken. The experiments were performed on a Connection Machine CM-200 and their analysis required a careful study of performance measures. The simplest cost function stood out as the best one for the three strategies. Parallel Mean Search, with several processors searching cooperatively in parallel, could solve a larger number of problems than the same number of processors working independently, but Simulated Annealing yielded overall the best results. Other conclusions, as detailed in the paper, could be drawn from the comparison, BIBDs remaining a challenging problem for neural optimization algorithms.
ESTABLISHING AN INDEX ARBITRAGE MODEL BY APPLYING NEURAL NETWORKS METHOD — A CASE STUDY OF NIKKEI 225 INDEX
Authors: CHEN AN-PIN , CHIANGLIN CHIEH-YOW , CHUNG HISU-PEI
Published: April 2003

This paper applies the neural network method to establish an index arbitrage model and compares the arbitrage performances to that from traditional cost of carry arbitrage model. From the empirical results of the Nikkei 225 stock index market, following conclusions can be stated:

(1) The basis will get enlarged for a time period, more profitability may be obtained from the trend.

(2) If the neural network is applied within the index arbitrage model, twofold of return would be obtained than traditional arbitrage model can do.

(3) If the T_basis has volatile trend, the neural network arbitrage model will ignore the peak. Although arbitrageur would lose the chance to get profit, they may reduce the market impact risk.
CONSTRUCTING A FUZZY RULE-BASED SYSTEM USING THE ILFN NETWORK AND GENETIC ALGORITHM
Authors: YEN GARY G. , MEESAD PHAYUNG
Published: April 2003

In this paper, a method for automatic construction of a fuzzy rule-based system from numerical data using the Incremental Learning Fuzzy Neural (ILFN) network and the Genetic Algorithm is presented. The ILFN network was developed for pattern classification applications. The ILFN network, which employed fuzzy sets and neural network theory, equips with a fast, one-pass, on-line, and incremental learning algorithm. After trained, the ILFN network stored numerical knowledge in hidden units, which can then be directly interpreted into if-then rule bases. However, the rules extracted from the ILFN network are not in an optimized fuzzy linguistic form. In this paper, a knowledge base for fuzzy expert system is extracted from the hidden units of the ILFN classifier. A genetic algorithm is then invoked, in an iterative manner, to reduce number of rules and select only discriminate features from input patterns needed to provide a fuzzy rule-based system. Three computer simulations using a simulated 2-D 3-class data, the well-known Fisher's Iris data set, and the Wisconsin breast cancer data set were performed. The fuzzy rule-based system derived from the proposed method achieved 100% and 97.33% correct classification on the 75 patterns for training set and 75 patterns for test set, respectively. For the Wisconsin breast cancer data set, using 400 patterns for training and 299 patterns for testing, the derived fuzzy rule-based system achieved 99.5% and 98.33% correct classification on the training set and the test set, respectively.
THE SIMULTANEOUS RECURRENT NEURAL NETWORK FOR ADDRESSING THE SCALING PROBLEM IN STATIC OPTIMIZATION
Authors: SERPEN GURSEL , PATWARDHAN AMOL , GEIB JEFF
Published: April 2003

A trainable recurrent neural network, Simultaneous Recurrent Neural network, is proposed to address the scaling problem faced by neural network algorithms in static optimization. The proposed algorithm derives its computational power to address the scaling problem through its ability to "learn" compared to existing recurrent neural algorithms, which are not trainable. Recurrent backpropagation algorithm is employed to train the recurrent, relaxation-based neural network in order to associate fixed points of the network dynamics with locally optimal solutions of the static optimization problems. Performance of the algorithm is tested on the NP-hard Traveling Salesman Problem in the range of 100 to 600 cities. Simulation results indicate that the proposed algorithm is able to consistently locate high-quality solutions for all problem sizes tested. In other words, the proposed algorithm scales demonstrably well with the problem size with respect to quality of solutions and at the expense of increased computational cost for large problem sizes.
ADAPTIVE CONTROL OF MOBILE ROBOTS USING A NEURAL NETWORK
Authors: de Sousa Celso , Hermerly Elder Moreira
Published: April 2003

A Neural Network - based control approach for mobile robot is proposed. The weight adaptation is made on-line, without previous learning. Several possible situations in robot navigation are considered, including uncertainties in the model and presence of disturbance. Weight adaptation laws are presented as well as simulation results.
BACKPROPAGATION ALGORITHM ADAPTATION PARAMETERS USING LEARNING AUTOMATA
Authors: BEIGY HAMID , MEYBODI MOHAMMAD R.
Published: April 2003

Despite of the many successful applications of backpropagation for training multi–layer neural networks, it has many drawbacks. For complex problems it may require a long time to train the networks, and it may not train at all. Long training time can be the result of the non-optimal parameters. It is not easy to choose appropriate value of the parameters for a particular problem. In this paper, by interconnection of fixed structure learning automata (FSLA) to the feedforward neural networks, we apply learning automata (LA) scheme for adjusting these parameters based on the observation of random response of neural networks. The main motivation in using learning automata as an adaptation algorithm is to use its capability of global optimization when dealing with multi-modal surface. The feasibility of proposed method is shown through simulations on three learning problems: exclusive-or, encoding problem, and digit recognition. The simulation results show that the adaptation of these parameters using this method not only increases the convergence rate of learning but it increases the likelihood of escaping from the local minima.
DESIGN OF A NEUROFUZZY CONTROLLER WITH SIMPLIFIED ARCHITECTURE
Authors: BORDON MÁRIO EDUARDO , DA SILVA IVAN NUNES , AVOLIO EDWIN
Published: April 2003

This work presents the design of a neurofuzzy controller with simplified architecture that minimizes the processing time used in several stages associated with systems and processes modelling. The basic procedures of fuzzification and defuzzification are very simplified, whereas the inference procedures are computed in a direct way. The simplified architecture has allowed a fast and easy configuration of the neurofuzzy controller, as consequence, the control rules that define the control actions are obtained automatically. To validate the proposed approach, this neurofuzzy system is used in an industrial application for fluid flow control.
CDMA & TDMA BASED NEURAL NETS
Authors: HERRERO JUAN CARLOS
Published: April 2003

CDMA and TDMA telecommunication techniques were established long time ago, but they have acquired a renewed presence due to the rapidly increasing mobile phones demand. In this paper, we are going to see they are suitable for neural nets, if we leave the concept "connection" between processing units and we adopt the concept "messages" exchanged between them. This may open the door to neural nets with a higher number of processing units and flexible configuration.
A STUDY ON RULE EXTRACTION FROM SEVERAL COMBINED NEURAL NETWORKS
Authors: BOLOGNA GUIDO
Published: April 2003

The problem of rule extraction from neural networks is NP-hard. This work presents a new technique to extract "if-then-else" rules from ensembles of DIMLP neural networks. Rules are extracted in polynomial time with respect to the dimensionality of the problem, the number of examples, and the size of the resulting network. Further, the degree of matching between extracted rules and neural network responses is 100%. Ensembles of DIMLP networks were trained on four data sets in the public domain. Extracted rules were on average significantly more accurate than those extracted from C4.5 decision trees.
TRAINING SVMs WITH EDR ALGORITHM
Authors: ALMEIDA M. B. , BRAGA A. P. , BRAGA J. P.
Published: April 2003

The aim of this work is to present a new training algorithm for SVMs based on the pattern selection strategy called Error Dependent Repetition (EDR). With EDR, the presentation frequency of a pattern depends on its error: patterns with larger errors are selected more frequently and patterns with smaller error (or learned) are presented with minor frequency. Using a simple iterative process based on gradient ascent, SVM-EDR can solve the dual problem without any assumption about support vectors or the Karush-Kuhn-Tucker (KKT) conditions.
RECENT ADVANCES IN THE MOBJ ALGORITHM FOR TRAINING ARTIFICIAL NEURAL NETWORKS
Authors: DE ALBUQUERQUE TEIXEIRA ROSELITO , BRAGA ANTÔNIO PADUA , TAKAHASHI RICARDO H. C. , SALDANHA RODNEY R.
Published: April 2003

This paper presents a new scheme for training MLPs which employs a relaxation method for multi-objective optimization. The algorithm works by obtaining a reduced set of solutions, from which the one with the best generalization is selected. This approach allows balancing between the training error and norm of network weight vectors, which are the two objective functions of the multi-objective optimization problem. The method is applied to classification and regression problems and compared with Weight Decay (WD), Support Vector Machines (SVMs) and standard Backpropagation (BP). It is shown that the systematic procedure for training proposed results on good generalization neural models, and outperforms traditional methods.
A PROPOSED MAX-PRODUCT THRESHOLD UNIT FOR CLASSIFICATION OF PATTERN VECTORS
Authors: BROUWER ROELOF K
Published: April 2003

This paper proposes a max-product threshold unit (maptu) that can successfully perform dichotomous classifications of pattern vectors. Maptu, with weight vector, w, classifies a pattern vector, x, by comparing x max-prod w to 0.5. Results obtained by other methods in classification of benchmark data are used for comparison to the method using maptu. The benchmark data consists of the Australian credit data set, cervical cell data set, diabetes data set and the iris data set.
A NOVEL APPROACH FOR SOLVING CONSTRAINED NONLINEAR OPTIMIZATION PROBLEMS USING NEUROFUZZY SYSTEMS
Authors: DA SILVA IVAN NUNES , DE SOUZA ANDRÉ NUNES , EDUARDO BORDON MÁRIO
Published: April 2003

A neural network model for solving constrained nonlinear optimization problems with bounded variables is presented in this paper. More specifically, a modified Hopfield network is developed and its internal parameters are computed using the valid-subspace technique. These parameters guarantee the convergence of the network to the equilibrium points. The network is shown to be completely stable and globally convergent to the solutions of constrained nonlinear optimization problems. A fuzzy logic controller is incorporated in the network to minimize convergence time. Simulation results are presented to validate the proposed approach.
EVOLUTIONARY OPTIMIZATION OF RBF NETWORKS
Authors: LACERDA E. , DE CARVALHO A. , LUDERMIR TERESA
Published: April 2003

One of the main obstacles to the widespread use of artificial neural networks is the difficulty of adequately defining values for their free parameters. This article discusses how Radial Basis Function (RBF) networks can have their parameters defined by genetic algorithms. For such, it presents an overall view of the problems involved and the different approaches used to genetically optimize RBF networks. A new strategy to optimize RBF networks using genetic algorithms is proposed, which includes new representation, crossover operator and the use of a multiobjective optimization criterion. Experiments using a benchmark problem are performed and the results achieved using this model are compared to those achieved by other approaches.
THREE CASE STUDIES OF THE GASNET MODEL IN DISCRETE DOMAINS
Authors: SANTOS CARMEN L. R. , DE OLIVEIRA PEDRO P. B. , HUSBANDS P. , SOUZA CELSO R.
Published: April 2003

A new neural network model – the GasNet – has been recently reported in the literature, which, in addition to the traditional electric type, point-to-point communication between units, also uses communication through a diffulsable chemical modulator. Here we assess the applicability of this model in three different scenarios, the XOR problem, a food gathering task for a simulated robot, and a docking task for a virtual spaceship. All of them represent discrete domains, a contrast with the one where the GasNet was originally introduced, which had an essentially continuous nature. These scenarios are well-known benchmark problems from the literature and, since they exhibit varying degrees of complexity, they impose distinct performance demands on the GasNet. The experiments were primarily intended to better understand the model, by extending the original problem domain where GasNet was introduced. The results reported point at some difficulties with the current GasNet model.
A LATE-STOPPING METHOD FOR OPTIMAL AGGREGATION OF NEURAL NETWORKS
Authors: GRANITTO PABLO M. , VERDES PABLO F. , NAVONE HUGO D. , CECCATTO H. ALEJANDRO
Published: April 2003

Ensembles of artificial neural networks have been used in the last years as classification/regression machines, showing improved generalization capabilities that outperform those of single networks. However, it has been recognized that for aggregation to be effective the individual networks must be as accurate and diverse as possible. An important problem is, then, how to tune the aggregate members in order to have an optimal compromise between these two conflicting conditions. We propose here a simple method for constructing regression/classification ensembles of neural networks that leads to overtrained aggregate members with an adequate balance between accuracy and diversity. The algorithm is favorably tested against other methods recently proposed in the literature, producing an improvement in performance on the standard statistical databases used as benchmarks. In addition, and as a concrete application, we apply our method to the sunspot time series and predict the remainder of the current cycle 23 of solar activity.
A GOAL-DRIVEN NEURAL PROPOSITIONAL INTERPRETER
Authors: LIMA PRISCILA M. V.
Published: April 2003

This work presents ARQ-PROP-II, the propositional version of a neural engine for finding proofs by refutation using the Resolution Principle. This neural architecture does not require special arrangements or modules to do forward or backward reasoning, being driven by the goal posed to it. ARQ-PROP-II is capable of integrated monotonic reasoning with complete and incomplete knowledge. The neural mechanism presented herein is the first to our knowledge that does not require that the knowledge base be either pre-encoded or learnt.
ARTIFICIAL NEURAL NETWORKS FOR BLUR IDENTIFICATION AND RESTORATION OF NONLINEARLY DEGRADED IMAGES
Authors: CHEEMA T. A. , QURESHI I. M. , JALIL A. , NAVEED A.
Published: April 2003

In this paper, an image restoration algorithm is proposed to identify noncausal blur function. Image degradation processes include both linear and nonlinear phenomena. A neural network model combining an adaptive auto-associative network with a random Gaussian process is proposed to restore the blurred image and blur function simultaneously. The noisy and blurred images are modeled as continuous associative networks, whereas auto-associative part determines the image model coefficients and the hetero-associative part determines the blur function of the system. The self-organization like structure provides the potential solution of the blind image restoration problem. The estimation and restoration are implemented by using an iterative gradient based algorithm to minimize the error function.
A RECURRENT NEURAL NETWORK FOR PARTITIONING OF HAND DRAWN CHARACTERS INTO STROKES OF DIFFERENT ORIENTATIONS
Authors: GOLTSEV ALEXANDER , RACHKOVSKIJ DMITRI
Published: April 2003

A neural network is described which is intended to extract orientation features that should be used for recognition of hand drawn characters. The network partitions the input hand drawn characters into separate line segments (strokes) according to their orientations. The network consists of several neural layers; each layer serves for extracting strokes of a certain orientation. Every neural layer has one-to-one correspondence with an input screen. The network uses an iterative update procedure which includes interactions of neurons inside each layer through oriented excitatory connections and inhibitory interrelations between the corresponding neurons of different layers. Computer simulation of the network was performed. Experiments showed that the network efficiently classifies all pixels of any hand drawn characters according to the orientations of the strokes constituting these characters and performs, as a result of that, a reasonable segmentation of characters.
STRING MATCHING ARTIFICIAL NEURAL NETWORKS
Authors: TAMBOURATZIS TATIANA
Published: April 2003

Three artificial neural networks (ANNs) are proposed for solving a variety of on- and off-line string matching problems. The ANN structure employed as the building block of these ANNs is derived from the harmony theory (HT) ANN, whereby the resulting string matching ANNs are characterized by fast match-mismatch decisions, low computational complexity, and activation values of the ANN output nodes that can be used as indicators of substitution, insertion (addition) and deletion spelling errors.
GLOBAL MAPPING ANALYSIS: STOCHASTIC APPROXIMATION FOR MULTIDIMENSIONAL SCALING
Authors: MATSUDA YOSHITATSU , YAMAGUCHI KAZUNORI
Published: April 2003

In this paper, we propose global mapping analysis (GMA) as a new method to solve multidimensional scaling (MDS). By GMA, MDS is done by an online learning rule based on stochastic approximation. GMA need not directly calculate the disparity matrix for carrying out MDS, as Oja's PCA network do not calculate the correlation matrix. So, GMA is expected to be useful for multivariate data analysis on a large scale. Actually, it was verified by numerical experiments based on artificial data that GMA can work well even if the number of the attribute N is quite large (N=10 000.)
PROBABILITY DENSITY FUNCTION LEARNING BY UNSUPERVISED NEURONS
Authors: FIORI SIMONE
Published: April 2003

In a recent work, we introduced the concept of pseudo-polynomial adaptive activation function neuron (FAN) and presented an unsupervised information-theoretic learning theory for such structure. The learning model is based on entropy optimization and provides a way of learning probability distributions from incomplete data. The aim of the present paper is to illustrate some theoretical features of the FAN neuron, to extend its learning theory to asymmetrical density function approximation, and to provide an analytical and numerical comparison with other known density function estimation methods, with special emphasis to the universal approximation ability. The paper also provides a survey of PDF learning from incomplete data, as well as results of several experiments performed on real-world problems and signals.
ALOPEX-B: A NEW, SIMPLER, BUT YET FASTER VERSION OF THE ALOPEX TRAINING ALGORITHM
Authors: BIA ALEJANDRO
Published: April 2003

Experimenting with some changes and simplifications to the Alopex algorithm, we obtained a new faster version (Alopex-B), that also shows lower failure rates on training attempts. Like Alopex, our version is network-architecture independent, does not require error or transfer functions to be differentiable, has a high potential for parallelism, and is stochastic (which helps avoid local minima), but unlike Alopex it follows no annealing scheme, and uses less parameters which makes it simpler to implement and to use.
AUTOMATIC DETERMINATION OF RADIAL BASIS FUNCTIONS: AN IMMUNITY-BASED APPROACH
Authors: NUNES DE CASTRO LEANDRO , VON ZUBEN FERNANDO J.
Published: April 2003

The appropriate operation of a radial basis function (RBF) neural network depends mainly upon an adequate choice of the parameters of its basis functions. The simplest approach to train an RBF network is to assume fixed radial basis functions defining the activation of the hidden units. Once the RBF parameters are fixed, the optimal set of output weights can be determined straightforwardly by using a linear least squares algorithm, which generally means reduction in the learning time as compared to the determination of all RBF network parameters using supervised learning. The main drawback of this strategy is the requirement of an efficient algorithm to determine the number, position, and dispersion of the RBFs. The approach proposed here is inspired by models derived from the vertebrate immune system, that will be shown to perform unsupervised cluster analysis. The algorithm is introduced and its performance is compared to that of the random, k-means center selection procedures and other results from the literature. By automatically defining the number of RBF centers, their positions and dispersions, the proposed method leads to parsimonious solutions. Simulation results are reported concerning regression and classification problems.
RECRUITMENT LEARNING OF BOOLEAN FUNCTIONS IN SPARSE RANDOM NETWORKS
Authors: HOGAN JAMES M. , DIEDERICH JOACHIM
Published: April 2003

This work presents a new class of neural network models constrained by biological levels of sparsity and weight-precision, and employing only local weight updates. Concept learning is accomplished through the rapid recruitment of existing network knowledge – complex knowledge being realised as a combination of existing basis concepts. Prior network knowledge is here obtained through the random generation of feedforward networks, with the resulting concept library tailored through distributional bias to suit a particular target class. Learning is exclusively local – through supervised Hebbian and Winnow updates – avoiding the necessity for backpropagation of error and allowing remarkably rapid learning. The approach is demonstrated upon concepts of varying difficulty, culminating in the well-known Monks and LED benchmark problems.
A BIOLOGICALLY INSPIRED NEURAL NETWORK FOR DYNAMIC PROGRAMMING
Authors: FRANCELIN ROMERO ROSELI A. , KACPRYZK JANUSZ , GOMIDE FERNANDO
Published: April 2003

An artificial neural network with a two-layer feedback topology and generalized recurrent neurons, for solving nonlinear discrete dynamic optimization problems, is developed. A direct method to assign the weights of neural networks is presented. The method is based on Bellmann's Optimality Principle and on the interchange of information which occurs during the synaptic chemical processing among neurons. The neural network based algorithm is an advantageous approach for dynamic programming due to the inherent parallelism of the neural networks; further it reduces the severity of computational problems that can occur in methods like conventional methods. Some illustrative application examples are presented to show how this approach works out including the shortest path and fuzzy decision making problems.
A FAST AND CONVERGENT STOCHASTIC MLP LEARNING ALGORITHM
Authors: SAKURAI AKITO
Published: April 2003

We propose a stochastic learning algorithm for multilayer perceptrons of linear-threshold function units, which theoretically converges with probability one and experimentally exhibits 100% convergence rate and remarkable speed on parity and classification problems with typical generalization accuracy. For learning the n bit parity function with n hidden units, the algorithm converged on all the trials we tested (n=2 to 12) after 5.8· 4.1n presentations for 0.23· 4.0n-6 seconds on a 533MHz Alpha 21164A chip on average, which is five to ten times faster than Levenberg-Marquardt algorithm with restarts. For a medium size classification problem known as Thyroid in UCI repository, the algorithm is faster in speed and comparative in generalization accuracy than the standard backpropagation and Levenberg-Marquardt algorithms.
FABRICATION OF ELECTRON FIELD EMITTERS USING CARBON NANOTUBES
Authors: CHOI YOUNG CHUL , PARK YOUNG SOO , LEE YOUNG HEE , CHOI WON BONG , LEE NAE SUNG , KIM JONG MIN , LEE CHEOL JIN , KIM DAE WOON , LEE TAE JAE
Published: April 2003

Carbon nanotube (CNT)-based field emission displays (FEDs) have been fabricated using well-aligned nanotubes on substrates in situ grown by thermal chemical vapor deposition (CVD), and paste squeeze and surface rubbing techniques. Although the former seems to be an ultimate approach for CNT-based FED, a large area synthesis and uniform field emission over the entire area is not yet easily accessible. On the other hand, the latter is fully scalable on glass substrates and shows very high luminance of 1800 cd/m2 at 4 V/μm. The degradation of emission currents for single-wall carbon nanotubes was less than 10% in electrical aging tests. Large field-enhancement factors (23,000–46,000) and low turn-on voltages (1.5-3 V/μm) were attributed to well-aligned carbon nanotubes on substrates and a large number density of carbon nanotubes of 5-10 μm-2, which was confirmed by high-resolution scanning electron microscopy.