nils-herrmann/clean_scirepeval huggingface dataset:
Sparse Kernel Canonical Correlation Analysis

We review the recently proposed method of Relevance Vector Machines which is a supervised training method related to Support Vector Machines. We also review the statistical technique of Canonical Correlation Analysis and its implementation in a Feature Space. We show how the technique of Relevance Vectors may be applied to the method of Kernel Canonical Correlation Analysis to gain a very sparse representation of a data set and discuss why such a representation may be bene cial to an organism.
Automating the construction of CBR systems using kernel methods

Instance‐based reasoning systems and, in general, case‐based reasoning systems are normally used in problems for which it is difficult to define rules. Although case‐based reasoning methods have proved their ability to solve different types of problems, there is still a demand for methods that facilitate their automation during their creation and the retrieval and reuse stages of their reasoning circle. This paper presents one method based on kernels, which can be used to automate some of the reasoning steps of instance‐based reasoning systems. Kernels were originally derived in the context of support vector machines, which identify the smallest number of data points necessary to solve a particular problem (e.g., regression or classification). Unsupervised kernel methods have been used successfully to identify the optimal instances to instantiate an instance base. The efficiency of the kernel model is shown on an oceanographic problem.
Seeking Independence Using Biologically-Inspired ANN’s

In this chapter, we make a case for research into the use of biologically-inspired artificial neural networks (BIANNs) in the search for independence. This may seem an odd case to have to make in a book derived from a workshop which followed the Seventh International Conference on Artificial Neural Networks (ICANN99), the most prestigious ANN conference in Europe. However, as this book demonstrates, many of the methods currently being investigated by the neural network community are very different from the biologically-inspired networks which we will advocate.
Canonical correlation analysis neural networks

We review a new method of performing canonical correlation analysis (CCA) with artificial neural networks. We have previously (1998, 1999) compared its capabilities with standard statistical methods on simple data sets such as an abstraction of random dot stereograms. In this paper, we show that this original rule is only one of a family of rules which use Hebbian and anti-Hebbian learning to find correlations between data sets. We derive slightly different rules from Becker's information theoretic criteria and from probabilistic assumptions. We then derive a robust version of this last rule and then compare the effectiveness of these rules on a standard data set.
Power load forecasting using neural canonical correlates

We (1998, 1999) have previously derived a neural network implementation of the statistical technique of canonical correlation analysis. We have then extended the network so that it may find nonlinear correlations in data sets. In this paper we demonstrate the capabilities of the network (both linear and nonlinear) on an artificial data set and demonstrate that the nonlinear network finds greater correlations than any lineal network. We then use both networks for forecasting the next day's power loading given the previous days' loads and forecasts of the temperature. We show that the nonlinear correlation method performs better than both a standard supervised learning neural network using backpropagation and a recent modification of that algorithm.
A comparison of sparse kernel principal component analysis methods

Presents a comparative study of a group of methods based on kernels which attempt to identify only the most significant cases with which to create the nonlinear feature space. Kernels were originally derived in the context of support vector machines, which identify the smallest number of data points necessary to solve a particular problem (e.g. regression or classification). We use extensions of kernel principal component analysis to identify the optimal cases to create a sparse representation in feature space. The efficiency of the kernel models is compared on an oceanographic problem.
Probability prediction using support vector machines

Reports on an investigation of the use of support vector machines in the problem of beating bookmakers' odds. The method used is to create nine regression lines, one for each of the possible half-time/full-time possibilities, e.g. a draw at half time with the home team winning at full time. We extend the support vector machine by incorporating variable-width permissible error bounds based on our knowledge of the sample size on which each of the data points is based. We show that the variable-width method improves the regression.
Technical Trading Versus Market Efficiency-A Genetic Programming Approach

In this paper genetic programming is used to investigate a number of long time series of price data for a stock exchange quoted share, in order to discern whether there are any patterns in the data which could be used for technical trading purposes. This extends the work done by the authors in a previous paper (Fyfe et al. 1999) which suggested that, although it was possible to find a rule which did outperform simple buy and hold, there were insufficient grounds for the rejection of the efficient market hypothesis. The purpose of the present paper is to investigate the robustness and generalisability of the conclusion reached by Fyfe et. al.
The Evolutionary Emergence of Language: Modelling Language-Physiology Coevolution

A feature of current computational models of language evolution is that the individuals in later populations are not structurally, ‘physiologically’, different from those in the first. Evolution may be working on the language itself, as learned by agents which do not evolve, or on an innate communication scheme. A number of models specifically demonstrate self-organisation of communication schemes and grammars in populations that are already capable of language.
A Probabilistic Agent Approach to the Traffic Jam Problem

Artificial life simulations of social situations are a relative new field which aims to model situations which are too complex to be analytically investigated. In this paper, we develop commuter-agents with simple probabilistic models of the world and show that such agents can develop cooperation which aids the society as a whole. We show that there are situations in which the more powerful agents are sometimes forced by their greater knowledge into taking a lower utility than the weaker ones. In the last series of experiments we show that agents which have the ability to predict others' road usage can materially improve the utility of the population as a whole.
A Gneral Class of Neural Networks for Principal Component Analysis and Factor Analysis

We review a recently proposed family of functions for finding principal and minor components of a data set. We extend the family so that the Principal Subspace of the data set is found by using a method similar to that known as the Bigradient algorithm. We then amend the method in a way which was shown to change a Principal Component Analysis (PCA) rule to a rule for performing Factor Analysis (FA) and show its power on a standard problem. We find in both cases that, whereas the one Principal Component family all have similar convergence and stability properties, the multiple output networks for both PCA and FA have different properties.
Kernel factor analysis with Varimax rotation

Kernel methods have recently become popular for the exploration of structure in data and one of the more commonly used methods is kernel principal components analysis (kernel PCA). This method is similar to nonlinear PCA in that PCA is performed in kernel space, which is a nonlinear transformation of the data into a higher dimension. We compare this method to a closely related statistic technique called factor analysis and show that, particularly when used in conjunction with a Varimax rotation of the factor axis, we can transform the kernel space so that the local variance in data clusters may be accounted for and not just the global variance across all of the data clusters. When the data is matched with an appropriate kernel then this method improves the interpretability of the results.
Generalised Canonical Correlation Analysis

Canonical Correlation Analysis  is used when we have two data sets which we believe have some underlying correlation. In this paper, we derive a new family of neural methods for finding the canonical correlation directions by solving a generalized eigenvalue problem. Based on the differential equation for the generalized eigenvalue problem, a family of CCA learning algorithms can be obtained. We compare our family of methods with a previously derived  CCA learning algorithm. Our results show that all the new learning algorithms of this family have the same order of convergence speed and in particular are much faster than existing algorithms; they are also shown to be able to find greater nonlinear correlations. They are also much more robust with respect to parameter selection.
Technical analysis versus market efficiency - a genetic programming approach

In the paper the authors maintain that the prevalence of technical analysis in professional investment argues that such techniques should perhaps be taken more seriously by academics. The new technique of genetic programming is used to investigate a long time series of price data for a quoted property investment company, to discern whether there are any patterns in the data which could be used for technical trading purposes. A successful buy rule is found which generates returns in excess of what would be expected from the best-fitting null time-series model. Nevertheless, this turns out to be a more sophisticated variant of the buy and hold rule, which the authors term timing specific buy and hold. Although the rule does outperform simple buy and hold, it really does not provide sufficient grounds for the rejection of the efficient market hypothesis, though it does suggest that further investigation of the specific conditions of applicability of the EMH may be appropriate.
Comparison of Kohonen, scale-invariant and GTM self-organising maps for interpretation of spectral data

We investigate the use of artificial neural networks in classifying hyperspectral data. Such data when collected from remote sensors provides extremely detailed coverage of e.g. the mineralogical composition of planetary surfaces, however the volume of data supplied often overwhelms traditional classifiers. When we wish to investigate such data sets in an open-ended manner, the use of unsupervised learning is a pre-requisite. A set of remotely sensed spectral images are use to train several different topology preserving neural networks. In each met hod, the data is projected onto a two dimensional grid designed to visualise the data set in a low dimensional space. Such mappings allow graceful degradation of the classifications given by the mappings since nearby data points are mapped to the same or similar classifications.
Rectified Gaussian distributions and the identification of multiple cause structure in data

We investigate the use of an unsupervised artificial neural network to form a sparse representation of the underlying causes in a data set. By using fixed lateral connections that are derived from the rectified generalised Gaussian distribution, we form a network that is capable of identifying the multiple cause structure of the data. We further show that some topology preservation of the input data is possible using this network and that related features may be coded in separate areas of the output space.
A Comparison of an Oligopoly Game and the N-Person Iterated Prisoner's Dilemma

We review and analyze a recently proposed model of an Oligopoly Game. From a theoretical investigation of the payoff scheme, we confirm that it cannot be a true representation of the N -Person Iterated Prisoner's Dilemma. From an empirical investigation of the model's parameters, we show which are crucial to the evolution of cooperation. We compare these results with a separate set of simulations of the abstract NIPD.
Unsupervised Detection of Illusory Contours

We present an unsupervised artificial neural network that is capable of detecting illusory objects in artificial image data. The network has previously been shown [4, 3] to self-organise to identify multiple causes in data distributions where each of the individual causes are independent and may be hidden from direct observation. Learning in the network is completely local and the form of coding that is achieved is sparse. The network is based on a non-linear Principal Components Analysis (PCA) network using a soft positive threshold function as a non-linearity instead of the typical sigmoid (or logistic) function. After the application of the non-linearity in the network we add zero-mean Gaussian noise. The non-linear PCA aspect of this network ensures an coding which is more accessable to human interpretation and the additive noise ensures a sparse representation in which only as many output neurons as are required are used. Illusory contours are detected when the probability of a feature appearing at the input to the network is so high that the most sparse response is to learn an illusory feature between the actual features. By adding lateral connections between the output neurons to provide temporal context in the network, we can achieve a temporal ordering of outputs in response to a particular feature moving across the input space.
Using noise to form a minimal overcomplete basis

The authors previously (1998) developed an extension of a principal component analysis artificial neural network which we have linked to the statistical technique of factor analysis. We have shown that the resulting network can identify the independent components of visual scenes. We now show that, in cases where the factor analysis network identifies factors of greater number than the inherent dimensionality of the input space, the addition of noise leads to an optimally sparse representation of the input data which we link to a minimal overcomplete basis. We show that in cases in which the data set is not itself inherently sparse, the method induces a very sparse description of the data set.
Neural networks which identify composite factors

We investigate the use of an artificial neural network to form a sparse distributed representation of the underlying factors in data sets. We extend the previously proposed [1] network so that it may identify composite causes in data sets by creating a hierarchical network. We use the network as a means of identifying individual faces when the network is trained on a mixture of faces and show both analytically and through experiments how noise allows us to find precisely the factors without prior assumptions of the number of factors.
Noise to extract independent causes

Noisy threshold activation functions are used to force sparse responses on the output neurons of an unsupervised neural network enabling the network to identify the underlying independent factors of visual data. The addition of noise into the network enables us to control the response of the network to the data so we can force only as many outputs to respond to the data as there are signi cant factors in the data. Noise is also used to modularise the response of the network so that factors with temporal correlation may be coded in the same module of the output space.
Fast formation of invariant feature maps

We present a neural network that uses competitive learning and a neighbourhood function in a similar way to the self-organising map (SOM). The network consists of a number of modules that are positioned in an array (normally in one or two dimensions) where each module performs a subspace projection and the learning rate used within each module is weighted by the neighbourhood function. By using a subspace method that always gives convergence of orthonormal basis vectors within the modules, we demonstrate that this method can allow fast formation of filters of basic invariant features. The adaptive subspace self-organising map (ASSOM) is in some ways similar and it has been proposed as a modular neural network for invariant filtering, providing invariances in scale, phase, rotation, translation etc. In this paper we show that our method can also provide these invariances with much improved performance in terms of the processing time required during training.
Convergence Properties of a Modified Temporal Anti-Hebbian Model

This paper proposes a modification to Girolami and Fyfe’s temporal variant of Foldiak’s first model. In the original temporal model asymmetric memory based lateral weights are adapted using a cross-correlation criterion and compared with information maximisation applied to the blind separation of sources which have experienced convolved mixing. In our new model the weights are adapted using a linearised discontinuous function which shares the same quadrature sign behaviour as the decorrelation function. The model provides separation and restoration of convolved mixes of signals where network weights converge very closely to ideal FIR deconvolving filter coefficients. We compare the convergence properties of a ‘linearised correlation’ model with that of the original model and examine potential for improvement of convergence behaviour.
Hierarchical and Parallel Models for Non-stationary IndependentComponent AnalysisC

We review the limited progress which has been made to date on the problem of identiication of independent sourcess from a linear mixture of such sources when the sources are non-stationary. In particular, we investigate the model of Matsuoka et al and show that their claim to have a linear model is in fact not correct. We demonstrate a hierarchical feedforward model which is more linear than that of Matsuoka et al. An extension which allows for parallel identiication of independent sources is also demonstrated. 1  Separation of a single signal from a mixture of source signals is an area of some current research interest. Most intensive work is being carried out on the problem of the blind separation of source signals-so called because we assume no prior knowledge of either the signals or the mixture model which created the observed mixture. This problem is closely related to that known as Independent Component Analysis, the analysis of independent components which together make up a set of observations. The problem of demixing linear mixtures has been largely solved using a number of often related, perhaps sometimes equivalent, techniques. The linear problem may be stated as: consider a set of independent zero mean signals, s = fs 1 ; s 2 at most one of which may be Gaussian. Let there be a non-singular mixing matrix A which is used to create observations, x = fx 1 ; x 2 ; :::; x m g = As. It is often assumed that A is square or at least that m n. Then the problem is to nd that matrix W such that y = Wx = s, up to a permutation and scaling. Attention is now shifting to more diicult problems such as those associated with non-linear mixing or those associated with non-stationarity. We will concentrate on the latter and consider the problem of linear mixtures of non-stationary signals. There has been relatively little research published in this area compared with the rather large volume of research published over the last two years on linear ICA. We will look at the model of Matsuoka et al 2] in some detail below. Other work includes that of Parga and Nadal 4] who extend the algebraic method of diagonalising both the covariance matrix C 0 = E(s i (t)s j (t)) and its time delayed second order equivalent, C = E(s i (t)s …
Neuro-adaptation method for a case-based reasoning system

A multi-agent approach is presented for identifying and forecasting the structure of the water ahead of an ongoing vessel. The work addresses the task of forecasting the behaviour of complex environments, in which the underling knowledge of the domain is not completely available, the rules governing the system are fuzzy and the available data sets are limited and incomplete. A hybrid approach is proposed that combines the ability of a case-based reasoning system for selecting previous similar situations and the generalising ability of artificial neural networks to guide the adaptation stage of the case-based reasoning system. The successful application of the approach to oceanographic forecasting in the Atlantic Ocean is described.
Invariant feature maps for analysis of orientations in image data

We present a method that uses competitive learning and a neighbourhood function in a similar way to the self-organising map (SOM) . The network consists of a number of modules that are positioned in an array (normally in one or two dimensions) where each module performs a subspace projection and the rotation of these subspaces is weighted by the neighbourhood function. Non-linear activation functions are introduced so that each node performs nonlinear PCA on the training data captured in its Voronoi region. We show that this network may be used for position invariant detection of bars at varying orientations.
Modelling multiple-cause structure using rectification constraints.

We present an artificial neural network which self-organizes in an unsupervised manner to form a sparse distributed representation of the underlying causes in data sets. This coding is achieved by introducing several rectification constraints to a PCA network, based on our prior beliefs about the data. Through experimentation we investigate the relative performance of these rectifications on the weights and/or outputs of the network. We find that use of an exponential function on the output to the network is most reliable in discovering all the causes in data sets even when the input data are strongly corrupted by random noise. Preprocessing our inputs to achieve unit variance on each is very effective in helping us to discover all underlying causes when the power on each cause is variable. Our resulting network methodologies are straightforward yet extremely robust over many trials.
Structured population-based incremental learning

We investigate a recently developed abstraction of genetic algorithms (GAs) in which a population of GAs in any generation is represented by a single vector whose elements are the probabilities of the corresponding bit positions being equivalent to 1. The process of evolution is represented by learning the elements of the probability vector; the method is clearly linked to the artificial neural network (ANN) method of competitive learning. We use techniques from ANNs to extend the applicability of the method to non-static problems, to multi-objective criteria, to multi-modal problems and to creating an order on a set of sub-populations.
Developing a Community Language

We describe simulations of a community of agents who live in an environment which has some structure that the agents can learn to identify and subsequently about which they learn to communicate. Each agent has two entirely separate arti-cial neural networks which learn to perform the two tasks: identiication of the structure in the environment and communication to others about the environment's structure. We show that a community of agents with similar representation capabilities is most successful in generating a common language and further that a community with diierent representation capabilities will evolve so that all members have the same representation capabilities.