nils-herrmann/clean_scirepeval huggingface dataset:
Feature extraction and fusion of wideband backscattered signals

Good discrimination result have been obtained with an active backscatter data set of mine-like objects, where the task as to distinguish between man-made and non-man-made objects. In this work we introduce a novel method for constructing best basis for discrimination from wavelet packets, and demonstrate the superiority of multiple ensembles of predictors. We achieve far better discrimination results using a wide band FM sweep of 80kHz compared with the earlier work that used a 40kHz FM sweep. We further show improved results by combining several discriminative methods with several wavelet packet representations.
Solutions of the BCM learning rule in a network of lateral interacting nonlinear neurons

We introduce a new method for obtaining the fixed points for neurons that follow the BCM learning rule. The new formalism, which is based on the objective function formulation, permits analysis of a laterally connected network of nonlinear neurons and allows explicit calculation of the fixed points under various network conditions. We show that the stable fixed points, in terms of the postsynaptic activity, are not altered by the lateral connectivity or nonlinearity. We show that the lateral connectivity alters the probability of attaining different states in a network of interacting neurons. We further show the exact alteration in presynaptic weights as a result of the neuronal nonlinearity.
Face Detection and Camouflage Breaking by Direct Convexity Estimation

Edge detection was, so far, the core of most state of the art techniques for attentional mechanisms as well as face detection.1,2 This excludes some recent works which utilize neural networks,3,4 colour histograms,5,6 or shape statistics7,8 for face detection. Though one cannot disregard their advantages, edge maps sustain severe flaws such as: sensitivity to changes in illumination, strong effect of surrounding objects, and inability to delineate objects in a cluttered scene. We overcome these problems of edge-based approaches by a novel attentional operator which detects smooth three dimensional convex or concave objects in the image. The operator is robust to face orientation, scale, and illumination, and is capable of detecting the subject in a strongly textured background. It is employed for face detection, namely to detect the eyes and hair, from which the scale of the face can be inferred. The operator answers the above problems as a whole, demands a relatively short running time, and its robustness leads to reliable results.
Boosted Mixture of Experts: An Ensemble Learning Scheme

We present a new supervised learning procedure for ensemble machines, in which outputs of predictors, trained on different distributions, are combined by a dynamic classifier combination model. This procedure may be viewed as either a version of mixture of experts (Jacobs, Jordan, Nowlan, & Hinton, 1991), applied to classification, or a variant of the boosting algorithm (Schapire, 1990). As a variant of the mixture of experts, it can be made appropriate for general classification and regression problems by initializing the partition of the data set to different experts in a boostlike manner. If viewed as a variant of the boosting algorithm, its main gain is the use of a dynamic combination model for the outputs of the networks. Results are demonstrated on a synthetic example and a digit recognition task from the NIST database and compared with classical ensemble approaches.
Feature extraction and fusion of wide-band backscattered signals

Good discrimination results have been obtained with an active backscatter data set of minelike objects [1], where the task was to distinguish between man-made and non-man-made objects. In this work we introduce a novel method for constructing best basis for discrimination from wavelet packets, and demonstrate the superiority of multiple ensembles of predictors. We achieve far better discrimination results using a wide band FM sweep of 80kHz compared with the earlier work that used a 40kHz FM sweep. We further show improved results by combing several discriminative methods with several wavelet packet representations.
Image enhancement for pattern recognition

We investigate various image enhancement techniques geared towards a specific detector. Our database consists of side- scan sonar images collected at the Naval Surface Warfare Center (NSWC), and the detector we use has proven to have excellent results on these data. We start by investigating various wavelet and wavelet packet denoising methods. Other methods we consider are based on more common filters (Gaussian and DOG filters). In wavelet based denoising we try different approaches, combining techniques that have been successfully used in signal and image denoising. We notice that the performance is mostly affected by the choice of the scale levels to which shrinkage is applied. We demonstrate that wavelet denoising can significantly improve detection performance while keeping low false alarm rates.
Classification of seismic signals by integrating ensembles of neural networks

We examine a classification problem in which seismic waveforms of natural earthquakes are to be distinguished from waveforms of man-made explosions. We present an integrated classification machine (ICM), which is a hierarchy of artificial neural networks (ANNs) that are trained to classify the seismic waveforms. In order to maximize the gain of combining the multiple ANNs, we suggest construction of a redundant classification environment (RCE) that consists of several "experts" whose expertise depends on the different input representations to which they are exposed. In the proposed scheme, the experts are ensembles of ANN, trained on different bootstrap replicas. We use various network architectures, different time-frequency decompositions of the seismic waveforms, and various smoothing levels in order to achieve an RCE. A confidence measure for the ensemble's classification is defined based on the agreement (variance) within the ensembles, and an algorithm for a nonlinear integration of the ensembles using this measure is presented. An implementation on a data set of 380 seismic events is described, where the proposed ICM had classified correctly 92% of the testing signals. The comparison we made with classical methods indicates that combining a collection of ensembles of ANNs can be used to handle complex high dimensional classification problems.
Classification of underwater mammals using feature extraction based on time-frequency analysis and BCM theory

Underwater mammal sound classification is demonstrated using a novel application of wavelet time-frequency decomposition and feature extraction using a Bienenstock, Cooper, and Munro (1982) (BCM) unsupervised network. Different feature extraction methods and different wavelet representations are studied. The system achieves outstanding classification performance even when tested with mammal sounds recorded at very different locations (from those used for training). The improved results suggest that nonlinear feature extraction from wavelet representations outperforms different linear choices of basis functions.
Receptive eld formation in natural scene environments: comparison of kurtosis, skewness, and the quadratic form of BCM

We study several statistically and biologically motivated learning rules using the same visual environment and neuronal architecture. This allows us to concentrate on the feature extraction and neuronal coding properties of these rules. We find that the quadratic form of the BCM rule behaves in a manner similar to a kurtosis maximization rule when the distribution contains kurtotic directions, although the BCM modification equations are computationally simpler.
Receptive Field Formation in Natural Scene Environments: Comparison of Single-Cell Learning Rules

We study several statistically and biologically motivated learning rules using the same visual environment: one made up of natural scenes and the same single-cell neuronal architecture. This allows us to concentrate on the feature extraction and neuronal coding properties of these rules. Included in these rules are kurtosis and skewness maximization, the quadratic form of the Bienenstock-Cooper-Munro (BCM) learning rule, and single-cell independent component analysis. Using a structure removal method, we demonstrate that receptive fields developed using these rules depend on a small portion of the distribution. We find that the quadratic form of the BCM rule behaves in a manner similar to a kurtosis maximization rule when the distribution contains kurtotic directions, although the BCM modification equations are computationally simpler.
Information theory and visual plasticity

The relevance of information theory to neural networks has become more apparent in recent years. This theory has become important in analysing and understanding the nature of the neuronal code that is relayed between cortical layers and the nature of the learning goals that guide neuronal learning and synaptic modification. The rapid advance in recent years of single and multiple electrode recording as well as other non-invasive techniques provides a window on neuronal activity and synaptic modification. However, the puzzle is still unsolved; we do not know what are neuronal learning goals, how they are being incorporated and most importantly, what is the nature of the neuronal code and how it is being formed and interpreted by successive layers. When we try to understand the nature of synaptic learning rules, we should be concerned with possible goals that may underly synaptic changes. It is conceivable that knowing what could be a useful goal under different input environments could serve for distinguishing between synaptic plasticity theories. Only after this distinction between goals is indicated, can one continue further and distinguish between learning rules aimed at achieving the same objective, on the basis of their detailed mathematical properties, or computational complexity etc. In this paper, we concentrate on the analysis of different neuronal goals and attempt to provide an update on the current related issues and future directions.
Conservation of total T-cell counts during HIV infection: alternative hypotheses and implications.

While CD4+ T-cell counts in the blood of HIV-infected individuals gradually decrease, there is a parallel increase in the number of blood CD8+ T cells such that the total number of T cells remains essentially constant for several years (1). The basis and significance of this phenomenon are not known. Based on a statistical analysis of longitudinal T-cell counts from the Transfusion Safety Study (TSS) database and on theoretical considerations, we evaluate several alternative models, including versions of the "blind homeostasis" (BH) hypothesis (1-3). At issue is the nature of the homeostatic regulation of lymphocytes and its apparent failure in HIV infection. The most plausible explanation for the conservation of total blood T-cell numbers while subset ratios change is that CD4+ and CD8+ T cells compete for a limited access to the blood compartment. Such interaction between the subsets implies, in particular, that changes in the number of CD4+ T cells occurring in other tissues cannot be reliably inferred from those observed in the blood. We reiterate propositions made earlier (4) that much of the apparent "depletion" of CD4+ lymphocytes during the asymptomatic phase of HIV infection may be attributed to redistribution between the tissues and the blood compartment.
Making a Low-Dimensional Representation Suitable for Diverse Tasks

We introduce a new approach to the training of classifiers for performance on multiple tasks. The proposed hybrid training method leads to improved generalization via a better low-dimensional representation of the problem space. The quality of the representation is assessed by embedding it in a 2D space using multidimensional scaling, allowing a direct visualization of the results. The performance of the approach is demonstrated on a highly nonlinear image classification task.
Feature extraction from acoustic backscattered signals using wavelet dictionaries

Discrimination problems differ in nature from reconstruction tasks. While in reconstruction, it is the mean squared error that is often used to measure the quality of the scheme, classification requires a different measure which often is not related to the former. The discrimination power of a certain basis or a set of basis function is not necessarily connected to the quality of reconstruction associated with this set. Furthermore, the degree of relevance of the orthonormality constraint to the quality of the discrimination is questionable. For example, linear discriminant analysis  searches for linear projections which maximize the between-class variance divided by the sum of within-class variance. Such projections do not necessarily coincide with the principal components of the data which are the directions that optimize MSE reconstruction. A successful approach to discrimination is based on an appropriate preprocessing to create an efficient signal representation, which then leads to an efficient dimensionality reduction. The next step is again some combination of feature extraction and classification. In this paper we briefly review several methods for finding data representation via optimal decomposition of wavelet basis functions and discuss their reconstruction properties. We then discuss some signal decomposition methods for the purpose of discrimination. This is followed by a brief discussion on a combination of feature extraction and classification scheme and with discrimination results on two acoustic data-sets.
Effect of binocular cortical misalignment on networks of BCM and OJA neurons

A two-eye visual environment, composed of natural images, is used in training a network of interacting BCM and Oja (PCA) neurons. This work is an extends our previous single cell model [10] to networks of interacting neurons. We study the effect of misalignment, between the synaptic density functions connecting both eyes to each single neuron, on the formation of orientation selectivity and ocular dominance. We show that for the BCM rule a natural image environment with binocular cortical misalignment is sufficient for producing networks of orientation selective cells with varying ocular dominance. Oja neurons in contrast are always perfectly binocular.
Robust Interpretation of Neural-Network Models

Na than Intrator Tel-Aviv University and Brown University nin©cns.brown.edu Artificial Neural Network seem very promising for regression and classification, especially for large covariate spaces. These methods represent a non-linear function as a composition of low dimensional ridge functions and therefore appear to be less sensitive to the dimensionality of the covariate space. However, due to non uniqueness of a global minimum and the existence of (possibly) many local minima, the model revealed by the network is non stable. We introduce a method to interpret neural network results which uses novel robustification techniques. This results in a robust interpretation of the model employed by the network. Simulated data from known models is used to demonstrate the interpretability results and to demonstrate the effects of different regularization methods on the robustness of the model. Graphical methods are introduced to present the interpretation results. We further demonstrate how interaction between covariates can be revealed. From this study we conclude that the interpretation method works well, but that NN models may sometimes be misinterpreted, especially if the approximations to the true model are less robust.
Learning Low Dimensional Representations of Visual Objects With Extensive Use of Prior Knowledge

Learning to recognize visual objects from examples requires the ability to find meaningful patterns in spaces of very high dimensionality. We present a method for dimensionality reduction which effectively biases the learning system by combining multiple constraints via an extensive use of class labels. The use of multiple class labels steers the resulting low-dimensional representation to become invariant to those directions of variation in the input space that are irrelevant to classification; this is done merely by making class labels independent of these directions. We also show that prior knowledge of the proper dimensionality of the target representation can be imposed by training a multiple-layer bottleneck network. A series of computational experiments involving parameterized fractal images and real human faces indicate that the low-dimensional representation extracted by our method leads to improved generalization in the learned tasks, and is likely to preserve the topology of the original space.
Complex cells and Object Recognition

Nearest-neighbor correlation-based similarity computation in the space of outputs of complex-type receptive fields can support robust recognition of 3D objects. Our experiments with four collections of objects resulted in mean recognition rates between 84% (for subordinate-level discrimination among 15 quadruped animal shapes) and 94% (for basic-level recognition of 20 everyday objects), over a 40deg X 40deg range of viewpoints, centered on a stored canonical view and related to it by rotations in depth (comparable figures were obtained for image-plane translations). This result has interesting implications for the design of a front end to an artificial object recognition system, and for the understanding of the faculty of object recognition in primate vision.
Face Detection by Direct Convexity Estimation

We suggest a novel attentional mechanism for detection of smooth convex and concave objects based on direct processing of intensity values. The operator detects the regions of the eyes and hair in a facial image, and thus allows us to infer the face location and scale. Our operator is robust to variations in illumination, scale, and face orientation. Invariance to a large family of functions, serving for lighting improvement in images, is proved. An extensive comparison with edgebased methods is delineated.
Optimal ensemble averaging of neural networks

Based on an observation about the different effect of ensemble averaging on the bias and variance portions of the prediction error, we discuss training methodologies for ensembles of networks. We demonstrate the effect of variance reduction and present a method of extrapolation to the limit of an infinite ensemble. A significant reduction of variance is obtained by averaging just over initial conditions of the neural networks, without varying architectures or training sets. The minimum of the ensemble prediction error is reached later than that of a single network. In the vicinity of the minimum, the ensemble prediction error appears to be flatter than that of the single network, thus simplifying optimal stopping decision. The results are demonstrated on sunspots data, where the predictions are among the best obtained, and on the 1993 energy prediction competition data set B.†School of Physics and Astronomy. ury@tarazan.tau.ac.il.‡School of Mathematical Sciences. nin@math.tau.ac.il.§School of Physics and ...
Learning low-dimensional representations via the usage of multiple-class labels

Learning to recognize visual objects from examples requires the ability to find meaningful patterns in spaces of very high dimensionality. We present a method for dimensionality reduction which effectively biases the learning system by combining multiple constraints via the use of class labels. The use of extensive class labels steers the resulting low-dimensional representation to become invariant to those directions of variation in the input space that are irrelevant to classification; this is done merely by making class labels independent of these directions. We also show that prior knowledge of the proper dimensionality of the target representation can be imposed by training a multi-layer bottleneck network. Computational experiments involving non-trivial categorization of parameterized fractal images and of human faces indicate that the low-dimensional representation extracted by our method leads to improved generalization in the learned tasks and is likely to preserve the topology of the original sp...
Bootstrapping with Noise: An Effective Regularization Technique

Bootstrap samples with noise are shown to be an effective smoothness and capacity control technique for training feedforward networks and for other statistical methods such as generalized additive models. It is shown that noisy bootstrap performs best in conjunction with weight-decay regularization and ensemble averaging. The two-spiral problem, a highly non-linear, noise-free data, is used to demonstrate these findings. The combination of noisy bootstrap and ensemble averaging is also shown useful for generalized additive modelling, and is also demonstrated on the well-known Cleveland heart data.
Classification of seismic waveforms by integrating ensembles of neural networks

The problem considered is the discrimination between natural and artificial seismic events, based on their waveform recording. We build a classification environment consists of several ensembles of neural networks trained on boot-strap sample sets, using various data representations and architectures. The integration of the different ensembles is made in a nonconstant signal adaptive manner, using a posterior confidence measure based on the agreement (variance) within the ensembles. The proposed integrated classification machine achieved 92.1% correct classification on the seismic test data. Cross validation tests and comparisons indicate that such integration of a collection of ANN's ensembles is a robust way for handling high dimensional problems with a complex nonstationary signal space as in the current seismic classification problem.
Effect of Binocular Cortical Misalignment on Ocular Dominance and Orientation Selectivity

We model a two-eye visual environment composed of natural images and study its effect on single cell synaptic modification. In particular, we study the effect of binocular cortical misalignment on receptive field formation after eye opening. We show that binocular misalignment affects principal component analysis (PCA) and Bienenstock, Cooper, and Munro (BCM) learning in different ways. For the BCM learning rule this misalignment is sufficient to produce varying degrees of ocular dominance, whereas for PCA learning binocular neurons emerge in every case.
Classification of seismic waveforms by integrating ensembles of neural networks

The problem considered is the discrimination between natural and artificial seismic events, based on their waveform recording. We build a classification environment consists of several ensembles of neural networks trained on bootstrap sample sets, using various data representations and architectures. The integration of the different ensembles is made in a non-constant signal adaptive manner, using a posterior confidence measure based on the agreement (variance) within the ensembles. The proposed integrated classification machine achieved 92.1% correct classification on the seismic test data. Cross validation tests and comparisons indicate that such integration of a collection of ANN's ensembles is a robust way for handling high dimensional problems with a complex non-stationary signal space as in the current seismic classification problem.
Bootstrapping with Noise : An E ective Regularization TechniqueYuval

Bootstrap samples with noise are shown to be an eeective smoothness and capacity control technique for training feed-forward networks and for other statistical methods such as generalized additive models. It is shown that noisy bootstrap performs best in conjunction with weight decay regularization and ensemble averaging. The two-spiral problem, a highly non-linear noise-free data, is used to demonstrate these ndings. The combination of noisy bootstrap and ensemble averaging is also shown useful for generalized additive modeling, and is also demonstrated on the well known Cleveland Heart Data 7].