nils-herrmann/clean_scirepeval huggingface dataset:
The robustness of the p-norm algorithms

We consider two on-line learning frameworks: binary classification through linear threshold functions and linear regression. We study a family of on-line algorithms, called p-norm algorithms, introduced by Grove, Littlestone and Schuurmans in the context of deterministic binary classification. We show how to adapt these algorithms for use in the regression setting, and prove worst-case bounds on the square loss, using a technique from Kivinen and Warmuth. As pointed out by Grove, et al., these algorithms can be made to approach a version of the classification algorithm Winnow as p goes to infinity; similarly they can be made to approach the corresponding regression algorithm EG in the limit. Winnow and EG are notable for having loss bounds that grow only logarithmically in the dimension of the instance space. Here we describe another way to use the p-norm algorithms to achieve this logarithmic behavior. With the way to use them that we propose, it is less critical than with Winnow and EG to retune the parameters of the algorithm as the learning task changes. Since the correct setting of the parameters depends on characteristics of the learning task that are not typically known a priori by the learner, this gives the p-norm algorithms a desireable robustness. Our elaborations yield various new loss bounds in these on-line settings. Some of these bounds improve or generalize known results. Others are incomparable.
Improved lower bounds for learning from noisy examples: an information-theoretic approach

This paper presents a general information-theoretic approach for obtaining lower bounds on the number of examples needed to PAC learn in the presence of noise. This approach deals directly with the fundamental information quantities, avoiding a Bayesian analysis. The technique is applied to several different models, illustrating its generality and power. The resulting bounds add logarithmic factors to (or improve the constants in) previously known lower bounds.
Linear Hinge Loss and Average Margin

We describe a unifying method for proving relative loss bounds for online linear threshold classification algorithms, such as the Perceptron and the Winnow algorithms. For classification problems the discrete loss is used, i.e., the total number of prediction mistakes. We introduce a continuous loss function, called the "linear hinge loss", that can be employed to derive the updates of the algorithms. We first prove bounds w.r.t. the linear hinge loss and then convert them to the discrete loss. We introduce a notion of "average margin" of a set of examples. We show how relative loss bounds based on the linear hinge loss can be converted to relative loss bounds i.t.o. the discrete loss using the average margin.
Max Horn SAT and the minimum cut problem in directed hypergraphs

In this paper we consider the Maximum Horn Satisfiability problem, which is reduced to the problem of finding a minimum cardinality cut on a directed hypergraph. For the latter problem, we propose different IP formulations, related to three different definitions of hyperpath weight. We investigate the properties of their linear relaxations, showing that they define a hierarchy. The weakest relaxation is shown to be equivalent to the relaxation of a well known IP formulation of Max Horn SAT, and to a max-flow problem on hypergraphs. The tightest relaxation, which is a disjunctive programming problem, is shown to have integer optimum. The intermediate relaxation consists in a set covering problem with a possible exponential number of constraints. This latter relaxation provides an approximation of the convex hull of the integer solutions which, as proven by the experimental results given, is much tighter than the one known in the literature.
SART: A system for supporting operators with contextual knowledge

The regulation of the subway line traffic is a domain where human controllers must deal with a huge quantity of knowledge pieces more or less implicit in the regulation itself. When an incident occurs on a subway line, the operator--the controller who has currently the responsibility of the subway line--must choose the best strategy to apply for moving from the incidental context to the operational one. An incident on the subway line may cause traffic delay or service interruption, and may last for a long or short time, depending on the nature of the incident and many other elements. Operators mainly focus on contextual information for incident solving. An operator said that "When an incident occurs, I first look at what the incident context is." We propose to support subway line traffic controllers in incident-solving with the SART system (French acronym for support system in the traffic regulation). SART is a decision support system based on the contextual analysis of events that arise at the time of the incident. It uses a context-based representation of incidents and applies a context-based reasoning. We discuss in this paper a context-based representation of incidents on the basis of the onion metaphor. The SART project now enters the second year of system design and development and implies two universities and two subway companies in France and Brazil.
A neuromorphic architecture for cortical multilayer integration of early visual tasks

The characteristics and performance of a hierarchical neural architecture, inspired by models of mammalian visual cortex, are considered. The visual pathway from sensory space to the intermediate (cortical) representation is structured in three layers, with intra and interlayer connections through feedforward and recurrent pathways. These interconnections provide a complex perceptual organization that integrates the specific functional tasks performed by each layer. This improves the capabilities of the architecture in feature extraction and segregation, further providing clues on the information content of the intermediate representation (primal sketch). Applications to preattentive vision tasks (edge and contour extractions, texture analysis and boundary completion, and defect detection) are presented with satisfactory results.
Form specifies function: robust spike-based computation in analog VLSI without precise synaptic weights

Recent studies demonstrated that sophisticated information processing can occur in spike-based computational systems that make use of synapses with only two states (potentiated or depressed). Here we present the hybrid software/hardware implementation of a model of the mammalian olfactory bulb using an analog VLSI device comprising an array of integrate and fire neurons with bistable synapses. Our implementation incorporates both software and hardware components, integrated using an asynchronous event-based spike representation. The model is able to perform highly selective simulated odor recognition, using induced synchronization within a population of neurons as the key to computation. The success of this scheme shows that the analog VLSI circuits used can perform sophisticated computation, taking advantage of the neuron dynamics and the topology of the network, without requiring precise analog synaptic weights.
The Cerebellum Chip: an Analog VLSI Implementation of a Cerebellar Model of Classical Conditioning

We present a biophysically constrained cerebellar model of classical conditioning, implemented using a neuromorphic analog VLSI (aVLSI) chip. Like its biological counterpart, our cerebellar model is able to control adaptive behavior by predicting the precise timing of events. Here we describe the functionality of the chip and present its learning performance, as evaluated in simulated conditioning experiments at the circuit level and in behavioral experiments using a mobile robot. We show that this aVLSI model supports the acquisition and extinction of adaptively timed conditioned responses under real-world conditions with ultra-low power consumption.
Characterizing the Firing Properties of an Adaptive Analog VLSI Neuron

We describe the response properties of a compact, low power, analog circuit that implements a model of a leaky–Integrate & Fire (I&F) neuron, with spike-frequency adaptation, refractory period and voltage threshold modulation properties. We investigate the statistics of the circuit’s output response by modulating its operating parameters, like refractory period and adaptation level and by changing the statistics of the input current. The results show a clear match with theoretical prediction and neurophysiological data in a given range of the parameter space. This analysis defines the chip’s parameter working range and predicts its behavior in case of integration into large massively parallel very–large–scale–integration (VLSI) networks.
A VLSI reconfigurable network of integrate-and-fire neurons with spike-based learning synapses

We present a VLSIdevice comprising an array of leaky integrate-and- fire (I&F) neurons and adaptive synapses with spike-timing dependent plasticity (STDP). The neurons transmit spikes off chip and the synapses receive spikes from external devices using a communication protocol based on the "Address- Event Representation" (AER). We studied the response properties of the neurons in the array to uniform input currents, and measured their AER outputs. We char- acterized the properties of the STDP synapses using AER input spike trains. Our results indicate that these circuits can be reliably used in massively parallel VLSI networks of I&F neurons to simulate real-time complex spike-based learning al- gorithms.
Robot Soccer using Optical Analog VLSI Sensors

In this paper we show how a combination of low dimensional vision sensors can be used to aid the higher level visual processing task of colour blob tracking, carried out by a conventional vision system. The processing elements are neuromorphic analog VLSI (aVLSI) vision sensors capable of computing motion and estimating the spatial position of high-contrast moving targets. In these devices both sensing and processing is done on the chip’s focal plane. The neuromorphic sensors can calculate optical flow, position of sharpest edge, and motion of sharpest edge, in real-time. The processing capability of the system is investigated in a mobile robotics application. Firstly, for visualization and evaluation purposes, a correlation analysis is performed between the data collected from the neuromorphic vision sensors and the standard vision system of the autonomous robot, then, we process the multiple neuromorphic sensory signals with a standard auto-regression method in order to achieve a higher level vision processing task at a much higher update rate. At the end we argue why this result is of great relevance for the application domain of reactive and lightweight mobile robotics, at the hands of a soccer robot, where the fastest sensory-motor feedback loop is imperative for a successful participation in a RoboCup soccer competition.
An event-based VLSI network of integrate-and-fire neurons

The growing interest in pulse-based neural networks is encouraging the development of hardware implementations of massively parallel, distributed networks of integrate-and-fire (I&F) neurons. We have developed a mixed-mode (analog/digital) VLSI device that comprises a reconfigurable network of I&F neurons and adaptive synapses. The synapses receive input spikes and the neurons transmit output spikes (events) using an asynchronous address-event representation (AER). We describe the network architecture, present experimental data demonstrating the characteristics of the single elements on the chip, and show that a competitive network configuration has winner-take-all (WTA) behaviour and produces spike synchronization.
Firing proprieties of an adaptive analog VLSI neuron

We describe the response properties of a compact, low power, analog circuit that implements a model of a leaky I&F neuron, with spike-frequency adaptation, refractory period and voltage threshold modulation properties. We investigate the statistics of the circuit's output response by modulation its operating parameters, like refractory period and adaptation level and by changing the statistics of the input current. The results show a clear match with theoretical and neurophysiological data in a given range of the parameter space. This analysis defines the chip's parameter working range and predicts its behavior in case of integration into large massively parallel VLSI networks.
Object Tracking Using Multiple Neuromorphic Vision Sensors

In this paper we show how a combination of multiple neuromorphic vision sensors can achieve the same higher level visual processing tasks as carried out by a conventional vision system. We process the multiple neuromorphic sensory signals with a standard auto-regression method in order to fuse the sensory signals and to achieve higher level vision processing tasks at a very high update rate. We also argue why this result is of great relevance for the application domain of reactive and lightweight mobile robotics, at the hands of a soccer robot, where the fastest sensory-motor feedback loop is imperative for a successful participation in a RoboCup soccer competition.
A low-power adaptive integrate-and-fire neuron circuit

We present a low-power analog circuit for implementing a model of a leaky integrate and fire neuron. Next to being optimized for low-power consumption, the proposed circuit includes elements for implementing spike frequency adaptation, for setting an arbitrary refractory period and for modulating the neuron's threshold voltage. We present experimental data from a prototype chip, implemented using a standard 1.5 /spl mu/m CMOS process.
Guest editorial - Special issue on neural networks hardware implementations

S INCE the very beginning of the neural network era, there has been the belief that to fully exploit the potential of this technology it would be necessary to also develop efficient hardware implementation techniques. In the last two decades, we have witnessed how the neural networks community is managing to acquire a more profound understanding of what neural networks can do and how. Neural networks knowledge has been enriched by developing more solid mathematical frameworks, by elaborating more efficient and powerful algorithms, by unfolding mysteries behind biological neural networks both at the individual neuron level and at different hierarchical levels of neuron ensembles and brain functions. In the last decade, we have seen how the initial " euphoria " of neural network expectations has settled down to a solid knowledge and research capable of undertaking complex problems and providing powerful solutions. In the field of hardware implementations , progress has been continuing as well. The challenge has been and still is to follow what biology does and how, to be able to build more complicated and larger size systems, to implement efficient hardware learning techniques, to improve speed and/or lower power consumptions, to efficiently exploit or develop new signal/state representation techniques (e.g., continuous time , pulse based, spiking nature, ...), to exploit noise and statistical properties, etc., and to apply all these ideas in real-world applications where efficient hardware solutions are necessary to provide compactness, high-speed and low-power consumptions. In recent years significant progress in hardware solutions is being achieved. For this reason we believe it is a good point in time for a Special Issue on Neural Networks Hardware Implementations for the IEEE TRANSACTIONS ON NEURAL NETWORKS. The Special Issue Call for Papers had an overwhelming response. Originally, 70 papers were considered for possible publication. From those, 35 made it finally into the Special Issue. The review process was carried out by splitting the load between the four guest editors. The review process went quite smoothly thanks to the close cooperation between guest editors and the fast efficient response of reviewers. The final set of papers can be grossly classified into two main groups of similar size: the mainly digital papers and the mainly analog papers. Within the analog group are included also the mixed Analog–Digital implementation papers. Besides this main classification the subsequent subgroups are many times fairly arbitrary, because most of the times the possible classifications …
Neuromorphic Bisable VLSI Synapses with Spike-Timing-Dependent Plasticity

We present analog neuromorphic circuits for implementing bistable synapses with spike-timing-dependent plasticity (STDP) properties. In these types of synapses, the short-term dynamics of the synaptic efficacies are governed by the relative timing of the pre- and post-synaptic spikes, while on long time scales the efficacies tend asymptotically to either a potentiated state or to a depressed one. We fabricated a prototype VLSI chip containing a network of integrate and fire neurons interconnected via bistable STDP synapses. Test results from this chip demonstrate the synapse's STDP learning properties, and its long-term bistable characteristics.
Silicon and Transistors

Neuromorphic engineers work to improve the performance of artificial systems through the development of chips and systems that process information collectively using primarily analog circuits. This book presents the central concepts required for the creative and successful design of analog VLSI circuits. The discussion is weighted toward novel circuits that emulate natural signal processing. Unlike most circuits in commercial or industrial applications, these circuits operate mainly in the subthreshold or weak inversion region. Moreover, their functionality is not limited to linear operations, but also encompasses many interesting nonlinear operations similar to those occurring in natural systems. Topics include device physics, linear and nonlinear circuit forms, translinear circuits, photodetectors, floating-gate devices, noise analysis, and process technology.
Silicon Retina Sensing guided by Omni-directional Vision

A way of combining a relatively new sensor-technology, that is optical analog VLSI devices, with a standard digital omni-directional vision system is investigated. The sensor used is a neuromorphic analog VLSI sensor that estimates the global visual image motion. The sensor provides two analog output voltages that represent the components of the global optical flow vector. The readout is guided by an omni-directional mirror that maps the location of the ball and directs the robot to align its position so that a sensor-actuator module that includes the analog VLSI optical flow sensor can be activated. The purpose of the sensoractuator module is to operate with a higher update rate than the standard vision system and thus increase the reactivity of the robot for very specific situations. This paper will demonstrate an application example where the robot is a goalkeeper with the task of defending the goal during a penalty kick.
Circuits for bistable spike-timing-dependent plasticity neuromorphic VLSI synapses

We present analog neuromorphic circuits for implementing bistable synapses with spike-timing-dependent plasticity (STDP) properties. In these types of synapses, the short-term dynamics of the synaptic efficacies are governed by the relative timing of the preand post-synaptic spikes, while on long time scales the efficacies tend asymptotically to either a potentiated state or to a depressed one. Test results from a prototype VLSI chip, containing a learning synapse connected to a preand post-synaptic integrate and fire neuron, demonstrate the synapse’s STDP learning properties, and its long-term bistable characteristics.
An adaptive visual tracking sensor with a hysteretic winner-take-all network

We present a CMOS VLSI image sensor containing a 26/spl times/26 array of differentiating adaptive photoreceptors, combined with a current-mode hysteretic winner-take-all (WTA) network, able to detect and track high-contrast targets moving within its field of view. The device, implemented using standard 0.8 /spl mu/m CMOS technology, dissipates 600 /spl mu/W, with a power supply voltage of 5 V. Its WTA outputs can be read out using either analog position-to-voltage circuits, or digital address encoders. Photoreceptor voltages with sustained and transient responses can be read out serially using an on-chip scanner, or in a random-access mode using address decoders. Combining digital output position encoders and input address decoders, it is possible to read only the location of the winning pixel, or a small window around the winning pixel (e.g. using a microcontroller), thus implementing a selective attention read-out mechanism.
Neuromorphic Bistable VLSI Synapses with Spike-Timing-Dependent Plasticity

We present analog neuromorphic circuits for implementing bistable synapses with spike-timing-dependent plasticity (STDP) properties. In these types of synapses, the short-term dynamics of the synaptic efficacies are governed by the relative timing of the preand post-synaptic spikes, while on long time scales the efficacies tend asymptotically to either a potentiated state or to a depressed one. We fabricated a prototype VLSI chip containing a network of integrate and fire neurons interconnected via bistable STDP synapses. Test results from this chip demonstrate the synapse’s STDP learning properties, and its long-term bistable characteristics.
Neuromorphic Sensory Processing using Analog Very Large-Scale Integration

Neuromorphic Sensory Processing Using Analog Very Large-Scale Integration Circuits Neuromorphic circuits are analog circuits that implement models of biological systems for sensory processing. Because the neuromorphic circuits share the same physical constraints as their biological counterparts, they have similar organizational structures and use similar strategies for optimizing robustness to noise and power consumption. Silicon retinas and other neuromorphic vision chips, built as hardware models of biological vision systems, represent efficient artificial sensory preprocessors. General processing networks which use detailed models of neurons are also being investigated. Simple neuromorphic systems have been built using networks of silicon neurons and neuromorphic chips as front ends for preprocessing incoming sensory signals.
Neuromorphic vision sensors for mobile robots

Real-time vision-based vehicle navigation tasks are typically computationally intensive and significantly complex in terms of resources used. Moreover, mobile applications of navigation systems place severe constraints on their size and power consumption. These constraints can be satisfied by using parallel image processing architectures and parallel control algorithms implemented with analog VLSI circuits. In this paper we present two examples of neuromorphic VLSI sensors that process motion signals to provide continuous time analog output signals that can be useful for real-time navigation tasks. Neuromorphic analog VLSI vision sensors are typically compact, low-power devices that implement parallel architectures to perform in real-time, image (pre)processing operations and provide results that would otherwise require CPU intensive computations . The two neuromorphic sensors we propose in this paper measure respectively the scene’s 2D optical flow field and the target’s position and its direction of motion in one dimension.