nils-herrmann/clean_scirepeval huggingface dataset:
FIR approximation of fractional sample delay systems

This brief examines the approximation of fractional delays by FIR systems using various techniques which have previously been reported in the literature. In particular, the equivalence of the time-domain Lagrangian interpolation, the frequency-domain maximally flat error criterion and the window method is shown, provided maximal flatness is specified at /spl omega//sub 0/=0 and the window used is a scaled binomial function. The first of these has been known before, the second equivalence is new.
The importance of convexity in learning with squared loss

We show that if the closure of a function class under the metric induced by some probability distribution is not convex, then the sample complexity for agnostically learning with squared loss (using only hypotheses in )i s where is the probability of success and is the required accuracy. In comparison, if the class is convex and has finite pseudodimension, then the sample complexity is . If a nonconvex class has finite pseudodimension, then the sample complexity for agnostically learning the closure of the convex hull of ,i s . Hence, for agnostic learning, learning the convex hull provides better approximation capabilities with little sample complexity penalty.
Local minima and attractors at infinity for gradient descent learning algorithms

In the paper Learning Nonlinearly Parametrized Decision Re gions an online scheme for learning a very general class of decision regions is given together with conditions on both the parametriza tion and on the sequence of input examples under which good learn ing can be guaranteed to occur In this paper we discuss these condi tions in particular the requirement that there be no non global local minima of the relevant error function and the more speci c problem of no attractor at in nity Somewhat simpler su cient conditions are given A number of examples are discussed
Efficient agnostic learning of neural networks with bounded fan-in

We show that the class of two-layer neural networks with bounded fan-in is efficiently learnable in a realistic extension to the probably approximately correct (PAC) learning model. In this model, a joint probability distribution is assumed to exist on the observations and the learner is required to approximate the neural network which minimizes the expected quadratic error. As special cases, the model allows learning real-valued functions with bounded noise, learning probabilistic concepts, and learning the best approximation to a target function that cannot be well approximated by the neural network. The networks we consider have real-valued inputs and outputs, an unlimited number of threshold hidden units with bounded fan-in, and a bound on the sum of the absolute values of the output weights. The number of computation steps of the learning algorithm is bounded by a polynomial in 1//spl epsiv/, 1//spl delta/, n and B where /spl epsiv/ is the desired accuracy, /spl delta/ is the probability that the algorithm fails, n is the input dimension, and B is the bound on both the absolute value of the target (which may be a random variable) and the sum of the absolute values of the output weights. In obtaining the result, we also extended some results on iterative approximation of functions in the closure of the convex hull of a function class and on the sample complexity of agnostic learning with the quadratic loss function.
Frequency invariant beamforming with exact null design

This paper extends earlier results by Ward, Kennedy and Williamson (1995, 1996) for the design of broadband arrays with frequency-invariant (FI) beam patterns to the case where it is desired to place an exact null in a given direction. The beamforming is done using appropriately selected FIR filters. First, the previous results for generating FI beam patterns using FIR filters are briefly summarised. Second, new results which give the conditions required for exact nulls in the beam pattern for all frequencies in any, possibly non-FI, beam pattern are given. Third, a method of generating beam patterns which possess an exact null and which are close, in an L/sub 2/ sense, to an arbitrary FI pattern is presented. Finally, some preliminary experimental results which corroborate the theoretical findings are presented.
Hasty Congregational Gradient Descent for Online Optimisation

Stepwise Gradient Descent (SGD) algorithms for online optimization converge to local minima of the relevant cost function. In this paper a globally convergent modification of SGD is proposed, in which several solutions of SGD are run in parallel, together with online estimates of the cost function and its gradient. As each SGD estimate reaches a local minimum of the cost, the fitness of the member is evaluated and the member is immediately restarted unless it is the current best estimate. A number of results concerning the convergence behaviour of the proposed algorithm are derived using results from dynamical systems theory and probability theory. The efficacy of the algorithm is demonstrated via simulations and heuristic argument.
The VC Dimension and Pseudodimension of Two-Layer Neural Networks with Discrete Inputs

We give upper bounds on the Vapnik-Chervonenkis dimension and pseudodimension of two-layer neural networks that use the standard sigmoid function or radial basis function and have inputs from {D, ,D}n. In Valiant's probably approximately correct (pac) learning framework for pattern classification, and in Haussler's generalization of this framework to nonlinear regression, the results imply that the number of training examples necessary for satisfactory learning performance grows no more rapidly than W log (WD), where W is the number of weights. The previous best bound for these networks was O(W4).
Learning Nonlinearly Parametrized

In this paper we present a deterministic analysis of an online scheme for learning very general classes of nonlinearly parametrized decision regions. The only input required is a sequence ((xk; yk)) k2Z + of data samples, where yk = 1 if xk belongs to the decision region of interest, and yk = ?1 otherwise. Averaging results and Lyapunov theory are used to prove the stability of the scheme. In the course of this proof, conditions on both the parametrization and the sequence of input examples arise which are suucient to guarantee convergence of the algorithm. A number of examples are presented, including the problem of learning an intersection of half spaces using only data samples.
Frequency Invariant Broadband Beamforming with Exact Null Placement

This paper extends earlier results by Ward, Kennedy and Williamson  for the design of broadband arrays with frequency-invariant (FI) beam patterns to the case where it is desired to place an exact null in a given direction. The beamforming is done using appropriately selected FIR filters. First, the previous results for generating FI beampatterns using FIR filters are briefly summarised. Second, new results which give the conditions required for exact nulls in the beam pattern for all frequencies in any, possibly non-FI, beam pattern are given. Third, a method of generating beam patterns which possess an exact null and which are close, in an L2 sense, to an arbitrary FI pattern is presented. Finally, some preliminary experimental results which corroborate the theoretical findings are presented.
Nearfield broadband frequency invariant beamforming

This paper presents a method for nearfield frequency invariant broadband array beamforming. This means that we can design an array with associated signal processing to achieve a desired frequency invariant beam pattern (as a function of direction) at any nominal finite distance over an arbitrarily wide range of frequencies. The challenge here is to compensate accurately for the nearfield distortions (both magnitude and phase) which are more acute at the lower frequencies of the operating bandwidth than the higher frequencies. The methodology uses spherical harmonics to transform the nearfield broadband frequency invariant beampattern specification to an equivalent farfield frequency varying beampattern specification. A simulation example is presented to demonstrate the effectiveness of this method in producing a nearfield beam pattern which is frequency invariant over an octave bandwidth.
Theory and design of broadband sensor arrays with frequency invariant far � eld beam patterns

The theory and design of a broadband array of sensors with a frequency invariant far eld beam pattern over an arbitrarily wide design bandwidth is presented The frequency invariant beam pattern property is de ned in terms of a continuously distributed sensor and the problem of designing a practical sensor array is then treated as an approximation to this continuous sensor using a discrete set of ltered broadband omni directional array elements The design methodology is suitable for one two and three dimensional sensor arrays it imposes no restrictions on the desired aperture distribution beam shape and can cope with arbitrarily wide bandwidths An important consequence of our results is that the frequency response of the lter applied to the output of each sensor can be factored into two components one component is related to a slice of the desired aperture distribution and the other is sensor independent The results also indicate that the locations of the sensors are not a crucial design consideration although we show that nonuniform spacings simultaneously avoid spatial aliasing and minimize the number of sensors An example design which covers a frequency range which is suitable for speech acquisition using a microphone array illustrates the utility of our method Finally the theory is generalized to cover a parameterized class of arrays in which the frequency dependence of the beam pattern can be controlled in a continuous manner from a classical single frequency design to a frequency invariant design
Neural networks, rational functions, and realization theory

The problem of parametrizing single hidden layer scalar neural networks with continuous activation functions is investigated. A connection is drawn between realization theory for linear dynamical systems, rational functions, and neural networks that appears to be new. A result of this connection is a general parametrization of such neural networks in terms of strictly proper rational functions. Some existence and uniqueness results are derived. Jordan decompositions are developed, which show how the general form can be expressed in terms of a sum of canonical second order sections. The parametrization may be useful for studying learning algorithms.
Existence and uniqueness results for neural network approximations

Some approximation theoretic questions concerning a certain class of neural networks are considered. The networks considered are single input, single output, single hidden layer, feedforward neural networks with continuous sigmoidal activation functions, no input weights but with hidden layer thresholds and output layer weights. Specifically, questions of existence and uniqueness of best approximations on a closed interval of the real line under mean-square and uniform approximation error measures are studied. A by-product of this study is a reparametrization of the class of networks considered in terms of rational functions of a single variable. This rational reparametrization is used to apply the theory of Pade approximation to the class of networks considered. In addition, a question related to the number of local minima arising in gradient algorithms for learning is examined.
Characterization of threshold for single tone maximum likelihood frequency estimation

This paper presents a simple and direct approach to understanding the threshold effect associated with maximum likelihood estimation of the frequency of a single complex tone. Motivation for the approach, stemming from known results in the field of phase locked loops, is given. It is shown both theoretically and experimentally that the onset of threshold can be directly characterized by a single, easily computed parameter, namely the Cramer-Rao bound on the phase estimation error variance.
Examples of learning curves from a modified VC-formalism

We examine the issue of evaluation of model specific parameters in a modified VC-formalism. Two examples are analyzed: the 2-dimensional homogeneous perceptron and the 1-dimensional higher order neuron. Both models are solved theoretically, and their learning curves are compared against true learning curves. It is shown that the formalism has the potential to generate a variety of learning curves, including ones displaying "phase transitions."
Learning curves from a modified VC-formalism: a case study

In this paper we present a case study of a 1-dimensional higher order neuron using a statistical approach to learning theory which incorporates some information on the distribution on the sample space and can be viewed as a modification of the Vapnik-Chervonenkis formalism (VC-formalism). We concentrate on learning curves defined as averages of the worst generalization error of binary hypothesis consistent with the target on training samples as a function of the training sample size. The true learning curve is derived and compared against estimates from the classical formalism and its modification. It is shown that the modified VC-formalism improves the VC-learning curve by a factor of and also produces a meaningful result for small training sample sizes where VC-bounds are void.
An efficient calculation of the moments of matched and mismatched hidden Markov models

Streit (1990) analysed the classification of an unknown hidden Markov model (HMM) using a set of prescribed HMMs. He proposed a suboptimal test statistic that can be approximated by certain moments for this classification. In the present paper the algorithm given by Streit to derive these moments is reformulated in a matrix algebra setting that gives a better insight into the algorithm. Also, an asymptotic analysis of the algorithm is derived using the reformulation.
Lower Bounds on the VC Dimension of Smoothly Parameterized Function Classes

We examine the relationship between the VC dimension and the number of parameters of a threshold smoothly parameterized function class. We show that the VC dimension of such a function class is at least k if there exists a k-dimensional differentiable manifold in the parameter space such that each member of the manifold corresponds to a different decision boundary. Using this result, we are able to obtain lower bounds on the VC dimension proportional to the number of parameters for several thresholded function classes including two-layer neural networks with certain smooth activation functions and radial basis functions with a gaussian basis. These lower bounds hold even if the magnitudes of the parameters are restricted to be arbitrarily small. In Valiant's probably approximately correct learning framework, this implies that the number of examples necessary for learning these function classes is at least linear in the number of parameters.
Theory and design of broadband sensor arrays with frequency invariant far‐field beam patterns

The theory and design of a broadband array of sensors with a frequency invariant far‐field beam pattern over an arbitrarily wide design bandwidth is presented. The frequency invariant beam pattern property is defined in terms of a continuously distributed sensor, and the problem of designing a practical sensor array is then treated as an approximation to this continuous sensor using a discrete set of filtered broadband omnidirectional array elements. The design methodology is suitable for one‐, two‐, and three‐dimensional sensor arrays; it imposes no restrictions on the desired aperture distribution (beam shape), and can cope with arbitrarily wide bandwidths. An important consequence of the results is that the frequency response of the filter applied to the output of each sensor can be factored into two components: One component is related to a slice of the desired aperture distribution, and the other is sensor independent. The results also indicate that the locations of the sensors are not a crucial desi...
On efficient agnostic learning of linear combinations of basis functions

We consider efficient agnostic learning of linear combinations of basis functions when the sum of absolute values of the weights of the linear combinations is bounded. With the quadratic loss function, we show that the class of linear combinations of a set of basis functions is efficiently agnostically learnable if and only if the class of basis functions is efficiently agnostically learnable. We also show that the sample complexity for learning the linear combinations grows polynomially if and only if a combinatorial property of the class of basis functions, called the fat-shattering function, grows at most polynomially. We also relate the problem to agnostic learning of f0 1g-valued function classes by showing that if a class of f0 1g-valued functions is efficiently agnostically learnable (using the same function class) with the discrete loss function, then the class of linear combinations of functions from the class is efficiently agnostically learnable with the quadratic loss function.
Audio frequency power amplifiers

An audio frequency amplifier having a switching converter comprising source and sink rectifiers configured for sourcing and sinking current, in which the source and sink supply voltages of the rectifiers track the audio frequency signal being amplified. The amplifier has a circuit or element for discharging the output capacitors of the source and sink rectifier as necessary to prevent floating of the source and sink supply voltages. The discharging element can be a locking capacitor or one or more Zener diodes connected across the source and sink supply voltages. The discharge circuit can be one or more switching transistors connected across the freewheeling diodes of the source and sink rectifiers to actively pull the source and sink voltages down to discharge the output capacitors and thereby prevent floating. Low voltage rated parts can be used in the output stage amplifier, and in certain configurations a high voltage low current preamplifier stage can be eliminated.
What Can a Neuron Learn with Spike-Timing-Dependent Plasticity?

Spiking neurons are very flexible computational modules, which can implement with different values of their adjustable synaptic parameters an enormous variety of different transformations F from input spike trains to output spike trains. We examine in this letter the question to what extent a spiking neuron with biologically realistic models for dynamic synapses can be taught via spike-timing-dependent plasticity (STDP) to implement a given transformation F. We consider a supervised learning paradigm where during training, the output of the neuron is clamped to the target signal (teacher forcing). The well-known perceptron convergence theorem asserts the convergence of a simple supervised learning algorithm for drastically simplified neuron models (McCulloch-Pitts neurons). We show that in contrast to the perceptron convergence theorem, no theoretical guarantee can be given for the convergence of STDP with teacher forcing that holds for arbitrary input spike patterns. On the other hand, we prove that average case versions of the perceptron convergence theorem hold for STDP in the case of uncorrelated and correlated Poisson input spike trains and simple models for spiking neurons. For a wide class of cross-correlation functions of the input spike trains, the resulting necessary and sufficient condition can be formulated in terms of linear separability, analogously as the well-known condition of learnability by perceptrons. However, the linear separability criterion has to be applied here to the columns of the correlation matrix of the Poisson input. We demonstrate through extensive computer simulations that the theoretically predicted convergence of STDP with teacher forcing also holds for more realistic models for neurons, dynamic synapses, and more general input distributions. In addition, we show through computer simulations that these positive learning results hold not only for the common interpretation of STDP, where STDP changes the weights of synapses, but also for a more realistic interpretation suggested by experimental data where STDP modulates the initial release probability of dynamic synapses.
A Criterion for the Convergence of Learning with Spike Timing Dependent Plasticity

We investigate under what conditions a neuron can learn by experimentally supported rules for spike timing dependent plasticity (STDP) to predict the arrival times of strong "teacher inputs" to the same neuron. It turns out that in contrast to the famous Perceptron Convergence Theorem, which predicts convergence of the perceptron learning rule for a simplified neuron model whenever a stable solution exists, no equally strong convergence guarantee can be given for spiking neurons with STDP. But we derive a criterion on the statistical dependency structure of input spike trains which characterizes exactly when learning with STDP will converge on average for a simple model of a spiking neuron. This criterion is reminiscent of the linear separability criterion of the Perceptron Convergence Theorem, but it applies here to the rows of a correlation matrix related to the spike inputs. In addition we show through computer simulations for more realistic neuron models that the resulting analytically predicted positive learning results not only hold for the common interpretation of STDP where STDP changes the weights of synapses, but also for a more realistic interpretation suggested by experimental data where STDP modulates the initial release probability of dynamic synapses.
Movement Generation with Circuits of Spiking Neurons

How can complex movements that take hundreds of milliseconds be generated by stereotypical neural microcircuits consisting of spiking neurons with a much faster dynamics? We show that linear readouts from generic neural microcircuit models can be trained to generate basic arm movements. Such movement generation is independent of the arm model used and the type of feedback that the circuit receives. We demonstrate this by considering two different models of a two-jointed arm, a standard model from robotics and a standard model from biology, that each generates different kinds of feedback. Feedback that arrives with biologically realistic delays of 50 to 280 ms turns out to give rise to the best performance. If a feedback with such desirable delay is not available, the neural microcircuit model also achieves good performance if it uses internally generated estimates of such feedback. Existing methods for movement generation in robotics that take the particular dynamics of sensors and actuators into account (embodiment of motor systems) are taken one step further with this approach, which provides methods for also using the embodiment of motion generation circuitry, that is, the inherent dynamics and spatial structure of neural circuits, for the generation of movement.