nils-herrmann/clean_scirepeval huggingface dataset:
Boosting Mixture Models for Semi-supervised Learning

This paper introduces MixtBoost, a variant of AdaBoost dedicated to solve problems in which both labeled and unlabeled data are available. We propose several definitions of loss for unlabeled data, from which margins are defined. The resulting boosting schemes implement mixture models as base classifiers. Preliminary experiments are analyzed and the relevance of loss choices is discussed. MixtBoost improves on both mixture models and AdaBoost provided classes are structured, and is otherwise similar to AdaBoost.
Injection de bruit adaptative pour la détermination de variables pertinentes

La selection de variables et les moindres carres regularises sont les deux archetypes des methodes utilisees pour regler la complexite d'un regresseur. Dans le domaine des reseaux de neurones, ces principes sont respectivement declines sous differentes formes de techniques d'elagage et de penalisation des poids. Elles peuvent etre utilisees dans l'objectif d'estimer l'ensemble des variables d'entree pertinentes pour predire les variables de sortie. Pour l'elagage, les indices de pertinence sont binaires (la variable est conservee ou supprimee); pour la penalisation, les indices sont continus. Cet article montre comment l'injection de bruit peut etre adaptee pour penaliser differement les entrees de maniere adaptative. L'algorithme propose conserve la simplicite de l'algorithme de base. En particulier, il ne necessite qu'un hyperparametre, dont le role est de regler a la fois la complexite du modele et les indices de pertinence de chaque variable. De nombreux resultats experimentaux sur des donnees simulees confirment la validite de cette approche.
Anisotropic noise injection for input variables relevance determination

There are two archetypal ways to control the complexity of a flexible regressor: subset selection and ridge regression. In neural-networks jargon, they are, respectively, known as pruning and weight decay. These techniques may also be adapted to estimate which features of the input space are relevant for predicting the output variables. Relevance is given by a binary indicator for subset selection, and by a continuous rating for ridge regression. This paper shows how to achieve such a rating for a multilayer perceptron trained with noise (or jitter). Noise injection (NI) is modified in order to penalize heavily irrelevant features. The proposed algorithm is attractive as it requires the tuning of a single parameter. This parameter controls the complexity of the model (effective number of parameters) together with the rating of feature relevances (effective input space dimension). Bounds on the effective number of parameters support that the stability of this adaptive scheme is enforced by the constraints applied to the admissible set of relevance indices. The good properties of the algorithm are confirmed by satisfactory experimental results on simulated data sets.
Bagging down-weights leverage points

Bagging is a procedure averaging estimators trained on bootstrap samples. Numerous experiments have shown that bagged estimates often yield better results than the original predictor, and several explanations have been given to account for this gain. However, six years from its introduction, bagging is still not fully understood. Most explanations given until now are based on global properties of the estimates. Here, we focus on the local effects on leverage points, i.e., on observations whose fitted values are largely determined by the corresponding response values. These points are shown experimentally to be down-weighted by bagging. The performance of the bagged estimate depends on the goodness of these points for the original estimator. Illustrative examples findings are supported by the study of smoothing matrix, and their consequences are discussed.
Local learning by sparse radial basis functions

The use of radial basis functions in supervised learning is well motivated by approximation theory. Computation issues have lead us to consider some approximations of this scheme, losing much of the mathematical foundation in the process. We show that basis pursuit denoising is a principled alternative to classical RBF, which leads to sparse expansions. This alternative is local in the sense that complexity is tuned locally. A further step in this direction is made by adapting the locality parameter of each basis function. The algorithm proposed to solve this problem is simple, and the resulting solution, although extremely flexible, is governed by a single hyperparameter.
Black-Box Software Sensor Design for Environmental Monitoring

Software sensor design consists in building a model to estimate an unknown quantity, with error bars, using other available measurements. In the environmental domain, due to a lack of physical model, non-linearities, and unknown time dependencies, black-box modelling is required. An application in river water quality monitoring illustrates a neural network based methodology. All stages of the method are described from data cleaning, and model selection, predictor estimation and prediction validity assessment. The originality of the approach is that it provides automatically an estimation of inputs relevance in merging the input selection and prediction estimation steps.
Outcomes of the Equivalence of Adaptive Ridge with Least Absolute Shrinkage

Adaptive Ridge is a special form of Ridge regression, balancing the quadratic penalization on each parameter of the model. It was shown to be equivalent to Lasso (least absolute shrinkage and selection operator), in the sense that both procedures produce the same estimate. Lasso can thus be viewed as a particular quadratic penalizer. 
 
From this observation, we derive a fixed point algorithm to compute the Lasso solution. The analogy provides also a new hyper-parameter for tuning effectively the model complexity. We finally present a series of possible extensions of lasso performing sparse regression in kernel smoothing, additive modeling and neural net training.
4 A new algorithm for finding the lasso solution

Adaptive ridge is a special form of ridge regression, balancing the quadratic penalization on each parameter of the model. This paper shows the equivalence between adaptive ridge and lasso (least absolute shrinkage and selection operator). This equivalence states that both procedures produce the same estimate. Least absolute shrinkage can thus be viewed as a particular quadratic penalization. From this observation, we derive an EM algorithm to compute the lasso solution. We finally present a series of applications of this type of algorithm in regression problems: kernel regression, additive modeling and neural net training.
Adaptive Noise Injection for Input Variables Relevance Determination

In this paper we consider the application of training with noise in multi-layer perceptron to input variables relevance determination. Noise injection is modified in order to penalize irrelevant features. The proposed algorithm is attractive as it requires the tuning of a single parameter. This parameter controls the penalization of the inputs together with the complexity of the model. After the presentation of the method, experimental evidences are given on simulated data sets.
Noise Injection: Theoretical Prospects

Noise injection consists of adding noise to the inputs during neural network training. Experimental results suggest that it might improve the generalization ability of the resulting neural network. A justification of this improvement remains elusive: describing analytically the average perturbed cost function is difficult, and controlling the fluctuations of the random perturbed cost function is hard. Hence, recent papers suggest replacing the random perturbed cost by a (deterministic) Taylor approximation of the average perturbed cost function. This article takes a different stance: when the injected noise is gaussian, noise injection is naturally connected to the action of the heat kernel. This provides indications on the relevance domain of traditional Taylor expansions and shows the dependence of the quality of Taylor approximations on global smoothness properties of neural networks under consideration. The connection between noise injection and heat kernel also enables controlling the fluctuations of the random perturbed cost function. Under the global smoothness assumption, tools from gaussian analysis provide bounds on the tail behavior of the perturbed cost. This finally suggests that mixing input perturbation with smoothness-based penalization might be profitable.
Control of complexity in learning with perturbed inputs

This paper considers the problem of function approximation from scattered data when using multilayered perceptrons. We present a new algorithm based on the principle of Inputs Perturbation, improving the generalization performances of backpropagation. In this algorithm, a new parameter is introduced in order to allow a control of the complexity of the t. This parameter balances the bias versus the variance independently of the smoothness of the inputs density estimate. The tuning capacity of the algorithm is illustrated by experimental evidences.
Comments on "Noise injection into inputs in back propagation learning"

The generalization capacity of neural networks learning from examples is important. Several authors showed experimentally that training a neural network with noise injected inputs could improve its generalization abilities. In the original paper (ibid., vol. 22,  1992), Matsuoka explained this fact in a formal way, claiming that using noise injected inputs is equivalent to reduce the sensitivity of the network. However, the author states that an error in Matsuoka's calculations lead him to inadequate conclusions. This paper corrects these calculations and conclusions.
Parallel high-throughput limited search trellis decoder VLSI design

Limited search trellis decoding algorithms have great potentials of realizing low power due to their largely reduced computational complexity compared with the widely used Viterbi algorithm. However, because of the lack of operational parallelism and regularity in their original formulations, the limited search decoding algorithms have been traditionally ruled out for applications demanding very high throughput. We believe that, through appropriate algorithm and hardware architecture co-design, certain limited search trellis decoding algorithms can become serious competitors to the Viterbi algorithm for high-throughout applications. Focusing on the well-known T-algorithm, this paper presents techniques at the algorithm and VLSI architecture levels to design fully parallel T-algorithm limited search trellis decoders. We first develop a modified T-algorithm, called SPEC-T, to improve the algorithmic parallelism. Then, based on the conventional state-parallel register exchange Viterbi decoder, we develop a parallel SPEC-T decoder architecture that can effectively transform the reduced computational complexity at the algorithm level to the reduced switching activities in the hardware. We demonstrate the effectiveness of the SPEC-T design solution in the context of convolutional code decoding. Compared with state-parallel register exchange Viterbi decoders, the SPEC-T convolutional code decoders can achieve almost the same throughput and decoding performance, while realizing up to 56% power savings. For the first time, this work provides an approach to exploit the low power potential of the T-algorithm in very high throughput applications.
Self-timed dynamically pipelined adaptive signal processing system: a case study of DLMS equalizer for read channel

Many pipelined adaptive signal processing systems are subject to a tradeoff between throughput and signal processing performance incurred by the pipelined adaptation feedback loops. In the conventional synchronous design regime, such throughput/performance tradeoff is typically fixed since the pipeline depth is usually determined in the design phase and remains unchanged in the run time. Nevertheless, in many real-life scenarios, the overall system performance can be potentially improved if we can run-time dynamically configure this tradeoff. With this motivation, we propose to apply self-timed pipeline, an alternative to synchronous pipeline, to implement the pipelined adaptive signal processing systems, in which the pipeline depth can be dynamically changed to realize run-time configurable throughput/performance tradeoffs. Based on a well-known high speed self-timed pipeline style, we developed architecture and circuit level design techniques to implement the self-timed pipelined adaptation feedback loop with configurable pipeline depth. We demonstrate the proposed design approach using a delayed least mean square (DLMS) adaptive equalizer for magnetic recording read channel. The data transfer rate in hard disk varies as the read head moves among tracks with different distance from the center of the disk platter. By adjusting the pipeline depth on-the-fly, the DLMS equalizer can dynamically track the best equalization performance allowed by the varying data transfer rates. Simulation result shows a significant performance improvement compared with its synchronous counterpart.
Parallel logic simulation of million-gate VLSI circuits

The complexity of today's VLSI chip designs makes verification a necessary step before fabrication. As a result, gate-level logic simulation has became an integral component of the VLSI circuit design process which verifies the design and analyzes its behavior. Since the designs constantly grow in size and complexity, there is a need for ever more efficient simulations to keep the gate-level logic verification time acceptably small. The focus of this paper is an efficient simulation of large chip designs. We present the design and implementation of a new parallel simulator, called DSIM, and demonstrate DSIM's efficiency and speed by simulating a million gate circuit using different numbers of processors.
Self-timed triple-rail MOS current mode logic pipeline for power-on-demand design

This paper presents an approach to address the high static power consumption problem of the MOS current mode logic (MCML) by integrating MCML with self-timed pipelines. We develop a new MCML style, called triple-rail MOS current mode logic (Tr-MCML), that supports seamless integration with a self-timed pipeline. A look-ahead self-timed Tr-MCML pipeline architecture is designed to realize the data-driven power-on-demand operation to reduce the static power consumption while maintaining the high-speed property of MCML. This design approach is particularly desirable for latency-critical digital systems that solely rely on high-speed circuit design to improve performance. We use ripple carry adders as a test vehicle to demonstrate the significant improvement on the energy-delay product (EDP) compared with conventional MCML design.
Parallel High-Throughput Limited Search Trellis

Limited search trellis decoding algorithms have great potentials of realizing low power due to their largely reduced computational complexity compared with the widely used Viterbi algorithm. However, because of the lack of operational parallelism and regularity in their original formulations, the limited search decoding algorithms have been traditionally ruled out for applica- tions demanding very high throughput. We believe that, through appropriate algorithm and hardware architecture co-design, certain limited search trellis decoding algorithms can become serious competitors to the Viterbi algorithm for high-throughout applications. Focusing on the well-known -algorithm, this paper presents techniques at the algorithm and VLSI architecture levels to design fully parallel -algorithm limited search trellis decoders. We first develop a modified -algorithm, called SPEC- , to im- prove the algorithmic parallelism. Then, based on the conventional state-parallel register exchange Viterbi decoder, we develop a par- allel SPEC- decoder architecture that can effectively transform the reduced computational complexity at the algorithm level to the reduced switching activities in the hardware. We demonstrate the effectiveness of the SPEC- design solution in the context of convolutional code decoding. Compared with state-parallel reg- ister exchange Viterbi decoders, the SPEC- convolutional code decoders can achieve almost the same throughput and decoding performance, while realizing up to 56% power savings. For the first time, this work provides an approach to exploit the low power potential of the -algorithm in very high throughput applications.
Parallelism/regularity-driven MIMO detection algorithm design

Efficient VLSI implementation of multiple-input multiple-output (MIMO) detectors plays an important role in the real-life implementation of MIMO communication systems. However, most high-performance MIMO detection algorithms developed so far largely lack the operational parallelism and regularity that are essential for high-throughput and low-power VLSI implementations. Following the theme of parallelism/regularity-driven algorithm design, we propose hard/soft-output MIMO detection algorithms that have high operational parallelism and a regular/static data flow structure with fixed detection delay. Besides those properties desirable for VLSI implementations, such algorithms can achieve superior detection performance, as demonstrated in the simulations.
Block-LDPC: a practical LDPC coding system design approach

This paper presents a joint low-density parity-check (LDPC) code-encoder-decoder design approach, called Block-LDPC, for practical LDPC coding system implementations. The key idea is to construct LDPC codes subject to certain hardware-oriented constraints that ensure the effective encoder and decoder hardware implementations. We develop a set of hardware-oriented constraints, subject to which a semi-random approach is used to construct Block-LDPC codes with good error-correcting performance. Correspondingly, we develop an efficient encoding strategy and a pipelined partially parallel Block-LDPC encoder architecture, and a partially parallel Block-LDPC decoder architecture. We present the estimation of Block-LDPC coding system implementation key metrics including the throughput and hardware complexity for both encoder and decoder. The good error-correcting performance of Block-LDPC codes has been demonstrated through computer simulations. With the effective encoder/decoder design and good error-correcting performance, Block-LDPC provides a promising vehicle for real-life LDPC coding system implementations.
Joint code-encoder-decoder design for LDPC coding system VLSI implementation

This paper presents a design approach for low-density parity-check (LDPC) coding system hardware implementation by jointly conceiving irregular LDPC code construction and VLSI implementations of encoder and decoder. The key idea is to construct good irregular LDPC codes subject to two constraints that ensure the effective LDPC encoder and decoder hardware implementations. We propose a heuristic algorithm to construct such implementation-aware irregular LDPC codes that can achieve very good error correction performance. The encoder and decoder hardware architectures are correspondingly presented.
A high throughput limited search trellis decoder for convolutional code decoding

Due to the lack of operational parallelism and structured data storage/retrieval, limited search trellis decoding algorithms have been traditionally ruled out for applications demanding high throughput convolutional code decoding. Among various limited search algorithms, the T-algorithm performs breadth-first limited search and has good potential for parallel decoding. In this paper, we propose two techniques at the algorithm and VLSI architecture levels for the T-algorithm to improve the decoding parallelism and tackle the data storage/retrieval problem, which enables the high throughput path-parallel T-algorithm decoder VLSI implementation. This work provides a vehicle for exploiting the merits of the T-algorithm, i.e., low computational complexity that is adaptive to the channel distortion, in high throughput applications.
Joint (3,k)-regular LDPC code and decoder/encoder design

Recently, low-density parity-check (LDPC) codes have attracted a lot of attention in the coding theory community. However, their real-world applications are still problematic mainly due to the lack of effective decoder/encoder hardware design approaches. In this paper, we present a joint (3,k)-regular LDPC code and decoder/encoder design technique to construct a class of (3,k)-regular LDPC codes that not only have very good error-correcting capability but also exactly fit to high-speed partly parallel decoder and low-complexity encoder implementations. We also develop two techniques to further modify this joint design scheme to achieve more flexible tradeoffs between decoder hardware complexity and decoding speed.
An FPGA Implementation of-Regular Low-Density Parity-Check Code Decoder

Because of their excellent error-correcting performance, low-density parity-check (LDPC) codes have recently attracted a lot of attention. In this paper, we are interested in the practical LDPC code decoder hardware implementations. The direct fully parallel decoder implementation usually incurs too high hardware complexity for many real applications, thus partly parallel decoder design approaches that can achieve appropriate trade-offs between hardware complexity and decoding throughput are highly desirable. Applying a joint code and decoder design methodology, we develop a high-speed-regular LDPC code partly parallel decoder architecture based on which we implement a-bit, rate--regular LDPC code decoder on Xilinx FPGA device. This partly parallel decoder supports a maximum symbol throughput of Mbps and achieves BER at 2 dB over AWGN channel while performing maximum decoding iterations.
Design of VLSI implementation-oriented LDPC codes

Recently, low-density parity-check (LDPC) codes have attracted much attention because of their excellent error-correcting performance and highly parallelizable decoding scheme. However, the effective VLSI implementation of an LDPC decoder remains a big challenge and is a crucial issue in determining how well we can exploit the benefits of the LDPC codes in real applications. In this paper, following the joint code and decoder design philosophy, we propose a semi-random design scheme to construct the LDPC codes that not only exhibit very good error-correcting performance but also effectively fit to partially parallel VLSI decoder implementations. The corresponding partially parallel decoder has a very regular structure and simple control mechanism. Our computer simulations show that such LDPC codes achieve very good performance comparable to their counterparts constructed in a fully random scheme, which however have little chance of fitting to partially parallel decoder implementations.
Implementation-Oriented LDPC Codes

Recently, low-density parity-check (LDPC) codes have attracted much attention because of their excellent errorcorrecting performance and highly parallelizable decoding scheme. However, the effective VLSI implementation of an LDPC decoder remains a big challenge and is a crucial issue in determining how well we can exploit the benefits of the LDPC codes in the real applications. In this paper, following the joint code and decoder design philosophy, we propose a semirandom design scheme to construct the LDPC codes that not only exhibit very good error-correcting performance but also effectively fit to partially parallel VLSI decoder implementations. The corresponding partially parallel decoder has a very regular structure and simple control mechanism. Our computer simulations show that such LDPC codes achieve very good performance comparable to their counterparts constructed in a fully random scheme, which however have little chance of fitting to partially parallel decoder implementations.