nils-herrmann/clean_scirepeval huggingface dataset:
Resource Allocation Using Sequential Auctions

Market-based mechanisms such as auctions are being studied as an appropriate means for resource allocation in distributed and multiagent decision problems. When agents value resources in combination rather than in isolation, one generally relies on combinatorial auctions where agents bid for resource bundles, or simultaneous auctions for all resources. We develop a different model, where agents bid for required resources sequentially. This model has the advantage that it can be applied in settings where combinatorial and simultaneous models are infeasible (e.g., when resources are made available at different points in time by different parties), as well as certain benefits in settings where combinatorial models are applicable. We develop a dynamic programming model for agents to compute bidding policies based on estimated distributions over prices. We also describe how these distributions are updated to provide a learning model for bidding behavior.
Information Extraction Research and Applications: Current Progress and Future Directions

Analysts face a daunting task: they must accurately analyze, categorize, and assimilate a large body of information from a variety of sources and for a variety of domains of interest. The complexity of the task necessitates a variety of information access and extraction tools which technology up to this point has not been able to provide. SRI's TIPSTER Phase III project has focused on two major obstacles to the development of such tools: inadequate degrees of accuracy and portability. We begin by providing an overview of SRI's information extraction (IE) system, FASTUS, and then describe our efforts in these two areas in turn. We then conclude with some thoughts concerning future directions.
Finite-connectivity spin-glass phase diagrams and low-density parity check codes.

We obtain phase diagrams of regular and irregular finite-connectivity spin glasses. Contact is first established between properties of the phase diagram and the performance of low-density parity check (LDPC) codes within the replica symmetric (RS) ansatz. We then study the location of the dynamical and critical transition points of these systems within the one step replica symmetry breaking theory (RSB), extending similar calculations that have been performed in the past for the Bethe spin-glass problem. We observe that the location of the dynamical transition line does change within the RSB theory, in comparison with the results obtained in the RS case. For LDPC decoding of messages transmitted over the binary erasure channel we find, at zero temperature and rate , an RS critical transition point at while the critical RSB transition point is located at , to be compared with the corresponding Shannon bound . For the binary symmetric channel we show that the low temperature reentrant behavior of the dynamical transition line, observed within the RS ansatz, changes its location when the RSB ansatz is employed; the dynamical transition point occurs at higher values of the channel noise. Possible practical implications to improve the performance of the state-of-the-art error correcting codes are discussed.
Coloring random graphs and maximizing local diversity.

We study a variation of the graph coloring problem on random graphs of finite average connectivity. Given the number of colors, we aim to maximize the number of different colors at neighboring vertices (i.e., one edge distance) of any vertex. Two efficient algorithms, belief propagation and Walksat, are adapted to carry out this task. We present experimental results based on two types of random graphs for different system sizes and identify the critical value of the connectivity for the algorithms to find a perfect solution. The problem and the suggested algorithms have practical relevance since various applications, such as distributed storage, can be mapped onto this problem.
Bayesian inference in densely connected networks applied to CDMA

Graphical models provide a powerful framework for modelling statistical dependencies between variables . Message passing techniques are typically used for inference in graphical models that can be represented by a sparse graph. Iterative message passing is guaranteed to converge to the globally correct estimate when the system is tree-like; there are no such guarantees for systems with loops. Two inherent limitations seem to prevent the use of message passing techniques in densely connected systems: 1) Their high connectivity implies an exponentially growing computational cost. 2) The existence of an exponential number of loops that render the method inconsistent. However, a new approach was suggested  for extending Belief Propagation (BP) techniques to densely connected systems. In this approach, messages are grouped together, giving rise to macroscopic random variables drawn from a different Gaussian distribution of varying mean and variance for each of the nodes. In a separate development , BP was extended to Survey Propagation (SP). This new algorithm has succeeded in solving hard computational problems [3], far beyond other existing approaches. Inspired by the extension of BP to SP we have extended the approach of , designed for inference in densely connected systems, in a similar manner by including an average over multiple pure states. However, for highlighting the advantages with respect to the original method , we apply it to the problem of signal detection in CDMA. Code Division Multiple Access  is based on spreading the signal by using K individual random binary spreading codes of spreading factor N . We consider the large-system limit N → ∞, K → ∞ with β = K/N ∼ O(1). The received aggregated, modulated and corrupted signal is of the form:
Improved message passing for inference in densely connected systems

An improved inference method for densely connected systems is presented. The approach is based on passing condensed messages between variables, representing macroscopic averages of microscopic messages. We extend previous work that showed promising results in cases where the solution space is contiguous to cases where fragmentation occurs. We apply the method to the signal detection problem of Code Division Multiple Access (CDMA) for demonstrating its potential. A highly efficient practical algorithm is also derived on the basis of insight gained from the analysis.
Ju l 2 00 5 Colouring random graphs and maximising local diversity

We study a variation of the graph colouring problem on random graphs of finite average connectivity. Given the number of colours, we aim to maximise the number of different colours at neighbouring vertices (i.e. one edge distance) of any vertex. Two efficient algorithms, belief propagation and Walksat are adapted to carry out this task. We present experimental results based on two types of random graphs for different system sizes and identify the critical value of the connectivity for the algorithms to find a perfect solution. The problem and the suggested algorithms have practical relevance since various applications, such as distributed storage, can be mapped onto this problem.
Statistical mechanics of low-density parity-check codes

We review recent theoretical progress on the statistical mechanics of error correcting codes, focusing on low-density parity-check (LDPC) codes in general, and on Gallager and MacKay-Neal codes in particular. By exploiting the relation between LDPC codes and Ising spin systems with multispin interactions, one can carry out a statistical mechanics based analysis that determines the practical and theoretical limitations of various code constructions, corresponding to dynamical and thermodynamical transitions, respectively, as well as the behaviour of error-exponents averaged over the corresponding code ensemble as a function of channel noise. We also contrast the results obtained using methods of statistical mechanics with those derived in the information theory literature, and show how these methods can be generalized to include other channel types and related communication problems.
The Theory of On-line Learning — A Statistical Physics Approach

In this paper we review recent theoretical approaches for analysing the dynamics of on-line learning in multilayer neural networks using methods adopted from statistical physics. The analysis is based on monitoring a set of macroscopic variables from which the generalisation error can be calculated. A closed set of dynamical equations for the macroscopic variables is derived analytically and solved numerically. The theoretical framework is then employed for defining optimal learning parameters and for analysing the incorporation of second order information into the learning process using natural gradient descent and matrix-momentum based methods. We will also briefly explain an extension of the original framework for analysing the case where training examples are sampled with repetition.
Critical noise levels for low-density parity check decoding.

We determine the critical noise level for decoding low-density parity check error-correcting codes based on the magnetization enumerator (M), rather than on the weight enumerator (W) employed in the information theory literature. The interpretation of our method is appealingly simple, and the relation between the different decoding schemes such as typical pairs decoding, MAP, and finite temperature decoding (MPM) becomes clear. In addition, our analysis provides an explanation for the difference in performance between MN and Gallager codes. Our results are more optimistic than those derived using the methods of information theory and are in excellent agreement with recent results from another statistical physics approach.
Critical Noise Levels for LDPC decoding

We determine the critical noise level for decoding low density parity check error correcting codes based on the magnetization enumerator ($\cM$), rather than on the weight enumerator ($\cW$) employed in the information theory literature. The interpretation of our method is appealingly simple, and the relation between the different decoding schemes such as typical pairs decoding, MAP, and finite temperature decoding (MPM) becomes clear. In addition, our analysis provides an explanation for the difference in performance between MN and Gallager codes. Our results are more optimistic than those derived via the methods of information theory and are in excellent agreement with recent results from another statistical physics approach.
J ul 1 99 9 Cascading Parity-Check Error-Correcting Codes

A method for improving the performance of sparse-matrix based parity check codes is proposed, based on insight gained from methods of statistical physics. The advantages of the new approach are demonstrated on an existing encoding/decoding paradigm suggested by Sourlas. We also discuss the application of the same method to more advanced codes of a similar type. Error-correcting codes are commonly used in most means of information transmission. The coding efficiency, measured in the percentage of informative transmitted bits, plays a crucial role in determining the speed of communication channels and the effective storage space on hard-disks. The question of channel capacity of noisy communication channels was addressed by Shannon  in his pioneering work from 1948. Rigorous bounds have been derived for the maximal transmission rate for which codes, capable of achieving arbitrarily small error probability, can be found. In a typical scenario, a message comprising N binary bits is transmitted through a noisy communication channel; the received string differs from the transmitted one due to noise (for instance, background radiation, defects in magnetic materials, thermal noise etc) the result of which is the flipping of some bits. Here, we denote the flipping rate of a bit in a binary symmetric channel (i.e., from 0 to 1 or from 1 to 0) by f ∈ [0 : 1]; other types of noise may also be considered, which may be more realistic in some scenarios. Error-correcting codes have been devised for retrieving the original message at the receiving end. The receiver can correct the flipped bits (or some fraction of them) in a retrieved message of length N only if the source transmits M(f) >N bits; the ratio between the original number of bits and those of the transmitted message R ≡ N/M is termed the code-rate. Shannon  obtained the optimal trade-off between the following three quantities in the thermodynamic limit: the maximal code rate Rc (termed channel capacity), the flip rate f (due to noise) and the coded bit error probability pb, given explicitly by
Random graph coloring: statistical physics approach.

The problem of vertex coloring in random graphs is studied using methods of statistical physics and probability. Our analytical results are compared to those obtained by exact enumeration and Monte Carlo simulations. We critically discuss the merits and shortcomings of the various methods, and interpret the results obtained. We present an exact analytical expression for the two-coloring problem as well as general replica symmetric approximated solutions for the thermodynamics of the graph coloring problem with p colors and K-body edges.
Public key cryptography and error correcting codes as Ising models

We employ the methods of statistical physics to study the performance of Gallager type error-correcting codes. In this approach, the transmitted codeword comprises Boolean sums of the original message bits selected by two randomly-constructed sparse matrices. We show that a broad range of these codes potentially saturate Shannon’s bound but are limited due to the decoding dynamics used. Other codes show sub-optimal performance but are not restricted by the decoding dynamics. We show how these codes may also be employed as a practical public-key cryptosystem and are of competitive performance to modern cyptographical methods.
Statistical Physics of Low Density Parity Check Error Correcting Codes

We study the performance of Low Density Parity Check (LDPC) error-correcting codes using the methods of statistical physics. LDPC codes are based on the generation of codewords using Boolean sums of the original message bits by employing two randomly-constructed sparse matrices. These codes can be mapped onto Ising spin models and studied using common methods of statistical physics. We examine various regular constructions and obtain insight into their theoretical and practical limitations. We also briefly report on results obtained for irregular code constructions, for codes with non-binary alphabet, and on how a finite system size effects the error probability.
Noise, regularizers, and unrealizable scenarios in online learning from restricted training sets.

We study the dynamics of online learning in multilayer neural networks where training examples are sampled with repetition and where the number of examples scales with the number of network weights. The analysis is carried out using the dynamical replica method aimed at obtaining a closed set of coupled equations for a set of macroscopic variables from which both training and generalization errors can be calculated. We focus on scenarios whereby training examples are corrupted by additive Gaussian output noise and regularizers are introduced to improve the network performance. The dependence of the dynamics on the noise level, with and without regularizers, is examined, as well as that of the asymptotic values obtained for both training and generalization errors. We also demonstrate the ability of the method to approximate the learning dynamics in structurally unrealizable scenarios. The theoretical results show good agreement with those obtained from computer simulations.
Weight vs. Magnetization Enumerator for Gallager Codes

We propose a method to determine the critical noise level for decoding Gallager type low density parity check error correcting codes. The method is based on the magnetization enumerator (M), rather than on the weight enumerator (W) presented recently in the information theory literature. The interpretation ofour method is appealingly simple, and the relation between the different decoding schemes such as typical pairs decoding, MAP, and finite temperature decoding (MPM) becomes clear. Our results are more optimistic than those derived via the methods ofinf ormation theory and are in excellent agreement with recent results from another statistical physics approach.
The TAP Approach to Intensive and Extensive Connectivity Systems

The Thouless-Anderson-Palmer (TAP) approach was originally developed for analysing the Sherrington-Kirkpatrick model in the study of spin glass models and has been employed since then mainly in the context of extensively connected systems whereby each dynamical variable interacts weakly with the others. Recently, we extended this method for handling general intensively connected systems where each variable has only O(1) connections characterised by strong couplings. However, the new formulation looks quite different with respect to existing analyses and it is only natural to question whether it actually reproduces known results for systems of extensive connectivity. In this chapter, we apply our formulation of the TAP approach to an extensively connected system, the Hopfield associative memory model, showing that it produces identical results to those obtained by the conventional formulation.
Statistical physics of irregular low-density parity-check codes

Low-density parity-check codes with irregular constructions have recently been shown to outperform the most advanced error-correcting codes to date. In this paper we apply methods of statistical physics to study the typical properties of simple irregular codes. We use the replica method to find a phase transition which coincides with Shannon's coding bound when appropriate parameters are chosen. The decoding by belief propagation is also studied using statistical physics arguments; the theoretical solutions obtained are in good agreement with simulation results. We compare the performance of irregular codes with that of regular codes and discuss the factors that contribute to the improvement in performance.
Finite-size effects and error-free communication in Gaussian channels

The efficacy of a specially constructed Gallager-type error-correcting code to communication in a Gaussian channel is examined. The construction is based on the introduction of complex matrices, used in both encoding and decoding, which comprise sub-matrices of cascading connection values. The finite-size effects are estimated for comparing the results with the bounds set by Shannon. The critical noise level achieved for certain code rates and infinitely large systems nearly saturates the bounds set by Shannon even when the connectivity used is low.
Statistical mechanics of low-density parity check error-correcting codes over Galois fields

A variation of low-density parity check (LDPC) error-correcting codes defined over Galois fields (GF(q)) is investigated using statistical physics. A code of this type is characterised by a sparse random parity check matrix composed of C non-zero elements per column. We examine the dependence of the code performance on the value of q, for finite and infinite C values, both in terms of the thermodynamical transition point and the practical decoding phase characterised by the existence of a unique (ferromagnetic) solution. We find different q-dependence in the cases of C = 2 and C ≥ 3; the analytical solutions are in agreement with simulation results, providing a quantitative measure to the improvement in performance obtained using non-binary alphabets.
M ar 2 00 0 The Statistical Physics of Regular Low-Density Parity-Check Error-Correcting Codes

A variation of Gallager error-correcting codes is investigated using statistical mechanics. In codes of this type, a given message is encoded into a codeword which comprises Boolean sums of message bits selected by two randomly constructed sparse matrices. The similarity of these codes to Ising spin systems with random interaction makes it possible to assess their typical performance by analytical methods developed in the study of disordered systems. The typical case solutions obtained via the replica method are consistent with those obtained in simulations using belief propagation (BP) decoding. We discuss the practical implications of the results obtained and suggest a computationally efficient construction for one of the more practical configurations. Typeset using
Cryptographical properties of Ising spin systems.

The relation between Ising spin systems and public-key cryptography is investigated using methods of statistical physics. The insight gained from the analysis is used for devising a matrix-based cryptosystem whereby the ciphertext comprises products of the original message bits; these are selected by employing two predetermined randomly constructed sparse matrices. The ciphertext is decrypted using methods of belief propagation. The analyzed properties of the suggested cryptosystem show robustness against various attacks and competitive performance to modern cryptographical methods.
On-line learning from restricted training sets in multilayer neural networks

We analyse the dynamics of on-line learning in multilayer neural networks where training examples are sampled with repetition and where the number of examples scales with the number of network weights. The analysis is based on monitoring a set of macroscopic variables from which the training and generalisation errors can be calculated. A closed set of dynamical equations is derived using the dynamical replica method and is solved numerically. The theoretical results are consistent with those obtained by computer simulations.
Error-correcting code on a cactus: A solvable model

An exact solution to a family of parity check error-correcting codes is provided by mapping the problem onto a Husimi cactus. The solution obtained in the thermodynamic limit recovers the replica-symmetric theory results and provides a very good approximation to finite systems of moderate size. The probability propagation decoding algorithm emerges naturally from the analysis. A phase transition between decoding success and failure phases is found to coincide with an information-theoretic upper bound. The method is employed to compare Gallager and MN codes.
A pr 2 00 0 Statistical Physics of Irregular Low-Density Parity-Check Codes [ Statistical Physics of Irregular Codes ]

Low-density parity-check codes with irregular constructions have been recently shown to outperform the most advanced error-correcting codes to date. In this paper we apply methods of statistical physics to study the typical properties of simple irregular codes. We use the replica method to find a phase transition which coincides with Shannon’s coding bound when appropriate parameters are chosen. The decoding by belief propagation is also studied using statistical physics arguments; the theoretical solutions obtained are in good agreement with simulations. We compare the performance of irregular with that of regular codes and discuss the factors that contribute to the improvement in performance. PACS numbers: 89.70+c 89.90+n 05.50+q
Error-correcting Codes on a Bethe-like Lattice

We analyze Gallager codes by employing a simple mean-field approximation that distorts the model geometry and preserves important interactions between sites. The method naturally recovers the probability propagation decoding algorithm as an extremization of a proper free-energy. We find a thermodynamic phase transition that coincides with information theoretical upper-bounds and explain the practical code performance in terms of the free-energy landscape.