nils-herrmann/clean_scirepeval huggingface dataset:
Deterministic annealing for clustering, compression, classification, regression, and related optimization problems

The deterministic annealing approach to clustering and its extensions has demonstrated substantial performance improvement over standard supervised and unsupervised learning methods in a variety of important applications including compression, estimation, pattern recognition and classification, and statistical regression. The application-specific cost is minimized subject to a constraint on the randomness of the solution, which is gradually lowered. We emphasize the intuition gained from analogy to statistical physics. Alternatively the method is derived within rate-distortion theory, where the annealing process is equivalent to computation of Shannon's rate-distortion function, and the annealing temperature is inversely proportional to the slope of the curve. The basic algorithm is extended by incorporating structural constraints to allow optimization of numerous popular structures including vector quantizers, decision trees, multilayer perceptrons, radial basis functions, and mixtures of experts.
Asymptotically optimal scalable coding for minimum weighted mean square error

We derive an asymptotically optimal multi-layer coding scheme for entropy-coded scalar quantizers (SQ) that minimizes the weighted mean-squared error (WMSE). The optimal entropy-coded SQ is non-uniform in the case of WMSE. The conventional multi-layer coder quantizes the base-layer reconstruction error at the enhancement-layer, and is sub-optimal for the WMSE criterion. We consider the compander representation of the quantizer, and propose to implement scalability in the compressed domain. We show that such a multi-layer coding system achieves the operational rate-distortion bound given by the non-scalable entropy-coded SQ, at the limit of high resolution. Simulation results for a synthetic memoryless Laplace source with /spl mu/-law companding are presented for various values of layer rates. Substantial gains are also achieved on the "real-world" sources of audio signals, when the optimal multi-layer approach is applied to a two-layer scalable MPEG-4 Advanced Audio Coder.
An achievable rate region for distributed source coding and dispersive information routing

This paper considers the problem of optimal multi-hop routing of correlated sources over a network with multiple sinks and arbitrary network demands. We recently introduced a new routing paradigm in  called ‘dispersive information routing’ (DIR), wherein the intermediate nodes are allowed to split a packet and forward a subset of the received bits on each forward path. DIR ensures that each sink receives just the information it requires to decode the sources it intends to reconstruct, and thereby outperforms conventional routing techniques in the literature. We proposed an encoding scheme called ‘power binning’ which achieves complete rate region and the minimum cost under this paradigm when each sink is allowed to receive packets only from the sources it wants to reconstruct. This paper considers the optimum encoding scheme when every source can (possibly) communicate with every sink irrespective of what the sinks reconstruct. This generalization happens to be considerably more complex and we derive an achievable rate region and an associated achievable cost using principles from distributed source coding and multiple descriptions encoding.
Encoder and decoder optimization for source-channel prediction in error resilient video transmission

Motion-compensated prediction that accounts for loss in the channel is achieved by the source-channel prediction (SCP) method, which is based on the expected decoder reconstruction of past frames (rather than their encoder reconstruction). The decoder reconstruction is estimated by exploiting the recursive optimal per-pixel estimate (ROPE), which explicitly accounts for the quantization distortion, channel loss, error propagation, as well as the decoder operation and achieves improved error resilience. We take this paradigm further by noting that the decoder can, in turn, be reoptimized to match the modification introduced to the encoder for SCP. Simulation results demonstrate substantial performance gains over conventional decoding. We then examine the benefits of re-optimizing the encoder for the newly matched decoder, and then reoptimizing the decoder, etc. and note that further incremental gains are minor. Hence, one complete round of SCP optimization offers significant gains, but multiple reoptimization iterations may not be cost-effective.
A deterministic annealing algorithm for entropy-constrained vector quantizer design

In this paper, we develop a deterministic annealing approach for the design of entropy-constrained vector quantizers. Here, the quantization problem is reformulated within a probabilistic setting and the principle of maximum entropy is invoked. The resulting method is connected to statistical physics in two important respects. First, there is annealing in the process controlled by a "temperature" parameter, useful for avoiding local optima of the cost. Second, the number of distinct codevectors grows by a sequence of phase transitions as the temperature is varied. When the temperature is lowered to zero, the method searches for the optimal entropy-constrained vector quantizer. Conditions for bifurcation in the process provide insights into the "right" codebook size for a given problem.<<ETX>>
A recursive extrapolation approach to intra prediction in video coding

A novel intra prediction scheme, based on recursive extrapolation filters, is introduced. Standard intra prediction largely consists of copying boundary pixels (or linear combinations thereof) along certain directions, which reflects an overly simplistic model for the underlying spatial correlations. As an alternative, we view the image signal as a 2-D non-separable Markov model, whose corresponding correlation model better captures the nuanced directionality effects within blocks. This viewpoint motivates the design of a set of prediction modes represented by three-tap extrapolation filters, which replace the standard “pixel-copying” prediction modes, and provide efficient prediction at modest complexity. The Markov property is exploited by recursive predictions from nearest neighbors without recourse to simplistic separability assumptions, and while effectively accounting for correlation decay with distance from available boundary pixels. Coefficients for the set of mode filters are first trained by an efficient “k-modes” iterative technique designed to monotonically decrease the mean squared prediction error, and are then adjusted to directly optimize the overall rate-distortion objective. This prediction scheme complements the hybrid (cosine and sine) transform coding approach developed by our group, to achieve consistent coding gains, as shown for standard and commercial intra coders such as H.264/AVC and VP8.
Deterministic annealing based design of error resilient predictive compression systems

This paper considers near optimal design of predictive compression system that accounts for packet loss over unreliable networks. Major challenges to address include, propagation of errors due to packet loss through the prediction loop, mismatch between statistics used for design and during operation, and above all a cost function that is fraught with poor local minima. Accurately estimating and minimizing the end-to-end distortion (EED), in combination with asymptotic closed-loop (ACL) design that employs open-loop iterations, but mimics closed-loop operation on convergence, was proposed to address the first two challenges. However the severe non-convexity of the cost function, especially due to the piece-wise linear nature of the quantizer function, makes this a particularly challenging optimization problem. We propose to tackle this via a new design approach in the deterministic annealing framework to avoid poor local minima, coupled with the ACL approach to minimize EED estimate. This effectively addresses all the major challenges, and leads to a near optimal design of error-resilient predictive compression system. Substantial performance improvement obtained in experimental evaluations demonstrates the efficacy of the proposed approach.
Multimode image coding for noisy channels

We attack the problem of designing robust image compression schemes for transmission over noisy channels. To achieve the dual goals of high compression efficiency and low sensitivity to channel noise, we introduce a multimode coding framework. Multimode coders are quasi-fixed length in nature, and this enables optimal tradeoff between the compression capability of variable-length coding and the robustness of fixed length coding. The framework lends itself to a natural incorporation of conventional source-channel coding mechanisms, and thus provides a joint design methodology for the source and channel coding components. We first derive a general Lagrangian based algorithm to design the multimode coder which optimizes the overall rate-distortion performance. We then apply our framework to develop multimode image coding (MIC) schemes for noisy channels. We demonstrate the benefits of the multimode approach by developing two specific image compression algorithms in conjunction with the following source-channel coding mechanisms: (i) channel optimized quantizers, and (ii) transmission energy allocation. Simulations demonstrate that substantial gains of up to 7 dB can be achieved by MICs over conventional approaches.
A mapping approach to rate-distortion computation and analysis

In rate-distortion theory, results are often derived and stated in terms of the optimizing density over the reproduction space. In this paper, the problem is reformulated in terms of the optimal mapping from the unit interval with Lebesgue measure that would induce the desired reproduction probability density. This results in optimality conditions that are "random relatives" of the known Lloyd (1982) optimality conditions for deterministic quantizers. The validity of the mapping approach is assured by fundamental isomorphism theorems for measure spaces. We show that for the squared error distortion, the optimal reproduction random variable is purely discrete at supercritical distortion (where the Shannon (1948) lower bound is not tight). The Gaussian source is thus the only source that produces continuous reproduction variables for the entire range of positive rate. To analyze the evolution of the optimal reproduction distribution, we use the mapping formulation and establish an analogy to statistical mechanics. The solutions are given by the distribution at isothermal statistical equilibrium, and are parameterized by the temperature in direct correspondence to the parametric solution of the variational equations in rate-distortion theory. The analysis of an annealing process shows how the number of "symbols" grows as the system undergoes phase transitions. Thus, an algorithm based on the mapping approach often needs but a few variables to find the exact solution, while the Blahut (1972) algorithm would only approach it at the limit of infinite resolution. Finally, a quick "deterministic annealing" algorithm to generate the rate-distortion curve is suggested. The resulting curve is exact as long as continuous phase transitions in the process are accurately followed.
A recursive optimal spectral estimate of end-to-end distortion in video communications

End-to-end distortion estimation is critical to effective error-resilient video coding. The recursive optimal per-pixel estimate (ROPE) is a known approach to compute up to second moments of decoder-reconstructed pixels, and thereby optimally estimate the distortion. ROPE accurately accounts for encoding/decoding operations that are recursive in the pixel domain, and their interaction with packet loss and decoder concealment. The premise of this work is that considerable gains could be recouped by a dual estimation technique that would perform its recursion in the transform domain. This opens the door to accurate distortion estimation in conjunction with estimation-theoretic source coding approaches that involve transform domain operations, including improved prediction in both single-layer and scalable video coding. We present a novel recursive optimal estimate that operates entirely in the transform domain, namely, the spectral coefficient-wise optimal recursive estimate (SCORE). The method overcomes intricacies due to motion compensation from “off-grid” blocks. We first demonstrate that its accuracy matches ROPE in the usual setting where ROPE is known to be optimal. Then we consider an enhanced encoding scenario involving spectral operations that cannot be accurately tracked by ROPE, but for which SCORE still maintains optimality and hence enables substantial end-to-end performance gains over a large range of packet loss rates.
Wafer-level three-dimensional monolithic integration for heterogeneous silicon ICs

A 3D IC technology platform is presented for high-performance, low-cost heterogeneous integration of silicon ICs. The platform uses dielectric adhesive bonding of fully-processed wafer-to-wafer aligned ICs, followed by a three-step thinning process and copper damascene patterning to form inter-wafer interconnects. Daisy-chain inter-wafer vias and compatibility of the process steps with 130 nm CMOS SOI devices and circuits indicate the viability of the process flow. Memory-intensive digital processors with large L2 caches have shorter access time and cycle time with 3D implementations. Performance advantages of recently designed SiGe BiCMOS pipelined A/D converters have promising figure-of-merits and illustrate partitioning issues for silicon RF ICs. Comparison with system-on-a-chip (SoC) and system-in-a-package (SiP) implementations is presented.
Transform domain temporal prediction with extended blocks

Traditional temporal prediction relies on motion compensated pixel copying operations. Such per-pixel temporal prediction was shown in prior work to be sub-optimal since it ignores the underlying spatial correlation. Transform domain temporal prediction (TDTP) was thus proposed previously to first achieve spatial decorrelation within the block using DCT, then apply an optimal one-to-one prediction per frequency. However, for sub-pixel motion compensation, the low-pass filter used for interpolation interferes with TDTP. In this paper, we propose an extended block based TDTP, which: i) completely disentangles spatial and temporal correlations to fully account for the sub-pixel interpolation effect, and ii) fully exploits spatial correlations around reference block boundary. Experimental evidence is provided for substantial coding gains over standard HEVC.
Generalized vector quantization: jointly optimal quantization and estimation

Given a pair of random vectors X, Y, we study the problem of finding an efficient or optimal estimator of Y given X when the range of the estimator is constrained to be a finite set of values. A generalized vector quantizer (GVQ), with input dimension k, output dimension m, and size N maps input X/spl isin//spl Rscr//sup k/, to output V(X)/spl isin//spl Rscr//sup m/. The output V(X) is constrained to be one of the estimation codevectors in the codebook, {y/sub 1/,y/sub 2/...y/sub N/}. The performance of the GVQ is measured by the average distortion, D=E[d(Y,V(X))] for a suitable output-space distortion measure d(.,.). A GVQ reduces to a conventional vector quantizer in the special case where X=Y. The GVQ problem has been approached in the information theory literature from many different standpoints. In particular, it appears in the context of noisy source coding, which is the special case where we quantize X, the observable, noisy version of a source, Y.
Multiple description quantization by deterministic annealing

We consider the design of vector quantizers for diversity-based communication over two channels of capacities R/sub 1/ and R/sub 2/ with possibly differing failure probabilities. A multiple description vector quantizer (MDVQ) maps an n-dimensional source vector x to n-dimensional code vectors x/sub ij//sup 0/, x/sub i//sup 1/ and x/sub  2/ from the code books /spl chi//sup 0/, /spl chi//sup 1/ and /spl chi//sup 2/ respectively. Let d(x,y) be a single-letter distortion measure, and random vectors X and X/sup m/, m=0,1,2 represent the source and decoder outputs respectively. For given values of R/sub 1/, R/sub 2/, /spl lambda/1 and /spl lambda/2 we then wish to find an MDVQ which minimizes the average distortion cost.
Impact of parameter accuracy on 3D design

Simulator accuracy is an important, but seldom examined, part of study on 3D integration. This paper examines inaccurate technology parameters as a source of error for the CACTI and PRACTICS cache simulation tools. By replacing the default parameters with ones derived from information made available in the literature, CACTI simulation error for the 90nm Itanium2 L3 cache is eliminated almost entirely. We evaluate the effect of this change in technology parameters on 3D design by comparing simulations that use the default parameter values to those using the newly derived ones.
A probabilistic framework for simultaneous segmentation and classification of multiple cells in multi-marker microscopy images

Segmentation and classification of cells in biological data are important problems in bio-medical image analysis. This paper outlines a novel probabilistic approach to simultaneously classify and segment multiple cells of different classes in a multi-variate setting. Superpixels are extracted from the input vector-valued image, and a 2D hidden Markov model (HMM) is set up on the superpixel graph. HMM emission probabilities are used to ensure high confidence in local class selection based on superpixel feature vectors. Spatial consistency of labels is enforced by proper choice of transition probabilities, which are conditioned on the feature vectors of neighboring superpixels at each location. Optimal superpixel-level class labels are inferred using the HMM, and are aggregated to obtain global multiple object segmentation. The performance is demonstrated on a challenging microscopy dataset. Experiments show, both quantitatively and qualitatively, that the proposed approach effectively segments and classifies cells, outperforming related techniques.
On conditions for linearity of optimal estimation

When is optimal estimation linear? It is well-known that, in the case of a Gaussian source contaminated with Gaussian noise, a linear estimator minimizes the mean square estimation error. This paper analyzes more generally the conditions for linearity of optimal estimators. Given a noise (or source) distribution, and a specified signal to noise ratio (SNR), we derive conditions for existence and uniqueness of a source (or noise) distribution that renders the L p norm optimal estimator linear. We then show that, if the noise and source variances are equal, then the matching source is distributed identically to the noise. Moreover, we prove that the Gaussian source-channel pair is unique in that it is the only source-channel pair for which the MSE optimal estimator is linear at more than one SNR values.
Computation and analysis of the N-Layer scalable rate-distortion function

Methods for determining and computing the rate-distortion (RD) bound for N-layer scalable source coding of a finite memoryless source are considered. Optimality conditions were previously derived for two layers in terms of the reproduction distributions q/sub y1/ and q/sub y2/|y/sub 1/. However, the ignored and seemingly insignificant boundary cases, where q/sub y1/=0 and q/sub y2/|y/sub 1/ is undefined, have major implications on the solution and its practical application. We demonstrate that, once the gap is filled and the result is extended to N-layers, it is, in general, impractical to validate a tentative solution, as one has to verify the conditions for all conceivable q/sub yi+1/,...,y/sub N/|y/sub 1/,...,y/sub i/ at each (y/sub 1/,...,y/sub i/) such that q/sub y1/,...,y/sub i/=0. As an alternative computational approach, we propose an iterative algorithm that converges to the optimal joint reproduction distribution q/sub y1/,...,y/sub N/, if initialized with q/sub y1/,...,y/sub N/>0 everywhere. For nonscalable coding (N=1), the algorithm specializes to the Blahut-Arimoto (1972) algorithm. The algorithm may be used to directly compute the RD bound, or as an optimality testing procedure by applying it to a perturbed tentative solution q. We address two additional difficulties due to the higher dimensionality of the RD surface in the scalable (N>1) case, namely, identifying the sufficient set of Lagrangian parameters to span the entire RD bound; and the problem of efficient navigation on the RD surface to compute a particular RD point.
On Conditions for Linearity of Optimal Estimation

When is optimal estimation linear? It is well known that when a Gaussian source is contaminated with Gaussian noise, a linear estimator minimizes the mean square estimation error. This paper analyzes, more generally, the conditions for linearity of optimal estimators. Given a noise (or source) distribution, and a specified signal-to-noise ratio (SNR), we derive conditions for existence and uniqueness of a source (or noise) distribution for which the Lp optimal estimator is linear. We then show that if the noise and source variances are equal, then the matching source must be distributed identically to the noise. Moreover, we prove that the Gaussian source-channel pair is unique in the sense that it is the only source-channel pair for which the mean square error (MSE) optimal estimator is linear at more than one SNR values. Furthermore, we show the asymptotic linearity of MSE optimal estimators for low SNR if the channel is Gaussian regardless of the source and, vice versa, for high SNR if the source is Gaussian regardless of the channel. The extension to the vector case is also considered where besides the conditions inherited from the scalar case, additional constraints must be satisfied to ensure linearity of the optimal estimator.
Vector quantization by deterministic annealing

A deterministic annealing approach is suggested to search for the optimal vector quantizer given a set of training data. The problem is reformulated within a probabilistic framework. No prior knowledge is assumed on the source density, and the principle of maximum entropy is used to obtain the association probabilities at a given average distortion. The corresponding Lagrange multiplier is inversely related to the 'temperature' and is used to control the annealing process. In this process, as the temperature is lowered, the system undergoes a sequence of phase transitions when existing clusters split naturally, without use of heuristics. The resulting codebook is independent of the codebook used to initialize the iterations.