nils-herrmann/clean_scirepeval huggingface dataset:
Joint Reconstruction of Multiview Compressed Images

Distributed representation of correlated multiview images is an important problem that arises in vision sensor networks. This paper concentrates on the joint reconstruction problem where the distributively compressed images are decoded together in order to take benefit from the image correlation. We consider a scenario where the images captured at different viewpoints are encoded independently using common coding solutions (e.g., JPEG) with a balanced rate distribution among different cameras. A central decoder first estimates the inter-view image correlation from the independently compressed data. The joint reconstruction is then cast as a constrained convex optimization problem that reconstructs total-variation (TV) smooth images, which comply with the estimated correlation model. At the same time, we add constraints that force the reconstructed images to be as close as possible to their compressed versions. We show through experiments that the proposed joint reconstruction scheme outperforms independent reconstruction in terms of image quality, for a given target bit rate. In addition, the decoding performance of our algorithm compares advantageously to state-of-the-art distributed coding schemes based on motion learning and on the DISCOVER algorithm.
Chebyshev polynomial approximation for distributed signal processing

Unions of graph Fourier multipliers are an important class of linear operators for processing signals defined on graphs. We present a novel method to efficiently distribute the application of these operators to the high-dimensional signals collected by sensor networks. The proposed method features approximations of the graph Fourier multipliers by shifted Chebyshev polynomials, whose recurrence relations make them readily amenable to distributed computation. We demonstrate how the proposed method can be used in a distributed denoising task, and show that the communication requirements of the method scale gracefully with the size of the network.
Consistent view synthesis in interactive multiview imaging

An important question in the design of interactive multiview systems consists in determining the information needed by the decoder for high quality navigation between the views. Most of the existing techniques focus on the captured sequences and only consider their transmission, which does not guarantee consistency among receiver-generated frames of chosen virtual views. In this work, we propose a solution that additional transmits auxiliary information in order to help the construction of synthesized views, especially in the occluded areas. Comparative results with existing approaches validate this novel representation of multiview data for interactive navigation. We show that decoding quality and consistency among frames are improved with only a small share of additional information.
Low-rate and flexible image coding with redundant representations

New breakthroughs in image coding possibly lie in signal decomposition through nonseparable basis functions that can efficiently capture edge characteristics, present in natural images. The work proposed in this paper provides an adaptive way of representing images as a sum of two-dimensional features. It presents a low bit-rate image coding method based on a matching pursuit (MP) expansion, over a dictionary built on anisotropic refinement and rotation of contour-like atoms. This method is shown to provide, at low bit rates, results comparable to the state of the art in image compression, represented here by JPEG2000 and SPIHT, with generally a better visual quality in the MP scheme. The coding artifacts are less annoying than the ringing introduced by wavelets at very low bit rate, due to the smoothing performed by the basis functions used in the MP algorithm. In addition to good compression performances at low bit rates, the new coder has the advantage of producing highly flexible streams. They can easily be decoded at any spatial resolution, different from the original image, and the bitstream can be truncated at any point to match diverse bandwidth requirements. The spatial adaptivity is shown to be more flexible and less complex than transcoding operations generally applied to state of the art codec bitstreams. Due to both its ability for capturing the most important parts of multidimensional signals, and a flexible stream structure, the image coder proposed in this paper represents an interesting solution for low to medium rate image coding in visual communication applications.
Progressive Coding of 3-D Objects Based on Overcomplete Decompositions

This paper presents a progressive coding scheme for 3-D objects, based on overcomplete signal expansions on the 2-D sphere. Due to increased freedom in the basis construction, redundant expansions have shown interesting approximation properties in the decomposition of signals with multidimensional singularities organized along embedded submanifolds. We propose to map simple 3-D models on 2-D spheres and then to decompose the signal over a redundant dictionary of oriented and anisotropic atoms living on the sphere. The signal expansion is computed iteratively with a matching pursuit algorithm, which greedily selects the most prominent components of the 3-D model. The decomposition therefore inherently represents a progressive stream of atoms, which is advantageously used in the design of scalable representations. An encoder is proposed that compresses the stream of atoms by adaptive coefficient quantization and entropy coding of atom indexes. Experimental results show that the novel coding strategy outperforms state-of-the-art progressive coders in terms of distortion, mostly at low bit rates. Furthermore, since the dictionary is built on structured atoms, the proposed representation simultaneously offers an increased flexibility for easy stream manipulations. We finally illustrate that advantage in the design of a view-dependent transmission scheme
Collaborative video streaming with Raptor network coding

We investigate the problem of collaborative video streaming with Raptor network coding over overlay networks. We exploit path and source diversity, as well as basic processing capabilities of network nodes to increase the overall throughput and improve the video quality at the clients. We consider an architecture where several streaming servers simultaneously deliver video information to a set of clients. The servers apply Raptor coding on the video packets for error resiliency, and the forwarding peer nodes further combine the Raptor coded video packets in order to increase the packet diversity in the network. We find the optimal source and channel rate allocation in such a collaborative streaming system. The resulting scheme efficiently exploits the available network resources for improved video quality. The experimental evaluation demonstrates that it typically outperforms Raptor video streaming systems that do not use network coding.
Randomized Network Coding for UEP video delivery in overlay networks

This paper presents a receiver-driven video delivery algorithm that exploits a novel Randomized Network Coding (RNC) scheme for unequal error protection (UEP). The main idea of our approach is to account for the unequal importance of media packets in the network coding algorithm for efficient stream delivery in lossy overlay networks. Based on the requests from their neighbours, the network nodes properly combine packets and forward them to their children nodes. The network coding operations at every node are formulated as a log-concave optimization problem, which is solved with a greedy algorithm in only a few iterations. Our experimental results demonstrate that the proposed scheme permits to respect the priorities between the different packet classes. It further outperforms baseline network coding techniques for video streaming in overlay networks.
Learning Laplacian Matrix in Smooth Graph Signal Representations

The construction of a meaningful graph plays a crucial role in the success of many graph-based representations and algorithms for handling structured data, especially in the emerging field of graph signal processing. However, a meaningful graph is not always readily available from the data, nor easy to define depending on the application domain. In particular, it is often desirable in graph signal processing applications that a graph is chosen such that the data admit certain regularity or smoothness on the graph. In this paper, we address the problem of learning graph Laplacians, which is equivalent to learning graph topologies, such that the input data form graph signals with smooth variations on the resulting topology. To this end, we adopt a factor analysis model for the graph signals and impose a Gaussian probabilistic prior on the latent variables that control these signals. We show that the Gaussian prior leads to an efficient representation that favors the smoothness property of the graph signals. We then propose an algorithm for learning graphs that enforces such property and is based on minimizing the variations of the signals on the learned graph. Experiments on both synthetic and real world data demonstrate that the proposed graph learning framework can efficiently infer meaningful graph topologies from signal observations under the smoothness prior.
Low-complexity multiview video coding

We consider the problem of complexity reduction in Multiview Video Coding (MVC). We provide a unique comprehensive study that integrates and compares the different low complexity encoding techniques that have been proposed at different levels of the MVC system. In addition, we propose a novel complexity reduction method that takes advantage of the relationship between disparity vectors along time. The relationship is exploited with respect to the motion activity in the frame, as well as with the position of the frame in the Group of Pictures. We integrate this technique into our unique comprehensive framework and evaluate the performance of the resulting system in different setups. We show that the effective combination of complexity reduction techniques results in saving up to 93% in encoding time at the cost of only 0.08 dB in peak signal-to-noise ratio (PSNR) and 1.64% increase in bitrate compared to the standard MVC implementation (JMVM 6.0).
Omnidirectional Views Selection for Scene Representation

This paper proposes a new method for the selection of sets of omnidirectional views, which contribute together to the efficient representation of a 3D scene. When the 3D surface is modelled as a function on a unit sphere, the view selection problem is mostly governed by the accuracy of the 3D surface reconstruction from non-uniformly sampled datasets. A novel method is proposed for the reconstruction of signals on the sphere from scattered data, using a generalization of the spherical Fourier transform. With that reconstruction strategy, an algorithm is then proposed to select the best subset of n views, from a predefined set of viewpoints, in order to minimize the overall reconstruction error. Starting from initial viewpoints determined by the frequency distribution of the 3D scene, the algorithm iteratively refines the selection of each of the viewpoints, in order to maximize the quality of the representation. Experiments show that the algorithm converges towards a minimal distortion, and demonstrate that the selection of omnidirectional views is consistent with the frequency characteristics of the 3D scene.
Distributed Image Processing [From the Guest Editors]

The emergence of image processing systems with multiple visual sensors has created the need for novel signal processing algorithms that are able to cope with the specific constraints imposed by distributed architectures. For example, large-scale camera networks have been deployed for a variety of applications including video surveillance in buildings and public spaces, as well as monitoring roadways and environmental conditions. Given the bandwidth limitations imposed by the network as well as computation constraints at nodes of such networks, solutions based on distributed processing become particularly attractive from the point of view of communication and computational costs as well as system scalability.
Near-optimal content replication for interactive multiview video streaming

In interactive multiview video streaming (IMVS), a client can watch the multiview video and interact with it by switching to different viewing angles at frozen moments. To provide scalable IMVS services, a content provider often deploys distributed content servers with heterogeneous storage capacities in order to collaboratively replicate video segments for their clients. IMVS presents a new challenge in content replication: which video segments to replicate to support interactive view-switching during an IMVS session. In this paper, we first formulate the content replication problem as an integer linear programming (ILP) problem, which is proven to be NP-hard. We then propose a fast algorithm to solve it with bounded approximation error. In this algorithm, we first solve the ILP problem as a relaxed LP problem, and then heuristically round the resulting fractional LP solution to integer towards ILP feasibility. Simulation results show that our replication strategy can substantially reduce access costs compared to a commonly used replication scheme and a state-of-the-art replication scheme.
R-D optimized auxiliary information for inpainting-based view synthesis

Texture and depth maps of two neighboring camera viewpoints are typically required for synthesis of an intermediate virtual view via depth-image-based rendering (DIBR). However, the bitrate overhead required for reconstruction of multiple texture and depth maps at decoder can be large. The performance of multiview video encoders such as MVC is limited by the simple fact that the chosen representation is inherently redundant: a texture or depth pixel visible from both camera viewpoints is represented twice. In this paper, we propose an alternative 3D scene representation without such redundancy, yet at decoder, one can still reconstruct texture and depth maps of two camera viewpoints for DIBR-based synthesis of intermediate views. In particular, we propose to first encode texture and depth videos of a single viewpoint, which are used to synthesize the uncoded viewpoint via DIBR at decoder. Then, we encode additional rate-distortion (RD) optimal auxiliary information (AI) to guide an inpainting-based hole-filling algorithm at decoder and complete the missing information due to disocclusion. For a missing pixel patch in the synthesized view, the AI can: i) be skipped and then let the decoder by itself retrieve the missing information, ii) identify a suitable spatial region in the reconstructed view for patch-matching, or iii) explicitly encode missing pixel patch if no satisfactory patch can be found in the reconstructed view. Experimental results show that our alternative representation can achieve up to 41% bit-savings compared to H.264/MVC implementation.
Optimal FEC rate for media streaming in active networks [video streaming]

This paper addresses the problem of optimal channel rate allocation for media streaming in active networks, where intermediate nodes are able to perform basic FEC decoding/encoding operations. FEC performance is analyzed in the case of hop-by-hop FEC protection, and compared with an end-to-end FEC scenario, in order to demonstrate the benefits of FEC operations in the intermediate nodes. An optimization problem is formulated, based on a distortion model for video streaming over lossy channels. Finally, the two streaming scenarios are compared in the particular case of MPEG-4 video, under a constrained end-to-end delay. FEC operations in intermediate nodes are shown to become especially useful when the links on the streaming path have quite heterogenous characteristics
Dependent Packet Transmission Policies in Rate-Distortion Optimized Media Scheduling

This paper addresses the problem of streaming packetized media over a lossy packet network, with sender-driven (re)transmissions and receiver acknowledgements. It extends the Markovian formulation of the rate-distortion optimized (RaDiO) streaming framework by allowing the transmission schedule of a media data unit to become contingent on the acknowledgements relative to other data units. Media decoding dependencies are generally considered in state-of-the-art rate-distortion optimized scheduling algorithms. However, the set of eligible packet schedules are restricted to independent streaming policies, where the transmission strategy envisioned for a data unit at future transmission opportunities only depends on its own acknowledgment, and not on the acknowledgments received for other data units. This paper questions the validity of this assumption in the design of rate-distortion optimal streaming solutions, and provides a first attempt in the formal derivation of the benefit offered by dependent policies. One of the main contributions of our paper is to propose a methodology that limits the search space of dependent policies to relevant dependencies that are likely to bring a rate-distortion benefit, in order to solve an optimization problem that is a priori computationally intractable. Extensive simulations validate the proposed approach that focuses on relevant dependencies between streaming policies. We further show that the benefit of dependent streaming policies is actually marginal in practical scenarios where the gain in distortion per unit of rate decreases along the media decoding dependency path. It represents the first demonstration that the common assumption of independent streaming policies is valid in many common streaming scenarios. However, experimental results also demonstrate significant benefits and encourage a careful investigation of dependent policies when the content is characterized by an increase of the benefit per transmission unit brought along the data unit dependency path.
Dictionary Learning

We describe methods for learning dictionaries that are appropriate for the representation of given classes of signals and multisensor data. We further show that dimensionality reduction based on dictionary representation can be extended to address specific tasks such as data analy sis or classification when the learning includes a class separability criteria in the objective function. The benefits of dictionary learning clearly show that a proper understanding of causes underlying the sensed world is key to task-specific representation of relevant information in high-dimensional data sets.
Classification-Specific Feature Sampling for Face Recognition

Feature extraction based on different types of signal filters has received a lot of attention in the context of face recognition. It generally results into extremely high dimensional feature vectors, and sampling of the coefficients is required to reduce their dimensionality. Unfortunately, uniform sampling that is commonly used to that aim, does not consider the specificities of the recognition task in selecting the most relevant features. In this paper, we propose to formulate the sampling problem as a supervised feature selection problem where features are carefully selected according to a well defined discrimination criterion. The sampling process becomes specific to the classification task, and further facilitates the face recognition operations. We propose to build features on random filters, and Gabor wavelets, since they present interesting characteristics in terms of discrimination, due to their high frequency components. Experimental results show that the proposed feature selection method outperforms uniform sampling, and that random filters are very competitive with the common Gabor wavelet filters for face recognition tasks
Adaptive entropy-constrained matching pursuit quantization

This paper proposes an adaptive entropy-constrained matching pursuit coefficient quantization scheme. The quantization scheme takes benefit of the inherent properties of matching pursuit streams where coefficient energy decreases along with the iteration number. The decay rate can moreover be upper-bounded with an exponential curve driven by the redundancy of the dictionary. An optimal entropy-constrained quantization scheme can thus be derived once the dictionary is known. We propose to approximate this optimal quantization scheme by adaptive quantization of successive coefficients whose actual values are used to update the quantization scheme parameters. This new quantization scheme is shown to outperform classical exponential quantization in the case of both random dictionaries and practical image coding with Gabor dictionaries.
Wide-baseline object interpolation using shape prior regularization of epipolar plane images

This paper considers the synthesis of intermediate views of an object captured by two calibrated and widely spaced cameras. Based only on those two very different views, our paper proposes to reconstruct the object Epipolar Plane Image Volume  (EPIV), which describes the object transformation when continuously moving the viewpoint of the synthetic view in-between the two reference cameras. This problem is clearly ill-posed since the occlusions and the foreshortening effect make the reference views significantly different when the cameras are far apart. Our main contribution consists in disambiguating this ill-posed problem by constraining the interpolated views to be consistent with an object shape prior. This prior is learnt based on images captured by the two reference views, and consists in a nonlinear shape manifold representing the plausible silhouettes of the object described by Elliptic Fourier Descriptors. Experiments on both synthetic and natural images show that the proposed method preserves the topological structure of objects during the intermediate view synthesis, while dealing effectively with the self-occluded regions and with the severe foreshortening effect associated to wide-baseline camera configurations.
Scale-Invariant Features and Polar Descriptors in Omnidirectional Imaging

We propose a method to compute scale-invariant features in omnidirectional images. We present a formulation based on the Riemannian geometry for the definition of differential operators on non-Euclidian manifolds that adapt to the mirror and lens structures in omnidirectional imaging. These operators lead to a scale-space analysis that preserves the geometry of the visual information in omnidirectional images. We then build a novel scale-invariant feature detection framework for omnidirectional images that can be mapped on the sphere. We further present a new descriptor and feature matching solution for these omnidirectional images. The descriptor builds on the log-polar planar descriptors and adapts the descriptor computation to the specific geometry and the nonuniform sampling density of omnidirectional images. We also propose a rotation-invariant matching method that eliminates the orientation computation during the feature detection phase and thus decreases the computational complexity. Experimental results demonstrate that the new feature computation method combined with the adapted descriptors offers promising detection and matching performance, i.e., it improves on the common scale-invariant feature transform (SIFT) features computed on the unwrapped omnidirectional images, as well as spherical SIFT features. Finally, we show that the proposed framework also permits to match features between images with different native geometry.