nils-herrmann/clean_scirepeval huggingface dataset:
Global Auto-Regressive Depth Recovery via Iterative Non-Local Filtering

Existing depth sensing techniques have many shortcomings in terms of resolution, completeness, and accuracy. The performance of 3-D broadcasting systems is therefore limited by the challenges of capturing high-resolution depth data. In this paper, we present a novel framework for obtaining high-quality depth images and multi-view depth videos from simple acquisition systems. We first propose a single depth image recovery algorithm based on auto-regressive (AR) correlations. A fixed-point iteration algorithm under the global AR modeling is derived to efficiently solve the large-scale quadratic programming. Each iteration is equivalent to a nonlocal filtering process with a residue feedback. Then, we extend our framework to an AR-based multi-view depth video recovery framework, where each depth map is recovered from low-quality measurements with the help of the corresponding color image, depth maps from neighboring views, and depth maps of temporally adjacent frames. AR coefficients on nonlocal spatiotemporal neighborhoods in the algorithm are designed to improve the recovery performance. We further discuss the connections between our model and other methods like graph-based tools, and demonstrate that our algorithms enjoy the advantages of both global and local methods. Experimental results on both the Middleburry datasets and other captured datasets finally show that our method is able to improve the performances of depth images and multi-view depth videos recovery compared with state-of-the-art approaches.
Guest Editorial Special Issue on Visual Computing in the Cloud: Mobile Computing

Recent advances in mobile devices (e.g., smartphones and wearables) and wireless technologies are fueling a new wave of user demands for an improved user experience. Indeed, users are not only expecting ubiquitous network connections for traditional services (e.g., messaging and calling), but also demanding extensive access to a wealth of video contents and services. However, this growing demand is seriously hindered by the fact that the onboard resources with mobile devices are inherently limited and their growth rate falls behind that of their desktop counterparts. It follows that new solutions should be in order to resolve this fundamental tussle. Fortunately, the emerging cloud computing offers a natural solution to extend the desktop visual experience to mobile devices. It actually provides both computational and storage support for media-rich applications with both front-end and back-end functionalities.
The virtue of patience when scheduling media in presence of feedback

We consider streaming of pre-encoded and packetized media over best-effort networks in presence of acknowledgment feedback. Given an estimation of future transmission resources and knowing about past transmissions and received acknowledgments, a scheduling algorithm is defined as a mechanism that selects the data to send over the network at any given time, so as to minimize the end-to-end distortion. Our work first reveals the sub-optimality of popular greedy schedulers, which might be strongly penalized by anticipated retransmissions. It then proposes an original scheduling algorithm that avoids premature retransmissions, while preserving the simplicity of the greedy paradigm. The proposed patient greedy (PG) scheduler appears to save up to 50% of rate in comparison with the conventional greedy approach.
Context-Adaptive Information Flow Allocation and Media Delivery in Online Social Networks

This paper investigates context-driven flow allocation and media delivery in online social networks. We exploit information on contacts and content preferences found in social networking applications to provide efficient network services and operation at the underlying transport layer. We formulate a linear programming framework that maximizes the information flow-cost ratio of the transport network serving the nodes in the social graph. For practical deployments, we also design a distributed version of the optimization framework that provides similar performance to its centralized counterpart, with lower complexity. In addition, we devise a tracker-based system for efficient content discovery in peer-to-peer (P2P) systems based on social network information. Finally, we design a context-aware packet scheduling technique that maximizes the utility of media delivery among the members of the social network. We provide a comprehensive investigation of the performance of our optimization strategies through both simulations and analysis. We demonstrate their significant advantages over several performance factors relative to conventional solutions that do not employ social network information in their operation.
Luminance coding in graph-based representation of multiview images

Multi-view video transmission poses great challenges because of its data size and dimension. Therefore, how to design efficient 3D scene representations and coding (of luminance and geometry) has become a critical research topic. Recently, the graph-based representation (GBR) is introduced, which provides a lossless compression of multi-view geometry by connecting informative pixels among views. This representation has been shown as a promising alternative to the classical depth-based representation, where the view synthesis accuracy is hard to control. In this work, we study the luminance compression under GBR, which is not well considered in existing literature. With a proper structural reformulation, we show that the graph-based transform can be applied on the GBR paradigm, hence better extracting the correlation among pixels along graph connections. Moreover, we extend the popular SPIHT coding scheme to further improve coding efficiency. The experimental results show that our method leads to better RD coding performance as compared the classical luminance coding algorithms.
ICon: Interference concentration for uplink in multicell OFDMA networks

In this work we propose a novel inter-cell interference coordination (ICIC) and resource allocation method. Our aim is to maximize the rate of the worst performing user in all cells. We solve the problem in two phases, inter-cell and intra-cell resource management: first we define an ICIC scheme called interference concentration (ICon) in order to manage resources across cells, then each cell independently performs resource allocation to its users while meeting the constraints imposed by ICon. Finally, we adapt the ICIC method in order to balance the performance achieved in neighboring cells. Differently than Soft Frequency Reuse (SFR), ICon assigns received interference power limits, or the Interference Power Profiles (IPPs) rather than transmit power limits. The IPP determines the interference level the cell tolerates on each band. The intuition behind this change is that the interference to a given cell can be concentrated on a small band, resulting in more efficient use of bandwidth. In the intra-cell resource management phase, each cell allocates power and sub-bands to its users given their location in the cell, maximizing the minimum rate such that the IPP of none of its neighboring cells is violated. In order to balance the performance across all cells we use gradient-like updates to IPPs of cells. Finally we simulate an LTE-like system and compare the performance of our method with reuse 1, static FFR and SFR with proportionally fair scheduling of users in each cell. Static ICon achieves 18% higher 5 percentile rate than reuse 1 which was the best of these methods. Adaptive ICon is found to converge almost immediately, and adds an additional 11% to this gain.
Video Packet Selection and Scheduling for Multipath Streaming

This paper addresses the problem of choosing the best streaming policy for distortion optimal multipath video delivery, under network bandwidth and playback delay constraints. The streaming policy consists in a joint selection of the network path and of the video packets to be transmitted, along with their sending time. A simple streaming model is introduced, which takes into account the video packet importance, and the dependencies between packets. A careful timing analysis allows to compute the quality perceived by the receiver for a constrained playback delay, as a function of the streaming policy. We derive an optimization problem based on a video abstraction model, under the assumption that the server knows, or can predict accurately the state of the network. A detailed analysis of constrained multipath streaming systems provides helpful insights to design an efficient branch and bound algorithm that finds the optimal streaming strategy. This solution allows to bound the performance of any scheduling strategy, but the complexity of the algorithm becomes rapidly intractable. We therefore propose a fast heuristic-based algorithm, built on load-balancing principles. It allows to reach close to optimal performance with a polynomial time complexity. The algorithm is then adapted to live streaming scenarios, where the server has only a partial knowledge of the packet stream, and the channel bandwidth. Extensive simulations show that the proposed algorithm only induces a negligible distortion penalty compared to the optimal strategy, even when the optimization horizon is limited, or the rate estimation is not perfect. Simulation results also demonstrate that the proposed scheduling solution performs better than common scheduling algorithms, and therefore represents a very efficient low-complexity multipath streaming algorithm, for both stored and live video services
Optimal Image Alignment With Random Projections of Manifolds: Algorithm and Geometric Analysis

This paper addresses the problem of image alignment based on random measurements. Image alignment consists of estimating the relative transformation between a query image and a reference image. We consider the specific problem where the query image is provided in compressed form in terms of linear measurements captured by a vision sensor. We cast the alignment problem as a manifold distance minimization problem in the linear subspace defined by the measurements. The transformation manifold that represents synthesis of shift, rotation, and isotropic scaling of the reference image can be given in closed form when the reference pattern is sparsely represented over a parametric dictionary. We show that the objective function can then be decomposed as the difference of two convex functions (DC) in the particular case where the dictionary is built on Gaussian functions. Thus, the optimization problem becomes a DC program, which in turn can be solved globally by a cutting plane method. The quality of the solution is typically affected by the number of random measurements and the condition number of the manifold that describes the transformations of the reference image. We show that the curvature, which is closely related to the condition number, remains bounded in our image alignment problem, which means that the relative transformation between two images can be determined optimally in a reduced subspace.
Learning Heat Diffusion Graphs

Information analysis of data often boils down to properly identifying their hidden structure. In many cases, the data structure can be described by a graph representation that supports signals in the dataset. In some applications, this graph may be partly determined by design constraints or predetermined sensing arrangements. In general though, the data structure is not readily available nor easily defined. In this paper, we propose to represent structured data as a sparse combination of localized functions that live on a graph. This model is more appropriate to represent local data arrangements than the classical global smoothness prior. We focus on the problem of inferring the connectivity that best explains the data samples at different vertices of a graph that is a priori unknown. We concentrate on the case where the observed data are actually the sum of heat diffusion processes, which is a widely used model for data on networks or other irregular structures. We cast a new graph learning problem and solve it with an efficient nonconvex optimization algorithm. Experiments on both synthetic and real world data finally illustrate the benefits of the proposed graph learning framework and confirm that the data structure can be efficiently learned from data observations only. We believe that our algorithm will help solving key questions in diverse application domains such as social and biological network analysis where it is crucial to unveil proper data structure for understanding and inference.
Dictionary Learning for Stereo Image Representation

One of the major challenges in multi-view imaging is the definition of a representation that reveals the intrinsic geometry of the visual information. Sparse image representations with overcomplete geometric dictionaries offer a way to efficiently approximate these images, such that the multi-view geometric structure becomes explicit in the representation. However, the choice of a good dictionary in this case is far from obvious. We propose a new method for learning overcomplete dictionaries that are adapted to the joint representation of stereo images. We first formulate a sparse stereo image model where the multi-view correlation is described by local geometric transforms of dictionary elements (atoms) in two stereo views. A maximum-likelihood (ML) method for learning stereo dictionaries is then proposed, where a multi-view geometry constraint is included in the probabilistic model. The ML objective function is optimized using the expectation-maximization algorithm. We apply the learning algorithm to the case of omnidirectional images, where we learn scales of atoms in a parametric dictionary. The resulting dictionaries provide better performance in the joint representation of stereo omnidirectional images as well as improved multi-view feature matching. We finally discuss and demonstrate the benefits of dictionary learning for distributed scene representation and camera pose estimation.
Complexity constrained representation selection for dynamic adaptive streaming

In this paper, we propose a representation selection optimization problem for complexity constrained adaptive video streaming that properly takes into account the different complexity-rate-distortion (C-R-D) characteristics of the videos when implementing rate control for desired representations. Our objective is to maximize the expected video distortion reduction of users, subject not only to encoding rate constraints, but also to complexity constraints. We prove that our optimization problem is a submodular maximization problem with two knapsack constraints. A weighted rate and complexity cost benefit greedy algorithm is then developed to obtain an approximate solution with polynomial time complexity and good approximation performance in simulations.
In-network view re-sampling for interactive free viewpoint video streaming

Interactive free viewpoint video offers the possibility for each user to independently choose the views of a 3D scene to be displayed at the decoder. The visual content is commonly represented by N texture and depth map pairs that capture different viewpoints. A server selects an appropriate subset of M ≤ N views for transmission, so that the user can freely navigate in the corresponding window of viewpoints without being affected by network delay. During navigation, a user can synthesize any intermediate virtual view image in the navigation window via depth-image-based rendering (DIBR) using two nearby camera views as references. When the available bandwidth is too small to transmit all camera views typically used to synthesize views in the navigation window, we propose to synthesize intermediate virtual views as new references for transmission - a resampling of viewpoints for the 3D scene - so that the synthesized view distortion within the navigation window is minimized. We formulate a combinatorial optimization problem to find the best set of M virtual views to synthesize as new references, and show that the problem is NP-hard. We approximate the original problem with a new reference view equivalence model and derive in this case an optimal dynamic programming algorithm to determine the best set of M views to be transmitted to each user. Experimental results show that synthesizing virtual views as new references for client-side view synthesis can outperform simple selection from camera views by up to 0.73dB in synthesized view quality.
Learning from sparse codes

In this paper we address the problem of learning image structures directly from sparse codes. We first model images as linear combinations of molecules, which are themselves groups of atoms from a redundant dictionary. We then formulate a new structure learning problem that learns molecules directly from image sparse codes, namely from the image representation in the atom domain. We build on a structural difference function that permits to compare molecules and we derive an algorithm that analyses sparse codes and estimates the most relevant signal structure without reconstructing the images. Experiments on both synthetic and real image datasets confirm the benefits of our new method compared to traditional learning methods.
Advanced Solutions for Quality-Oriented Multimedia Broadcasting

Multimedia content is increasingly being delivered via different types of networks to viewers in a variety of locations and contexts using a variety of devices. The ubiquitous nature of multimedia services comes at a cost, however. The successful delivery of multimedia services will require overcoming numerous technological challenges many of which have a direct effect on the quality of the multimedia experience. For example, due to dynamically changing requirements and networking conditions, the delivery of multimedia content has traditionally adopted a best effort approach. However, this approach has often led to the end-user perceived quality of multimedia-based services being negatively affected. Yet the quality of multimedia content is a vital issue for the continued acceptance and proliferation of these services. Indeed, end-users are becoming increasingly quality-aware in their expectations of multimedia experience and demand an ever-widening spectrum of rich multimedia-based services. As a consequence, there is a continuous and extensive research effort, by both industry and academia, to find solutions for improving the quality of multimedia content delivered to the users; as well, international standards bodies, such as the International Telecommunication Union (ITU), are renewing their effort on the standardization of multimedia technologies. There are very different directions in which research has attempted to find solutions in order to improve the quality of the rich media content delivered over various network types. It is in this context that this special issue on broadcast multimedia quality of the IEEE Transactions on Broadcasting illustrates some of these avenues and presents some of the most significant research results obtained by various teams of researchers from many countries. This special issue provides an example, albeit inevitably limited, of the richness and breath of the current research on multimedia broadcasting services. The research issues addressed in this special issue include, among others, factors that influence user perceived quality, encoding-related quality assessment and control, transmission and coverage-based solutions and objective quality measurements.
Prioritized Distributed Video Delivery With Randomized Network Coding

We address the problem of prioritized video streaming over lossy overlay networks. We propose to exploit network path diversity via a novel randomized network coding (RNC) approach that provides unequal error protection (UEP) to the packets conveying the video content. We design a distributed receiver-driven streaming solution, where a client requests packets from the different priority classes from its neighbors in the overlay. Based on the received requests, a node in turn forwards combinations of the selected packets to the requesting peers. Choosing a network coding strategy at every node can be cast as an optimization problem that determines the rate allocation between the different packet classes such that the average distortion at the requesting peer is minimized. As the optimization problem has log-concavity properties, it can be solved with low complexity by an iterative algorithm. Our simulation results demonstrate that the proposed scheme respects the relative priorities of the different packet classes and achieves a graceful quality adaptation to network resource constraints. Therefore, our scheme substantially outperforms reference schemes such as baseline network coding techniques as well as solutions that employ rateless codes with built-in UEP properties. The performance evaluation provides additional evidence of the substantial robustness of the proposed scheme in a variety of transmission scenarios.
Image inpainting through neural networks hallucinations

We consider in this paper the problem of image inpainting, where the objective is to reconstruct large continuous regions of missing or deteriorated parts of an image. Traditional in-painting algorithms are unfortunately not well adapted to handle such corruptions as they rely on image processing techniques that cannot properly infer missing information when the corrupted holes are too large. To tackle this problem, we propose a novel approach where we rely on the hallucinations of pre-trained neural networks to fill large holes in images. To generate globally coherent images, we further impose smoothness and consistency regularization, thereby constraining the neural network hallucinations. Through illustrative experiments, we show that pre-trained neural networks contain crucial prior information that can effectively guide the reconstruction process of complex inpainting problems.
Distributed Media Rate Allocation in Overlay Networks

We address the problem of distributed path selection and rate allocation for media streaming in overlay networks. Under the assumption that each node has only a local view of the network, we propose a distributed algorithm for joint path selection, and rate allocation, in order to minimize the end-to-end media distortion. The distributed algorithm performs iteratively, by greedy rate allocation for all incoming media flows on the outgoing links at each intermediate node. Our algorithm is shown to converge to the optimal rate allocation solution in a very small number of iterations, and to outperform heuristic distributed rate allocation mechanisms for a number of random network topologies
Compressed classification of observation sets with linear subspace embeddings

We consider the problem of classification of a pattern from multiple compressed observations that are collected in a sensor network. In particular, we exploit the properties of random projections in generic sensor devices and we take some first steps in introducing linear dimensionality reduction techniques in the compressed domain. We design a classification framework that consists in embedding the low dimensional classification space given by classical linear dimensionality reduction techniques in the compressed domain. The measurements of the multiple observations are then projected onto the new classification subspace and are finally aggregated in order to reach a classification decision. Simulation results verify the effectiveness of our scheme and illustrate that compressed measurements combined with information diversity lead to efficient dimensionality reduction in simple sensing architectures.