nils-herrmann/clean_scirepeval huggingface dataset:
A novel upsampling scheme for depth map compression in 3DTV system

In 3D video transmission, the depth map is normally compressed by resolution reduction to save bandwidth. The lost information in resolution reduction is recovered by an appropriate upsampling algorithm in decoding step. Most previous work considers the depth upsampling problem as common 2D map upsampling problem and do not take the intrinsic property of depth map into consideration. In this paper, we propose a novel two-step depth map upsampling scheme to address the problem on 3D videos. The first step utilizes the full resolution 2D color map to direct the reconstruction of a more accurate full resolution depth map. And the second step further flats the reconstructed depth map to ensure its local uniformity. Test results show that the proposed novel upsam-pling scheme achieves up to 2dB coding gains for the rendering of free-viewpoint video, and improves its perceptual quality significantly.
A Matrix-Based Approach to Unsupervised Human Action Categorization

Human action, as the basic unit of most human-relevant video content, bridges the gap between low-level visual features and high-level semantics. Human action recognition is of great significance in the applications of human-computer interaction, intelligent video surveillance, video retrieval and search. In this paper, we propose a novel unsupervised approach to mining categories from action video sequences, which consists of two modules: action representation for video data structurization and learning model for unsupervised categorization. In action representation, a novel view of video decomposition is presented. Videos are regarded as spatially distributed dynamic pixel time series, and these dynamic pixels are first quantized into pixel prototypes. After replacing the pixel time series with their corresponding prototype labels, the video sequences are compressed into two-dimensional action matrices. In the learning model, we put these matrices together to form an multi-action tensor, and propose the joint matrix factorization method to simultaneously cluster the pixel prototypes into pixel signatures, and matrices into action classes with the consideration of the duality between pixel clustering and action clustering. The approach is tested on public and popular Weizmann, and KTH datasets, and promising results are achieved.
Joint Optimization of Data Sponsoring and Edge Caching for Mobile Video Delivery

In this work, we study the joint optimization of edge caching and data sponsoring for a video content provider (CP), aiming at reducing the content delivery cost and increasing the CP's revenue. Specifically, we formulate the joint optimization problem as a two-stage decision problem for the CP. In Stage I, the CP determines the edge caching policy (for a relatively long time period). In Stage II, the CP decides the real-time data sponsoring strategy for each content request within the period. We first propose a Lyapunov- based online sponsoring strategy in Stage II, which reaches 90% of the offline maximum performance (benchmark). We then solve the edge caching problem in Stage I based on the online sponsoring strategy proposed in Stage II, and show that the optimal caching policy depends on the aggregate user request for each content in each location. Simulations show that such a joint optimization can increase the CP's revenue by 30%↑100%, comparing with the purely data sponsoring (i.e., without edge caching).
A Novel Event-Oriented Segment-of-Interest Discovery Method for Surveillance Video

During recent years, the quick development of computer techniques has witnessed the ever-increasing surveillance video data, which essentially pose great challenge on the data storage, management, analysis and even retrieval. Considering that most of the high volume of data is with no interest, we mainly investigate the problem of effectively and efficiently discovering segments-of-interest (SoI) in this paper. To do so, we propose a novel event-oriented Sol discovery method in two steps: first, we represent an event by modeling pixels' change in temporal-spatial space, aiming to unify both the inter-frames and frames-background changes; second, with the benefit of unsupervised learning, the prototype-event models could be learned from these detected events and in turn exploited to measure the interest factor of each prototype-event. The experiment results demonstrate that the proposed method precisely discriminate different events and effectively discover Sols.
Social-Aware Video Recommendation for Online Social Groups

Group recommendation plays a significant role in today's social media systems, where users form social groups to receive multimedia content together and interact with each other, instead of consuming the online content individually. Limitations of traditional group recommendation approaches are as follows. First, they usually infer group members’ preferences by their historical behaviors, failing to capture inactive users’ preferences from the sparse historical data. Second, relationships between group members are not studied by these approaches, which fail to capture the inherent personality of members in a group. To address these issues, we propose a social-aware group recommendation framework that jointly utilizes both social relationships and social behaviors to not only infer a group's preference, but also model the tolerance and  altruism characteristics of group members. Based on the observation that the following relationship in the online social network reflects common interests of users, we propose a group preference model based on external experts of group members. Furthermore, we model users’ tolerance (willingness to receive content not preferred) and altruism (willingness to receive content preferred by friends). Finally, based on the group preference model, we design recommendation algorithms for users under different social contexts. Experimental results demonstrate the effectiveness of our approach, which significantly improves the recommendation accuracy against traditional approaches, especially in the cases of inactive group members.
Guiding internet-scale video service deployment using microblog-based prediction

Online microblogging has been very popular in today's Internet, where users exchange short messages and follow various contents shared by people that they are interested in. Among the variety of exchanges, video links are a representative type on a microblogging site. More and more viewers of an Internet video service are coming from microblog recommendations. It is intriguing research to explore the connections between the patterns of microblog exchanges and the popularity of videos, in order to potentially use the propagation patterns of microblogs to guide proactive service deployment of a video sharing system. Based on extensive traces from Youku and Tencent Weibo, a popular video sharing site and a favored microblogging system in China, we explore how patterns of video link propagation in the microblogging system are correlated with video popularity on the video sharing site, at different times and in different geographic regions. Using influential factors summarized from the measurement studies, we further design neural network-based learning frameworks to predict the number of potential viewers of different videos and the geographic distribution of viewers. Experiments show that our neural network-based frameworks achieve better prediction accuracy, as compared to a classical approach that relies on historical numbers of views. We also briefly discuss how proactive video service deployment can be effectively enabled by our prediction frameworks.
Longer, Better: On Extending User Online Duration to Improve Quality of Streaming Service in P2P Networks

During recent years, the Internet has witnessed a rapid growth in deployment of peer to peer (P2P) based live media streaming systems. In this paper, we are motivated to investigate the problem of improving the streaming quality of service (QoS) of those systems by two universal recognitions: one is that the understanding to practical service experiences would benefit the performance enhancement, while the other is "the more, the better" design philosophy in P2P networks. As an interesting approach, we first retrieve the practical online duration information of end users from service traces in our GridMedia system, and then intentionally extend the online duration of each peer, which would result in a more stable peer overlay. The comparative simulations with original real-world traces and modified ones demonstrate that the quality of streaming service would be better if peers stay longer in the community. We claim that, this simple but positive result essentially validates both the influence of end users' behaviors and the need of an incentive to encourage users to spend more time in P2P live streaming systems.
Frame-level heuristic scheduling Multi-view Video Coding on symmetric multi-core architecture

In this paper, we propose a frame-level heuristic scheduling parallel emerging Multi-view Video Coding (MVC) using Directed Acyclic Graph (DAG) on Intel multi-core processor. We illustrate the reason to choose heuristic scheduling and formulate the problem. Through defining dependent degree and concurrent degree, we demonstrate why to choose frame as parallel granularity. Experimental results demonstrate the effectiveness and scalability of our parallel MVC.
Strategies of Collaboration in Multi-Swarm Peer-to-Peer Content Distribution

In modern Peer-to-Peer (P2P) content distribution applications, multiple swarms typically exist, each corresponding to the dissemination of one content among interested peers. A common design in the existing P2P applications is to allow peers in one swarm to help each other, while different swarms are only coupled when sharing the upload bandwidth at the dedicated content servers/publishers. In recent years, a number of proposals have emerged which advocate inter-swarm collaboration and resource sharing, where peers in one swarm may contribute their storage and bandwidth resources to help peers in the swarm of another content. Such inter-swarm collaboration can improve content availability and optimize resource utilization in the entire system, at the cost of additional overhead for content preloading and inter-swarm coordination. This paper presents a survey of studies on effective inter-swarm collaboration mechanisms in the existing literature. This paper first discusses strategies of collaboration in P2P file sharing applications, and then presents multi-channel collaborative design for P2P live and Video-on-Demand (VoD) streaming. In particular, this paper elaborates our recent design of collaboration strategies among multiple streaming channels in a P2P VoD system, and shows that the server cost can be reduced by up to 25% while high streaming qualities are guaranteed in the entire system, even during extreme scenarios such as unexpected flash crowds. This paper also discusses representative approaches to implement inter-swarm collaborations in various P2P content distribution systems.
Visualizing Wi-Fi accesses from city-scale population for urban analysis

With the concept of big data being widely adopted in various areas, smart urban computing based on big data analysis has been a hot topic in recent years. As an important part of most people's daily life, Wi-Fi accessing can provide large-scale valuable user mobility data for urban computing, which indicates a new research field. A multi-layer visualizing model is proposed in this paper, based on which urban regional function and feature analysis is performed to find out valuable patterns from Wi-Fi accessing records generated by city-scale population. Moreover, by partitioning the city into region blocks, the association between different regions can be characterized by spatio-temporal population flows between them. Potential problems caused by inappropriate urban planning can be revealed by inspecting and comparing the multi-scale mobility visualizations. The experimental results show that important urban dynamic patterns can be visualized and this can be utilized to interpret potential urban planning problems reflected by large scale Wi-Fi connection data.
A Framework for Heuristic Scheduling for Parallel Processing on Multicore Architecture: A Case Study With Multiview Video Coding

In this paper, using the Intel multicore architectures and the emerging multiview video coding standard, we introduce a framework for performing analysis, simulation, and evaluation of heuristics scheduling algorithms for implementing computationally intensive algorithms on multicore processors. The framework allows for accurate and quantitative characterization of the performance of dynamic scheduling algorithms for multimedia applications on different multicore processors without actual implementation of the scheduling algorithm and application on the actual platform. Experimental results demonstrate the effectiveness and scalability of our framework.
Building semi-immersing human-computer interaction environment of virtual teleconferencing based on multi-screen general projection

Building a human-computer interaction environment which supports natural communication and interaction among participant group is the key issue of virtual teleconferencing system, which needs to reestablish the correct space relationship of participants and synthesizing real size video into the virtual environment. We introduce a method that uses multiple projection screens driven by network-connected PCs to construct the spatial and realism semi-immerse virtual teleconferencing environment based on general projection transformation. The experiment shows that participants can interact and collaborate with natural interaction model in there.
MOMD: A multi-object multi-dimensional auction for crowdsourced mobile video streaming

Crowdsourced mobile video streaming enables nearby mobile video users to aggregate their network resources to improve the video streaming performance. However, users are often selfish and may not be willing to cooperate without proper incentives. Designing an incentive mechanism for such a scenario is challenging due to the users' asynchronous downloading behaviors as well as their private valuations for multi-bitrate encoded videos. In this work, we propose a multi-object multi-dimensional auction-based incentive framework, through which users can download multiple video segments with different bitrates for multiple nearby users (and themselves). Based on this incentive framework, we propose a Vickrey-score auction, which is the first multi-object multi-dimensional auction that achieves both truthfulness and efficiency. Simulations with real traces show that crowdsourced mobile streaming outperforms noncooperative streaming by 48.6% (on average) in terms of social welfare. We further implement our proposed auction mechanism in a demostration system, and show that the crowdsourced framework together with the auction mechanism can substantially increase mobile user's welfare and video service stability.
MUSA: Wi-Fi AP-assisted video prefetching via Tensor Learning

Driven by the exponentially increasing amount of mobile video traffic, caching videos closer to the end users has become an appealing solution to reduce the traffic through the backbone network while improving users' perceived quality-of-experience (e.g., better video quality and reduced service delay). This research interest has been gaining lots of momentums due to the emergence of smart Access Points (APs), which are equipped with large storage space (several GBs). To address the “small population” problem involved in the prefetching at the edge, we propose to prefetch videos to APs ahead of users' requests via tensor learning: We first adopt the weighted tensor model to mine the hidden semantic pattern to characterize both users' preference for different types of videos and the dynamic video popularity over time; Then, based on the resulting low-dimension matrixes generated by the tensor factorization, we adopt an exponential smoothing model to capture the temporal pattern to predict users' propensity to unwatched videos; Finally, based on the predicted video popularity, we proactively replicate videos from the original CDN server to the APs at the edge. Through trace-driven simulations, we show that the proposed prefetching solution can outperform the baseline algorithms: compared with the SVD-based prefetching strategy, our design achieves a better hit ratio (e.g., surpassing about 10%) and accuracy (e.g., surpassing about 15%); compared with the history based strategy, our design also have about 40% (resp. 20%) improvement in terms of hit ratio (resp. accuracy).
A Virtual MIMO Transmission Scheme in Wireless Video Sensor Network

In order to improve the real-time video transmission distortion performance over wireless video sensor network (WVSN), we propose a novel cooperative transmission scheme by using virtual multiple-input-multiple-output (MIMO) technology in this paper. The video transmission distortion, energy consumption and the end-to-end delay performance are analyzed with virtual MIMO transmission manner. According to the distinctive characteristic of real-time video data, different type coded frames are transmitted by virtual MIMO manner with different priority as soon as possible in our approach. The greedy algorithm is used to optimize the video distortion performance with the total energy and end-to-end delay constraints. From the simulation results, our approach can get better distortion performance than the traditional manner when the intra cluster distance is set to be appropriate relative to the inter cluster distance. Virtual MIMO; wireless video sensor network; realtime video transmission
High-quality non-blind motion deblurring

Traditional non-blind motion deblurring methods are sensitive to kernel estimate errors and image noise, thus suffering from either ringing artifacts, enlarged image noise, or over-smoothed image details. We introduce a robust non-blind deblurring algorithm that produces high quality results even from many challenging images with noisy kernels. We adopt the Gaussian Scale Mixture Fields of Experts (GSM FOE) model and the smoothness constraint as image prior, and use the iterative re-weight least-square (IRLS) algorithm to produce the temporal result. The residual deconvolution suite is used to restore the lost image details. We denoise the result using our std-controlled cross bilateral filter. The experimental results are much better than those of previous approaches.
A Joint Online Transcoding and Delivery Approach for Dynamic Adaptive Streaming

Dynamic adaptive streaming has emerged as a popular approach for video services in today's Internet. To date, the two important components in dynamic adaptive streaming, video transcoding that generates the adaptive bitrates of a video and video delivery that streams the videos to users, have been separately studied, resulting in a huge waste of computation and storage resource due to producing and caching different versions of videos regardless of their demands. We conduct extensive measurement studies of video sharing systems, including an IPTV service which streams regular, professionally made videos and an instant video clip sharing service which provides extremely short user-generated videos, as well as the availability of computation resource in conventional content delivery networks (CDNs). Based on the measurement insights, we propose an online joint transcoding and delivery approach for adaptive video streaming. We formulate optimization problems to enable high streaming quality for the users, and low computation and replication costs for the system. In particular, our strategy connects video transcoding and video delivery based on users' preferences of CDN regions and regional preferences of video versions. We analyze hardness of these problems and design distributed solutions. Extensive trace-driven experiments further demonstrate the superiority of our design.
A selective transport framework for delivery MVC video over MPEG-2 TS

Multiview video has gained significant interest in recent years. Generally speaking, Multiview video has more than one views from slightly different angles of the same scene. The huge amount of video data and the heterogeneous network condition pose great challenge for the transport of multiview video through internet or broadcast networks. In order to support variety of 3D application scenarios, selective transport of multiview video has to be deployed. This paper proposes a priority-based selective transport framework of multiview video over MPEG-2 Transport Stream. The proposed framework achieves selective transport by extracting the related view components from MVC video as well as discarding lowest priority view components without obstructing the decoding process at the receiver client when the transport bandwidth is insufficient. To the best of our knowledge, this paper is the first to investigate a priority-based selective transport framework for delivery MVC video over MPEG-2 transport stream.
Enhancements of Representation and Interactivity for Multi-view Video Based on Layered Depth Image

The feature of rendering arbitrary view make layered depth image (LDI) be able to present multi-view video data and provide enhanced interactivity, such as free view video (FVV). However, visual artifacts occurred in the rendered result limit the practicality of LDI. In this paper, we proposed an approach to deal with this problem, which refines colors of the depth pixels by clustering to remove matting effects during generating of LDI, and applies median filter to rendered result to remove the gaps or holes caused by disocclusion and undersample. Experimental results show that our approach is practicable and efficient.
Joint Sponsor Scheduling in Cellular and Edge Caching Networks for Mobile Video Delivery

The explosive growth of mobile video traffic introduces new challenges for the network infrastructure. Edge caching, as one of the key technologies in 5G wireless networks, has shown great potential to improve the quality of mobile video services by reducing the transmission overhead over backhaul links. With edge caching, content providers (CPs) need to decide not only the traditional data sponsoring strategy on cellular networks (where CPs cover part or all of the mobile users’ cellular data cost), but also a novel cache sponsoring strategy on the edge caching networks (where CPs place part of contents on edge networks in advance). In this paper, we study the joint optimization of both sponsors on cellular and edge caching networks for a single CP, aiming at maximizing the CP's revenue. Specifically, we formulate the joint optimization problem as a two-stage sequential decision problem. In stage I, the CP determines the edge caching policy (for a relatively long time period). In stage II, the CP decides the real-time data sponsoring strategy for each content request within the period. We analyze this two-stage decision problem systematically. First, we propose an online sponsoring strategy in stage II based on Lyapunov optimization framework. Then, we propose an edge caching strategy in stage I via predicting the number of aggregate user requests. Simulations on real data traces show that such a joint optimization policy can increase the CP's revenue by 124%–154%, comparing with the traditional data sponsoring policy (i.e., without edge caching). Moreover, the proposed online strategy can achieve 90% of the maximum revenue in the offline benchmark.