nils-herrmann/clean_scirepeval huggingface dataset:
Fractal Dimension Invariant Filtering and Its CNN-Based Implementation

Fractal analysis has been widely used in computer vision, especially in texture image processing and texture analysis. The key concept of fractal-based image model is the fractal dimension, which is invariant to bi-Lipschitz transformation of image, and thus capable of representing intrinsic structural information of image robustly. However, the invariance of fractal dimension generally does not hold after filtering, which limits the application of fractal-based image model. In this paper, we propose a novel fractal dimension invariant filtering (FDIF) method, extending the invariance of fractal dimension to filtering operations. Utilizing the notion of local self-similarity, we first develop a local fractal model for images. By adding a nonlinear post-processing step behind anisotropic filter banks, we demonstrate that the proposed filtering method is capable of preserving the local invariance of the fractal dimension of image. Meanwhile, we show that the FDIF method can be re-instantiated approximately via a CNN-based architecture, where the convolution layer extracts anisotropic structure of image and the nonlinear layer enhances the structure via preserving local fractal dimension of image. The proposed filtering method provides us with a novel geometric interpretation of CNN-based image model. Focusing on a challenging image processing task &#x2014; detecting complicated curves from the texture-like images, the proposed method obtains superior results to the state-of-art approaches.
Group Event Detection With a Varying Number of Group Members for Video Surveillance

This paper presents a novel approach for automatic recognition of group activities for video surveillance applications. We propose to use a group representative to handle the recognition with a varying number of group members, and use an asynchronous hidden Markov model (AHMM) to model the relationship between people. Furthermore, we propose a group activity detection algorithm which can handle both symmetric and asymmetric group activities, and demonstrate that this approach enables the detection of hierarchical interactions between people. Experimental results show the effectiveness of our approach.
Inferring user image-search goals by mining query logs with semi-supervised spectral clustering

Inferring user search goals for a query can be very useful in improving search engine relevance and user experience. Although the research on analyzing user goals or intents for text search has received much attention, little has been proposed for image search. In this paper, we propose a novel approach to infer user search goals in image search by mining search engine query logs with semi-supervised spectral clustering. We combine the visual information of the clicked images with user click information by using graph-based models and then cluster the images with spectral clustering to capture user image-search goals. Experimental results based on a popular commercial search engine demonstrate the effectiveness of the proposed method.
A representative-based framework for parsing and summarizing events in surveillance videos

This paper presents a novel representative-based framework for parsing and summarizing events in long surveillance videos. The proposed framework first extracts object blob sequences and utilizes them to represent events in a surveillance video. Then, a sequence filtering strategy is introduced which detects and eliminates noisy blob sequences based on their spatial and temporal characteristics. After clustering the blob sequences into different event types, we further introduce a representative-based model which integrates location, size, and appearance cues to select a representative blob sequence from each cluster, and creates a snapshot image for each representative blob sequence. Based on the blob-sequence clustering and representative-sequence selection results, two schemes are further proposed to summarize contents of the input surveillance video: (1) type-based scheme which shows snapshot images to users and creates a summary video for a specific event cluster according to user-selected snapshot image; (2) representative-based scheme which creates a summary video only with the extracted representative blob sequences. Experimental results show that our approach can create more effective and well-organized summarization results compared with the state-of-the-art methods.
Human activity recognition for video surveillance

This paper presents a novel approach for automatic recognition of human activities from video sequences. We first group features with high correlations into category feature vectors (CFVs). Each activity is then described by a combination of GMMs (Gaussian mixture models) with each GMM representing the distribution of a CFV. We show that this approach offers flexibility to add new events and to deal with the problem of lacking training data for building models for unusual events. For improving the recognition accuracy, a confident-frame-based Recognizing algorithm (CFR) is proposed to recognize the human activity, where the video frames which have high confidence for recognition an activity (confident-frames) are used as a specialized model for classifying the rest of the video frames. Experimental results show the effectiveness of the proposed approach.
Text localization in natural scene images with stroke width histogram and superpixel

A novel stroke-based method to localize text in unconstrained natural scene images is proposed. Firstly, in order to improve the edge detection in tough situations where the texts are partially occluded or noisy, we use stroke width histogram as guidance to generate a series of superpixels. Secondly, we present a novel way of using distance transform and sobel operator to extract character skeleton and then use the skeleton to improve stroke-width accuracy. Our method was evaluated on two standard datasets: ICDAR 2005 and ICDAR 2011, and the experimental results show that it achieves state-of-the-art performance.
Kill two birds with one stone: Boosting both object detection accuracy and speed with adaptive patch-of-interest composition

Object detection is an important yet challenging task in video understanding & analysis, where one major challenge lies in the proper balance between two contradictive factors: detection accuracy and detection speed. In this paper, we propose a new adaptive patch-of-interest composition approach for boosting both the accuracy and speed for object detection. The proposed approach first extracts patches in a video frame which have the potential to include objects-of-interest. Then, an adaptive composition process is introduced to compose the extracted patches into an optimal number of sub-frames for object detection. With this process, we are able to maintain the resolution of the original frame during object detection (for guaranteeing the accuracy), while minimizing the number of inputs in detection (for boosting the speed). Experimental results on various datasets demonstrate the effectiveness of the proposed approach.
An algorithm of texture mapping based on boundary adjustment and image blending

In this paper, A new texture mapping algorithm for 3D model is proposed. The proposed algorithm starts with a 3D model and several color images of this model. First, we match pixels of each image with the 3D model. Then the boundaries between images are determined by minimizing a boundary-based cost function. In this paper, two different cost functions are proposed. Finally, an image blending method is proposed to smooth the color around the boundaries. The proposed algorithm is robust to the illuminant difference among images and can generate texture map with much less texture discontinuity. Experimental results demonstrate the effectiveness of the proposed algorithm.
A new Temporal-Constraint-Based algorithm by handling temporal qualities for video enhancement

Video enhancement has played very important roles in many applications. However, most existing enhancement methods only focus on the spatial quality within a frame while the temporal qualities of the enhanced video are often unguaranteed. In this paper, a new algorithm is proposed for video enhancement. The proposed algorithm introduces new temporal constraints and combines them with the spatial constraints such that both the spatial and temporal qualities of the video can be improved. Two strategies are proposed for including the temporal constraints. Experimental results demonstrate the effectiveness of the proposed algorithm.
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression

We propose an efficient and unified framework, namely ThiNet, to simultaneously accelerate and compress CNN models in both training and inference stages. We focus on the filter level pruning, i.e., the whole filter would be discarded if it is less important. Our method does not change the original network structure, thus it can be perfectly supported by any off-the-shelf deep learning libraries. We formally establish filter pruning as an optimization problem, and reveal that we need to prune filters based on statistics information computed from its next layer, not the current layer, which differentiates ThiNet from existing methods. Experimental results demonstrate the effectiveness of this strategy, which has advanced the state-of-the-art. We also show the performance of ThiNet on ILSVRC-12 benchmark. ThiNet achieves 3.31 x FLOPs reduction and 16.63× compression on VGG-16, with only 0.52% top-5 accuracy drop. Similar experiments with ResNet-50 reveal that even for a compact network, ThiNet can also reduce more than half of the parameters and FLOPs, at the cost of roughly 1% top-5 accuracy drop. Moreover, the original VGG-16 model can be further pruned into a very small model with only 5.05MB model size, preserving AlexNet level accuracy but showing much stronger generalization ability.
Efficient pose machine based on parameter-sensitive hashing

In this paper, we propose an efficient pose machine using Parameter-Sensitive Hashing(PSH) techniques. Based on the original pose machine, which is a sequential prediction framework, we employ the Convolutional Neural Network(CNN) to extract features. To handle the high dimensional feature vectors and conduct similarity search efficiently, we use the Parameter-Sensitive Hashing Function(PSHF) to map the feature vectors into binary values. The property of the PSHF ensures that the collisions happen when two vectors are near to each other and the search can be completed in a fractional power time. We apply our approach to the popular datasets including LSP and FLIC and make a comparison with previous methods based on a criterion of strict Percentage of Correct Parts(PCP). Experimental results reflect that our approach outperforms previous methods in accuracy.
Flexible Image Similarity Computation Using Hyper-Spatial Matching

Spatial pyramid matching (SPM) has been widely used to compute the similarity of two images in computer vision and image processing. While comparing images, SPM implicitly assumes that: in two images from the same category, similar objects will appear in similar locations. However, this is not always the case. In this paper, we propose hyper-spatial matching (HSM), a more flexible image similarity computing method, to alleviate the mis-matching problem in SPM. The match between corresponding regions, HSM considers the relationship of all spatial pairs in two images, which includes more meaningful match than SPM. We propose two learning strategies to learn SVM models with the proposed HSM kernel in image classification, which are hundreds of times faster than a general purpose SVM solver applied to the HSM kernel (in both training and testing). We compare HSM and SPM on several challenging benchmarks, and show that HSM is better than SPM in describing image similarity.
A New Network-Based Algorithm for Human Activity Recognition in Videos

In this paper, a new network-transmission-based (NTB) algorithm is proposed for human activity recognition in videos. The proposed NTB algorithm models the entire scene as an error-free network. In this network, each node corresponds to a patch of the scene and each edge represents the activity correlation between the corresponding patches. Based on this network, we further model people in the scene as packages, while human activities can be modeled as the process of package transmission in the network. By analyzing these specific package transmission processes, various activities can be effectively detected. The implementation of our NTB algorithm into abnormal activity detection and group activity recognition are described in detail in this paper. Experimental results demonstrate the effectiveness of our proposed algorithm.
Bit rate reduction of H.264/AVC video coding for mobile applications

The increasing use of mobile applications requires higher compression efficiency for video coding. However, in mobile applications, the capability of the processor is limited and the channel is unreliable. Since the emerging H.264/AVC video coding standard provides superior compression performance compared to all the existing video coding standards, many research efforts concentrate on the compression efficiency of intra-frame coding in H.264/AVC. In this paper, we propose a novel algorithm to reduce the bit rate compressed by H.264/AVC intra-frame coding. We focus on the method to code the intra-frame prediction modes. Our proposed method significantly reduces the bits used to code the modes and do not change the syntax of the bit stream. Experimental results demonstrate it is effective and efficient
CPCDN: Content Delivery Powered by Context and User Intelligence

There is an unprecedented trend that content providers (CPs) are building their own content delivery networks (CDNs) to provide a variety of content services to their users. By exploiting powerful CP-level information in content distribution, these CP-built CDNs open up a whole new design space and are changing the content delivery landscape. In this paper, we adopt a measurement-based approach to understanding why, how, and how much CP-level intelligences can help content delivery. We first present a measurement study of the CDN built by Tencent, a largest content provider based in China. We observe new characteristics and trends in content delivery which pose great challenges to the conventional content delivery paradigm and motivate the proposal of CPCDN, a CDN powered by CP-aware information. We then reveal the benefits obtained by exploiting two indispensable CP-level intelligences, namely context intelligence and user intelligence, in content delivery. Inspired by the insights learnt from the measurement studies, we systematically explore the design space of CPCDN and present the novel architecture and algorithms to address the new content delivery challenges that have arisen. Our results not only demonstrate the potential of CPCDN in pushing content delivery performance to the next level, but also identify new research problems calling for further investigation.
CP-operated dash caching via reinforcement learning

In recent years, Dynamic Adaptive Streaming over HTTP (DASH) has gained momentum as an effective solution for delivering videos on the Internet. This trend is further driven by the deployment of existing HTTP cache infrastructures in DASH systems to reduce the traffic load as well as to serve clients better. However, deploying conventional cache servers in DASH systems still suffers from low cache hit ratio and bitrate oscillations, which makes it challenging for content providers (CPs) to balance the user-perceived quality-of-experience (QoE) and the operating cost in cache-enabled DASH systems. To address this challenge, we propose a CP-operated DASH caching framework to provide good user QoE with low cost. In particular, we first formulate the caching decision problem as a stochastic optimization problem over a finite time horizon. The objective of this problem is to maximize a weighted sum of the user QoE and the operating cost, termed as the utility. Then we design a reinforcement learning based online algorithm which can obtain approximately optimal solution of this problem. Through extensive trace-driven experiments, we show that our approach not only achieves 40% average improvement of the overall utility compared to baseline approaches, but also adapts to the server load.
Strategies of Collaboration in Multi-Channel P2P VoD Streaming

As compared to live peer-to-peer (P2P) streaming, modern P2P video-on-demand (VoD) systems have brought much larger volumes of videos and more interactive controls to the Internet users. Nevertheless, the larger number of available videos and the flexibility of allowing users to jump back and forth in a video, have led to much fewer numbers of concurrent peers watching at a similar pace, that reduces the chance for collaborative chunk supply among peers and thus significantly increases the server bandwidth cost. Towards the ultimate goal of maximizing peer resource utilization, in this paper, we design effective strategies for both cross-channel and intra-channel collaborations in multi- channel P2P VoD systems, such that individual peer's resources, including download/upload bandwidths and the cache capacity, are effectively utilized to maximize the streaming qualities in all the channels. In particular, each peer actively and strategically determines the supply-and-demand imbalance in different channels, as well as that among different chunks within each video, makes use of its surplus download capacity to fetch chunks with the most need, and then serves those chunks using its idle upload bandwidth, all without impairing its own streaming quality. Our extensive trace-driven simulations show the effectiveness of our strategies in reducing the server cost while guaranteeing high streaming qualities in the entire system, even during extreme scenarios such as unexpected flash crowds.
Understanding the Power of Smartrouter-Based Peer CDN for Video Streaming

Recent years have witnessed a new video delivery paradigm: smartrouter-based video delivery network, which is enabled by smartrouters deployed at users' homes, together with the conventional video servers deployed in the datacenters. Recently, ChinaCache (a largest CDN provider) and Youku (a video service provider using smartrouters to assist video delivery) announced their cooperation to create a new paradigm of content delivery based on householders' network resources. This new paradigm is different from the conventional peer-to-peer (P2P) approach, because such dedicated smartrouters are inherently operated by the centralized video service provides in a coordinative manner. It is intriguing to study the strategies, performance and potential impact on the content delivery ecosystem of such peer CDN systems. In this paper, we study the Youku peer CDN, which has deployed over 300K smartrouter devices for its video streaming. In our measurement, 78K videos were investigated and 3TB traffic has been analyzed, over controlled peer nodes and players. Our contributions are the following measurement insights. First, a global replication and caching strategy is essential for the peer CDN systems, and proactively scheduling replication and caching on a daily basis can guarantee their performance. Second, such peer CDN deployment can itself form an effective QoS monitoring sub-system, which can be used for fine-grained user request redirection. We also provide our analysis on the performance issues and potential improvement to the peer CDN systems.
Joint online transcoding and geo-distributed delivery for dynamic adaptive streaming

Dynamic adaptive video streaming has emerged as a popular approach for video streaming in today's Internet. To date the two important components in dynamic adaptive streaming, video transcoding which generates the adaptive bitrates of a video and video delivery which streams the videos to users, have been separately studied, resulting in a huge waste of computation and storage resource due to transcoding useless videos and suboptimal streaming quality due to homogeneous video replication. In this paper, we propose to jointly perform video transcoding and video delivery for adaptive streaming in an online manner. We conduct extensive measurement studies of a video sharing system and a CDN to motivate our design. We formulate and solve optimization problems to enable high streaming quality for the users, and low computation and replication costs for the system. In particular, our design connects video transcoding and video delivery based on users' preferences of CDN regions and regional preferences of video versions. Extensive trace-driven experiments further confirm the superiority of our design.
Dispersing Instant Social Video Service Across Multiple Clouds

Instant social video sharing which combines the online social network and user-generated short video streaming services, has become popular in today's Internet. Cloud-based hosting of such instant social video contents has become a norm to serve the increasing users with user-generated contents. A fundamental problem of cloud-based social video sharing service is that users are located globally, who cannot be served with good service quality with a single cloud provider. In this paper, we investigate the feasibility of dispersing instant social video contents to multiple cloud providers. The challenge is that inter-cloud social propagation is indispensable with such multi-cloud social video hosting, yet such inter-cloud traffic incurs substantial operational cost. We analyze and formulate the multi-cloud hosting of an instant social video system as an optimization problem. We conduct large-scale measurement studies to show the characteristics of instant social video deployment, and demonstrate the trade-off between satisfying users with their ideal cloud providers, and reducing the inter-cloud data propagation. Our measurement insights of the social propagation allow us to propose a heuristic algorithm with acceptable complexity to solve the optimization problem, by partitioning a propagation-weighted social graph in two phases: a preference-aware initial cloud provider selection and a propagation-aware re-hosting. Our simulation experiments driven by real-world social network traces show the superiority of our design.
Contextual Mixture Tracking

Multiple object tracking (MOT) poses three challenges to conventional well-studied single object tracking (SOT) algorithms: 1) multiple targets lead the configuration space to be exponential to the number of targets; 2) multiple motion conditions due to multiple targets' entering, exiting and intersection make the prediction process degrade in precision; 3) visual ambiguities among nearby targets make the trackers error prone. In this paper, we address the MOT problem by embedding contextual proposal distributions and contextual observation models into a mixture tracker which is implemented in a Particle Filter framework. The proposal distributions are adaptively selected by motion conditions of targets which are determined by context information, and the multiple features are combined according to their discriminative power between ambiguity prone objects. The induction of contextual proposal distribution and observation model can help to surmount the incapability of conventional mixture tracker in handling object occlusions, meanwhile retain its merits of flexibility and high efficiency. The final experiments show significant improvement in variable number objects tracking scenarios compared with other methods.