nils-herrmann/clean_scirepeval huggingface dataset:
Intra-and-Inter-Constraint-Based Video Enhancement Based on Piecewise Tone Mapping

Video enhancement plays an important role in various video applications. In this paper, we propose a new intra-and-inter-constraint-based video enhancement approach aiming to: 1) achieve high intraframe quality of the entire picture where multiple regions-of-interest (ROIs) can be adaptively and simultaneously enhanced, and 2) guarantee the interframe quality consistencies among video frames. We first analyze features from different ROIs and create a piecewise tone mapping curve for the entire frame such that the intraframe quality can be enhanced. We further introduce new interframe constraints to improve the temporal quality consistency. Experimental results show that the proposed algorithm obviously outperforms the state-of-the-art algorithms.
Adaptive scheduling for multicasting hard deadline constrained prioritized data via network coding

Network coding offers a promising platform for multicast transmission by approaching its min-cut capacity. However, pushing the network throughput toward this upper bound comes with a sacrifice in delivery delay due to the decoding procedure that requires performing batch of coded packets. Further, in some transmission scenarios where the receivers experience deep fading or unable to collect a full set of the transmitted data, no useful information is recovered. The effect is more severe in the networks where the transmitted information has priority structure with hard deadline constraint due to the limited delivery time and data interdependencies. In this paper, we consider single-hop wireless networks where the transmitter wishes to multicast hard deadline constrained prioritized data to many receivers over lossy channels. We first study the network performance of a variety of transmission techniques, depending on how the transmitter schedules transmission in each time slot. We then propose an adaptive encoding and scheduling technique to maximize the network throughput. To find the optimal transmission scheduling at the presence of the network dynamics, we cast the problem in the framework of Markov Decision Processes (MDP) and use backward induction method to find an optimal solution. We further propose simulation-based algorithm and greedy scheduling technique that obtain high performance with much lower time complexity. Both analytical and simulation results have been provided to corroborate the effectiveness of the proposed techniques.
A new global-based video enhancement algorithm by fusing features of multiple region-of-interests

Video enhancement plays an important role in various video applications. It is desirable to achieve high visual quality of the entire picture where multiple region-of-interests (ROIs) within the frame can be adaptively and simultaneously enhanced. In this paper, a new global-based video enhancement algorithm is proposed. The proposed algorithm first analyzes features from different ROIs. Then, a ‘global’ tone mapping curve is created for the entire picture which can adaptively enhance different regions at the same time. According to the statistics of ROIs, two fusion strategies, i.e., piecewise-based and factor-based fusions, are proposed for creating the global tone mapping curve. Experimental results show that the proposed algorithm can obtain more appealing perceptual quality than the state-of-the-art algorithms.
Region-Based Rate Control for H.264/AVC for Low Bit-Rate Applications

Rate control plays an important role in video coding. However, in the conventional rate control algorithms, the number and position of macroblocks (MBs) inside one basic unit for rate control is inflexible and predetermined. The different characteristics of the MBs are not fully considered. Also, there is no overall optimization of the coding of basic units. This paper proposes a new region-based rate control scheme for H.264/advanced video coding to improve the coding efficiency. The inter-frame information is explored to objectively divide one frame into multiple regions based on their rate-distortion (R-D) behaviors. The MBs with similar characteristics are classified into the same region, and the entire region, instead of a single MB or a group of contiguous MBs, is treated as a basic unit for rate control. A linear rate-quantization stepsize model and a linear distortion-quantization stepsize model are proposed to accurately describe the R-D characteristics for the region-based basic units. Moreover, based on the above linear models, an overall optimization model is proposed to obtain suitable quantization parameters for the region-based basic units. Experimental results demonstrate that the proposed region-based rate control approach can achieve both better subjective and objective quality by performing the rate control adaptively with the content, compared to the conventional rate control approaches.
Person Re-Identification with Correspondence Structure Learning

This paper addresses the problem of handling spatial misalignments due to camera-view changes or human-pose variations in person re-identification. We first introduce a boosting-based approach to learn a correspondence structure which indicates the patch-wise matching probabilities between images from a target camera pair. The learned correspondence structure can not only capture the spatial correspondence pattern between cameras but also handle the viewpoint or human-pose variation in individual images. We further introduce a global-based matching process. It integrates a global matching constraint over the learned correspondence structure to exclude cross-view misalignments during the image patch matching process, hence achieving a more reliable matching score between images. Experimental results on various datasets demonstrate the effectiveness of our approach.
An enhanced factorized model based on user and item features

Nowadays, people rely on the Recommender System (RS) to make their decisions on the Internet. Most previous techniques only focused on the user bias and item bias, while the detailed features of users or items have been overlooked. In this paper, we propose a new algorithm to provide more precise suggestions for users in RS. The proposed algorithm introduces a User-Item-Feature-based (UIF) model to dig out concealed factors that influence the users' rating preferences. With this method, more detailed information, including the users' gender, age and occupation features and the items' decade features, can be well considered when recommending to users. Experimental results demonstrate that by adding the user and items' features, our proposed UIF model can provide more accurate and scalable results compared with previous methods.
Improved human head and shoulder detection with local main gradient and tracklets-based feature

In this paper, a new approach is proposed which extracts local main gradients and tracklet-based features for describing human head-and-shoulders. Firstly, local main gradient is extracted for each sliding window such that only gradient features fitting a reasonable orientation are detected as candidate head-and-shoulders. Secondly, given that the shape of head-and-shoulder satisfies a specific curve, we model head-and-shoulder shapes as the combination of short trajectories (tracklets) and utilize the statistics of tracklets to describe head-and-shoulder shapes. Experiments show that by the introduction of our new features, we can achieve better detection results than existing head-and-shoulder detection methods.
A Fast Sub-Pixel Motion Estimation Algorithm for H.264/AVC Video Coding

Motion estimation (ME) is one of the most time-consuming parts in video coding. The use of multiple partition sizes in H.264/AVC makes it even more complicated when compared to ME in conventional video coding standards. It is important to develop fast and effective sub-pixel ME algorithms since: 1) the computation overhead by sub-pixel ME has become relatively significant while the complexity of integer-pixel search has been greatly reduced by fast algorithms, and 2) reducing sub-pixel search points can greatly save the computation for sub-pixel interpolation. In this letter, a novel fast sub-pixel ME algorithm is proposed which performs a “rough” sub-pixel search before the partition selection, and performs a “precise” sub-pixel search for the best partition. By reducing the searching load for the large number of non-best partitions, the computation complexity for sub-pixel search can be greatly decreased. Experimental results show that our method can reduce the sub-pixel search points by more than 50% compared to existing fast sub-pixel ME methods with negligible quality degradation.
Improved 3D-model texture mapping with region-of-interest weighting and iterative boundary-texture updating

The target of 3D-model texture mapping is to select suitable texture contents from a set of 2D images and map them onto their corresponding 3D model for creating a “textured” 3D object. The existing texture mapping methods still suffer from large illumination differences and the mismatch between images and 3D models. In this paper, we proposed a new approach for seamless texture mapping on 3D models. The proposed approach first introduces an region-of-interest-based weighting strategy to guide the texture-selection process, such that suitable patches can be selected from images to create a more smooth and natural texture-mapping layout. Then, an iterative update strategy is further proposed, which iteratively minimizes the boundary differences between image patches and texture-boundary fitness inside each patch, such that the patch-wise texture differences on a 3D model can be deleted. Experimental results show that our approach can effectively reduce the texture difference and achieve a more satisfactory texture-mapping result.
Unsupervised Trajectory Clustering via Adaptive Multi-kernel-Based Shrinkage

This paper proposes a shrinkage-based framework for unsupervised trajectory clustering. Facing to the challenges of trajectory clustering, e.g., large variations within a cluster and ambiguities across clusters, we first introduce an adaptive multi-kernel-based estimation process to estimate the 'shrunk' positions and speeds of trajectories' points. This kernel-based estimation effectively leverages both multiple structural information within a trajectory and the local motion patterns across multiple trajectories, such that the discrimination of the shrunk point can be properly increased. We further introduce a speed-regularized optimization process, which utilizes the estimated speeds to regularize the optimal shrunk points, so as to guarantee the smoothness and the discriminative pattern of the final shrunk trajectory. Using our approach, the variations among similar trajectories can be reduced while the boundaries between different clusters are enlarged. Experimental results demonstrate that our approach is superior to the state-of-art approaches on both clustering accuracy and robustness. Besides, additional experiments further reveal the effectiveness of our approach when applied to trajectory analysis applications such as anomaly detection and route analysis.
Spectrum Optimization for OFDMA Based Hybrid Wireless Networks

This paper proposes a new scheme for cooperative hybrid network resource allocation. Different from the traditional cognitive radio networks that aim to utilize the spectrum holes (such as white space in TV spectrum) in an uncoordinated way, this paper proposes an OFDMA based collaborative hybrid network. We study the joint resource allocation problem for both the primary users and the secondary users. Results show that by cooperatively allocating resources in the primary network and the secondary network, we can achieve higher spectral efficiency while provide satisfactory admission control and QoS for all users.
A network-based algorithm for recognizing group activities with multiple people and crowded scenes

In this paper, a network-based algorithm is proposed for group activity recognition with multiple people/objects and crowded scenes. This algorithm models the entire scene as an error-free network. With this network, we model objects in the scene as packages while activities as package transmission in the network. By analyzing these “package transmission” processes, activities can be detected. Based on the proposed network-based algorithm, we also propose two implementations to deal with the scenarios of group activities with multiple people/objects and crowded scenes. Experimental results demonstrate the effectiveness of our proposed algorithm.
A new algorithm for compressing massive region-of-interest location information in videos

Region-of-Interest (ROI) location information in videos is of increasing importance in many applications including user interest analysis and user experience improvement. Although ROI-based coding has been studied by many researchers, most of the works only focus on using the ROI information for improving coding efficiency while the ROI location information itself is seldom coded or transmitted. Though some methods can recover the ROI locations at the decoder by some parameters such as the quantization parameter, they will fail to work when the number of ROIs becomes large or the ROIs become overlapping. In this paper, we propose a new algorithm to compress this massive ROI location information in videos. The proposed algorithm introduces the region position information extracted from the reconstructed frame as the reference to reduce the ROI location data. Furthermore, the temporal correlations among ROIs in neighboring frames are also utilized for compressing the ROI locations. By suitably integrating the extracted region position as well as the temporal correlation, the proposed algorithm can reduce the data by about 20% for videos with 30 ROI regions. Experimental results demonstrate the effectiveness of the proposed algorithm.
Enriched Long-Term Recurrent Convolutional Network for Facial Micro-Expression Recognition

Facial micro-expression (ME) recognition has posed a huge challenge to researchers for its subtlety in motion and limited databases. Recently, handcrafted techniques have achieved superior performance in micro-expression recognition but at the cost of domain specificity and cumbersome parametric tunings. In this paper, we propose an Enriched Long-term Recurrent Convolutional Network (ELRCN) that first encodes each micro-expression frame into a feature vector through CNN module(s), then predicts the micro-expression by passing the feature vector through a Long Short-term Memory (LSTM) module. The framework contains 2 different network variants: (1) Channel-wise stacking of input data for spatial enrichment, (2) Feature-wise stacking of features for temporal enrichment. We demonstrate that the proposed approach is able to achieve reasonably good performance, without data augmentation. In addition, we also present ablation studies conducted on the framework and visualizations of what CNN "sees" when predicting the micro-expression classes.
Macroblock Classification Method for Video Applications Involving Motions

In this paper, a macroblock classification method is proposed for various video processing applications involving motions. Based on the analysis of the Motion Vector field in the compressed video, we propose to classify Macroblocks of each video frame into different classes and use this class information to describe the frame content. We demonstrate that this low-computation-complexity method can efficiently catch the characteristics of the frame. Based on the proposed macroblock classification, we further propose algorithms for different video processing applications, including shot change detection, motion discontinuity detection, and outlier rejection for global motion estimation. Experimental results demonstrate that the methods based on the proposed approach can work effectively on these applications.
A Heat-Map-Based Algorithm for Recognizing Group Activities in Videos

In this paper, a new heat-map-based algorithm is proposed for group activity recognition. The proposed algorithm first models human trajectories as series of heat sources and then applies a thermal diffusion process to create a heat map (HM) for representing the group activities. Based on this HM, a new key-point-based (KPB) method is used for handling the alignments among HMs with different scales and rotations. A surface-fitting (SF) method is also proposed for recognizing group activities. Our proposed HM feature can efficiently embed the temporal motion information of the group activities while the proposed KPB and SF methods can effectively utilize the characteristics of the HM for activity recognition. Section IV demonstrates the effectiveness of our proposed algorithms.
Learning Correspondence Structures for Person Re-Identification

This paper addresses the problem of handling spatial misalignments due to camera-view changes or human-pose variations in person re-identification. We first introduce a boosting-based approach to learn a correspondence structure, which indicates the patchwise matching probabilities between images from a target camera pair. The learned correspondence structure can not only capture the spatial correspondence pattern between cameras but also handle the viewpoint or human-pose variation in individual images. We further introduce a global constraint-based matching process. It integrates a global matching constraint over the learned correspondence structure to exclude cross-view misalignments during the image patch matching process, hence achieving a more reliable matching score between images. Finally, we also extend our approach by introducing a multi-structure scheme, which learns a set of local correspondence structures to capture the spatial correspondence sub-patterns between a camera pair, so as to handle the spatial misalignments between individual images in a more precise way. Experimental results on various data sets demonstrate the effectiveness of our approach.
Layout-expectation-based Model for Image Search Re-ranking

In this paper, a new algorithm is proposed for image re-ranking in Web search applications. The proposed algorithm introduces a new layout expectation model for improving the image search results. The motivation for using the expectation model is that users may often have potential expectations about the desired image during the search process. By including the layout expectation model to describe users' expectation on image layouts, the re-ranked search results can become more satisfactory to users. Experimental results demonstrate that our proposed algorithm can significantly improve the re-ranking precision compared with the state-of-the-art algorithms.
IEEE Access Special Section Editorial: Intelligent Sensing On Mobile and Social Media Analytics

The proliferation of mobile devices has led to an increasingly interconnected era, and a recent trend is the migration of multimedia search and analysis from the desktop to the cloud. This is, perhaps, partly due to the ease of capturing images and videos on a mobile device and associating them with social and contextual metadata such as comments and GPS tags. Such a hybrid data structure can benefit a wide variety of potential (mobile) multimedia applications, such as location recognition, landmark search, augmented reality, and commercial recommendations. One intrinsic potential solution is to explore large-scale social multimedia to assist and facilitate location related applications on the go and in real-time. With the combination of social and mobile cues, several problems that are previously difficult to be tackled in multimedia content analysis have also become more tractable. Current research in Intelligent Sensing is addressing the following:
Integrating visual saliency and consistency for re-ranking image search results

The paper investigates two mechanisms, visual consistency and visual saliency, in web image search: (1) In most current web image search engines, such as Google Image Search and Yahoo Image Search, the images that closely related to the search query are typically visually similar. These visually consistent images which occur most frequently in the first few web pages will be given higher ranks. (2) From visual aspect, it is obvious that salient images would be easier to catch users' eyes and more likely to be clicked than the cluttered ones in low-level vision. In addition, we also observe the fact that the visually salient images in the front pages are often relevant to the user's query. The principal novelty of this paper is in combining visual saliency and consistency to re-rank the results from search engines to make the re-ranked images more satisfying in both vision and content. The experimental results on a real world web image dataset demonstrate that our approach can effectively improve the performance of image retrieval.
Hypothesis comparison guided cross validation for unsupervised signer adaptation

Signer adaptation is important to sign language recognition systems in that a one-size-fits-all model set can not perform well on all kinds of signers. Supervised signer adaptation must utilize the labeled adaptation data that are collected explicitly. To skip the data collecting process in signer adaptation, we propose an unsupervised adaptation method called hypothesis comparison guided cross validation (HC-CV) algorithm. The algorithm not only addresses the problem of overlap between the data set to be labeled and the data set for adaptation, but also employs an additional hypothesis comparison step to decrease the noise rate of the adaptation data set. Experimental results show that the HC-CV adaptation algorithm is superior to the CV adaptation algorithm and the conventional self-teaching algorithm. Though the algorithm is proposed for signer adaptation, it can also be applied to speaker adaptation and writer adaptation straightforwardly.
Learning-based super-resolution method with a combining of both global and local constraints

Learning-based super-resolution (SR) methods are popular in many applications recently. In these methods, the high-frequency details are usually found or combined through patch matching from training database. However, the representation ability of small patch is limited and it is difficult to guarantee that the super-resolved image is the best under the global view. To this end, the authors propose a statistical learning method for SR with both global and local constraints. More specifically, they introduce a mixture model into maximum a posteriori (MAP) estimation, which combines a global parametric constraint with a patch-based local non-parametric constraint. The global parametric constraint guarantees the super-resolved global image to agree with the sparse property of natural images, and the local non-parametric constraint is used to infer the residues between the image derived from the global constraint and the ground truth high-resolution (HR) image. Compared with the traditional patch-based learning methods without the global constraint, our method can not only preserve global image structure, but also restore the local details more effectively. Experiments verify the effectiveness of the proposed method.
Facial expression mapping based on elastic and muscle-distribution-based models

In this paper, a new algorithm is proposed for facial expression mapping. The proposed algorithm first introduces a new elastic model to balance the global and local warping effects such that the impacts from facial feature differences between people can be avoided, thus more reasonable geometric warping results can be created. Furthermore, a muscle-distribution-based (MD) model is also proposed. The proposed MD model utilizes the muscle distribution information of the human face to evaluate and strengthen the facial illumination details. By this way, the impacts from human face difference as well as the effects of unsuitable noise filtering can be effectively alleviated. Experimental results show that our proposed algorithm can create obviously better facial expression results than the existing methods.