nils-herrmann/clean_scirepeval huggingface dataset:
Automatic Face Annotation in Personal Photo Collections Using Context-Based Unsupervised Clustering and Face Information Fusion

In this paper, a novel face annotation framework is proposed that systematically leverages context information such as situation awareness information with current face recognition (FR) solutions. In particular, unsupervised situation and subject clustering techniques have been developed that are aided by context information. Situation clustering groups together photos that are similar in terms of capture time and visual content, allowing for the reliable use of visual context information during subject clustering. The aim of subject clustering is to merge multiple face images that belong to the same individual. To take advantage of the availability of multiple face images for a particular individual, we propose effective FR methods that are based on face information fusion strategies. The performance of the proposed annotation method has been evaluated using a variety of photo sets. The photo sets were constructed using 1385 photos from the MPEG-7 Visual Core Experiment 3 (VCE-3) data set and approximately 20000 photos collected from well-known photo-sharing websites. The reported experimental results show that the proposed face annotation method significantly outperforms traditional face annotation solutions at no additional computational cost, with accuracy gains of up to 25% for particular cases.
Semantic annotation of personal video content using an image folksonomy

The increasing popularity of user-generated content (UGC) requires effective annotation techniques in order to facilitate precise content search and retrieval. In this paper, we propose a new approach for the semantic annotation of personal video content, taking advantage of user-contributed tags available in an image folksonomy. Video shots and folksonomy images are first represented by a semantic vector. Next, the semantic vectors are used to measure the semantic similarity between each video shot and the folksonomy images. Tags assigned to semantically similar folksonomy images are then used to annotate the video shots. To verify the effectiveness of the proposed annotation method, experiments were performed with video sequences retrieved from YouTube and images downloaded from Flickr. Our experimental results demonstrate that the proposed method is able to successfully annotate personal video content with user-contributed tags retrieved from an image folksonomy. In addition, the size of our tag vocabulary is significantly higher than the size of the tag vocabulary used by conventional annotation methods.
Investigating experienced quality factors in synthesized multi-view stereo images

In this study, we investigated the quality factors in view synthesis that might lead to the visual discomfort and degradation of the overall viewing quality of the synthesized multi-view stereo or free viewpoint images. In particular, we focused on the investigation of the left and right (LR) image mismatch that might be one of the most important quality factors in the stereoscopic viewing. In order to measure how severely this factor influences the visual comfort and overall quality, we conducted a series of subjective assessment experiments on the visual comfort and viewing preference. In the subjective experiments, we compared the stereo view synthesis results of three different hole filling methods, because incorrectly filled holes could cause severe mismatches between the left- and right-view images. The subjective results revealed that the LR image mismatch could induce more visual discomfort and lower overall quality of stereoscopic images. Also, the results indicated that the visual comfort and overall quality were able to be improved by considering the inter-view consistency in multi-view stereo generation.
Full-Reference Video Quality Metric for Fully Scalable and Mobile SVC Content

Universal Multimedia Access (UMA) aims at enabling a straightforward consumption of multimedia content in heterogeneous usage environments. These usage environments may range from mobile devices in a wireless network to high-end desktop computers with wired network connectivity. Scalable video content can be used to deal with the restrictions and capabilities of diverse usage environments. However, in order to optimally tailor scalable video content along the temporal, spatial, or perceptual quality axis, a metric is needed that reliably models subjective video quality. The major contribution of this paper is the development of a novel full-reference quality metric for scalable video bit streams that are compliant with the H.264/AVC Scalable Video Coding (SVC) standard. The scalable video bit streams are intended to be used in mobile usage environments (e.g., adaptive video streaming to mobile devices). The proposed quality metric allows modeling the temporal, spatial, and perceptual quality characteristics of SVC-compliant bit streams by taking into account several properties of the compressed bit streams. These properties include the temporal and spatial variance of the video content, the frame rate, the spatial resolution, and PSNR values. An extensive number of subjective experiments have been conducted to construct and validate our quality metric. Experimental results show that the average correlation coefficient for the video sequences tested is as high as 0.95 (compared to a value of 0.60 when only using the traditional PSNR quality metric). The proposed quality metric also shows a performance that is a uniformly high for video sequences with different temporal and spatial characteristics.
Versatile video on demand system

In this paper, we proposed versatile VoD (video on demand) system concentrated on managing the services in heterogeneous network and terminal. To deliver and consume the multimedia contents in heterogeneous network and terminal, we adopted MPEG-21 multimedia framework. The proposed system is organized into four components: education content server including authoring module to generate the form of digital item, contents player located on set-top box, content adaptation server and tool server. To verify the usefulness of the proposed system, experiments are performed with the educational video contents.
Efficient modeling of video-to-image conversion for QoS management in networked video service

Modality conversion (also called transmoding) currently emerges as an important issue in universal multimedia access. Intuitively, when content scaling cannot provide the acceptable QoS, modality conversion may be the good choice to maintain the quality. For this purpose, we need to answer a QoS-related question "at which resource constraint should the current modality be converted?" In our previous works, we have proposed the overlapped content value model that represents the relationship between quality and resource for different modalities, thus helping to find the conversion points between modalities. In this paper, we propose an efficient method to build the OCV model in the case of video-to-image conversion.
Combining multiple feature representations and AdaBoost ensemble learning for reducing false-positive detections in Computer-aided Detection of masses on mammograms

One of the drawbacks of current Computer-aided Detection (CADe) systems is a high number of false-positive (FP) detections, especially for detecting mass abnormalities. In a typical CADe system, classifier design is one of the key steps for determining FP detection rates. This paper presents the effective classifier ensemble system for tackling FP reduction problem in CADe. To construct ensemble consisting of correct classifiers while disagreeing with each other as much as possible, we develop a new ensemble construction solution that combines data resampling underpinning AdaBoost learning with the use of different feature representations. In addition, to cope with the limitation of weak classifiers in conventional AdaBoost, our method has an effective mechanism for tuning the level of weakness of base classifiers. Further, for combining multiple decision outputs of ensemble members, a weighted sum fusion strategy is used to maximize a complementary effect for correct classification. Comparative experiments have been conducted on benchmark mammogram dataset. Results show that the proposed classifier ensemble outperforms the best single classifier in terms of reducing the FP detections of masses.
Image tag refinement along the ‘what’ dimension using tag categorization and neighbor voting

Online sharing of images is increasingly becoming popular, resulting in the availability of vast collections of user-contributed images that have been annotated with usersupplied tags. However, user-supplied tags are often not related to the actual image content, affecting the performance of multimedia applications that rely on tag-based retrieval of user-contributed images. This paper proposes a modular approach towards tag refinement, taking into account the nature of tags. First, tags are automatically categorized in five categories using WordNet: ‘where’, ‘when’, ‘who’, ‘what’, and ‘how’. Next, as a start towards a full implementation of our modular tag refinement approach, we use neighbor voting to learn the relevance of tags along the ‘what’ dimension. Our experimental results show that the proposed tag refinement technique is able to successfully differentiate correct tags from noisy tags along the ‘what’ dimension. In addition, we demonstrate that the proposed tag refinement technique is able to improve the effectiveness of image tag recommendation for non-tagged images.
An Evaluation of Bitrate Adaptation Methods for HTTP Live Streaming

HTTP streaming has become a cost-effective means for multimedia delivery nowadays. For adaptivity to networks and terminals, a provider should generate multiple representations of an original video as well as the related metadata. Recently, there have been various adaptation methods to support adaptive HTTP streaming. In this paper, we investigate typical adaptation methods in the context of live video streaming. We first discuss the trade-off among typical adaptation methods. The evaluation and comparison are then carried out not only in terms of bitrate and buffer behaviors but also in terms of the perceptual impact on end users. It is found that the perceptual impact depends not only on adaptation method but also on the content itself. We also show that the preparation of representation sets may affect the behaviors of some adaptation methods.
Towards an automatic face indexing system for actor-based video services in an IPTV environment

Automatic face indexing is an important technique for realizing actor-based video services in an IPTV environment. This paper proposes a novel face indexing system that takes advantages of the internet connection of an STB to construct a FR engine that is equipped with a high number of training face images. In addition, we use a face clustering technique to obtain multiple face images of the same subject from a sequence of video frames. The clustered face images are combined using a weighted feature fusion scheme, resulting in a considerable enhancement in face indexing accuracy. The effectiveness of the proposed system is validated using more than 300,000 video frames, collected from five video clips containing drama or movie content. The experimental results show that the proposed system can achieve a face annotation accuracy that is feasible for practical applications.
Breast tissue removal for enhancing microcalcification cluster detection in mammograms

In this paper, we propose a novel normal breast tissue removal approach using sparse representation (SR), in order to emphasize subtle microcalcifications (MCs) for MC cluster (MCC) detection in mammograms. The proposed method adopts SR to estimate normal breast tissue texture only; such that the difference between estimated image and the original image can emphasize subtle MCs. Comparative experiments have been conducted to validate the effectiveness of the proposed preprocessing with the publicly available DDSM database. The experimental results showed that the MCC detection performances in terms of FROC were improved with the proposed approach compared with the commonly used wavelet decomposition approach. Furthermore, the improved detection performance increases the overall performance for the malignant MCC classification.
Using colour local binary pattern features for face recognition

In this paper, we propose a novel feature representation based on color-based Local Binary Pattern (LBP) texture analysis for face recognition (FR). The proposed method exploits both color and texture discriminative features of a face image for FR purpose. We evaluate the proposed feature using three public face databases: CMU-PIE, Color FERET, and XM2VTSDB. Experimental results show that the results of the proposed feature impressively better than the results of grayscale LBP and color features. In particular, it is shown that the proposed feature is highly robust against severe variations in illumination and spatial resolution.
Feature subspace determination in video-based mismatched face recognition

In video-based face recognition (FR) applications such as surveillance security, the resolution of a facial image could significantly impact the reliability of recognition system. In most practical FR applications, it is reasonable to assume that the face resolution used during training is higher than the resolution used during the identification or verification process. This dimensional mismatch negatively impacts the performance of the traditional subspace recognition solutions. To address resolution mismatch problem, this paper introduces a novel estimation method capable of determining feature subspace of the lower resolution probe given an eigenspace pre-trained with higher resolution facial images. The effectiveness of the proposed solution has been successfully tested on standard CMU PIE face dataset. Experimental results and comparative evaluations provided in this work demonstrate the benefits of the proposed solution.
Human brain response to visual fatigue caused by stereoscopic depth perception

In stereopsis, excessive screen disparity is known as one of the main causes of visual fatigue. Although the visual fatigue caused by stereoscopic depth perception has been reported in a large number of subjective assessments, there has been little effort to measure objective indicators for the visual fatigue. In this study, we investigated the relationship between the visual fatigue and the amount of binocular disparity by using functional magnetic resonance imaging (fMRI). As a result, we detected strong neuronal activities in frontal eye field (FEF) when the presented visual stimuli had disparities beyond the comfort zone. Since FEF is known to be involved in eye movement control, it also appears to have correlation with the visual fatigue caused by the stereoscopic depth perception.
Fast content-based MPEG video indexing using object motion

We propose a new object motion histogram for content-based indexing of MPEG video data. Video data are segmented into shots. Then object motions are extracted and clustered after removing background motion. We adopted polar coordinates to describe the clustered object motion. An object motion histogram in a shot (OMHS) is defined and obtained on the proposed polar division technique. The first moment of object motion is utilized as a descriptor of object motion.
Visual content adaptation according to user perception characteristics

Adapting multimedia content to users' preferences and perceptual characteristics is a key direction for enabling personalized multimedia services. In this paper, we address the problem of tailoring visual content within the MPEG-21 Digital Item Adaptation (DIA) framework to meet users' visual perception characteristics. In particular, we present methods for adapting visual content to accommodate color vision deficiency and low-vision capabilities. In addition, we present methods for adapting visual content according to user preferences for color temperature. Finally, we report on experiments that adapt visual content within the MPEG-21 DIA framework.
Investigating Cascaded Face Quality Assessment for Practical Face Recognition System

Recently, the development of practical face recognition (FR) system has received much attention. Despite of its extensive study, the FR performance could be severely degraded in real-life scenario (e.g., CCTV surveillance), due to uncontrolled face image conditions of pose/alignment, blur, and brightness. This paper proposes new automated face quality assessment (FQA) framework built-in to a practical FR system. In the proposed framework, three quality factors in face images are rapidly evaluated owing to a cascaded classification. Only face images that have been verified by the FQA are used in recognition phase. Our experiment shows that the cascaded FQA can successfully discard face images that could negatively affect FR.
Critical Binocular Asymmetry Measure for the Perceptual Quality Assessment of Synthesized Stereo 3D Images in View Synthesis

In human vision, excessive binocular asymmetry between the left- and right-eye images can be very problematic to perceive single binocular vision (causing visual discomfort) in the viewing of stereoscopic images. In this paper, we propose a critical binocular asymmetry (CBA) measure for objectively assessing the perceptual quality of the synthesized stereo 3D images generated through a depth-image-based rendering (DIBR) process. The proposed method detects critical regions that are likely to induce excessive binocular asymmetry. In particular, this paper considers view extrapolation since it introduces much more artifacts due to the lack of data. We measure structural similarity on the critical regions between the left and right images to quantify the perceptual effects of binocular asymmetry. The effectiveness of the proposed quality measure has been successfully evaluated by subjective assessment experiments using various types of synthesized stereoscopic images generated by four different DIBR-based view synthesis algorithms. We demonstrate the validity of the proposed quality measure by the comparison of subjective ratings and existing objective methods. Experimental results show that the combined use of the proposed binocular asymmetry measure and existing quality measures substantially improves the performance of the quality measures by explicitly considering the perceptual effects of CBA.
Home video management system with semantic information

The increasing amount of home video content has resulted in the need of efficient systems for the management of home video content. In this paper, we discuss the design of a home video management system that relies on the use of several semantic web technologies. The proposed system first segments video into shots. In a next step, semantic information is automatically extracted from key frames that are representative for the shots found. The extracted information is subsequently represented using the resource description framework (RDF), satisfying the constraints of a web ontology language (OWL) schema. The use of RDF facilitates interoperability among different sources of metadata and makes the information understandable for machines. Further, the use of RDF in our system design also allows using the SPARQL protocol and RDF query language (SPARQL) to query our metadata and to retrieve the video content the user is interested in.
Graph-Based Perceptual Quality Model for Audiovisual Contents

Quality is an essential factor in multimedia communication, especially in content adaptation/compression. This paper deals with the perceptual quality of audiovisual contents. Existing audiovisual quality models, which are based on intuitive formulas to combine individual audio and video qualities, cannot clearly identify the contributions of different factors in audiovisual quality. In this paper, we present a graph-based formulation of audiovisual quality. A key advantage of our approach is that it can quantify the contributions of modalities as well as the contribution of their relation in audiovisual perceptual quality.
Analysis of detection performance in spectrum sensing optimisation for long sensing periods

Spectrum sensing optimisation techniques maximise the efficiency of spectrum sensing while satisfying a number of constraints. Many optimisation models consider the possibility of the primary user changing activity state during the secondary user's transmission period. However, most ignore the possibility of activity change during the sensing period and assume the activity state is constant without validating or enforcing this assumption. The observed primary user signal during sensing can exhibit a duty cycle which has been shown to severely degrade detection performance. This paper shows that (a) the probability of state change during sensing cannot be neglected and (b) the true detection performance obtained when incorporating the duty cycle of the primary user signal can deviate significantly from the results expected with the assumption of no such duty cycle.
Chromatic colour spaces for skin detection using GMMS

Skin detection often forms part of both face detectors and pornography detectors for Internet filters. This paper introduces a trivial skin detection algorithm that uses Gaussian Mixture Models to classify 8 × 8 blocks according to their skin-likeness. The absence of structural features in skin regions means luminance information alone is unsuitable for accurate skin detection. Therefore, the affect of the chosen colour components on the classification accuracy is investigated. Fused features vectors extracted from the Cb and Cr chrominance components were found to give the best results, achieving an equal error rate of 5.5%. By utilising these components and an 8 × 8 block size, skin regions in JPEG and MPEG files can be located without full decompression.
Periocular recognition under expression variation using Higher Order Spectral features

Periocular recognition promises attractive trade-off between iris recognition and face recognition because it provides a longer range of imaging than iris and could achieve a higher recognition performance than face. This benefit is critical to the success of a biometric system under unconstrained and less cooperative operating conditions. A number of feature encoding techniques have been proposed for periocular recognition including global features (SIFT, HOG) and local features (LBP). However, these features perform poorly for images captured in non-ideal conditions with the presence of facial expressions. In this paper, we investigate periocular recognition dealing with the deformation caused by facial expressions. In addition, we also investigate a novel feature encoding technique, called Higher Order Spectral features, on periocular images. We show that our proposed approach toward features for the periocular recognition under facial expressions using Higher Order Spectra is effective in encoding discriminant features. The proposed approach is validated on the JAFFE dataset.
Higher order spectral analysis of Chua's circuit

Higher order spectral analysis is used to investigate nonlinearities in time series of voltages measured from a realization of Chua's circuit. For period-doubled limit cycles, quadratic and cubic nonlinear interactions result in phase coupling and energy exchange between increasing numbers of triads and quartets of Fourier components as the nonlinearity of the system is increased, For circuit parameters that result in a chaotic, Rossler-type attractor, bicoherence and tricoherence spectra indicate that both quadratic and cubic nonlinear interactions are important to the dynamics. When the circuit exhibits a double-scroll chaotic attractor the bispectrum is zero, but the tricoherences are high, consistent with the importance of higher-than-second order nonlinear interactions during chaos associated with the double scroll.