nils-herrmann/clean_scirepeval huggingface dataset:
Experimental Investigation of Facial Expressions Associated With Visual Discomfort: Feasibility Study Toward an Objective Measurement of Visual Discomfort Based on Facial Expression

This paper aims to investigate facial expressions associated with visual discomfort induced by excessive screen disparities of stereoscopic three-dimensional (S3D) contents. For this purpose, we constructed a novel facial expression database regarding the visual discomfort. While viewing the realistic stereoscopic stimuli with screen disparities varying from 0° to 4.66°, each viewer's face was captured. The database consisted of face videos and associated comfort scores obtained by self-reporting, which might be only a publicly available database regarding the facial expressions associated with visual discomfort. Using the database, for the quantitative investigation, the facial expressions associated with visual discomfort were compared with basic emotional expressions that were well defined and universal. As a result, we observed that the emotional expression of “stressed” (i.e., anger or disgust) was highly correlated with the perceived visual discomfort (Pearson correlation coefficient: 0.91). Furthermore, the feasibility of the discomfort measurement using facial expressions obtained while viewing S3D contents was verified. Experimental results showed that the discomfort measurement using facial expression recognition could achieve a feasible performance (classification accuracy of 81.42%).
Image compression mismatch effect on color image based face recognition system

Face recognition (FR) for emerging applications such as face tagging for social networking, consumer products, and gamming utilize color images stored in distributed repositories. Such images are often in compressed format and of different dimensions. This compression mismatch problem may adversely affect the performance of the face recognition engine. In this paper, we present a comparative investigation of the image compression mismatch problem. Two commonly used color image based face recognition solutions are utilized. More than three thousand images of 341 subjects, typical of the problem, are collected from three public databases. The experimental results support the main thesis of the paper that recognition performance depends critically on the color image properties.
Use of semantic features for filtering of malicious content in an IPTV environment

The increasing amount of available User-Created Content (UCC) has resulted in the need of efficient systems for the filtering of malicious content. In this paper, we discuss a system capable of detecting malicious content by relying on the use of semantic features. The semantic features are represented by the likelihood rate of elementary concepts (e.g., ‘breast’, ‘bottom’). These elementary concepts together constitute a high-level concept (e.g., ‘naked people’). To do so, the proposed system first segments video into shots. In a next step, semantic features are automatically extracted from key frames that are representative for the shots found. The system then determines the nature of the video content using the extracted semantic features. To verify the usefulness of the proposed filtering system, experiments were performed with UCC available on social network sites.
Crosstalk reduction in stereoscopic displays: A combined approach of disparity adjustment and crosstalk cancellation

This paper proposes a novel crosstalk reduction method for stereoscopic 3D displays by using a combined approach of disparity adjustment and subtractive crosstalk cancellation. Specifically, we propose a disparity adjustment method that can minimize the perceived crosstalk and negative effects of the crosstalk cancellation on the image quality. In addition, we also provide a contrast reduction method optimized for the subtractive crosstalk cancellation. The experimental results showed that the proposed method could provide higher image quality than existing crosstalk cancellation methods while successfully reducing the perceived crosstalk.
Noise reduction in DEXA image based on system noise modeling

Denoising X-ray image is an important preprocessing process to measure BMD (Bone Mineral Density) correctly. In this paper, we deal with denoising X-ray image in DEXA (Dual-Energy X-ray Absorptiometry) system. Image noise in DEXA is modeled by separating source and detector noises. Using the noise model in DEXA, a noise reduction method is proposed. The proposed denoising method consists of Wiener filter to reduce the detector noise, and logarithmic scale-wavelet BayesShrink filter to reduce the source noise. Experimental results verified the proposed DEXA noise model and showed that the proposed noise reduction method outperformed the conventional method.
Local age group modeling in unconstrained face images for facial age classification

Age classification in the real world is a very challenging task due to the large variation of face appearances (e.g., a variety of human races, genders, facial expressions, poses etc.). In this paper, we propose a new age classification method using local modeling of age group to deal with large variation problem. The local modeling is built by clustering training faces within an age group. Nearest face clusters in the local modeling to a test face contribute in determining the age group of the test face. This enables us to reduce the effect of the variation unrelated to age. For comparing the test face with the face clusters, we combine two complementary similarities that consider the cluster centroid and the intra-cluster variation. Experimental result on a real-world dataset shows that our local modeling based approach is superior to global modeling based approach, achieving state-of-the-art performance.
Meaningful scene filtering for TV terminals

Digital broadcasting has been in progress for user-friendly TV watching environment by personalized broadcasting services. As one of the services, a meaningful scene filtering for TV terminals including a set-top box and a personal video recorder (PVR) is proposed in this paper. Using the proposed filtering, TV viewer can find desired scenes from multiple programs of multiple channels simultaneously. The usefulness of the proposed content filtering is verified with sport video on a broadcasting system with intelligent agent platforms
Fast MPEG-7 visual descriptor extraction using DCT coefficient

TMPEG-7 descriptors are known as standard descriptors for the content-based indexing and retrieval. To extract visual features from a compressed image, the spatial domain information should be extracted from the compression domain information. In this paper, we propose the fast feature extraction method that can be applied to MPEG-7 visual descriptor from DCT compressed image. In the proposed method, fast extraction of descriptor is achieved by combination of down sampling and direct calculation of spatial information in DCT domain. Experiment is performed with MPEG-7 reference database. The results show that the proposed method can be applied to fast feature extraction with large size of database in Internet.
Enhanced classification of focal hepatic lesions in ultrasound images using novel texture features

This paper discusses novel texture features that allow providing enhanced classification accuracy for focal hepatic lesions. The proposed texture features takes advantage of the rotation and scale invariant nature of Gabor wavelets, as well as the gray-level co-occurrence matrix (GLCM) for analyzing the spatial distribution of the pixel intensity in the lesion. To verify the effectiveness of the proposed texture features, experiments were performed with 150 ultrasound images containing 150 focal hepatic lesions, consisting of 50 cysts, 50 hemangiomas, and 50 malignancies. Experimental results show that the proposed texture features allow for an improved classification performance, compared to the use of other features.
Semantic Home Photo Categorization

A semantic categorization method for generic home photo is proposed. The main contribution of this paper is to exploit a two-layered classification model incorporating camera metadata with low-level features for multilabel detection. The two-layered support vector machine (SVM) classifiers operate to detect local and global photo semantics in a feed-forward way. The first layer aims to predict likelihood of predefined local photo semantics based on camera metadata and regional low-level visual features. In the second layer, one or more global photo semantics is detected based on the likelihood. To construct classifiers producing a posterior probability, we use a parametric model to fit the output of SVM classifiers to posterior probability. A concept merging process based on a set of semantic-confidence maps is also presented to cope with selecting more likelihood photo semantics on spatially overlapping local regions. Experiment was performed with 3086 photos that come from MPEG-7 visual core experiment two official databases. Results showed that the proposed method would much better capture multiple semantic meanings of home photos, compared to other similar technologies
Using color texture sparsity for facial expression recognition

This paper presents a new facial expression recognition (FER) which exploits the effectiveness of color information and sparse representation. For extracting face feature, we compute color vector differences between color pixels so that they can effectively capture change of face appearance (e.g., skin texture). Through comparative and extensive experiment using two public FER databases (DBs), we validate that our color texture features are suited to the sparse representation for improving FER accuracy. Specifically, our color texture features can considerably improve the recognition accuracy obtained by sparse representation compared with other features (e.g., Local Binary Pattern (LBP)) under realistic recognition conditions (e.g., low-resolution faces). It is also shown that the use of our features can yield high discrimination capability and sparsity, justifying the high recognition accuracies obtained. Further, the proposed FER outperforms five other state-of-the-art FER methods.
Visual content adaptation for low vision users in MPEG-21 framework

This paper deals with the visual content adaptation, in the context of MPEG-21 standard, to help low vision users have better accessibility to the contents. The proposed adaptation targets at two low vision symptoms, loss of fine detail and lack of contrast. Specifically, we present an adaptation framework describing the problem space and then a systematic contrast-enhancement method to improve the content visibility for low vision users. The experiment results show that the proposed framework and method are effective for the low vision users.
Semantic Detection of Adult Image Using Semantic Features

Recently, in the fields of internet and social networking, the classification and filtering of naked images has been receiving a significant amount of attention. In this paper, we propose a novel naked image classification which can make effective use of semantic features of a naked image. In addition, a novel measurement, termed accumulated distance ratio (ADR), is proposed in order to systematically analyze the effect of semantic features on improving classification performance, compared to the approach relying on low-level visual features. Extensive experiments have been carried out to assess the effectiveness of semantic features in naked image classification with realistic and challenging data set. The experimental result of the proposed approach using semantic features, for challenging data set, shows improvement up to 14% than the approach using low-level visual feature. Further, the proposed ADR measure has proven to be useful measure for analyzing the effect of semantic features for naked image classification.
Color Face Recognition for Degraded Face Images

In many current face-recognition (FR) applications, such as video surveillance security and content annotation in a Web environment, low-resolution faces are commonly encountered and negatively impact on reliable recognition performance. In particular, the recognition accuracy of current intensity-based FR systems can significantly drop off if the resolution of facial images is smaller than a certain level (e.g., less than 20 times 20 pixels). To cope with low-resolution faces, we demonstrate that facial color cue can significantly improve recognition performance compared with intensity-based features. The contribution of this paper is twofold. First, a new metric called ldquovariation ratio gainrdquo (VRG) is proposed to prove theoretically the significance of color effect on low-resolution faces within well-known subspace FR frameworks; VRG quantitatively characterizes how color features affect the recognition performance with respect to changes in face resolution. Second, we conduct extensive performance evaluation studies to show the effectiveness of color on low-resolution faces. In particular, more than 3000 color facial images of 341 subjects, which are collected from three standard face databases, are used to perform the comparative studies of color effect on face resolutions to be possibly confronted in real-world FR systems. The effectiveness of color on low-resolution faces has successfully been tested on three representative subspace FR methods, including the eigenfaces, the fisherfaces, and the Bayesian. Experimental results show that color features decrease the recognition error rate by at least an order of magnitude over intensity-driven features when low-resolution faces (25 times 25 pixels or less) are applied to three FR methods.
Efficient and effective human action recognition in video through motion boundary description with a compact set of trajectories

Human action recognition (HAR) is at the core of human-computer interaction and video scene understanding. However, achieving effective HAR in an unconstrained environment is still a challenging task. To that end, trajectory-based video representations are currently widely used. Despite the promising levels of effectiveness achieved by these approaches, problems regarding computational complexity and the presence of redundant trajectories still need to be addressed in a satisfactory way. In this paper, we propose a method for trajectory rejection, reducing the number of redundant trajectories without degrading the effectiveness of HAR. Furthermore, to realize efficient optical flow estimation prior to trajectory extraction, we integrate a method for dynamic frame skipping. Experiments with four publicly available human action datasets show that the proposed approach outperforms state-of-the-art HAR approaches in terms of effectiveness, while simultaneously mitigating the computational complexity.
Predicting Visual Discomfort of Stereoscopic Images Using Human Attention Model

We introduce a new objective assessment method for visual discomfort of stereoscopic images that makes effective use of the human visual attention model. The proposed method takes into account visual importance regions that play an important role in determining the overall degree of visual discomfort of a stereoscopic image. After obtaining a saliency-based visual importance map for an image, perceptually significant disparity features are extracted to predict the overall degree of visual discomfort. Experimental results show that the proposed method can achieve significantly higher prediction accuracy than the state-of-the-art methods.
Improvement of speckle noise reduction using multi-resolutional coherence measurement in ultrasound image

Ultrasound (US) images are degraded by speckle noise that reduces the contrast and details of images. But the effective method for speckle noise reduction with keeping edge has been still a challenging point. Coherence measurement has been used to distinguish homogeneous region and coherence regions, e.g. edges, and the edge map obtained from coherence measurement has limitation such as noisy or discontinued edge detection. In this paper, to overcome these problems, the enhanced edge map generation using multi-resolutional coherence measurement is proposed and adaptive wavelet based-speckle noise reduction is performed using the enhanced edge map. The experimental results showed that the enhanced edge map by proposed method contains the robust and linked edge information. Moreover, the results show that the proposed method outperforms the conventional method in preserving edge details.
Multiview Stereoscopic Video Hole Filling Considering Spatiotemporal Consistency and Binocular Symmetry for Synthesized 3D Video

This paper proposes a new hole-filling method with spatiotemporal consistency and binocular symmetry for synthesized 3D videos in view extrapolation. Disocclusion regions in the synthesized views at virtual viewpoints result in regions with missing content. These regions will be referred to as hole regions. To provide the high-quality synthesized 3D videos via 3D display, the hole regions need to be filled considering the characteristics of human visual perception. From the perceptual point of view, binocular asymmetry between synthesized left- and right-eye videos (i.e., stereo pair video) is one of the most important factors that induce visual discomfort in stereoscopic viewing. In addition, binocular symmetry without temporal consistency between temporally neighboring frames could cause visual discomfort by annoying flickering artifacts. In this paper, to maintain the spatiotemporal consistency and binocular symmetry in synthesized 3D videos at multiple virtual viewpoints, we propose a global optimization-based hole-filling method using the information from the already filled adjacent view and previous frame. Furthermore, to reduce the computational cost of the global optimization, we propose a label propagation method, which propagates reliable labels used in the adjacent view and previous frame to the target image to be filled. The performance of the proposed method has been evaluated by objective assessments of 3D image quality, temporal consistency, and computational efficiency. In addition, subjective assessment is conducted for measuring the visual comfort and overall quality. The experimental results proved that the proposed method provides hole filling results with spatiotemporal consistency and binocular symmetry.
Adaptive attention fusion network for visual question answering

Automatic understanding of the content of a reference image and natural language questions is needed in Visual Question Answering (VQA). Generating a visual attention map that focuses on the regions related to the context of the question can improve performance of VQA. In this paper, we propose adaptive attention-based VQA network. The proposed method utilizes the complementary information from the attention maps depending on three levels of word embedding (word level, phrase level, and question level embedding), and adaptively fuses the information to represent the image-question pair appropriately. Comparative experiments have been conducted on the public COCO-QA database to validate the proposed method. Experimental results have shown that the proposed method outperforms previous methods in terms of accuracy.
Facial dynamic modelling using long short-term memory network: Analysis and application to face authentication

According to the supplementary information hypothesis in psychology, facial motion benefits the perception of identity for human. In this study, we propose a face authentication framework which exploits facial dynamics with appearance to effectively improve the authentication performance. In our face authentication scenario, users are guided to make smile expression and the identity behind smile dynamics has been utilized. In order to model the facial dynamics, the recurrent neural network with long short-term memory cells is adopted and the facial dynamics from onset to offset duration is encoded. Comparative experiment has showed that the combination of facial dynamic features with appearance features improves the accuracy of the face authentication system compared to conventional appearance features and spatio-temporal features by effectively capturing facial dynamic and appearance features.
Bilateral hemiface feature representation learning for pose robust facial expression recognition

We propose a bilateral hemiface feature representation learning via convolutional neural networks (CNNs) for pose robust facial expression recognition. The proposed method considers two characteristics of facial expressions. First, features from local patches are more robust to pose variations. Second, human faces are bilaterally symmetrical on left and right hemifaces. To incorporate those characteristics, a CNN is devised to learn feature representations from local patches. Then, feature representations are learned from each hemiface separately. To reduce the effect of self-occlusion, a shared feature representation is learned by combining both hemiface feature representations. The shared feature representation adaptively learns to utilize the hemiface feature representations according to the head pose. Experiments conducted on the Multi-PIE dataset showed that the proposed bilateral hemiface feature representation is pose robust and compares favorably to state-of-the-art methods.
Adaptive video streaming over HTTP with dynamic resource estimation

Adaptive hypertext transfer protocol (HTTP) streaming has become a new trend to support adaptivity in video delivery. An HTTP streaming client needs to estimate exactly resource availability and resource demand. In this paper, we focus on the most important resource which is bandwidth. A new and general formulation for throughput estimation is presented taking into account previous values of instant throughput and round trip time. Besides, we introduce for the first time the use of bitrate estimation in HTTP streaming. The experiments show that our approach can effectively cope with drastic changes in connection throughput and video bitrate.
Exploiting collective knowledge in an image folksonomy for semantic-based near-duplicate video detection

An increasing number of duplicates and near-duplicates can be found on websites for video sharing. These duplicates and near-duplicates often infringe copyright or clutter search results. Consequently, a high need exists for techniques that allow identifying duplicates and near-duplicates. In this paper, we propose a semantic-based approach towards the task of identifying near-duplicates. Our approach makes use of semantic video signatures that are constructed by detecting semantic concepts along the temporal axis of video sequences. Specifically, we make use of an image folksonomy (i.e., a set of user-contributed images annotated with user-supplied tags) to detect semantic concepts in video sequences, making it possible to exploit an unrestricted concept vocabulary. Comparative experiments using the MUSCLE-VCD-2007 dataset and folksonomy images retrieved from Flickr show that our approach is successful in identifying near-duplicates.
Face image assessment learned with objective and relative face image qualities for improved face recognition

Considerable research efforts have been made for face recognition in various real-world applications. However, degraded face images, acquired in the real-world, make face recognition difficult. In this paper, we propose a new face image quality assessment that aims to realize a robust and reliable face recognition system. The proposed method considers two factors for face image quality, i.e., visual quality and mismatch between training and test face images. A face image quality assessor is learned based on the two factors to discriminate useful faces from unuseful ones. The proposed face image quality assessment model is robust and adaptive to face recognition systems by employing a learned assessment. Our experimental results on a challenging database show significant improvement in face recognition accuracy by the proposed method.