nils-herrmann/clean_scirepeval huggingface dataset:
Real-time quality evaluation of adaptation strategies in VoD streaming

HTTP Adaptive Streaming (HAS) has become a popular trend for multimedia delivery nowadays. Because of throughput variations, video adaptation methods are needed to avoid buffer underflows. In this context, it is also important to evaluate the overall video quality of a session. In this paper, we investigate a quality model that can evaluate a session quality as well as different adaptation strategies in real time. We use the histogram of segment quality values and the histogram of quality gradients in a session to model the overall video quality. Then, our quality model is employed to evaluate, for the first time, the cumulative quality of typical adaptation methods in real time. It is found that, to provide a high quality level, the client should avoid changing versions frequently and drastically.
Content sharing between home networks by using personal information and associated fuzzy vault scheme

Content sharing is getting popular in home network as well as in social media. Content sharing inside home is usually done among family members, while that between different home networks is done by friends and even anonymous users. Especially, anonymous users could access contents in home networks which could act like home portals. To protect content in home servers, the contents can be encrypted by key and the key can be shared among users. However it is quite challenging to distribute the key to anonymous users. In this paper, we present a method for sharing contents securely among different home networks. We propose a method of content sharing between anonymous users, who have similar personal preferences about the contents. The personal preferences include user profiles and contents preference. With the proposed method, a content sharing community is created for content delivery between different home networks. The community allows content creators to deliver contents to other users, who have similar personal information and can consume the contents without any leakage of personal information. In order to verify the usefulness of the proposed method, experiments were performed. The results showed that contents in one home server were securely shared with users of the other home server, if both have similar personal information.
Visual Importance- and Discomfort Region-Selective Low-Pass Filtering for Reducing Visual Discomfort in Stereoscopic Displays

In human vision, the depth of focus (DOF) blur is important to perceive single binocular vision. Unfortunately, conventional stereoscopic displays cannot provide a natural DOF blur effect. Synthetic DOF blur that simulates the natural DOF blur effect could be useful for reducing the visual discomfort in stereoscopic displays. However, the incorrect introduction of artificial blur in the image might also have negative effects. This paper proposes a selective DOF blur technique that is applied only to local regions that induce high visual discomfort but are less visually important. We employ a visual importance analysis to obtain reasonable estimates of the areas/objects that are likely to be attended using saliency maps. After identifying the regions likely to cause discomfort because of excessive relative disparity, we apply low-pass filtering for the less-salient neighboring pixels of the discomfort regions. We demonstrate that the DOF blur simulation for less important background regions that induce visual discomfort is helpful in reducing the visual discomfort in stereoscopic viewing.
Color channel-wise recurrent learning for facial expression recognition

Facial expression recognition is increasingly gaining importance in emerging affective computing applications. In practice, achieving accurate facial expression recognition is still challenging due to environmental variations. In this paper, we propose a color channel-wise recurrent facial feature learning. The proposed method adopts recurrent neural network to learn expression features sequentially along color channels. The proposed network preserves discriminative expression feature through a long short-term memory for the sequence of color spatial features. Comprehensive experiments have been conducted on the publically available CMU Multi-PIE dataset under illumination variations. Experimental results showed that the proposed method achieved higher recognition rates compared to the state-of-the-art methods.
Video Copy Detection Using Inclined Video Tomography and Bag-of-Visual-Words

Techniques for video fingerprinting are helpful in managing vast libraries of video clips. Recent advances have shown that video tomography and Bag-of-Visual-Words (BoVW) can be successfully used for the purpose of video fingerprinting. In this paper, we introduce a novel video signature (i.e., a novel video fingerprint) that takes advantage of both video tomography and BoVW. Specifically, the proposed video signature is created by first extracting inclined tomography images from the video content, and by subsequently applying the BoVW approach to the inclined tomography images obtained. The key to our approach is that we make the angle of inclination of the tomography images dependent on the amount of motion in the video content. That way, the proposed video signature is able to capture both spatial and temporal information. Experimental results obtained for the publicly available TREVID-2009 video set indicate that video copy detection by means of the proposed video signature is robust against spatial and temporal transformations.
Visual discomfort visualizer using stereo vision and time-of-flight depth cameras

Visual discomfort monitoring is an important technique for comfortable viewing of stereoscopic images. This paper proposes a novel monitoring system of visual discomfort that takes advantages of a fusion camera system equipped with stereo vision cameras and a time-of-flight camera. We improve disparity map enhancement technique to obtain a high-quality disparity map that is aligned to the right view color image coordinate. The captured stereo-plusdisparity images are used for visual discomfort visualizer that automatically predicts overall degree of perceived visual discomfort and visualizes which regions are problematic in terms of visual discomfort. The proposed visual discomfort visualizer is based on visual importance of image scenes, resulting in considerable improvement in prediction performance of visual discomfort. The experimental results showed that the proposed system could generate enhanced disparity maps and achieve visual discomfort prediction accuracy that is feasible for practical applications.
Real-time filtering system on content semantics for live broadcasting

Currently, digital broadcasting runs to make more user-friendly TV watching environment by offering personalized broadcasting services that can adapt to viewer's intention or preference. As one of the services, a real-time broadcasting content filtering could provide a useful function that a TV viewer can find or store desired scenes from programs of multiple channels while watching his/her favorite program on a main channel. In this paper, therefore, we propose a real-time filtering system of broadcasting content semantics and implement it in a TV terminal such as a set-top box or a personal video recorder. The usefulness of the broadcasting content filtering is verified with sport videos on an agent-based intelligent broadcasting system.
Multispectral Texture Features from Visible and Near-Infrared Synthetic Face Images for Face Recognition

Recently, high-performance face recognition has attracted research attention in real-world scenarios. Thanks to the advances in sensor technology, face recognition system equipped with multiple sensors has been widely researched. Among them, face recognition system with near-infrared imagery has been one important research topic. In this paper, complementary effect resided in face images captured by nearinfrared and visible rays is exploited by combining two distinct spectral images (i.e., face images captured by near-infrared and visible rays). We propose a new texture feature (i.e., multispectral texture feature) extraction method with synthesized face images to achieve high-performance face recognition with illumination-invariant property. The experimental results show that the proposed method enhances the discriminative power of features thanks the complementary effect.
Near-Duplicate Video Clip Detection Using Model-Free Semantic Concept Detection and Adaptive Semantic Distance Measurement

Motivated by the observation that content transformations tend to preserve the semantic information conveyed by video clips, this paper introduces a novel technique for near-duplicate video clip (NDVC) detection, leveraging model-free semantic concept detection and adaptive semantic distance measurement. In particular, model-free semantic concept detection is realized by taking advantage of the collective knowledge in an image folksonomy (which is an unstructured collection of user-contributed images and tags), facilitating the use of an unrestricted concept vocabulary. Adaptive semantic distance measurement is realized by means of the signature quadratic form distance (SQFD), making it possible to flexibly measure the similarity between video shots that contain a varying number of semantic concepts, and where these semantic concepts may also differ in terms of relevance and nature. Experimental results obtained for the MIRFLICKR-25000 image set (used as a source of collective knowledge) and the TRECVID 2009 video set (used to create query and reference video clips) demonstrate that model-free semantic concept detection and SQFD can be successfully used for the purpose of identifying NDVCs.
Face Feature Weighted Fusion Based on Fuzzy Membership Degree for Video Face Recognition

This paper proposes a new video face recognition (FR) method that is designed for significantly improving FR via adaptive fusion of multiple face features (belonging to the same subject) acquired from a face sequence of video frames. In this paper, we derive an upper bound for recognition error arising from the proposed weighted feature fusion to justify theoretically its effectiveness for recognition from videos. In addition, in order to compute the optimal weights of face features to be fused, we develop a novel weight determination solution based on fuzzy membership function and quality measurement for face images. Using four public video databases, the effectiveness of the proposed method has been successfully evaluated under the conditions that are similar to those in real-world video FR applications. Furthermore, our method is simple and straightforward to implement.
Mass detection based on pooled mass probability map of 3D reconstructed slices in digital breast tomosynthesis

In this paper, we propose a novel approach for automated detection of breast masses in three-dimensional (3D) reconstructed slices on digital breast tomosynthesis (DBT). The 3D reconstructed slices provide quasi-3D information with limited resolution along the depth direction due to insufficient sampling in depth direction. This problem could cause an error of general 3D segmentation approaches which have to process information with different resolution at the same time. In order to resolve the problem, this paper proposes an effective mass detection method based on pooled mass probability map. The proposed pooled mass probability map contains slice plane information by fusing mass probabilities of initially detected regions along slices. Extensive and comparative experiments have been conducted using clinical data set to validate the effectiveness of proposed mass detection approach. Experimental results demonstrate the feasibility of proposed pooled mass probability map based approach for detecting masses on 3D reconstructed slices.
A deep facial landmarks detection with facial contour and facial components constraint

In this paper, we propose a new facial landmarks detection method based on deep learning with facial contour and facial components constraints. The proposed deep convolutional neural networks (DCNNs) for facial landmark detection consists of two deep networks: one DCNN is to detect landmarks constrained on the facial contour and the other is to detect landmarks constrained on facial components. A novel DCNN structure for the landmarks detection with facial component constraints is proposed, which branches the network at higher layers in order to capture the intricate local facial components features. Moreover, a novel learning strategy is proposed to learn the DCNN for detecting the landmarks on the facial contour by exploiting the relationship between facial contour landmarks and those on facial components. Experimental results have shown that the proposed method outperforms the state-of-the-art FLD methods.
Iterative deep convolutional encoder-decoder network for medical image segmentation.

In this paper, we propose a novel medical image segmentation using iterative deep learning framework. We have combined an iterative learning approach and an encoder-decoder network to improve segmentation results, which enables to precisely localize the regions of interest (ROIs) including complex shapes or detailed textures of medical images in an iterative manner. The proposed iterative deep convolutional encoder-decoder network consists of two main paths: convolutional encoder path and convolutional decoder path with iterative learning. Experimental results show that the proposed iterative deep learning framework is able to yield excellent medical image segmentation performances for various medical images. The effectiveness of the proposed method has been proved by comparing with other state-of-the-art medical image segmentation methods.
Near-Duplicate Video Detection Using Temporal Patterns of Semantic Concepts

Methods for video copy detection are typically based on the use of low-level visual features. However, low-level features may vary significantly for near-duplicates, which are video sequences that have been the subject of spatial or temporal modifications. As such, the use of low-level visual features may be inadequate for detecting near-duplicates. In this paper, we present a new video copy detection method that aims to identify near-duplicates for a given query video sequence. More specifically, the proposed method is based on identifying semantic concepts along the temporal axis of a particular video sequence, resulting in the construction of a so-called semantic video signature. The semantic video signature is then used for the purpose of similarity measurement. The main advantage of the proposed method lies in the fact that the presence of semantic concepts is highly robust to spatial and temporal video transformations. Our experimental results show that the use of a semantic video signature allows for the efficient and effective detection of near-duplicates.
Multi-Objective Based Spatio-Temporal Feature Representation Learning Robust to Expression Intensity Variations for Facial Expression Recognition

Facial expression recognition (FER) is increasingly gaining importance in various emerging affective computing applications. In practice, achieving accurate FER is challenging due to the large amount of inter-personal variations such as expression intensity variations. In this paper, we propose a new spatio-temporal feature representation learning for FER that is robust to expression intensity variations. The proposed method utilizes representative expression-states (e.g., onset, apex and offset of expressions) which can be specified in facial sequences regardless of the expression intensity. The characteristics of facial expressions are encoded in two parts in this paper. As the first part, spatial image characteristics of the representative expression-state frames are learned via a convolutional neural network. Five objective terms are proposed to improve the expression class separability of the spatial feature representation. In the second part, temporal characteristics of the spatial feature representation in the first part are learned with a long short-term memory of the facial expression. Comprehensive experiments have been conducted on a deliberate expression dataset (MMI) and a spontaneous micro-expression dataset (CASME II). Experimental results showed that the proposed method achieved higher recognition rates in both datasets compared to the state-of-the-art methods.
Pose-Robust and Discriminative Feature Representation by Multi-task Deep Learning for Multi-view Face Recognition

Automatic face recognition (FR) under uncontrolled environments has attracted considerable research attention. In the uncontrolled environments, pose variation is known as one of the crucial factors that influences FR performance. In this paper, we propose a discriminative and pose-robust feature representation using the multi-task learning in deep convolutional neural networks (ConvNet). We introduce four tasks (i.e., maximizing inter-class variation, minimizing intraclass variation, minimizing intra-pose variation, and preserving pose continuity) to learn the ConvNet. Moreover, two-stage learning strategy is proposed to minimize the error functions in learning the deep ConvNet. The extensive experimental results (with the challenging CMU MultiPIE dataset containing pose variations) show that the proposed method outperform stateof-the-art in terms of FR accuracy. Furthermore, the proposed method shows significant improvement even for the face images whose poses are not included in training set.
Binocular Fusion Net: Deep Learning Visual Comfort Assessment for Stereoscopic 3D

In this paper, we propose a novel deep learning-based visual comfort assessment (VCA) for stereoscopic images. To assess the overall degree of visual discomfort in stereoscopic viewing, we devise a binocular fusion deep network (BFN) learning binocular characteristics between stereoscopic images. The proposed BFN learns the latent binocular feature representations for the visual comfort score prediction. In the BFN, the binocular feature is encoded by fusing the spatial features extracted from left and right views. Finally, the visual comfort score is predicted by projecting the binocular feature onto the subjective score space. In addition, we devise a disparity regularization network (DRN) for improving the prediction results. The proposed DRN takes the binocular feature from the BFN and estimates disparity maps from the feature in order to embed disparity relations between left and right views into the deep network. The proposed deep network with BFN and DRN is end-to-end trained in a unified framework in which the DRN acts as disparity regularization. We evaluated the prediction performance of the proposed deep network for VCA by the comparison of existing objective VCA metrics. Further, we demonstrated that the proposed BFN showed various factors causing visual discomfort by using network visualization.
Spatio-temporal representation for face authentication by using multi-task learning with human attributes

For human identification, facial motion is useful in representing specific dynamic signature. In this paper, we present an effective spatio-temporal representation from facial motion as well as appearance by devising a 3D convolutional neural network (CNN). To maintain the intra-class invariance with limited number of training samples, a multi-task learning approach with human attributes, which are high-level semantic descriptions for identity, has been proposed. Identity-related human attributes can be leveraged to learn the 3D CNN. Comparative experiment has showed that the proposed method improves the performance of the face-based authentication system compared to conventional methods by effectively encoding facial appearance and motions with identity-related human attributes.
A novel mammographic mass detection approach to combining suprevised and unsuprevised detection algorithms

In this paper, we propose the combination of different mass detection algorithms to increase overall mass detection sensitivity for various types of breast masses on mammograms. In particular, supervised and unsupervised mass detection algorithms are effectively combined to maximize complementary effects of both approaches. By combining the aforementioned mass detection algorithms, we can arrive at a combined mass detection approach that makes stronger and accurate detection results. Comparative experiments have been conducted on public mammogram data set. Our results show that the proposed detection system can considerably improve the mass detection sensitivity with relatively small number of false positives, compared to the implementation of using only a single detection solution.
Combining multiresolution local binary pattern texture analysis and variable selection strategy applied to computer-aided detection of breast masses on mammograms

In this paper, we propose new texture features, so-called multiresolution local binary pattern (LBP), for classifying breast masses and normal tissue on mammograms. The proposed texture features are able to well characterize the texture patterns of mass core region and its margin. To this end, two individual LBP patterns are extracted from the core region and the ribbon region of pixels of a given mass region, respectively. Further, in order to improve classification accuracy, SVM-RFE based variable selection solution is applied for selecting an optimal subset of variables of multiresolution LBP texture features. Extensive and comparative experiments have been conducted to evaluate our multiresolution LBP features in conjunction with SVM-RFE based variable selection on public benchmark mammogram data set. Our results demonstrate the feasibility of combining our multiresolution LBP features with variable selection strategy for classification of masses and normal tissue on mammograms.
An automatic face indexing framework for actor-based video services in an IPTV environment

Automatic face indexing is a promising technique for personalized IPTV applications. This paper proposes a novel framework that makes use of an STB's Internet connection in order to realize face indexing for a personal media library. In particular, given a drama or movie title, actor names are first retrieved from a drama or movie information web service. The obtained actor names are subsequently used to acquire training and gallery face images from the Internet. More than 20,000 video frames collected from five drama video sequences were used to evaluate our face indexing framework. Experimental results show that the obtained face indexing accuracy is promising for realizing actor-based video services in an IPTV environment.
Watermarking requirement for QoS adaptive transcoding

In ubiquitous environment, seamless video consumption requires content adaptation technologies to satisfy QoS (quality of service). But video transcoding that is one of the content adaptations would distort the watermark signal embedded in the video content. In this paper, we analyze the effect of video transcoding for frequency based watermarking and propose watermarking requirement to keep watermark even after video transcoding. Experiments were performed with widely used watermark algorithms to verify that the proposed watermarking requirement is useful.
Quality Measurement Modeling on Scalable Video Applications

For various mobile applications, measuring the grade of the video quality is needed in order to guarantee the optimal quality of video streaming service. As H.264/AVC scalable video coding (SVC) has been emerged and developed to support full scalability including spatial, temporal, and signal-to-noise ratio (SNR) scalability, each of which shows different visual effect, it is necessary to measure video quality with full scalability. In this paper, we develop a novel video quality metric allowing full scalability through the subjective quality assessment. Experimental results show that the proposed quality metric has high correlation with subjective quality and is useful to determine the video quality of SVC.