nils-herrmann/clean_scirepeval huggingface dataset:
Identifying Source Cell Phone using Chromatic Aberration

Chromatic aberration is the phenomenon where light of different wavelengths fail to converge at the same position on the focal plane. There are two kinds of chromatic aberration: longitudinal aberration causes different wavelengths to focus at different distances from the lens while lateral aberration is attributed to different wavelengths focusing at different positions on the sensor. In this paper, we estimate the parameters of lateral chromatic aberration by maximizing the mutual information between the corrected R and B channels with the G channel. The extracted parameters are then used as input features to a SVM classifier for identifying source cell phone of images. By considering only a certain part of the image when estimating the parameters, we reduce the runtime complexity of the algorithm dramatically while preserving the accuracy at a high level.
Content based editing of semantic video metadata

Bridging the 'signal-symbol gap' existing between multimedia signals generated through audio, video or other multimedia streams and the high level symbols (metadata) which describe them is presently one of the most vital areas of multimedia research. The paper attempts to bridge this significant gap by proposing a novel automatic mechanism for XML based video metadata editing, in tandem with video editing operations. An implementation framework for editing metadata in accordance with the video editing operations is demonstrated. Conflicting resolution and regularization operations are defined and implemented with respect to video metadata editing operations.
Semantic video summarization in compressed domain MPEG video

In this paper, we present a semantic summarization algorithm that interfaces with the metadata and that works in compressed domain, in particular MPEG-1 and MPEG-2 videos. In enabling a summarization algorithm through high-level semantic content, we try to address two major problems. First, we present the facility provided in the DVA system that allows the semi-automatic creation of this metadata. Second, we address the main point of this system which is the utilization of this metadata to filter out frames, creating an abstract of a video summary quality survey indicates that the proposed method performs satisfactorily.
Unsupervised classification of music genre using hidden Markov model

Music genre classification can be of great utility to musical database management. Most current classification methods are supervised and tend to be based on contrived taxonomies. However, due to the ambiguities and inconsistencies in the chosen taxonomies, these methods are not applicable for a much larger database. We proposed an unsupervised clustering method, based on a given measure of similarity which can be provided by hidden Markov models. In addition, in order to better characterize music content, a novel segmentation scheme is proposed, based on music intrinsic rhythmic structure analysis and features are extracted based on these segments. The performance of this feature segmentation scheme performs better than the traditional fixed-length method, according to experimental results. Our preliminary results also suggest that the proposed method is comparable to the supervised classification method.
Video Modeling Using Strata-Based Annotation

Our project on digital video modeling aims to achieve efficient browsing and retrieval. Video is not merely a huge collection of still images, it is a complex temporal medium capturing high-level semantic ideas. For almost a decade researchers have worked on developing efficient techniques to model, index, and retrieve digital video information. To support intuitive video access, the video data must be properly modeled and indexed based on its characteristics and contents. One such annotation-based model is the stratification model, which focuses on segmenting the video's contextual information into multiple layers or strata. Each stratum describes the temporal occurrences of a simple concept such as the appearance of an anchor person in a news video. It thus focuses on segmenting the contextual information into chunks rather than dividing physically contiguous frames into shots, as is traditionally done.
Soccer video event detection with visual keywords

In this paper, we propose a new two-level framework to analyze high-level structure of video and to detect useful events automatically based on visual keywords. The first level extracts low-level features such as motion, color, texture etc to detect video segments boundaries and label segments as visual keywords. We then apply an event detection grammar to the visual keywords sequence at the second level to detect video segments that match the predefined event model. The exact position of the segment that the event occurs can also be spotted. We have applied the proposed approach to the detection of goal and corner-kick events in the portions of 4 FIFA World Cup 2002 soccer videos (1666 segments) with more than 80% accuracy.
Multi-view action recognition by cross-domain learning

This paper proposes a novel multi-view human action recognition method by discovering and sharing common knowledge among different video sets captured in multiple viewpoints. To our knowledge, we are the first to treat a specific view as target domain and the others as source domains and consequently formulate the multi-view action recognition into the cross-domain learning framework. First, the classic bag-of-visual word framework is implemented for visual feature extraction in individual viewpoints. Then, we propose a cross-domain learning method with block-wise weighted kernel function matrix to highlight the saliency components and consequently augment the discriminative ability of the model. Extensive experiments are implemented on IXMAS, the popular multi-view action dataset. The experimental results demonstrate that the proposed method can consistently outperform the state of the arts.
Super-resolution de-fencing: Simultaneous fence removal and high-resolution image recovery using videos

In real-world scenarios, images or videos taken at public places using inexpensive low-resolution cameras, such as smartphones are also often degraded by the presence of occlusions such as fences/barricades. Finer details in images captured using such low-end equipment are lost due to blurring and under-sampling. Compounding this problem is missing data due to the presence of an intervening occlusion between the scene and the camera such as a fence. To recover a fence-free high-resolution image, we use videos of the scene captured by panning a hand-held camera and model the effects of various degradations. Initially, we obtain the spatial locations of the fence/occlusions and the global shifts of the degraded background image. The underlying high-resolution fence-free image is modeled as a discontinuity-adaptive Markov random field and its maximum a-posteriori estimate is obtained using an optimization approach. The advantage of using this prior is that high-frequency information is preserved during the reconstruction of the super-resolved image. Specifically, we use the fast graduated non-convexity algorithm to minimize a non-convex energy function. Experiments with both synthetic and real-world data demonstrate the efficacy of the proposed algorithm.
Experiential Sampling on Multiple Data Streams

Multimedia systems must deal with multiple data streams. Each data stream usually contains significant volume of redundant noisy data. In many real-time applications, it is essential to focus the computing resources on a relevant subset of data streams at any given time instant and use it to build the model of the environment. We formulate this problem as an experiential sampling problem and propose an approach to utilize computing resources efficiently on the most informative subset of data streams. In this paper, we generalize our experiential sampling framework to multiple data streams and provide an evaluation measure for this technique. We have successfully applied this framework to the problems of traffic monitoring, face detection and monologue detection
A Collection-Oriented Metadata Framework for Digital Images

A digital photo can "tell a thousand words" through the use of its metadata and as it is usually part of a collection, metadata management, reuse, propagation&inference could be achieved via its association with a collection. However, there is not much work on metadata management, reuse, propagation&inference, particularly on a group basis. In this paper, we proposed a collection-oriented metadata framework which provides a basis for metadata management, reuse, propagation&inference and demonstrated the utility of such a framework
Goal based optimal selection of media streams

A multimedia system utilizes a set of correlated media streams each of which partially help in achieving the system goal. However, since not all of the streams always contribute towards the goal, there is a need for determining the most informative subset from the available set of media streams at any instant. For example, a subset of two video cameras and two microphones could be better than any other subset of multimedia sensors at some time instance. This paper presents a novel framework to find the optimal subset of media streams that achieves the system goal under specified constraints. The proposed framework uses a dynamic programming approach to find the optimal subset of media streams based on two criteria; first, by maximizing the probability of achieving the goal under the specified maximum cost, and second by minimizing the cost of using the streams so that the goal is achieved with a specified minimum probability. To show the utility of our framework, we provide the simulation results for hypothesis testing.
Audio Matters in Visual Attention

There is a dearth of information on how perceived auditory information guides image-viewing behavior. To investigate auditory-driven visual attention, we first generated a human eye-fixation database from a pool of 200 static images and 400 image-audio pairs viewed by 48 subjects. The eye tracking data for the image-audio pairs were captured while participants viewed images, which took place immediately after exposure to coherent/incoherent audio samples. The database was analyzed in terms of time to first fixation, fixation durations on the target object, entropy, AUC, and saliency ratio. It was found that coherent audio information is an important cue for enhancing the feature-specific response to the target object. Conversely, incoherent audio information attenuates this response. Finally, a system predicting the image-viewing with the influence of different audio sources was developed. The detailedly discussed top-down module in the system is composed of auditory estimation based on Gaussian mixture model-maximum a posteriori algorithm-universal background model structure, as well as visual estimation based on the conditional random field model and sparse latent variables. The evaluation experiments show that the proposed models in the system exhibit strong consistency with eye fixations.
Joint Watermarking Scheme for Multiparty Multilevel DRM Architecture

Multiparty multilevel digital rights management (DRM) architecture involving several levels of distributors in between an owner and a consumer has been suggested as an alternative business model to the traditional two-party (buyer-seller) DRM architecture for digital content delivery. In the two-party DRM architecture, cryptographic techniques are used for secure delivery of the content, and watermarking techniques are used for protecting the rights of the seller and the buyer. The cryptographic protocols used in the two-party case for secure content delivery can be directly applied to the multiparty multilevel case. However, the watermarking protocols used in the two-party case may not directly carry over to the multiparty multilevel case, as it needs to address the simultaneous security concerns of multiple parties such as the owner, multiple levels of distributors, and consumers. Towards this, in this paper, we propose a joint digital watermarking scheme using Chinese remainder theorem for the multiparty multilevel DRM architecture. In the proposed scheme, watermark information is jointly created by all the parties involved; then a watermark signal is generated out of it and embedded into the content. This scheme takes care of the security concerns of all parties involved. Further, in the event of finding an illegal copy of the content, the violator(s) can be traced back.
Progressive Audio Scrambling in Compressed Domain

Audio scrambling can be employed to ensure confidentiality in audio distribution. We first describe scrambling for raw audio using the discrete wavelet transform (DWT) first and then focus on MP3 audio scrambling. We perform scrambling based on a set of keys which allows for a set of audio outputs having different qualities. During descrambling, the number of keys provided and the number of rounds of descrambling performed will decide the audio output quality. We also perform scrambling by using multiple keys on the MP3 audio format. With a subset of keys, we can descramble to obtain a low quality audio. However, we can obtain the original quality audio by using all of the keys. Our experiments show that the proposed algorithms are effective, fast, simple to implement while providing flexible control over the progressive quality of the audio output. The security level provided by the scheme is sufficient for protecting MP3 music content.
Image Re-Attentionizing

In this paper, we propose a computational framework, called Image Re-Attentionizing, to endow the target region in an image with the ability of attracting human visual attention. In particular, the objective is to recolor the target patches by color transfer with naturalness and smoothness preserved yet visual attention augmented. We propose to approach this objective within the Markov Random Field (MRF) framework and an extended graph cuts method is developed to pursue the solution. The input image is first over-segmented into patches, and the patches within the target region as well as their neighbors are used to construct the consistency graphs. Within the MRF framework, the unitary potentials are defined to encourage each target patch to match the patches with similar shapes and textures from a large salient patch database, each of which corresponds to a high-saliency region in one image, while the spatial and color coherence is reinforced as pairwise potentials. We evaluate the proposed method on the direct human fixation data. The results demonstrate that the target region(s) successfully attract human attention and in the meantime both spatial and color coherence is well preserved.
Fractional scaling of image and video in DCT domain

An algorithm for scaling image and video with fractional factors of 1.25 and 1.50 directly in compressed (DCT) domain without explicit decompression and recompression is presented. It differs from those compressed-domain sampling methods that work only with integer factors, and provides more flexibility in changing image sizes. We employ a simple, consistent and extensible way to implement the algorithm. The resulting scheme ensures that the compressed domain algorithms always use fewer arithmetic operations than their conventional spatial domain counterparts. Experimental results in terms of visual quality and objective evaluation are provided.
Hazy image enhancement based on the full-saturation assumption

In this paper, we present an algorithm that can successfully improve the quality of hazy images and offer visually-pleasant haze-free results with vivid colors. We propose the full-saturation assumption (FSA) based on the aesthetic photographic effect: photos of vivid colors are visually pleasant, and first recover the degraded saturation layer. The depth image is also obtained as a by-product. We then apply an example-based approach to avoid over-saturation. The proposed assumption can also serve as a criterion to evaluate the degree of haze removal. Results are shown on various images to demonstrate its competence.
A DCT domain visible watermarking technique for images

The growth of computer networks has boosted the growth of the information technology sector to a greater extent. There is a trend to move from conventional libraries to digital libraries. In digital libraries images and text are made available through the Internet for scholarly research. At the same time care is taken to prevent the unauthorized use of the images commercially. In some cases the observer is encouraged to patronize the institution that owns the material. To satisfy both these needs simultaneously the owner needs to use visible watermarking. Visible watermarking is a type of digital watermarking used for protection of publicly available images. We describe a visible watermarking scheme that is applied into the host image in the DCT domain. A mathematical model has been developed for this purpose. We also propose a modification of the algorithm to make the watermark more robust.
Compressed-domain scrambler/descrambler for digital video

Multimedia data security is very important for multimedia commerce on the Internet and real-time video multicast. However, traditional encryption algorithms for data secrecy, such as DES, may not be suitable for multimedia applications because they are unable to meet the real-time constraints required by the multimedia applications. For multimedia applications, lightweight encryption algorithms are attractive. This paper examines the joint encryption and compression framework in which video data are scrambled in the frequency domain by employing selective bit scrambling, block shuffling and block rotation of the transformed coefficients and motion vectors. In addition a lightweight MPEG video encryption algorithm is proposed whose primary motivation is to save the encryption computation by taking the advantage of combining MPEG compression and data encryption, and at the same time avoids adverse effects on the video compression rate.
Robust invisible watermarking of volume data using the 3D DCT

Proposes a novel watermarking algorithm for 3D volume data based on the spread-spectrum communication technique, which is invisible and robust. "Invisible" means that the 2D rendered image of this watermarked volume is perceptually indistinguishable from that of the original volume. "Robust" watermarking implies that the watermark is resistant to most intentional or unintentional attacks. We have implemented the algorithm in a software package and have conducted experiments showing that the watermark is invisible in the volume-rendered 2D images. This is further confirmed by computing the signal-to-noise ratio (SNR) and the peak signal-to-noise ratio (PSNR) of the 3D watermarked volume data. We addressed different attacks, putting more emphasis on those attacks that are most commonly performed by attackers. The experiments show that the watermarking scheme is robust.
A hierarchical signature scheme for robust video authentication using secret sharing

Ensuring the integrity of a digital video is an important and challenging research problem arising out of many video applications. In this paper, we present a hierarchical framework for video authentication based on cryptographic secret sharing that protects a video from spatial cropping and temporal jittering, yet is robust against frame dropping in the streaming video scenario. Our algorithm provides a tradeoff between security and robustness by having configurable inputs. The authentication signature is compact and very sensitive against spatial attacks such as region tampering, and interframe attacks like frame replacement, major frame dropping, and frame reordering. Given a video, we identify the key frames based on different energy between the frames. Considering video frames as shares, we compute the secret at three hierarchical levels. The master secret is used as digital signature to authenticate the video. We present extensive experimental results which show the utility of our technique.
Robust Watermarking of Compressed and Encrypted JPEG2000 Images

Digital asset management systems (DAMS) generally handle media data in a compressed and encrypted form. It is sometimes necessary to watermark these compressed encrypted media items in the compressed-encrypted domain itself for tamper detection or ownership declaration or copyright management purposes. It is a challenge to watermark these compressed encrypted streams as the compression process would have packed the information of raw media into a low number of bits and encryption would have randomized the compressed bit stream. Attempting to watermark such a randomized bit stream can cause a dramatic degradation of the media quality. Thus it is necessary to choose an encryption scheme that is both secure and will allow watermarking in a predictable manner in the compressed encrypted domain. In this paper, we propose a robust watermarking algorithm to watermark JPEG2000 compressed and encrypted images. The encryption algorithm we propose to use is a stream cipher. While the proposed technique embeds watermark in the compressed-encrypted domain, the extraction of watermark can be done in the decrypted domain. We investigate in detail the embedding capacity, robustness, perceptual quality and security of the proposed algorithm, using these watermarking schemes: Spread Spectrum (SS), Scalar Costa Scheme Quantization Index Modulation (SCS-QIM), and Rational Dither Modulation (RDM).
Goal detection in soccer video using audio/visual keywords

In this paper, we propose a two-level framework to detect interesting events automatically based on audio and visual keywords (AVKs). The first level extracts low-level features such as motion, color, texture, pitch etc to detect video segments boundaries and label segments as audio and visual keywords. Next, we extract the exciting break portions from the AVK sequence. Then, we use two hidden Markov models (HMM) to model the exciting break portions with and without goal event respectively. We have applied the proposed approach to the detection of goal event in six half matches of soccer videos (270 minutes, 14 goals) from FlFA 2002 and UEFA 2002 and achieve 90% precision and 100% recall respectively.
Automatically generating summaries for musical video

In this paper, we propose a novel approach to automatically summarize musical videos. The proposed summarization scheme is different from the current methods used for video summarization. The musical video is separated into the musical and visual tracks. A music summary is created by analyzing the music content based on music features, adaptive clustering algorithm and musical domain knowledge. Then, shots are detected and clustered in the visual track. Finally, the music video summary is created by aligning the music summary and clustered video shots. Subjective studies by experienced users have been conducted to evaluate the quality of summarization. The experiments on different genres of musical video and comparisons with the summaries only based on music track and video track indicate that the results of summarization using proposed method are significant and effective to help realize user's expectation.