nils-herrmann/clean_scirepeval huggingface dataset:
Multimedia signatures for vehicle forensics

The use of dashboard-mounted video cameras is rapidly spreading in many countries around the world. Widespread usage of dash-cams brings new problems, for example, dash-cam videos are uploaded on public websites which contain footage of other cars with the number-plates visible. This can potentially compromise privacy. Further, dash-cam videos can be used as evidence in case of accidents. There have been even cases of usage of dash-cam videos for insurance claims. Not only genuine claims can be made but fraudulent claims using some other cars footage can be used. The use as well as misuse of dash-cam videos is going to be widely prevalent in the near future. In this paper, we present a solution to problem of identifying the vehicle in which the dashboard camera is mounted. Our technique can be used by insurance companies for authenticating the source of origin (the vehicle on which the camera is mounted) of video before processing the insurance claim. We make use of features extracted from motion blur as a feature which is generated due to the unique motion of a vehicle. We have found that the subtle motion pattern of every vehicle acts as unique signature. To the best of our knowledge, ours is the first work that describes this new area of research.
ClickSmart: A Context-Aware Viewpoint Recommendation System for Mobile Photography

In this paper, we propose ClickSmart, a viewpoint recommendation system that can assist a user in capturing high-quality photographs at well-known tourist locations. ClickSmart can provide real-time viewpoint recommendation based on the preview on the user's camera, current time, and user's geolocation. It makes use of publicly available geotagged images along with the associated metadata for learning a recommendation model. We define view-cells, macroblocks in geospace, and propose the concepts of popularity, quality, and uniqueness of view-cells from the viewpoint perspective. Viewpoint recommendation is generated at the granularity of a view-cell and is based on its popularity, quality, and uniqueness, which are estimated using social media cues associated with images. We further observe that contextual information such as time and weather conditions play an important role in photography, and therefore augment the recommendation system with the associated context. ClickSmart also takes into account the presence of people in the view for making the recommendation. It can provide two kinds of recommendations, quality based and uniqueness based. Although both were found effective in the experimental evaluation, our user study showed that uniqueness-based recommendation was preferred more by skilled photographers compared with amateurs.
Anonymous secure routing in mobile ad-hoc networks

Although there are a large number of papers on secure routing in mobile ad-hoc networks, only a few consider the anonymity issue. We define more strict requirements on the anonymity and security properties of the routing protocol, and notice that previous research works only provide weak location privacy and route anonymity, and are vulnerable to specific attacks. Therefore, we propose the anonymous secure routing (ASR) protocol that can provide additional properties on anonymity, i.e. identity anonymity and strong location privacy, and at the same time ensure the security of discovered routes against various passive and active attacks. Detailed analysis shows that ASR can achieve both anonymity and security properties, as defined in the requirements, of the routing protocol in mobile ad-hoc networks.
Colorizing infrared home videos

A color video always conveys more vivid sentiments than a grayscale one. Obtaining a grayscale video from a color video is almost trivial but the converse is known to be hard. Nowadays, digital camcorders come equipped with an infrared device for night shot that enables one to shoot home videos in the dark. Unfortunately, the infrared lighting device used generates a "green-scale" video which is akin to a grayscale video albeit possessing all tints of green. In this paper, we present a novel technique for colorizing infrared home videos. We first convert the green scale video into grayscale, afterwards our technique involves generating key-frames for every shot and then building up a one to one correspondence map between the key frames and the designated color images. These pairs are used to generate the color palette table for the video segment, which is then utilized to colorize that segment of the home video. Our novel technique could also be applied for colorizing X-ray videos generated by diagnostic imaging devices as well as surveillance videos generated by baggage scanners at airports.
A Flexible Surveillance System Architecture

Traditional multimedia surveillance systems are task specific and tightly coupled to the environment. Moreover, system designs generally start with the assumption that the environment, context, and sensors always remain static. With such a tight coupling, it becomes very difficult to port the system to new environments. Furthermore, for most of the systems, there is no straightforward way to upgrade the existing system to incorporate technological advancements such as new sensors or novel feature extraction techniques. We propose a flexible surveillance system architecture which can be easily ported in different environments, is dynamic without any significant compromise in system performance, and can be extended to integrate newer technological developments. We also introduce the notion of Environment Model (EM), which completely defines the coupling between system and the physical environment. The isolation of environment specific variables in EM makes the system easily portable in different environments. We present results of a prototype implementation of the system that highlights our design goals.
Mosaic based view enlargement for moving objects in moving pictures

Conventional mosaicing techniques convert a video from frame-based representation to scene-based representation, but they usually lack dynamic information so that their mosaic is not complete. In this paper, we present a novel method to detect moving objects in the video sequences, then add them into the static background mosaic to represent the scene completely. This novel algorithm separates static and dynamic information in a video sequence, builds the background mosaic from static part and reconstructs moving objects on the static mosaic. We have implemented our techniques and the experimental results demonstrate the effectiveness of our approach
Seeing through the fence: Image de-fencing using a video sequence

Tourists and amateur photographers are often hindered in capturing their cherished images/videos by a fence/occlusion that limits accessibility to the scene of interest. The situation has been exacerbated by growing concerns of security at public places and a need exists to provide a tool that can be used for post-processing such “fenced videos” to produce a “de-fenced” image. There are several challenges in this problem and in this work, we identify them as 1. Robust detection of the fence/occlusions. 2. Estimating pixel motion of background scene. 3. Filling in the fence/occlusions by utilizing information in multiple frames of the input video. We use a video captured by a camera panning the scene containing a fence and obtain a “de-fenced” image. Our method can effectively remove fences from images as demonstrated for several synthetic and real-world cases.
Context-Based Multimedia Sensor Selection Method

Modern multimedia systems have large number of sensors spread across a wide area. In a time-shared multimedia system, many people will be making queries to the system simultaneously which requires sharing of computing resources. In such scenarios, processing information from all the sensors for each query will make the system inefficient. Considering the fact that only few sensors provide information relevant to the query, we can reduce the cost incurred in query evaluation by efficiently selecting a subset of sensors to be processed without compromising the system performance. This paper demonstrates a two-stage sensor selection method which uses contextual information and confidence in individual sensors to select sensors which provide more reliable answers to the queries.
ICSC 2015 message from the program chairs

Welcome to the Proceedings of the Ninth IEEE International Conference on Semantic Computing (ICSC 2015) in Anaheim, California, USA. ICSC 2015 continues to foster the growth of a new research community. This edition of the conference builds upon the tradition and success of the past ICSC series as an international forum for researchers as well as practitioners in academia and industry to present research that not only advances the state of the art and practice of Semantic Computing but also identifies the emerging research topics, and defines the future of the field.
Analyzing Abnormal Events from Spatio-temporal Trajectories

Advances in RFID based sensor technologies has been used in applications which requires the tracking of assets, products and individuals. The recording of such movements is captured in a trajectory database and can be analyzed for the monitoring of abnormal events. In this paper, we describe a system called InViTA for analyzing abnormal events from spatio-temporal trajectories captured during an office evacuation after an explosion. InViTA utilizes a trajectory representation scheme and extract the features to derive a set of rules that label each person's trajectory as belonging to a suspect, witness, or victim, etc. We run the system on the office evacuation data provided in VAST 2008 challenge and obtain comparable results with that obtained from visualization and human analysis. The system includes a user-friendly graphical interface for parameter tuning and intuitive result analysis.
Integrated Detect-Track Framework for Multi-view Face Detection in Video

An Experiential sampling and Meanshift tracker based Multi-view face detection in video is proposed in this paper. In this framework, instead of performing face detection at every position in a frame, we determine certain key positions to run the multi-view face detectors. These key positions are statistical samples drawn from a density function that is estimated based on color cues, past detection results, Meanshift tracker results and a temporal continuity model. These samples are then propogated using a Particle filter framework. We use a Meanshift tracker to track faces that are missed by the multiview face detectors. Our framework results in a significant reduction in computation time and accounts for the detection of complete 180 degree pose of the face. We also come up with a novel likelihood measure for track termination, which becomes important when used for detection purposes.
Adaptive visible watermarking of images

Digital watermarks are emerging as a tool to provide copyright protection for high quality images and video. Though a lot of work has been done in the area of invisible watermarks, relatively less earlier work exists for visible watermarks. As a consensus on the various issues for invisible watermarks to be legally admissible eludes us, a visible watermark could serve as a deterrence to theft and also to provide instantaneous recognition of the owner or creator of an image. We propose a technique in which the location and strength of the watermark image to be embedded is varied in accordance with the underlying content of the image to be watermarked. We propose a new algorithm that classifies each block of 8/spl times/8 of pixels into one of 8 classes depending on the sensitivity of the block to distortion. We analyze the texture, edge and luminance information in the block for this purpose. The embedding process is automated and the bits are embedded in the DCT transform domain. Since the strength of the watermark in a block depends on the class to which the block belongs, the result is a pleasant and unobtrusively watermarked image irrespective of the type of image.
Event detection in basketball video using multiple modalities

Semantic sports video analysis has attracted more and more attention recently. In this paper, we present a basketball event detection method by using multiple modalities. Instead of using low-level features, the proposed method is built upon visual and auditory mid-level features i.e. semantic shot classes and audio keywords. Promising event detection results have been achieved. By heuristically mapping semantic shot classes and by aligning audio keywords with semantic shot classes, we have been able to detect nine basketball events. Experimental results have shown our proposed method is effective for basketball event detection.
Progressive scrambling for MP3 audio

Audio scrambling can be employed in audio distribution for the purpose of guaranteeing the confidentiality. Electronic commerce in audio products would be facilitated by the development of solutions which can ensure security/privacy, have efficiency in the compressed domain to reduce the bandwidth requirements, flexibility of implementing progressive audio quality control and be computationally inexpensive for use in real-time systems. This paper proposes an effective progressive scrambling algorithm that works with MP3 audio. We perform multiple-level scrambling based on keys to produce a set of audio outputs of differing qualities. During decoding, the keys provided, and rounds of descrambling performed, decide the quality of the audio output. Our experiments show that the proposed algorithm is effective, fast, simple to be implemented, and at the same time it does provide flexibility to control the quality of audio output progressively. We also present some experimental results to show its utility in real-time systems.
Dynamic workload assignment in video surveillance systems

Current surveillance systems consist of large numbers of cameras. The video feeds from cameras are automatically processed for threat detection, which is a computationally intensive task. In order to meet the real-time requirements of surveillance, we need to distribute the video processing over multiple computers. Generally the cameras are statically assigned to the processors; we show that this is not a desirable solution as the workload for a particular camera may vary over time depending on the number of the targets in its view. In future, this uneven distribution of workload will become more critical as the sensing infrastructures are being deployed on the cloud. In this work, we model the camera workload as a function of the number of targets, and use that to dynamically assign video feeds to the processors. Experimental results show that the proposed model successfully captures the variability of the workload, and that dynamic workload assignment provides better results than a static assignment.
Affective Video Summarization and Story Board Generation Using Pupillary Dilation and Eye Gaze

We propose a semi-automated, eye-gaze based method for affective analysis of videos. Pupillary Dilation (PD) is introduced as a valuable behavioural signal for assessment of subject arousal and engagement. We use PD information for computationally inexpensive, arousal based composition of video summaries and descriptive story-boards. Video summarization and story-board generation is done offline, subsequent to a subject viewing the video. The method also includes novel eye-gaze analysis and fusion with content based features to discover affective segments of videos and Regions of interest (ROIs) contained therein. Effectiveness of the framework is evaluated using experiments over a diverse set of clips, significant pool of subjects and comparison with a fully automated state-of-art affective video summarization algorithm. Acquisition and analysis of PD information is demonstrated and used as a proxy for human visual attention and arousal based video summarization and story-board generation. An important contribution is to demonstrate usefulness of PD information in identifying affective video segments with abstract semantics or affective elements of discourse and story-telling, that are likely to be missed by automated methods. Another contribution is the use of eye-fixations in the close temporal proximity of PD based events for key frame extraction and subsequent story board generation. We also show how PD based video summarization can to generate either a personalized video summary or to represent a consensus over affective preferences of a larger group or community.
SalAd: A Multimodal Approach for Contextual Video Advertising

The explosive growth of multimedia data on Internet has created huge opportunities for online video advertising. In this paper, we propose a novel advertising technique called SalAd, which utilizes textual information, visual content and the webpage saliency, to automatically associate the most suitable companion ads with online videos. Unlike most existing approaches that only focus on selecting the most relevant ads, SalAd further considers the saliency of selected ads to reduce intentional ignorance. SalAd consists of three basic steps. Given an online video and a set of advertisements, we first roughly identify a set of relevant ads based on the textual information matching. We then carefully select a sub-set of candidates based on visual content matching. In this regard, our selected ads are contextually relevant to online video content in terms of both textual information and visual content. We finally select the most salient ad among the relevant ads as the most appropriate one. To demonstrate the effectiveness of our method, we have conducted a rigorous eye-tracking experiment on two ad-datasets. The experimental results show that our method enhances the user engagement with the ad content while maintaining users' quality of video viewing experience.
Video content representation on tiny devices

The perceptual satisfaction of a user watching video on a tiny mobile device is constrained by the display capability and network bandwidth. To maximize the user's perceptual satisfaction in this constrained environment, we propose a new method to represent the video content adaptively in real-time on tiny devices according to the user's attention. First, a sampling based dynamic attention model is proposed to obtain and maintain the user's attention in the video streams. Second, based on the most attended regions and sequences extracted, the attention based representation is introduced to achieve a higher perceptual satisfaction on a small display. Experiments with users show the effectiveness of our proposed method in a video surveillance application.
View-invariant feature discovering for multi-camera human action recognition

Intelligent video surveillance system is built to automatically detect events of interest, especially on object tracking and behavior understanding. In this paper, we focus on the task of human action recognition under surveillance environment, specifically in a multi-camera monitoring scene. Despite many approaches have achieved success in recognizing human action from video sequences, they are designed for single view and generally not robust against viewpoint invariant. Human action recognition across different views remains challenging due to the large variations from one view to another. We present a framework to solve the problem of transferring action models learned in one view (source view) to another view (target view). First, local space-time interest point feature and global shape-flow feature are extracted as low-level feature, followed by building the hybrid Bag-of-Words model for each action sequence. The data distribution of relevant actions from source view and target view are linked via a cross-view discriminative dictionary learning method. Through the view-adaptive dictionary pair learned by the method, the data from source and target view can be respectively mapped into a common space which is view-invariant. Furthermore, We extend our framework to transfer action models from multiple views to one view when there are multiple source views available. Experiments on the IXMAS human action dataset, which contains videos captured with five viewpoints, show the efficacy of our framework.
EMD and psychoacoustic model based watermarking for audio

The audio watermarking method proposed in this paper offers the copyright protection to an audio without the use of the original signal for watermark detection. The analysis filterbank decomposition, the psychoacoustic model and the empirical mode decomposition (EMD) are the three key techniques used in the novel audio watermarking method. Unlike the traditional audio watermarking algorithms where the watermark bits are embedded directly in the signal either by time domain or transform domain processing, the novel blind audio watermarking algorithm proposed in this paper embeds the watermark bits in the final residue of the subbands in the transform domain. Four watermark messages are embedded into the proposed audio watermarking system. The inaudibility, capacity and robustness of the audio watermarking system are evaluated, in order to optimize the system performance. The experimental results show that the proposed blind watermarking scheme is robust against MP3 compression and adding Gaussian noise attacks.
Dual-Glance Model for Deciphering Social Relationships

Since the beginning of early civilizations, social relationships derived from each individual fundamentally form the basis of social structure in our daily life. In the computer vision literature, much progress has been made in scene understanding, such as object detection and scene parsing. Recent research focuses on the relationship between objects based on its functionality and geometrical relations. In this work, we aim to study the problem of social relationship recognition, in still images. We have proposed a dualglance model for social relationship recognition, where the first glance fixates at the individual pair of interest and the second glance deploys attention mechanism to explore contextual cues. We have also collected a new large scale People in Social Context (PISC) dataset, which comprises of 22,670 images and 76,568 annotated samples from 9 types of social relationship. We provide benchmark results on the PISC dataset, and qualitatively demonstrate the efficacy of the proposed model.
Semantic labeling of soccer video

As traditional shot segmentation may not produce video segments that possess one-to-one correspondence to semantic views, we present an integrated segmentation and classification approach to label soccer video into semantic units in this paper. In our system, each P frame is divided to a 6 /spl times/ 4 blocks with color and motion features extracted on both block and frame levels. First, a threshold is used to divide the video stream into relatively static parts and active parts. Then every active part is segmented into sub-parts according to 4 view types and the motion features are used to classify segments with support vector machines. Finally, static parts are merged with classified active sub-parts to form labeled segments. Four 10-minute test clips from the World Cup 2002 are used to evaluate our system resulting in a promising classification rate of 79.8%.
A CRT based watermark for multiparty multilevel DRM architecture

In this paper, we propose a joint digital watermarking protocol for the multiparty multilevel DRM architecture using Garner's algorithm for the Chinese remainder theorem (CRT). Our protocol exploits the incremental nature of the computation of CRT by the Garner's algorithm. The proposed joint watermarking protocol embeds a single watermark signal into the content while taking care of the various security concerns such as proof of involvement in the distribution chain, nonrepudiation of the involvement and protection against false framing of the different parties involved. Further, in the event of finding an illegal copy of the content, the identities of all the parties involved in that content distribution chain can be traced back by extracting the watermark information.