nils-herrmann/clean_scirepeval huggingface dataset:
Music Key Detection for Musical Audio

The key or the scale information of a piece of music provides important clues on its high level musical content, like harmonic and melodic context, which can be useful for music classification, retrieval or further content analysis. Researchers have previously addressed the issue of finding the key for symbolically encoded music (MIDI); however, very little work has been done on key detection for acoustic music. In this paper, we present a method for estimating the root of diatonic scale and the key directly from acoustic signals (waveform) of popular and classical music. We propose a method to extract pitch profile features from the audio signal, which characterizes the tone distribution in the music. The diatonic scale root and key are estimated based on the extracted pitch profile by using a tone clustering algorithm and utilizing the tone structure of keys. Experiments on 72 music pieces have been conducted to evaluate the proposed techniques. The success rate of scale root detection for pop music pieces is above 90%.
Functionality Delegation in Distributed Surveillance Systems

The utilization of multimedia devices is growing rapidlyin surveillance and monitoring applications. These multimediasurveillance systems need to process large amountsof multimodal sensor data in order to detect events and objects.While processing this large amount of data, the systemfaces many processing and network bottlenecks. Thedesign of efficient multimedia surveillance system requiresintelligent architectural decisions and performance evaluationto cope with these resource demands. One critical issueamong all these architectures is task assignment amongprocessing units. To study the effect of this task assignmenton system performance with quantifiable performancemeasures is very useful and challenging. We define a FunctionalityDelegation Coefficient which abstracts the delegationof functionality among processing units of a distributedsurveillance system and show its effect on event blockingprobability and response time. Simulation and real implementationresults are provided to validate the model.
Concept Based Hybrid Fusion of Multimodal Event Signals

Recent years have seen a significant increase in the number of sensors and resulting event related sensor data, allowing for a better monitoring and understanding of real-world events and situations. Event-related data come from not only physical sensors (e.g., CCTV cameras, webcams) but also from social or microblogging platforms (e.g., Twitter). Given the wide-spread availability of sensors, we observe that sensors of different modalities often independently observe the same events. We argue that fusing multimodal data about an event can be helpful for more accurate detection, localization and detailed description of events of interest. However, multimodal data often include noisy observations, varying information densities and heterogeneous representations, which makes the fusion a challenging task. In this paper, we propose a hybrid fusion approach that takes the spatial and semantic characteristics of sensor signals about events into account. For this, we first adopt the concept of an image-based representation that expresses the situation of particular visual concepts (e.g. "crowdedness", "people marching") called Cmage for both physical and social sensor data. Based on this Cmage representation, we model sparse sensor information using a Gaussian process, fuse multimodal event signals with a Bayesian approach, and incorporate spatial relations between the sensor and social observations. We demonstrate the effectiveness of our approach as a proof-of-concept over real-world data. Our early results show that the proposed approach can reliably reduce the sensor-related noise, locate the event place, improve event detection reliability, and add semantic context so that the fused data provides a better picture of the observed events.
Visual keywords labeling in soccer video

In this paper, we propose a framework to label soccer video shots with pre-defined visual keywords to bridge the semantic gap between low-level features and semantic understanding. Based on these visual keywords, further structure analysis and event detection are facilitated. In our system, a MPEG-1 soccer video stream is partitioned into shots, from which each P-Frame is converted into color-based and edge-based binary maps. Then, we detect the playing field and segment the regions of interest (ROIs) inside the playing field. Finally two support vector machine classifiers and some decision rules are applied to the properties of the ROIs such as size, position, texture ratio, etc., and the position of the playing field to label the video shot with visual keyword. We have applied the proposed method to 3495 soccer video shots and achieved 94.3% and 97.2% average precision and recall, respectively.
Multi-Camera Action Dataset for Cross-Camera Action Recognition Benchmarking

Action recognition has received increasing attention from the computer vision and machine learning communities in the last decade. To enable the study of this problem, there exist a vast number of action datasets, which are recorded under controlled laboratory settings, real-world surveillance environments, or crawled from the Internet. Apart from the "in-the-wild" datasets, the training and test split of conventional datasets often possess similar environments conditions, which leads to close to perfect performance on constrained datasets. In this paper, we introduce a new dataset, namely Multi-Camera Action Dataset (MCAD), which is designed to evaluate the open view classification problem under the surveillance environment. In total, MCAD contains 14,298 action samples from 18 action categories, which are performed by 20 subjects and independently recorded with 5 cameras. Inspired by the well received evaluation approach on the LFW dataset, we designed a standard evaluation protocol and benchmarked MCAD under several scenarios. The benchmark shows that while an average of 85% accuracy is achieved under the closed-view scenario, the performance suffers from a significant drop under the cross-view scenario. In the worst case scenario, the performance of 10-fold cross validation drops from 87.0% to 47.4%.
Relevance feedback techniques for image retrieval using multiple attributes

The paper proposes a relevance feedback (RF) approach to content based image retrieval using multiple attributes. The proposed approach has been applied to images' text and color attributes. In order to ensure that meaningful features are extracted, a pseudo object model based on color coherence vector has been adopted to model color content. The RF approach employs techniques developed in the fields of information retrieval and machine learning to extract pertinent features from each of the attributes. It then uses the user's relevance judgments to estimate the importance of different attributes in an integrated content based image retrieval. The system developed has been tested on a large image collection containing over 12000 images. The results demonstrate that the proposed RF approaches and pseudo object based color model are effective.
Copyright protection for MPEG-2 compressed broadcast video

We describe a novel compressed domain (MPEG-2) method to achieve video security and copyright protection in a broadcasting or a multicasting environment. The proposed method is an efficient solution for resolving the conflicting requirements of single copy transmission for broadcasting and multiple unique copies for copyright protection by the broadcaster. The new method supports dynamic join and leave and it relies upon robust invisible watermarking, masking and conditional access techniques to achieve the desired result.
An Anonymous Routing Protocol with The Local-repair Mechanism for Mobile Ad Hoc Networks

In this paper, we first define the requirements on anonymity and security properties of the routing protocol in mobile ad hoc networks, and then propose a new anonymous routing protocol with the local-repair mechanism. Detailed analysis shows that our protocol achieves both anonymity and security properties defined. A major challenge in designing anonymous routing protocols is to reduce computation and communication costs. To overcome this challenge, our protocol is design to require neither asymmetric nor symmetric encryption/decryption while updating the flooding route requests; more importantly, once a route is broken, instead of re-launching a new costly flooding route discovery process like previous work, our protocol provides a local-repair mechanism to fix broken parts of a route without compromising anonymity
Action and Interaction Recognition in First-Person Videos

In this work, we evaluate the performance of the popular dense trajectories approach on first-person action recognition datasets. A person moving around with a wearable camera will actively interact with humans and objects and also passively observe others interacting. Hence, in order to represent real-world scenarios, the dataset must contain actions from first-person perspective as well as third-person perspective. For this purpose, we introduce a new dataset which contains actions from both the perspectives captured using a head-mounted camera. We employ a motion pyramidal structure for grouping the dense trajectory features. The relative strengths of motion along the trajectories are used to compute different bag-of-words descriptors and concatenated to form a single descriptor for the action. The motion pyramidal approach performs better than the baseline improved trajectory descriptors. The method achieves 96.7% on the JPL interaction dataset and 61.8% on our NUS interaction dataset. The same is used to detect actions in long video sequences and achieves average precision of 0.79 on JPL interaction dataset.
Mode of teaching based segmentation and annotation of video lectures

Online education is becoming more and more prevalent these days. Many universities provide pre-recorded classroom lectures for distance learning and remote users can access these lectures over Internet. With the available indexing techniques, users can search and retrieve videos related to their topic of interest in these stored databases. However, sometimes the `mode of teaching' impacts the viewer's perception for the retrieved video lecture or snippet. In this work we make use of visual concepts in the video lecture to identify the mode of teaching and generate annotations for the video. The developed approach uses low-level features like color and edges to classify video frames into high level semantic concepts. The system performs frame-by-frame classification and mode of teaching can be inferred for each segment as well as the complete video. Experimental results show high accuracy of proposed method and demonstrate its potential for relevant applications.
Eye-2-I: Eye-tracking for just-in-time implicit user profiling

For many applications, such as targeted advertising and content recommendation, knowing users' traits and interests is a prerequisite. User profiling is a helpful approach for this purpose. However, current methods, i.e. self-reporting, web-activity monitoring and social media mining are either intrusive or require data over long periods of time. Recently, there is growing evidence in cognitive science that a variety of users' profile is significantly correlated with eye-tracking data. A novel just-in-time implicit profiling method, Eye-2-I, which learns the user's demographic and personality traits from the eye-tracking data while the user is watching videos is proposed. Although seemingly conspicuous by closely monitoring the user's eye behaviors, the proposed method is unobtrusive and privacy-preserving owing to its unique combination of speed and implicitness. As a proof-of-concept, the proposed method is evaluated in a user study with 51 subjects.
Visual cryptography for print and scan applications

Visual cryptography is not much in use in spite of possessing several advantages. One of the reasons for this is the difficulty of use in practice. The shares of visual cryptography are printed on transparencies which needed to be superimposed. However, it is not very easy to do precise superposition due to the fine resolution as well as printing noise. Furthermore, many visual cryptography applications need to print shares on paper in which scanning of the share is necessary. The print and scan process can introduce noise as well which can make the alignment difficult. We consider the problem of precise alignment of printed and scanned visual cryptography shares. Due to vulnerabilities in the spatial domain, we have developed a frequency domain alignment scheme. We employ the Walsh transform to embed marks in both of the shares so as to find the alignment position of these shares. Our experimental results show that our technique can be useful in print and scan applications.
Automatic music summarization based on music structure analysis

In this paper, we present a novel approach for music summarization based on music structure analysis. From the audio signal, we first extract the note onset representing the time tempo of the song and the music structure analysis can be performed based on this tempo information. After music content has been structured into different semantic regions such as introduction (intro), verse, chorus, ending (outro), etc., the final music summary can be created with chorus and music phrases which are included anterior or posterior to the selected chorus to get the desired length of the final summary. In this way, we can guarantee that the summaries begin and end at meaningful music phrase boundaries, which is a difficult problem for existing music summarization methods. Experiments show our proposed method can capture the main theme of the music compared to the ideal summaries selected by music experts and user subjective evaluation indicates our proposed method has a good performance.
Anonymous surveillance

Video surveillance is a very effective tool of surveillance that enables a single security agent to monitor wide areas. However, it compromises the privacy of the individuals. There have been attempts to obfuscate face and silhouette regions of the images to hide the identity of individuals. We recognize that in traditional surveillance systems, the viewer generally has sufficient contextual knowledge about location of the camera, time, and activity patterns; which can lead to identity leakage even when the visual cues (face and appearance) are not present. In this way, the viewer can relate the identity of individuals to the sensitive information in the video causing privacy loss. In order to provide robust privacy preservation, the context knowledge needs to be decoupled from the video; however, human monitoring of the videos is also necessary for the assessment of the situation. In this paper we propose anonymous surveillance framework that decouples the contextual knowledge and video to the minimal extent required for situation assessment. The experimental results confirm that the proposed framework is very effective in protecting the privacy, yet does not affect much of the surveillance utility of the data.
Providing efficient certification services against active attacks in ad hoc networks

Most of previous research work in key management can only resist passive attacks, such as dropping the certificate request, and are vulnerable under active attacks, such as returning a fake reply to the node requesting the certification service. In this paper, we propose two algorithms to address both security and efficiency issues of certification services in ad hoc networks. Both of the algorithms can resist active attacks. In addition, simulation results show that, compared to the previous works, our second algorithm is not only much faster in a friendly environment, but it also works well in a hostile environment in which existing schemes work poorly. Furthermore, the process of generating partial certificates in our second algorithm is extremely fast. Such advantage is critical in ad hoc networks where by nature the less help a node requests from its neighbors, the higher is the chance of obtaining the help. Consequently, using our second algorithm, a node can easily find enough neighboring nodes which provide the certification service.
CAVVA: Computational Affective Video-in-Video Advertising

Advertising is ubiquitous in the online community and more so in the ever-growing and popular online video delivery websites (e.g., YouTube). Video advertising is becoming increasingly popular on these websites. In addition to the existing pre-roll/post-roll advertising and contextual advertising, this paper proposes an in-stream video advertising strategy-Computational Affective Video-in-Video Advertising (CAVVA). Humans being emotional creatures are driven by emotions as well as rational thought. We believe that emotions play a major role in influencing the buying behavior of users and hence propose a video advertising strategy which takes into account the emotional impact of the videos as well as advertisements. Given a video and a set of advertisements, we identify candidate advertisement insertion points (step 1) and also identify the suitable advertisements (step 2) according to theories from marketing and consumer psychology. We formulate this two part problem as a single optimization function in a non-linear 0-1 integer programming framework and provide a genetic algorithm based solution. We evaluate CAVVA using a subjective user-study and eye-tracking experiment. Through these experiments, we demonstrate that CAVVA achieves a good balance between the following seemingly conflicting goals of (a) minimizing the user disturbance because of advertisement insertion while (b) enhancing the user engagement with the advertising content. We compare our method with existing advertising strategies and show that CAVVA can enhance the user's experience and also help increase the monetization potential of the advertising content.
A Survey on Digital Camera Image Forensic Methods

There are two main interests in digital camera image forensics, namely source identification and forgery detection. In this paper, we first briefly provide an introduction to the major processing stages inside a digital camera and then review several methods for source digital camera identification and forgery detection. Existing methods for source identification explore the various processing stages inside a digital camera to derive the clues for distinguishing the source cameras while forgery detection checks for inconsistencies in image quality or for presence of certain characteristics as evidence of tampering.
Adaptive monitoring for video surveillance

Adaptability is one of the key issues in the important area of surveillance systems. Based on attention and sensor samples, the experiential sampling technique provides a general framework for analyzing video data. In this paper, we present a scheme for adaptive monitoring of surveillance objects by utilizing the feedback of the experiential sampling based video surveillance results to change the video camera parameters. Our framework first detects the moving objects in the surveillance video. We then analyze the output of this step to determine the state of the video camera settings. The relevant parameters of the video camera are continuously adjusted based on a proportional feedback control system. The fixed camera is thus adoptively tuned so as to obtain a good quality surveillance video output. We centrally frame the target object by doing panning and zooming operations. Moreover, we utilize the experiential sampling approach to capture the moving objects by utilizing the context which is a function of the current state of the environment as well as past experiences. This adds tremendous flexibility to the surveillance process which can be applied in a wide variety of monitoring situations.
Content-based representative frame extraction for digital video

We consider a content-based adaptive clustering algorithm (CBAC) to extract a fixed pre-determined number of representative frames to summarize a given digital video. In our algorithm, shot boundary detection is not needed. The inter-frame changes are compared globally for the extraction of representative frames. Small units of the video are dynamically clustered into two clusters based on the amount of change in content within the unit. One cluster, which has units of low content change, is designated for deletion and the other cluster, which has units of high content change, is for retention. The algorithm adaptively converges to the desired number of frames by deleting some redundant frames from each unit of the deletion cluster during every iteration.
Performance Modeling of Multimedia Surveillance Systems

Automated surveillance is critically important in the current scenario of heightened security concerns. Therefore, there has been a surge in the development of surveillance systems. Surveillance systems employ sensors to capture various environmental aspects in order to reason about the dynamically changing situation. Due to their cheap availability, the number of sensors used in modern systems is quite large. While processing the large amount of sensor data, the system faces many bottlenecks in terms of processor and memory requirements. Despite many efforts to propose efficient system architectures, they all ignore the study of the dynamic behavior of the system and the impact of various factors on system performance. In this work, we develop an analytical model to evaluate the performance of surveillance systems. Using the proposed model, we obtain closed form equations for event miss probability and response time as a function of system parameters. The results obtained from the model are validated with those obtained from the simulator. Finally we explore the different trade-offs among the system parameters and performance metrics.
A robust framework for aligning lecture slides with video

We propose a robust approach for aligning lecture slides with lecture videos using a combination of Hough transform, optical flow and Gabor analysis. A Markov Decision Process model is used to incorporate prior knowledge for enhanced recognition. We demonstrate synchronization of slides with videos containing de-focused slide content, speaker occlusion as well as camera pan, tilt and zoom sequences. Experimental results confirm the effectiveness of our approach for multimedia indexing applications.
Scrambling of engineering drawings

Engineering drawings are ubiquitously used for capturing, conveying and archiving innovative engineering designs. Many engineering companies' core intellectual property resides in their proprietary engineering drawings. Therefore, protection of such vital data is extremely important. This paper provides a swap-transformation matrix based approach to scramble engineering drawings in order to enable confidentiality. An engineering drawing involves the topological information and vertex information. The vertex information is more valuable than the topological information, since the vertices information primarily determines the content of engineering drawings. We argue that the vertex information is more valuable than the topological information, even if some topological information is lost, a drawing may be reconstructed from the vertex positions. We provide for three keys to ensure the security of the drawing. The technique can facilitate digital rights management of engineering drawings. The advantages of our technique are that scrambling is computationally less intensive than encryption and it allows for partial obfuscation.