nils-herrmann/clean_scirepeval huggingface dataset:
Fast kernel learning for spatial pyramid matching

Spatial pyramid matching (SPM) is a simple yet effective approach to compute similarity between images. Similarity kernels at different regions and scales are usually fused by some heuristic weights. In this paper, we develop a novel and fast approach to improve SPM by finding the optimal kernel fusing weights from multiple scales, locations, as well as codebooks. One unique contribution of our approach is the novel formulation of kernel matrix learning problem leading to an efficient quadratic programming solution, with much lower complexity than those associated with existing solutions (e.g., semidefinite programming). We demonstrate performance gains of the proposed methods by evaluations over well-known public data sets such as natural scenes and TRECVID 2007.
Quality-Optimized and Secure End-to-End Authentication for Media Delivery

The need for security services, such as confidentiality and authentication, has become one of the major concerns in multimedia communication applications, such as video on demand and peer-to-peer content delivery. Conventional data authentication cannot be directly applied for streaming media when an unreliable channel is used and packet loss may occur. This paper begins by reviewing existing end-to-end media authentication schemes, which can be classified into stream-based and content-based techniques. We then motivate and describe how to design authentication schemes for multimedia delivery that exploit the unequal importance of different packets. By applying conventional cryptographic hashes and digital signatures to the media packets, the system security is similar to that achievable in conventional data security. However, instead of optimizing packet verification probability, we optimize the quality of the authenticated media, which is determined by the packets that are received and able to be decoded and authenticated. The quality of the authenticated media is optimized by allocating the authentication resources unequally across streamed packets based on their relative importance, thereby providing unequal authenticity protection. The effectiveness of this approach is demonstrated through experimental results on different media types (image and video), different compression standards (JPEG, JPEG2000, and H.264), and different channels (wired with packet erasures and wireless with bit errors).
Hybrid block-based/segment-based video compression at very low bit rate

An algorithmic architecture of video compression at very low bit rate is presented. This architecture incorporates block-based as well as segment-based coding algorithms and manages them in a cooperative way in which different approaches are applied to different image frames iteratively: I and P frames are coded by block-based algorithm and O frames are coded by segment-based algorithm. An extra degree of freedom in video coding is introduced in order to achieve higher compression ratios while maintaining good visual quality. Every other frame in the image sequence is coded using block-based coding (ITU-T H.263) with motion (block-matching, 2D translations) and color (DCT) coefficients. These I and P frames are allocated with more bits, generating images with higher spatial accuracy, and providing the coder with generic coding capability. Between every two P frames, the O frame is coded using segment-based algorithm providing the motion (rigid 2D source models, 3D motions) and shape (polygon or spline approximation) information. Since only motion prediction is provided, pixel values are copied or interpolated from neighboring P frames. With good segmentation and affine transform, the prediction can preserve the natural shapes of the objects and give natural motion information which is sufficient for very low bitrate coding. These frames are allocated with less bits, generating images with less temporal accuracy, but providing the coder with higher compression efficiency.
Element Rearrangement for Tensor-Based Subspace Learning

The success of tensor-based subspace learning depends heavily on reducing correlations along the column vectors of the mode-k flattened matrix. In this work, we study the problem of rearranging elements within a tensor in order to maximize these correlations, so that information redundancy in tensor data can be more extensively removed by existing tensor-based dimensionality reduction algorithms. An efficient iterative algorithm is proposed to tackle this essentially integer optimization problem. In each step, the tensor structure is refined with a spatially-constrained Earth Mover's Distance procedure that incrementally rearranges tensors to become more similar to their low rank approximations, which have high correlation among features along certain tensor dimensions. Monotonic convergence of the algorithm is proven using an auxiliary function analogous to that used for proving convergence of the Expectation-Maximization algorithm. In addition, we present an extension of the algorithm for conducting supervised subspace learning with tensor data. Experiments in both unsupervised and supervised subspace learning demonstrate the effectiveness of our proposed algorithms in improving data compression performance and classification accuracy.
Content-based utility function prediction for real-time MPEG-4 video transcoding

Utility function based transcoding is an efficient systematic solution for choosing optimal media transcoding operation to meet dynamic resource constraints (such as bandwidth). However, to date the real-time generation of utility function is not feasible due to computational complexity. In this paper we present a content-based utility function prediction framework for real-time MPEG-4 video transcoding. We develop a statistical approach combining real-time compressed-domain feature extraction, content-based pattern classification and regression. Our extensive experiment results demonstrate that the proposed method achieves very promising prediction accuracy - up to 89% in choosing the optimal transcoding operation with the highest quality from multiple alternatives meeting the same target bitrate.
VideoQ: a fully automated video retrieval system using motion sketches

The rapidity with which digital information, particularly video, is being generated, has necessitated the development of tools for efficient search of these media. Content based visual queries have been primarily focused on still image retrieval. In this paper we propose a novel interactive system on the Web, based on the visual paradigm, with spatio-temporal attributes playing a key role in video retrieval. The resulting system VideoQ, is the first on-line video search engine supporting automatic object based indexing and spatio-temporal queries.
General and domain-specific techniques for detecting and recognizing superimposed text in video

We have developed generic and domain-specific video algorithms for caption text extraction and recognition in digital video. Our system includes several unique features: for caption box location, we combine the compressed-domain features derived from DCT coefficients and motion vectors. Long-term temporal consistency is employed to enhance localization performance. For character segmentation, we use a single-pass threshold free approach combining classification and projection to address noisy segmentation, text intensity variation, and algorithm complexity. In recognition, we use Zernike moments to achieve more accurate recognition performance. Finally, domain knowledge is explored and a statistical transition graph model is used to enhance recognition of domain-specific characters, such as ball counts and game score of baseball videos. The algorithms achieved real-time speed and significantly improved recognition accuracy. Furthermore, although the experiments were conducted in baseball videos only, these algorithms (except the transition model) are general and can be used in other applications, such as news and films.
Robust Object Co-detection

Object co-detection aims at simultaneous detection of objects of the same category from a pool of related images by exploiting consistent visual patterns present in candidate objects in the images. The related image set may contain a mixture of annotated objects and candidate objects generated by automatic detectors. Co-detection differs from the conventional object detection paradigm in which detection over each test image is determined one-by-one independently without taking advantage of common patterns in the data pool. In this paper, we propose a novel, robust approach to dramatically enhance co-detection by extracting a shared low-rank representation of the object instances in multiple feature spaces. The idea is analogous to that of the well-known Robust PCA~\cite{rpca}, but has not been explored in object co-detection so far. The representation is based on a linear reconstruction over the entire data set and the low-rank approach enables effective removal of noisy and outlier samples. The extracted low-rank representation can be used to detect the target objects by spectral clustering. Extensive experiments over diverse benchmark datasets demonstrate consistent and significant performance gains of the proposed method over the state-of-the-art object co-detection method and the generic object detection methods without co-detection formulations.
Semi-supervised hashing for scalable image retrieval

Large scale image search has recently attracted considerable attention due to easy availability of huge amounts of data. Several hashing methods have been proposed to allow approximate but highly efficient search. Unsupervised hashing methods show good performance with metric distances but, in image search, semantic similarity is usually given in terms of labeled pairs of images. There exist supervised hashing methods that can handle such semantic similarity but they are prone to overfitting when labeled data is small or noisy. Moreover, these methods are usually very slow to train. In this work, we propose a semi-supervised hashing method that is formulated as minimizing empirical error on the labeled data while maximizing variance and independence of hash bits over the labeled and unlabeled data. The proposed method can handle both metric as well as semantic similarity. The experimental results on two large datasets (up to one million samples) demonstrate its superior performance over state-of-the-art supervised and unsupervised methods.
Learning component-level sparse representation using histogram information for image classification

A novel component-level dictionary learning framework which exploits image group characteristics within sparse coding is introduced in this work. Unlike previous methods, which select the dictionaries that best reconstruct the data, we present an energy minimization formulation that jointly optimizes the learning of both sparse dictionary and component level importance within one unified framework to give a discriminative representation for image groups. The importance measures how well each feature component represents the image group property with the dictionary by using histogram information. Then, dictionaries are updated iteratively to reduce the influence of unimportant components, thus refining the sparse representation for each image group. In the end, by keeping the top K important components, a compact representation is derived for the sparse coding dictionary. Experimental results on several public datasets are shown to demonstrate the superior performance of the proposed algorithm compared to the-state-of-the-art methods.
DYNAMIC LOCAL TRACING FOR 3D AXON CURVILINEAR STRUCTURE DETECTION FROM MICROSCOPIC IMAGE STACK

To study the morphologic structure of axons can help neuro-scientists understand the neuronal function and development. The modern microscopes provide the fundamental tool for visual inspection of axonal structure. Due to the high volume of generated microscopic axon image data, it is critical to develop an automated technique for robustly and rapidly detecting 3D axonal structure. In this paper, we present a pure 3D approach to extract the curvilinear structure of axonal axes from microscopic image stacks. The method mimics the axon tracing procedure in 3D space as walking along a path with minimized cost value, which corresponds to the shortest path problem (SPP) in graph theory. The global solution for SPP, such as Dijkstra's algorithm, is infeasible for the real axon tracing problem because of the computation cost. We simplify this problem using a dynamic local tracing technique with linear computation complexity. The merits of the proposed method lie in that it can handle the short turn and non-vertical problems and also can separate closely distributed axons from each other
Real-time pose estimation of deformable objects using a volumetric approach

Pose estimation of deformable objects is a fundamental and challenging problem in robotics. We present a novel solution to this problem by first reconstructing a 3D model of the object from a low-cost depth sensor such as Kinect, and then searching a database of simulated models in different poses to predict the pose. Given noisy depth images from 360-degree views of the target object acquired from the Kinect sensor, we reconstruct a smooth 3D model of the object using depth image segmentation and volumetric fusion. Then with an efficient feature extraction and matching scheme, we search the database, which contains a large number of deformable objects in different poses, to obtain the most similar model, whose pose is then adopted as the prediction. Extensive experiments demonstrate better accuracy and orders of magnitude speed-up compared to our previous work. An additional benefit of our method is that it produces a high-quality mesh model and camera pose, which is necessary for other tasks such as regrasping and object manipulation.
Zero-error information hiding capacity of digital images

We derive a theoretical capacity for digital image watermarking with zero transmission errors. We present three different discrete memoryless channel model to represent the watermarking process. Given the magnitude bound of noise set by applications and the acceptable watermark magnitude determined by the just-noticeable distortion, we estimate the zero-error capacity by applying Shannon's (1948) adjacency-reducing mapping technique. The capacity we estimate here corresponds to a deterministic guarantee of zero error, different from the traditional theorem approaching zero error asymptotically.
A new approach to decoding and compositing motion-compensated DCT-based images

A novel decoding algorithm for the MC-DCT (motion-compensation discrete-cosine-transform)-based video, which performs inverse MC before inverse DCT, is designed. This algorithm can be applied in compositing compressed video within the network, which may take multiple compressed video sources and combine them into a single compressed output stream. The proposed algorithm convers all MC-DCT compressed video into the DCT domain and performs compositing in the DCT domain. This DCT-domain approach can reduce the required computations with a speedup factor depending on the compression ratio and the nonzero motion vector percentage. However, dropping some least-significant DCT coefficients may be necessary for the worst case of high-motion video in real-time implementations. Some issues of networked video compositing are also discussed. Another direct application of the proposed decoding algorithm is converting MC-DCT compressed video to the DCT compressed format directly in the DCT domain.<<ETX>>
Exploiting Feature and Class Relationships in Video Categorization with Regularized Deep Neural Networks

In this paper, we study the challenging problem of categorizing videos according to high-level semantics such as the existence of a particular human action or a complex event. Although extensive efforts have been devoted in recent years, most existing works combined multiple video features using simple fusion strategies and neglected the utilization of inter-class semantic relationships. This paper proposes a novel unified framework that jointly exploits the feature relationships and the class relationships for improved categorization performance. Specifically, these two types of relationships are estimated and utilized by imposing regularizations in the learning process of a deep neural network (DNN). Through arming the DNN with better capability of harnessing both the feature and the class relationships, the proposed regularized DNN (rDNN) is more suitable for modeling video semantics. We show that rDNN produces better performance over several state-of-the-art approaches. Competitive results are reported on the well-known Hollywood2 and Columbia Consumer Video benchmarks. In addition, to stimulate future research on large scale video categorization, we collect and release a new benchmark dataset, called FCVID, which contains 91,223 Internet videos and 239 manually annotated categories.
Adaptive bandwidth balancing on DQDB networks

The bandwidth balancing (BWB) scheme has been used on distributed queue dual bus (DQDB) networks to improve the fairness of bandwidth allocation. Flexible dynamic bandwidth allocation can be achieved by using nonuniform BWB modular values. However, the local load condition in each station is ignored. The authors change the BWB scheme to be adaptive based on the local traffic intensity in each station. No additional hardware is required except a simple queue length monitor. The fairness of access delay time for heavily loaded networks is improved, while the original static BWB scheme remains more effective for lower load levels. The transient time for reaching throughput steady state is reduced at the cost of some overshooting. Variations of adaptive BWB schemes are discussed and analyzed numerically.<<ETX>>
A content based video traffic model using camera operations

We present our work on content based video (CBV) traffic modeling of variable bit rate (VBR) sources. The CBV approach differs from previous works in that it is not based only on matching of various statistics of the original source, but rather on modeling and mapping its visual content into the corresponding bit rate. We show that the CBV model is fully compatible with current and future compression algorithms including those of very low bit rate video coding. We introduce the separation principle between the visual content and encoder dependent bit rate mapping. We construct and verify two experimental CBV models for basic camera operations. The results obtained show that the CBV model can closely match the various statistics of the MPEG-2 VBR stream.
Condensing computable scenes using visual complexity and film syntax analysis

In this paper, we present a novel algorithm to condense computable scenes. A computable scene is a chunk of data that exhibits consistencies with respect to chromaticity, lighting and sound. We attempt to condense such scenes in two ways. First, we define visual complexity of a shot to be its Kolmogorov complexity. Then, we conduct experiments that help us map the complexity of a shot into the minimum time required for its comprehension. Second, we analyze the grammar of the film language, since it makes the shot sequence meaningful. These grammatical rules are used to condense scenes, in parallel to the shot level condensation. Weve implemented a system that generates a skim given a time budget. Our user studies show good results on skims with compression rates between 60~80%.
Spatio-temporal video search using the object based video representation

Object-based video representation provides great promises for new search and editing functionalities. Feature regions in video sequences are automatically segmented, tracked, and grouped to form the basis for content-based video search and higher levels of abstraction. We present a new system for video object segmentation and tracking using feature fusion and region grouping. We also present efficient techniques for spatio-temporal video query based on the automatically segmented video objects.
Temporal Action Localization in Untrimmed Videos via Multi-stage CNNs

We address temporal action localization in untrimmed long videos. This is important because videos in real applications are usually unconstrained and contain multiple action instances plus video content of background scenes or other activities. To address this challenging issue, we exploit the effectiveness of deep networks in temporal action localization via three segment-based 3D ConvNets: (1) a proposal network identifies candidate segments in a long video that may contain actions, (2) a classification network learns one-vs-all action classification model to serve as initialization for the localization network, and (3) a localization network fine-tunes the learned classification network to localize each action instance. We propose a novel loss function for the localization network to explicitly consider temporal overlap and achieve high temporal localization accuracy. In the end, only the proposal network and the localization network are used during prediction. On two largescale benchmarks, our approach achieves significantly superior performances compared with other state-of-the-art systems: mAP increases from 1.7% to 7.4% on MEXaction2 and increases from 15.0% to 19.0% on THUMOS 2014.
Pattern Mining in Visual Concept Streams

Pattern mining algorithms are often much easier applied than quantitatively assessed. In this paper we address the pattern evaluation problem by looking at both the capability of models and the difficulty of target concepts. We use four different data mining models: frequent itemset mining, k-means clustering, hidden Markov model, and hierarchical hidden Markov model to mine 39 concept streams from the a 137-video broadcast news collection from TRECVID-2005. We hypothesize that the discovered patterns can reveal semantics beyond the input space, and thus evaluate the patterns against a much larger concept space containing 192 concepts defined by LSCOM. Results show that HHMM has the best average prediction among all models, however different models seem to excel in different concepts depending on the concept prior and the ontological relationship. Results also show that the majority of the target concepts are better predicted with temporal or combination hypotheses, and there are novel concepts found that are not part of the original lexicon. This paper presents the first effort on temporal pattern mining in the large concept space. There are many promising directions to use concept mining to help construct better concept detectors or to guide the design of multimedia ontology
MPEG video compositing in the compressed domain

Compositing several MPEG video sources into a single displayed streams is important for advanced MPEG video applications such as advanced multimedia terminals and multi-point video conferencing. Compositing video directly in the compressed domain can reduce computational complexity by processing less data and avoiding the conversion process back and forth between the compressed and uncompressed data formats. We address the computational speedup of compositing MPEG video directly in the DCT domain, compared to the straightforward spatial domain approach. In this paper, we propose a new compositing algorithm for MPEG1 compressed video which converts compressed MPEG1 video to the DCT domain and enables video compositing in the DCT compressed domain. Typical compositing operations include overlapping, scaling, translation, filtering, etc. Simulations using picture-in-picture video compositing sequence are presented to quantify some performance factors concerned.