nils-herrmann/clean_scirepeval huggingface dataset:
Randomized Spatial Context for Object Search

Searching visual objects in large image or video data sets is a challenging problem, because it requires efficient matching and accurate localization of query objects that often occupy a small part of an image. Although spatial context has been shown to help produce more reliable detection than methods that match local features individually, how to extract appropriate spatial context remains an open problem. Instead of using fixed-scale spatial context, we propose a randomized approach to deriving spatial context, in the form of spatial random partition. The effect of spatial context is achieved by averaging the matching scores over multiple random patches. Our approach offers three benefits: 1) the aggregation of the matching scores over multiple random patches provides robust local matching; 2) the matched objects can be directly identified on the pixelwise confidence map, which results in efficient object localization; and 3) our algorithm lends itself to easy parallelization and also allows a flexible tradeoff between accuracy and speed through adjusting the number of partition times. Both theoretical studies and experimental comparisons with the state-of-the-art methods validate the advantages of our approach.
Spatial and angular resolution measurement of a tensor light field display

Dependent on the way in which the images are formed, some light field three-dimensional (3D) displays are less straightforward to characterize than two-dimensional (2D) displays. They do not have an effective screen position that physically exists, they do not have pixels in the conventional sense that can be simply measured with a microscope and, in common with all 3D displays they have light emission that varies with angle, giving them the characteristic of angular resolution. In our previous work on light field characterisation, subjective evaluation was used to determine perceived spatial resolution and some other characteristics; however, the measurement of angular resolution eluded us. As this is the vital characteristic that distinguishes 3D displays from 2D displays, we set out to determine a method of obtaining this parameter from existing data collected. Crucial to this was the realisation that, first, the meanings of the characteristics must be defined. It was observed that the overall display system, including; capture, processing and display, behaves as a low-pass filter. Regarding the system as such enables the spatial resolution to be expressed in terms of spatial cut-off frequency and angular resolution in terms of angular cut-off frequency. This led us to determine the angular resolution indirectly from the variation of spatial resolution with depth. In this paper, we describe the rationale behind our method, which also dispenses with the need to conduct lengthy and unreliable user trials, and give the results of the measurements.
Person Reidentification Using Multiple Egocentric Views

Development of a robust and scalable multicamera surveillance system is the need of the hour to ensure public safety and security. Being able to reidentify and track one or more targets over multiple nonoverlapping camera field of views in a crowded environment remains an important and challenging problem because of occlusions, large change in the viewpoints, and illumination across cameras. However, the rise of wearable imaging devices has led to new avenues in solving the reidentification (re-id) problem. Unlike static cameras, where the views are often restricted or low resolution and occlusions are common scenarios, egocentric/first person views (FPVs) mostly get zoomed in, unoccluded face images. In this paper, we present a person re-id framework designed for a network of multiple wearable devices. The proposed framework builds on commonly used facial feature extraction and similarity computation methods between camera pairs and utilizes a data association method to yield globally optimal and consistent re-id results with much improved accuracy. Moreover, to ensure its utility in practical applications where a large amount of observations are available every instant, an online scheme is proposed as a direct extension of the batch method. This can dynamically associate new observations to already observed and labeled targets in an iterative fashion. We tested both the offline and online methods on realistic FPV video databases, collected using multiple wearable cameras in a complex office environment and observed large improvements in performance when compared with the state of the arts.
Temporally enhanced image object proposals for videos

Despite the recent success of image object proposals (IOPs) for image applications, the per-frame IOPs are also important for video applications. However, the existing IOPs are extracted from each frame separately and may exhibit inconsistencies across the frames. In this paper, we propose to improve the existing IOPs by enforcing the temporal consistency through a video sequence in an on-line manner. To achieve this, we propose a novel spatio-temporal objectness measure considering both the frame level objectness as well as the temporal consistency across frames. An on-line dynamic programing technique is proposed to efficiently compute such spatio-temporal objectness. In addition, compared with the spatio-temporal video object proposals(VOPs), the proposed method supports on-line applications and provides more accurate per-frame localizations. Experiments on benchmark datasets validate its superior performance compared with the existing IOPs and VOPs.
Flexible Trajectory Indexing for 3D Motion Recognition

Motion trajectory analysis is important for human motion recognition and human computer interaction. In this paper, we propose a flexible 3D trajectory indexing method for complex 3D motion recognition. Based on both point level and primitive-level descriptors, trajectories are represented in the sub-primitive level, the level between the point level and primitive level. Primitives are flexibly segmented into sub-primitives in various scales, and the sub-primitives retain more detailed information than primitives. The detailed level of sub-primitives can be adjusted by controlling segmentation scales according to motion complexities. The proposed approach is suitable for spatial motion trajectory, which is view-invariant in 3D space. A cluster model is also proposed to represent motion classes and motion recognition performed based on maximum a posteriori (MAP) criterion. The experiments on benchmark datasets validate the effectiveness of the proposed approach.
Duality properties of light field capture and display on Lytro camera and multi-layer display

Light field capture and multi-layer light field displays are increasingly concerned topics for academia and industry. However, an integrated discussion connecting capture and display has been long neglected. Therefore, this paper proposes a duality analysis of capture and display to comprehensively analyze and define two basic attributes; depth range and angular resolution. The main contribution of this paper is to discuss the display properties based on the configurations of light field capture and its effect on displayed image quality rather than focusing on the display itself.
Depth camera based hand gesture recognition and its applications in Human-Computer-Interaction

Of various Human-Computer-Interactions (HCI), hand gesture based HCI might be the most natural and intuitive way to communicate between people and machines, since it closely mimics how human interact with each other. Its intuitiveness and naturalness have spawned many applications in exploring large and complex data, computer games, virtual reality, health care, etc. Although the market for hand gesture based HCI is huge, building a robust hand gesture recognition system remains a challenging problem for traditional vision-based approaches, which are greatly limited by the quality of the input from optical sensors.  proposed a novel dissimilarity distance metric for hand gesture recognition using Kinect sensor, called Finger-Earth Mover's Distance (FEMD). In this paper, we compare the performance in terms of speed and accuracy between FEMD and traditional corresponding-based shape matching algorithm, Shape Context. And then we introduce several HCI applications built on top of a accurate and robust hand gesture recognition system based on FEMD. This hand gesture recognition system performs robustly despite variations in hand orientation, scale or articulation. Moreover, it works well in uncontrolled environments with background clusters. We demonstrate that this robust hand gesture recognition system can be a key enabler for numerous hand gesture based HCI systems.
Unsupervised random forest indexing for fast action search

Despite recent successes of searching small object in images, it remains a challenging problem to search and locate actions in crowded videos because of (1) the large variations of human actions and (2) the intensive computational cost of searching the video space. To address these challenges, we propose a fast action search and localization method that supports relevance feedback from the user. By characterizing videos as spatio-temporal interest points and building a random forest to index and match these points, our query matching is robust and efficient. To enable efficient action localization, we propose a coarse-to-fine sub-volume search scheme, which is several orders faster than the existing video branch and bound search. The challenging cross-dataset search of several actions validates the effectiveness and efficiency of our method.
Profit Maximization for Viral Marketing in Online Social Networks: Algorithms and Analysis

Information can be disseminated widely and rapidly through Online Social Networks (OSNs) with “word-of-mouth” effects. Viral marketing is such a typical application in which new products or commercial activities are advertised by some seed users in OSNs to other users in a cascading manner. The selection of initial seed users yields a tradeoff between the expense and reward of viral marketing. In this paper, we define a general profit metric that naturally combines the benefit of influence spread with the cost of seed selection in viral marketing. We carry out a comprehensive study on finding a set of seed nodes to maximize the profit of viral marketing. We show that the profit metric is significantly different from the influence metric in that it is no longer monotone. This characteristic differentiates the profit maximization problem from the traditional influence maximization problem. We develop new seed selection algorithms for profit maximization with strong approximation guarantees. We also derive several upper bounds to benchmark the practical performance of an algorithm on any specific problem instance. Experimental evaluations with real OSN datasets demonstrate the effectiveness of our algorithms and techniques.
Non-iterative SLAM

The goal of this paper is to create a new framework for dense SLAM that is light enough for micro-robot systems based on depth camera and inertial sensor. Feature-based and direct methods are two mainstreams in visual SLAM. Both methods minimize photometric or reprojection error by iterative solutions, which are computationally expensive. To overcome this problem, we propose a non-iterative framework to reduce computational requirement. First, the attitude and heading reference system (AHRS) and axonometric projection are utilized to decouple the 6 Degree-of-Freedom (DoF) data, so that point clouds can be matched in independent spaces respectively. Second, based on single key-frame training, the matching process is carried out in frequency domain by Fourier transformation, which provides a closed-form non-iterative solution. In this manner, the time complexity is reduced to O(n log n), where n is the number of matched points in each frame. To the best of our knowledge, this method is the first non-iterative and online trainable approach for data association in visual SLAM. Compared with the state-of-the-arts, it runs at a faster speed and obtains 3-D maps with higher resolution yet still with comparable accuracy.
Robust Discriminative Tracking via Landmark-Based Label Propagation

The appearance of an object could be continuously changing during tracking, thereby being not independent identically distributed. A good discriminative tracker often needs a large number of training samples to fit the underlying data distribution, which is impractical for visual tracking. In this paper, we present a new discriminative tracker via landmark-based label propagation (LLP) that is nonparametric and makes no specific assumption about the sample distribution. With an undirected graph representation of samples, the LLP locally approximates the soft label of each sample by a linear combination of labels on its nearby landmarks. It is able to effectively propagate a limited amount of initial labels to a large amount of unlabeled samples. To this end, we introduce a local landmarks approximation method to compute the cross-similarity matrix between the whole data and landmarks. Moreover, a soft label prediction function incorporating the graph Laplacian regularizer is used to diffuse the known labels to all the unlabeled vertices in the graph, which explicitly considers the local geometrical structure of all samples. Tracking is then carried out within a Bayesian inference framework, where the soft label prediction value is used to construct the observation model. Both qualitative and quantitative evaluations on the benchmark data set containing 51 challenging image sequences demonstrate that the proposed algorithm outperforms the state-of-the-art methods.
Multi-layer light field display characterisation

Light field 3D displays, where a hologram-like image is produced by using geometrical optical techniques as opposed to interference of light, require different measurement procedures to conventional stereoscopic displays for the measurement of certain parameters. This paper covers methods that are particularly applicable to multi-layer light field displays using two or more cascaded display layers where the images are produced by computationally-intensive algorithms. As the overall system performance is dependent on capture and computation as well as display hardware, we have developed methods that allow for all the components in the complete chain. In addition to describing these we also cover other selected measurement techniques here.
Topical Video Object Discovery from Key Frames by Modeling Word Co-occurrence Prior

A topical video object refers to an object that is frequently highlighted in a video. It could be, e.g., the product logo and the leading actor/actress in a TV commercial. We propose a topic model that incorporates a word co-occurrence prior for efficient discovery of topical video objects from a set of key frames. Previous work using topic models, such as Latent Dirichelet Allocation (LDA), for video object discovery often takes a bag-of-visual-words representation, which ignored important co-occurrence information among the local features. We show that such data driven co-occurrence information from bottom-up can conveniently be incorporated in LDA with a Gaussian Markov prior, which combines top down probabilistic topic modeling with bottom up priors in a unified model. Our experiments on challenging videos demonstrate that the proposed approach can discover different types of topical objects despite variations in scale, view-point, color and lighting changes, or even partial occlusions. The efficacy of the co-occurrence prior is clearly demonstrated when comparing with topic models without such priors.
Efficient search of Top-K video subvolumes for multi-instance action detection

Action detection was formulated as a subvolume mutual information maximization problem in , where each subvolume identifies where and when the action occurs in the video. Despite the fact that the proposed branch-and-bound algorithm can find the best subvolume efficiently for low resolution videos, it is still not efficient enough to perform multi-instance detection in videos of high spatial resolution. In this paper we develop an algorithm that further speeds up the subvolume search and targets on real-time multi-instance action detection for high resolution videos (e.g. 320 × 240 or higher). Unlike the previous branch-and-bound search technique which restarts a new search for each action instance, we find the Top-K subvolumes simultaneously with a single round of search. To handle the larger spatial resolution, we downsample the volume of videos for a more efficient upper-bound estimation. To validate our algorithm, we perform experiments on a challenging dataset of 54 video sequences where each video consists of several actions performed by different people in a crowded environment. The experiments show that our method is not only efficient, but also capable of handling action variations caused by performing speed and style changes, spatial scale changes, as well as cluttered and moving background.
Hierarchical sparse coding based on spatial pooling and multi-feature fusion

We propose a novel hierarchical sparse coding algorithm with spatial pooling and multi-feature fusion, to construct the low-level visual primitives, e.g., local image patches or regions, into high-level visual phrases, e.g., image patterns. In the first layer we learn the sparse codes for the visual primitives and then pass them into the second layer by spatial pooling and multi-feature fusion. In the second layer we further learn the sparse codes for the visual phrases. In order to obtain the high-quality representations for visual phrases, our proposed algorithm iteratively optimizes over the two-layer sparse codes, as well as the two-layer codebooks. Since we have explored both the spatial and multi-feature contextual information, more representative sparse codes of the visual phrases can be obtained. The experiments on image pattern discovery, image scene clustering and image classification justify the advantages of the proposed algorithm.
Modelling Multi-Party Interactions among Virtual Characters, Robots, and Humans

3D virtual humans and physical human-like robots can be used to interact with people in a remote location in order to increase the feeling of presence. In a telepresence setup, their behaviors are driven by real participants. We envision that in the absence of the real users, when they have to leave or they do not want to do a repetitive task, the control of the robots can be handed to an artificial intelligence component to sustain the ongoing interaction. At the point when human-mediated interaction is required again, control can be returned to the real users. One of the main challenges in telepresence research is the adaptation of 3D position and orientation of the remote participants to the actual physical environment to have appropriate eye contact and gesture awareness in a group conversation. In case the human behind the robot and/or virtual human leaves, multi-party interaction should be handed to an artificial intelligence component. In this paper, we discuss the challenges in autonomous multi-party interaction among virtual characters, human-like robots, and real participants, and describe a prototype system to study these challenges.
Object instance search in videos

In this paper, we propose a novel approach for object instance search in videos. Employing discriminative mutual information score and inferring the location of target object centers from matched local feature descriptors using Hough voting, we achieve robust matching and per-frame localization despite orientation and scale variations. We then leverage Max-Path search  to efficiently find the globally optimal spatio-temporal trajectory of the object center in each video sequence. Experimental results on a collection of mobile-captured videos in real-world environments demonstrate the effectiveness and accuracy of our method.
Propagative Hough Voting for Human Activity Detection and Recognition

Generalized Hough voting (HV) has shown promising results in both object and action detection. However, most existing HV methods will suffer when insufficient training data are provided. We propose propagative HV to address this limitation and apply it to human activity analysis. Instead of training a discriminative classifier for local feature voting, we match individual local features to propagate the label and spatiotemporal configuration information of local features via HV. To enable a fast local feature matching, we index the local features using random projection trees (RPTs). RPTs can reveal the low-dimension manifold structure to provide adaptive local feature matching. Moreover, as the RPT index can be built in either labeled or unlabeled dataset, it can be applied to different tasks, such as activity search (limited training) and recognition (sufficient training). The superior performances on benchmarked datasets validate that our propagative HV can outperform state-of-the-art techniques in various activity analysis tasks, such as activity search, recognition, and prediction.
Hough Forest With Optimized Leaves for Global Hand Pose Estimation With Arbitrary Postures

Vision-based hand pose estimation is important in human–computer interaction. While many recent works focus on full degree-of-freedom hand pose estimation, robust estimation of global hand pose remains a challenging problem. This paper presents a novel algorithm to optimize the leaf weights in a Hough forest to assist global hand pose estimation with a single depth camera. Different from traditional Hough forest, we propose to learn the vote weights stored at the leaf nodes of a forest in a principled way to minimize average pose prediction error, so that ambiguous votes are largely suppressed during prediction fusion. Experiments show that the proposed method largely improves pose estimation accuracy with optimized leaf weights on both synthesis and real datasets and performs favorably compared to state-of-the-art convolutional neural network-based methods. On real-world depth videos, the proposed method demonstrates improved robustness compared to several other recent hand tracking systems from both industry and academy. Moreover, we utilize the proposed method to build virtual/augmented reality applications to allow users to manipulate and examine virtual objects with bare hands.
Efficient Mining of Optimal AND/OR Patterns for Visual Recognition

The co-occurrence features are the composition of base features that have more discriminative power than individual base features. Although they show promising performance in visual recognition applications such as object, scene, and action recognition, the discovery of optimal co-occurrence features is usually a computationally demanding task. Unlike previous feature mining methods that fix the order of the co-occurrence features or rely on a two-stage frequent pattern mining to select the optimal co-occurrence feature, we propose a novel branch-and-bound search-based co-occurrence feature mining algorithm that can directly mine both optimal conjunctions (AND) and disjunctions (OR) of individual features at arbitrary orders simultaneously. This feature mining process is integrated into a multi-class boosting framework Adaboost.MH such that the weighted training error is minimized by the discovered co- occurrence features in each boosting step. Experiments on UCI benchmark datasets, the scene recognition dataset, and the action recognition dataset validate both the effectiveness and efficiency of our proposed method.