nils-herrmann/clean_scirepeval huggingface dataset:
Effective Codebooks for Human Action Representation and Classification in Unconstrained Videos

Recognition and classification of human actions for annotation of unconstrained video sequences has proven to be challenging because of the variations in the environment, appearance of actors, modalities in which the same action is performed by different persons, speed and duration, and points of view from which the event is observed. This variability reflects in the difficulty of defining effective descriptors and deriving appropriate and effective codebooks for action categorization. In this paper, we propose a novel and effective solution to classify human actions in unconstrained videos. It improves on previous contributions through the definition of a novel local descriptor that uses image gradient and optic flow to respectively model the appearance and motion of human actions at interest point regions. In the formation of the codebook, we employ radius-based clustering with soft assignment in order to create a rich vocabulary that may account for the high variability of human actions. We show that our solution scores very good performance with no need of parameter tuning. We also show that a strong reduction of computation time can be obtained by applying codebook size reduction with Deep Belief Networks with little loss of accuracy.
Distant targets identification as an on-line dynamic vehicle routing problem using an active-zooming camera

This paper considers the problem of modeling an active observer to plan a sequence of decisions regarding what target to look at, through a foveal-sensing action. The gathered images by the active observer provides meaningful identification imagery of distant targets which are not recognizable in a wide angle view. We propose a framework in which a pan/tilt/zoom (PTZ) camera schedules saccades in order to acquire high resolution images of as many moving targets as possible before they leave the scene. We cast the whole problem as a particular kind of dynamic discrete optimization, specially as a novel on-line dynamic vehicle routing problem (DVRP) with deadlines. We show that using an optimal choice for the sensing order of targets the total time spent in visiting the targets by the active camera can be significantly reduced. To show the effectiveness of our approach we apply congestion analysis to a dual camera system in a master-slave configuration. We report that our framework gives good results in monitoring wide areas with little extra costs with respect to approaches using a large number of cameras.
A fuzzy object retrieval system for image understanding

The authors address the problem of object recognition in the field of image understanding systems. They show how the problem can be approached exploiting information retrieval techniques. That is, given a collection of object models, the target object is regarded as a query and object recognition is regarded as a query processing task aimed at locating the models which best fulfil the given one. Since the problem is fuzzy in nature, the inference mechanism is based on a fuzzy set theoretical framework. The authors also show how the object-oriented paradigm provides the right framework to represent effectively and efficiently the knowledge about the objects. Each real world entity in the domain is mapped on a collection of composite objects, hierarchically organized, each level representing a specific abstraction level at which the entity is regarded. The authors discuss implementation issues with reference, to the retrieval component and related techniques that have been devised to improve the computational efficiency of the system.<<ETX>>
Local Pyramidal Descriptors for Image Recognition

In this paper, we present a novel method to improve the flexibility of descriptor matching for image recognition by using local multiresolution pyramids in feature space. We propose that image patches be represented at multiple levels of descriptor detail and that these levels be defined in terms of local spatial pooling resolution. Preserving multiple levels of detail in local descriptors is a way of hedging one's bets on which levels will most relevant for matching during learning and recognition. We introduce the Pyramid SIFT (P-SIFT) descriptor and show that its use in four state-of-the-art image recognition pipelines improves accuracy and yields state-of-the-art results. Our technique is applicable independently of spatial pyramid matching and we show that spatial pyramids can be combined with local pyramids to obtain further improvement. We achieve state-of-the-art results on Caltech-101 (80.1%) and Caltech-256 (52.6%) when compared to other approaches based on SIFT features over intensity images. Our technique is efficient and is extremely easy to integrate into image recognition pipelines.
Sensor Fusion for Cooperative Head Localization

In modern video surveillance systems, pan–tilt– zoom (PTZ) cameras certainly have the potential to allow the coverage of wide areas with a much smaller number of sensors, compared to the common approach of fixed camera networks. This paper describes a general framework that aims at exploiting the capabilities of modern PTZ cameras in order to acquire high resolution images of body parts, such as the head, from the observation of pedestrians moving in a wide outdoor area. The framework allows to organize the sensors in a network with arbitrary topology, and to establish pairwise master–slave relationship between them. In this way a slave camera can be steered to acquire imagery of a target keeping into account both target and zooming uncertainties. Experiments show good performance in localizing target’s head, independently from the zooming factor of the slave camera.
Pyramidal retrieval by color perceptive regions

The minor expressiveness of text with respect to visual features has determined the prevalence of visual interaction paradigms for specifying queries in systems for image retrieval by content. To cope with user's subjectivity, the description of image visual contents should exploit different levels of detail in image representation. We present a system which supports image indexing and retrieval based on color regions. The system exploits a pyramidal representation of the images which are segmented at different levels of resolution. Multiple descriptions of image regions are created at different levels of resolution thus allowing effective retrieval through specific queries as well as imprecise ones.
David: Discriminant analysis for verification of monuments in image data

In the past few years, several research works have addressed the problems posed by vision-assisted navigation systems. Basically, these systems allow a tourists navigating in an urban environment to take the photograph of a scene and submit it to the navigation system that will recognize what monument is represented in the photograph. However, solutions proposed so far focus on the recognition of approximately planar structures (e.g. building facades) and are inadequate to support the recognition task if the object of interest has a generic 3D structure such as a statue. In this paper we present a model to support the recognition of generic monuments that is based on the information-theoretic notion of mutual information to quantify the saliency of invariant local descriptors. Results are demonstrated on a real-database also in comparison with the baseline method that performs matching without any measure of saliency.
A SIFT-Based Forensic Method for Copy–Move Attack Detection and Transformation Recovery

One of the principal problems in image forensics is determining if a particular image is authentic or not. This can be a crucial task when images are used as basic evidence to influence judgment like, for example, in a court of law. To carry out such forensic analysis, various technological instruments have been developed in the literature. In this paper, the problem of detecting if an image has been forged is investigated; in particular, attention has been paid to the case in which an area of an image is copied and then pasted onto another zone to create a duplication or to cancel something that was awkward. Generally, to adapt the image patch to the new context a geometric transformation is needed. To detect such modifications, a novel methodology based on scale invariant features transform (SIFT) is proposed. Such a method allows us to both understand if a copy-move attack has occurred and, furthermore, to recover the geometric transformation used to perform cloning. Extensive experimental results are presented to confirm that the technique is able to precisely individuate the altered area and, in addition, to estimate the geometric transformation parameters with high reliability. The method also deals with multiple cloning.
Blackboard-based concurrent object recognition using an object-oriented database

A blackboard-based framework is presented for concurrent object recognition in the presence of a large model database. A new object model is defined with a knowledge organization with two-object hierarchies. Concurrency has been introduced in order to provide a separate execution thread for each scheduled model. The system performs recognition by comparing graphical entities with models that are stored in an object-oriented database. Results are presented in terms of performance figures on a multiprocessor architecture.<<ETX>>
Bringing the Wiki Collaboration Model to the Tabletop World

We present an interactive workspace that integrates wiki collaboration in knowledge-building activities with face-to-face scenarios like brainstorming or problem solving sessions. A wiki serves as the repository for knowledge elements, which are presented to co-located users in form of a concept map projected on a table. A computer vision module tracks multiple hands and fingers on the table surface and map elements can be manipulated using a simple gesture language. Also smart devices, like PDAs or tablets, can be connected to the system and participate in the interaction with their local input. The concept map and the wiki are synchronized in real-time, providing notifications to both co-located and distributed users and allowing a community shared awareness that enhances and enriches the knowledge building experience
Navigation and visualization of movies content

The description and manipulation of the information embedded in video data is the key challenge for future multimedia technologies. The multilevel messages conveyed by time-varying visual data can be suitably handled only when a structured conceptual schema is built, which takes into account the multiple links among different elements of visual speech. In the paper a new schema for film data is introduced and a hypermedia navigation system is proposed, which is based upon this conceptual model. Such a system integrates the technical and visual content of a film into a comprehensive framework and makes it possible to navigate and display the different classes of content in a unconstrained sequence. The navigation task does not require any knowledge of complex query languages, since a query is structured as a form of data filtering through the formal operator of "perspective".
On the perspective projection of 3D planar-faced junctions

A method to determine the spatial orientation of a 3D planar-faced object from a single perspective view is presented. In this method the junctions, that is the concurrences of three object edges in a vertex, are considered as key features. A geometric constraint, related to the perspective projection of the object junctions, is shown and exploited to make the process faster and more efficient. In this way, the knowledge of only two parameters is sufficient to verify the object orientation in 3D space. A system of equations is proposed which define the orientations space for a junction as a subset of R/sup 2/, instead of R/sup 3/ as in the previous literature.<<ETX>>
Image indexing using shape-based visual features

Efficient image retrieval by contents from database requires that selective access methods are provided to prune out uninteresting items during the search process. Image indexing based on visual features is particularly challenging, owing to the difficulty to derive a representation of the shapes that closely models the visual appearance perceived by humans. In this paper we present a novel approach for indexing planar and closed curves, on the basis on their visual appearance. A hierarchical model of the curve is derived from its multi-scale analysis, and it is used to provide a description of the curve which is able to distinguish between its structural parts and its details. To cope with the inherent uncertainty of shape appearance, fuzzy sets are used to represent the visual attributes of the shapes. In this way the shapes which share similar structural parts can be gathered and embedded into an index structure which retains the imprecision of shape description by embedding fuzziness in the index itself.
Accurate evaluation of HER-2 amplification in FISH images

Fluorescence in situ hybridization (FISH) is a cytogenetic technique used to detect the presence of specific DNA sequences on chromosomes. An application of FISH targets the measurement of the amplification of the HER-2 gene within the chromosomes. This requires computation of the ratio of HER-2 over centromere 17 (CEP-17) markers detected within a representative number of nuclei. Selection of the nuclei that are used to compute this ratio is a crucial step to yield an accurate estimation of HER-2 amplification. In fact, errors deriving from overlapping, clustered and oversegmented nuclei can bias the estimation of the ratio and yield inaccurate evaluation of HER-2 amplification. In this paper, a model is presented to associate a reliability score with each nucleus. The reliability score enables refinement of segmentation results—yielding more accurate identification of the boundary of each nucleus—as well as ranking of detected nuclei so as to use just the most reliable nuclei to compute the ratio. Experimental results demonstrate that the adoption of the proposed model improves the estimation of HER-2 amplification.
Unsupervised incremental learning of deep descriptors from video streams

We present a novel unsupervised method for face identity learning from video sequences. The method exploits the ResNet deep network for face detection and VGGface fc7 face descriptors together with a smart learning mechanism that exploits the temporal coherence of visual data in video streams. We present a novel feature matching solution based on Reverse Nearest Neighbour and a feature forgetting strategy that supports incremental learning with memory size control, while time progresses. It is shown that the proposed learning procedure is asymptotically stable and can be effectively applied to relevant applications like multiple face tracking.
Real-time tracking and reproduction of 3D human body motion

Non-intrusive human body tracking is a key issue in advanced human-computer interaction, with key applications ranging from virtual reality to videoconferencing and telepresence. This paper describes a system for vision-based tracking of body posture. The system is explicitly designed to provide a robust yet simple and inexpensive solution to real-time body tracking through a careful choice of visual and kinematic models. Human posture representation is fully compatible with the MPEG-4 standard. Results of system application to a computer graphics scenario (animation of 3D avatars) are presented and discussed.
Fisher Vectors over Random Density Forests for Object Recognition

In this paper we describe a Fisher vector encoding of images over Random Density Forests. Random Density Forests (RDFs) are an unsupervised variation of Random Decision Forests for density estimation. In this work we train RDFs by splitting at each node in order to minimize the Gaussian differential entropy of each split. We use this as generative model of image patch features and derive the Fisher vector representation using the RDF as the underlying model. Our approach is computationally efficient, reducing the amount of Gaussian derivatives to compute, and allows more flexibility in the feature density modelling. We evaluate our approach on the PASCAL VOC 2007 dataset showing that our approach, that only uses linear classifiers, improves over bag of visual words and is comparable to the traditional Fisher vector encoding over Gaussian Mixture Models for density estimation.
Matching Faces with Textual Cues in Soccer Videos

In soccer videos, most significant actions are usually followed by close-up shots of players that take part in the action itself. Automatically annotating the identity of the players present in these shots would be considerably valuable for indexing and retrieval applications. Due to high variations in pose and illumination across shots however, current face recognition methods are not suitable for this task. We show how the inherent multiple media structure of soccer videos can be exploited to understand the players' identity without relying on direct face recognition. The proposed method is based on a combination of interest point detector to "read" textual cues that allow to label a player with its name, such as the number depicted on its jersey, or the superimposed text caption showing its name. Players not identified by this process are then assigned to one of the labeled faces by means of a face similarity measure, again based on the appearance of local salient patches. We present results obtained from soccer videos taken from various recent games between national teams
Compact representation and probabilistic classification of human actions in videos

This paper addresses the problem of classifying human actions in a video sequence. A representation eigenspace approach based on the PCA algorithm is used to train the classifier according to an incremental learning scheme based on a "one action, one eigenspace" approach. Before dimensionality reduction, a high dimensional description of each frame of the video sequence is constructed, based on foreground blob analysis. Classification is performed by matching incrementally the reduced representation of the test image sequence against each of the learned ones, and accumulating matching scores according to a probabilistic framework, until a decision is obtained. Experimental results with real video sequences are presented and discussed.
Context-Dependent Logo Matching and Recognition

We contribute, through this paper, to the design of a novel variational framework able to match and recognize multiple instances of multiple reference logos in image archives. Reference logos and test images are seen as constellations of local features (interest points, regions, etc.) and matched by minimizing an energy function mixing: 1) a fidelity term that measures the quality of feature matching, 2) a neighborhood criterion that captures feature co-occurrence/geometry, and 3) a regularization term that controls the smoothness of the matching solution. We also introduce a detection/recognition procedure and study its theoretical consistency. Finally, we show the validity of our method through extensive experiments on the challenging MICC-Logos dataset. Our method overtakes, by 20%, baseline as well as state-of-the-art matching/recognition procedures.
Improving evidential quality of surveillance imagery through active face tracking

We report on a system for automatically obtaining high-resolution images of surveillance targets. The system, beginning from a framed target, uses the presence of faces as a cue to initiate a real-time active face tracking process. Using a single pan-tilt-zoom (PTZ) camera, the system tracks the target while zooming in for a closeup image of the face. The system tracks feature points on a target face, and a simple yet reliable control strategy is used in conjunction with robust target localization to guide a PTZ camera to a facial closeup of the target. We report on experiments with our system in real surveillance environments
Motion capture based on color error maps in a distributed collaborative environment

In this paper a composite framework for collaborative working is presented. The framework includes real-time motion tracking based on computer vision from standard Web-cams situated at different locations, data transmission and real-time animation of 3D avatars in a virtual world. Motion tracking is obtained without using markers, with weak constraints on users' clothes and environment lighting, ft is based on a model fitting process that compares the 2D processed images supplied by cameras with a set of artificially generated views of a human model.
Uncalibrated 3D metric reconstruction and flattened texture acquisition from a single view of a surface of revolution

We describe a geometric approach for reconstructing 3D textured graphical models of surface of revolution (SOR) objects from a single uncalibrated view Our approach is based on the fact that, for the object class of interest, the structure of the scene provides enough constraints for camera calibration even from a single view. Reconstruction (up to a scaling factor) of 3D shape is complemented with the extraction of flattened 2D texture, so as to support visual retrieval from 2D/3D cues anti to generate realistic 3D visualization models. The approach developed is quite simple, yet accurate and robust; its applications range from the preservation, analysis and classification of cultural heritage, to advanced graphics and multimedia.
The photographer metaphor for content based image retrieval

Image database systems have been widely studied in the last few years. Having to deal with visual information, a great effort has also been put into designing and implementing user interfaces for retrieval engines. Most of those interfaces allow the user to specify the query through visual examples, either by feeding a complete image to the system or by drawing some coloured/textured patches on a sketchpad. However, the "query by example" paradigm can be inadequate in many contexts, due to the difficulty for many users to paint significant examples and the impossibility, in many cases, of reproducing complex scenes. 3D interfaces and virtual reality-based interaction paradigms offer new possibilities to overcome these limitations. In the prototype system presented here, the user is allowed to navigate a 3D scene. While navigating the virtual world, the user is given the opportunity of taking some photographs used to query by content a database of images.
Effective image retrieval using deformable templates

Image retrieval by contents from database is a major research subject in advanced multimedia systems. Effective image retrieval by contents requires that visual image properties are used instead of textual labels to properly index and recover pictorial data. Retrieval by shape similarity given a user-sketched template is particularly challenging, owing to the difficulty to derive a similarity measure that closely conforms to the common perception of similarity by humans. In this paper we present a technique which takes into account global shape properties and is based on elastic matching of sketched templates over the shapes in the images to evaluate similarity ranks. The degree of matching achieved and the elastic deformation energy spent by the sketch to achieve such a match are used to derive a measure of similarity between the sketch and the images in the database and to rank images to be displayed.