nils-herrmann/clean_scirepeval huggingface dataset:
iCoseg: Interactive co-segmentation with intelligent scribble guidance

This paper presents an algorithm for Interactive Co-segmentation of a foreground object from a group of related images. While previous approaches focus on unsupervised co-segmentation, we use successful ideas from the interactive object-cutout literature. We develop an algorithm that allows users to decide what foreground is, and then guide the output of the co-segmentation algorithm towards it via scribbles. Interestingly, keeping a user in the loop leads to simpler and highly parallelizable energy functions, allowing us to work with significantly more images per group. However, unlike the interactive single image counterpart, a user cannot be expected to exhaustively examine all cutouts (from tens of images) returned by the system to make corrections. Hence, we propose iCoseg, an automatic recommendation system that intelligently recommends where the user should scribble next. We introduce and make publicly available the largest co-segmentation datasetyet, the CMU-Cornell iCoseg Dataset, with 38 groups, 643 images, and pixelwise hand-annotated groundtruth. Through machine experiments and real user studies with our developed interface, we show that iCoseg can intelligently recommend regions to scribble on, and users following these recommendations can achieve good quality cutouts with significantly lower time and effort than exhaustively examining all cutouts.
Video categorization using object of interest detection

Object of Interest (OOI) detection has been widely used in many recent works in video analysis, especially in video similarity and video retrieval. In this paper, we describe a generic video classification algorithm using object of interest detection. We use online user-submitted videos and aim to categorize the videos into six broad categories hot star, news, anime, pets, sports and commercials. We show through our experiments that, detecting and describing the object of interest improves the video classification accuracy by about 10 percentage points.
Kinship classification by modeling facial feature heredity

We propose a new, challenging, problem in kinship classification: recognizing the family that a query person belongs to from a set of families. We propose a novel framework for recognizing kinship by modeling this problem as that of reconstructing the query face from a mixture of parts from a set of families. To accomplish this, we reconstruct the query face from a sparse set of samples among the candidate families. Our sparse group reconstruction roughly models the biological process of inheritance: a child inherits genetic material from two parents, and therefore may not appear completely similar to either parent, but is instead a composite of the parents. The family classification is determined based on the reconstruction error for each family. On our newly collected “Family101” dataset, we discover links between familial traits among family members and achieve state-of-the-art family classification performance.
Unsupervised learning of hierarchical spatial structures in images

The visual world demonstrates organized spatial patterns, among objects or regions in a scene, object-parts in an object, and low-level features in object-parts. These classes of spatial structures are inherently hierarchical in nature. Although seemingly quite different these spatial patterns are simply manifestations of different levels in a hierarchy. In this work, we present a unified approach to unsupervised learning of hierarchical spatial structures from a collection of images. Ours is a hierarchical rule-based model capturing spatial patterns, where each rule is represented by a star-graph. We propose an unsupervised EM-style algorithm to learn our model from a collection of images. We show that the inference problem of determining the set of learnt rules instantiated in an image is equivalent to finding the minimum-cost Steiner tree in a directed acyclic graph. We evaluate our approach on a diverse set of data sets of object categories, natural outdoor scenes and images from complex street scenes with multiple objects.
Where do emotions come from? Predicting the Emotion Stimuli Map

Which parts of an image evoke emotions in an observer? To answer this question, we introduce a novel problem in computer vision - predicting an Emotion Stimuli Map (ESM), which describes pixel-wise contribution to evoked emotions. Building a new image database, EmotionROI, as a benchmark for predicting the ESM, we find that the regions selected by saliency and objectness detection do not correctly predict the image regions which evoke emotion. Although objects represent important regions for evoking emotion, parts of the background are also important. Based on this fact, we propose using fully convolutional networks for predicting the ESM. Both qualitative and quantitative experimental results confirm that our method can predict the regions which evoke emotion better than both saliency and objectness detection.
Exploring Tiny Images: The Roles of Appearance and Contextual Information for Machine and Human Object Recognition

Typically, object recognition is performed based solely on the appearance of the object. However, relevant information also exists in the scene surrounding the object. In this paper, we explore the roles that appearance and contextual information play in object recognition. Through machine experiments and human studies, we show that the importance of contextual information varies with the quality of the appearance information, such as an image's resolution. Our machine experiments explicitly model context between object categories through the use of relative location and relative scale, in addition to co-occurrence. With the use of our context model, our algorithm achieves state-of-the-art performance on the MSRC and Corel data sets. We perform recognition tests for machines and human subjects on low and high resolution images, which vary significantly in the amount of appearance information present, using just the object appearance information, the combination of appearance and context, as well as just context without object appearance information (blind recognition). We also explore the impact of the different sources of context (co-occurrence, relative-location, and relative-scale). We find that the importance of different types of contextual information varies significantly across data sets such as MSRC and PASCAL.
Image retrieval with geometry-preserving visual phrases

The most popular approach to large scale image retrieval is based on the bag-of-visual-word (BoV) representation of images. The spatial information is usually re-introduced as a post-processing step to re-rank the retrieved images, through a spatial verification like RANSAC. Since the spatial verification techniques are computationally expensive, they can be applied only to the top images in the initial ranking. In this paper, we propose an approach that can encode more spatial information into BoV representation and that is efficient enough to be applied to large-scale databases. Other works pursuing the same purpose have proposed exploring the word co-occurrences in the neighborhood areas. Our approach encodes more spatial information through the geometry-preserving visual phrases (GVP). In addition to co-occurrences, the GVP method also captures the local and long-range spatial layouts of the words. Our GVP based searching algorithm increases little memory usage or computational time compared to the BoV method. Moreover, we show that our approach can also be integrated to the min-hash method to improve its retrieval accuracy. The experiment results on Oxford 5K and Flicker 1M dataset show that our approach outperforms the BoV method even following a RANSAC verification.
Floating-point error analysis based on affine arithmetic

During the development of floating-point signal processing systems, an efficient error analysis method is needed to guarantee the output quality. We present a novel approach to floating-point error bound analysis based on affine arithmetic. The proposed method not only provides a tighter bound than the conventional approach, but also is applicable to any arithmetic operation. The error estimation accuracy is evaluated across several different applications which cover linear operations, nonlinear operations, and feedback systems. The accuracy decreases with the depth of computation path and also is affected by the linearity of the floating-point operations.
Face recognition through mismatch driven representations of the face

Performance of face verification systems can be adversely affected by a number of different mismatches (e.g. illumination, expression, alignment, etc.) between gallery and probe images. In this paper, we demonstrate that representations of the face used during the verification process should be driven by their sensitivity to these mismatches. Two representation categories of the face are proposed, parts and reflectance, each motivated by their own properties of invariance and sensitivity to different types of mismatches (i.e. spatial and spectral). We additionally demonstrate that the employment of the sum rule gives approximately equivalent performance to more exotic combination strategies based on support vector machine (SVM) classifiers, without the need for training on a tuning set. Improved performance is demonstrated, with a reduction in false reject rate of over 30% when compared to the single representation algorithm. Experiments were conducted on a subset of the challenging face recognition grand challenge (FRGC) v1.0 dataset.
Improving object color categorization with shapes

We explore the problem of object color categorization from natural images. Previous works use the histograms of RGB values of images with learning base methods. We propose to use shape information to help to localize the foreground areas of an image that determine the color of the object (such as car hoods), and focus the color learning and prediction on these areas. A novel Co-PLSA model is proposed to jointly learn the color and shape detectors in weakly supervised manner, where training images are only labeled with the color categories, while the locations of the foreground areas are not provided.
Highly Efficient Performance Portable Tracking of Evolving Surfaces

In this paper we present a framework to obtain highly efficient implementations for the narrow band level set method on commercial off-the-shelf (COTS) multicore CPU systems with a cache-based memory hierarchy such as Intel Xeon and Atom processors. The narrow-band level set algorithm tracks wave-fronts in discretized volumes (for instance, explosion shock waves), and is computationally very demanding. At the core of our optimization framework is a novel projection-based approach to enhance data locality and enable reuse for sparse surfaces in dense discretized volumes. The method reduces stencil operations on sparse and changing sets of pixels belonging to an evolving surface into dense stencil operations on meta-pixels in a lower-dimensional projection of the pixel space. These meta-pixels are then amenable to standard techniques like time tiling. However, the complexity introduced by ever-changing meta-pixels requires us to revisit and adapt all other necessary optimizations. We apply adapted versions of SIMDization, multi-threading, DAG scheduling for basic tiles, and specialization through code generation to extract maximum performance. The system is implemented as highly parameterized code skeleton that is auto-tuned and uses program generation. We evaluated our framework on a dual-socket 2.8 GHz Xeon 5560 and a 1.6 GHz Atom N270. Our single-core performance reaches 26%-35% of the machine peak on the Xeon, and 12%- 20% on the Atom across a range of image sizes. We see up to 6.5x speedup on 8 cores of the dual-socket Xeon. For cacheresident sizes our code outperforms the best available thirdparty code (C pre-compiled into a DLL) by about 10x and for the largest out-of-cache sizes the speedup approaches around 200x. Experiments fully explain the high speedup numbers.
Speech-assisted video processing: interpolation and low-bitrate coding

We utilize speech information to improve the quality of audio/visual communications, such as videotelephony, videoconferencing, and multimedia. In particular, marriage of speech processing and image processing can solve problems related to lip synchronization. Two main techniques proposed in this paper are: speech-assisted interpolation and speech-assisted coding of talking head video. Audio/video sequences are presented to demonstrate our techniques.<<ETX>>
An approach to region coding for content-based scalable video

We propose a scheme for coding sub-regions in video scenes to provide content-based scalable video. For each region, a special color is used to represent the non-object area, and the resulting frames are coded using conventional video coding algorithms. At the decoder, the region shape is recovered based on chroma keying, and hence content-based manipulations are made possible. A number of techniques that eliminate boundary artifacts common to region-based coding are presented. In this scheme, no explicit shape coding is needed, and the advantages of existing coding algorithms are retained. This scheme was submitted to ISO MPEG-4, and performed very well in the subjective tests.
Compression of medical volumetric data in a video-codec framework

We propose to use video-codec framework for the compression of volumetric medical images by treating interslice variations as "motion" of the tissues. As compression is necessary for telemedicine and archive, such an approach needs only slight modifications of existing codecs and is able to re-use most of the existing software and hardware. This is our main motivation and we seek modifications of the MPEG codec. An obvious modification is to replace block matching (BM) by more sophisticated methods. Specifically, we use the affine transformation (AT) on triangular patches. We implemented and compared our scheme with a simplified MPEG-1 encoder under identical parameters on a set of MRI images of a human hand. The blockiness is reduced dramatically and structures are better preserved. The images also appear less granular and show improvements of several dB in SNR. We expect further improvements when more features of MPEG codec are incorporated in our ongoing work.
A framework of changing image emotion using emotion prediction

Most works about affective image classification in computer vision treat each emotion category independently and predict hard labels, ignoring the correlation between emotion categories. In this work, inspired by psychological theories, we adopt a dimensional emotion model to model the correlation among certain emotion categories. We also propose a framework of changing image emotion by using our emotion predictor. Easily extendable to other feature transformations, our framework changes image emotion by color histogram specification, relaxing the limitation of the previous method that each emotion is associated with a monotonic palette. Effective and comparable to the previous work of changing image emotion shown by user study, our proposed framework provides users with more flexible control in changing image emotion compared with the previous work.
GPS Refinement and Camera Orientation Estimation from a Single Image and a 2D Map

A framework is presented for refining GPS location and estimate the camera orientation using a single urban building image, a 2D city map with building outlines, given a noisy GPS location. We propose to use tilt-invariant vertical building corner edges extracted from the building image. A location-orientation hypothesis, which we call an LOH, is a proposed map location from which an image of building corners would occur at the observed positions of corner edges in the photo. The noisy GPS location is refined and orientation is estimated using the computed LOHs. Experiments show the framework improves GPS accuracy significantly, generally produces reliable orientation estimation, and is computationally efficient.
Progressive browsing of 3D models

High-speed desktop computers, capable of rendering complex 3D models, are becoming more commonplace, resulting in increasing interest in virtual reality systems. Virtual Reality Modelling Language (VRML) has become the de-facto standard for the description of virtual worlds over the Internet, yet the current version (VRML97) lacks capabilities for progressive transfer. This paper describes the implementation of a browser that allows the progressive transfer and simultaneous viewing of 3D models, stored in a progressive format on a Web server, using a codec based on MPEG-4 verification model source code. The authors then implement their own codec that is progressive per-vertex rather than per-level. This approach also has the benefit that it does not alter the vertex coordinates. Finally a watermarking scheme, based on the second codec, is proposed.
Defocus-based image segmentation

Foreground and background features are focused (or defocused) differently in an image because corresponding objects are at different depths in the scene. This paper presents a novel approach for segmenting foreground and background in video images based on feature defocus. A modified defocus measurement that distinguishes between high-contrast defocused edges and low-contrast focused edges is presented. Defocus-based segmentation is desirable because defocus techniques are computationally simple. Results indicate that the foreground is easily segmented from moving background. This approach, coupled with motion detection, can segment complex scenes containing both moving background and stationary foreground.
Real-Time Pedestrian Detection using Eigenflow

We propose a novel learning algorithm to detect moving pedestrians from a stationary camera in real-time. The algorithm learns a discriminative model based on eigenflow, i.e., the eigenvectors derived from applying principal component analysis to the optical flow of moving objects, to differentiate between human motion patterns from other kind of motions like of cars etc. The learned model is a cascade of Adaboost classifiers of increasing complexity, with eigenflow vectors as the weak classifiers. Unlike some recent attempts to use motion for pedestrian detection, this system works in real-time. Moreover, the system is robust to small camera motion and slow illumination changes, and can detect moving children even though the training data had only adult pedestrians.
Video Streaming Over 802.11 WLAN with Content-Aware Adaptive Retry

Robust video streaming over error-prone wireless LANs (WLANs) poses many challenges. In this paper, we propose a timestamp-based content-aware adaptive retry (CAR) mechanism for MPEG video streaming over 802.11 WLANs, where the MAC dynamically determines whether to send or discard a packet based on its retransmission deadline. The retransmission deadline is assigned to each packet according to its temporal relationship and error propagation characteristics with respect to other video packets in the same GOP. The proposed scheme avoids late packets by eliminating the impact of random backoff deference and co-channel interference with proper initial delay introduced at the receiver. Simulation results show CAR significantly improves video quality and saves channel bandwidth
Pictorial structures for object recognition and part labeling in drawings

Although the sketch recognition and computer vision communities attempt to solve similar problems in different domains, the sketch recognition community has not utilized many of the advancements made in computer vision algorithms. In this paper we propose using a pictorial structure model for object detection, and modify it to better perform in a drawing setting as opposed to photographs. By using this model we are able to detect a learned object in a general drawing, and correctly label its parts. We show our results on 4 categories.
Using Group Prior to Identify People in Consumer Images

While face recognition techniques have rapidly advanced in the last few years, most of the work is in the domain of security applications. For consumer imaging applications, person recognition is an important tool that is useful for searching and retrieving images from a personal image collection. It has been shown that when recognizing a single person in an image, a maximum likelihood classifier requires the prior probability for each candidate individual. In this paper, we extend this idea and describe the benefits of using a group prior for identifying people in consumer images with multiple people. The group prior describes the probability of a group of individuals appearing together in an image. In our application, we have a subset of ambiguously labeled images for a consumer image collection, where we seek to identify all of the people in the collection. We describe a simple algorithm for resolving the ambiguous labels. We show that despite errors in resolving ambiguous labels, useful classifiers can be trained with the resolved labels. Recognition performance is further improved with a group prior learned from the ambiguous labels. In summary, by modeling the relationships between the people with the group prior, we improve classification performance.
Light weight multiview capturing

The framework of capturing multiview images has received a great attention from industry as well as academy due to its usability in various applications; 3D scene reconstruction, super-resolution, scene understanding, surveillance etc. This paper presents a multiview capturing system that uses a single camera with multiple mirrors. We place multiple mirrors pointing to the target scene and record an image of those mirrors. From this image of mirrors, we can extract the multiview images of the target scene, images reflected by mirrors. Based on the proposed method, we achieve some success in geometry reconstruction and view synthesis using multiview images, which shows the usability of the proposed work. The main contribution of this work is to introduce a light weight implementation for multiview capturing with a number of useful properties; the automatic synchronization across multiple views, the consistent color/white balance setting across multiple views, the capability of recording a wide range of view and the cost-effective implementation.
Pose invariant face recognition

We describe a novel neural network architecture, which can recognize human faces with any view in a certain viewing angle range (from left 30 degrees to right 30 degrees out of plane rotation). View-specific eigenface analysis is used as the front-end of the system to extract features, and the neural network ensemble is used for recognition. Experimental results show that the recognition accuracy of our network ensemble is higher than conventional methods such as using a single neural network to recognize faces of a specific view.
A Probabilistic Framework for Geometry Reconstruction using Prior Information

In this paper, we propose a probabilistic framework for reconstructing scene geometry utilizing prior knowledge of a class of scenes, for example, scenes captured by a camera mounted on a vehicle driving through city streets. In this framework, we assume the video camera is calibrated, i.e., the intrinsic and extrinsic parameters are known all the time. While we assume a single camera moving during capturing, the framework can be generalized to multiple cameras as well. Traditional approaches try to match the points, lines or patches in multiple images to reconstruct scene geometry. The proposed framework also takes advantage of each patch's appearance and location to infer its orientation using prior information based on statistical learning from training data. The prior hence enhances the geometry reconstruction performance. We show that prior-based 3D reconstruction outperforms traditional 3D reconstruction with both synthetic data and real data, especially in the textureless areas.