nils-herrmann/clean_scirepeval huggingface dataset:
Compression with mosaic prediction for image-based rendering applications

The authors present a mosaic based compression scheme for image based rendering applications in which a sequence of images is captured by a camera located at different positions along a circle. The mosaic image is constructed first and then used to predictively encode the original images. Furthermore, motion compensation is applied to provide a closer match at the block level between the prediction image and the original image. We examine this compression scheme and compare its performance with intra and inter coding using a general video codec. Our experimental result shows that mosaic based compression with motion compensation provides a better performance in the rate-distortion sense compared with intra coding, while at the same time it has the advantage of random access over inter coding.
In the Shadows, Shape Priors Shine: Using Occlusion to Improve Multi-region Segmentation

We present a new algorithm for multi-region segmentation of 2D images with objects that may partially occlude each other. Our algorithm is based on the observation that human performance on this task is based both on prior knowledge about plausible shapes and taking into account the presence of occluding objects whose shape is already known - once an occluded region is identified, the shape prior can be used to guess the shape of the missing part. We capture the former aspect using a deep learning model of shape, for the latter, we simultaneously minimize the energy of all regions and consider only unoccluded pixels for data agreement. Existing algorithms incorporating object shape priors consider every object separately in turn and can't distinguish genuine deviation from the expected shape from parts missing due to occlusion. We show that our method significantly improves on the performance of a representative algorithm, as evaluated on both preprocessed natural and synthetic images. Furthermore, on the synthetic images, we recover the ground truth segmentation with good accuracy.
Classification based mode decisions for video over network

A video encoder has to make many mode decisions in order to achieve the goal of a low bit rate, high quality, and fast implementation. We propose a general classification based approach to making such mode decisions accurately and efficiently. We first illustrate the approach using the Intra-Inter coding mode decision. We then focus on the decision to skip or code a frame for rate control of video over networks. Using the classification used approach we show improvement in the rate-distortion sense. We then extend the work to scalable video coding in choosing between scalability modes and examine the performance of our approach over error prone networks, using simulated packet losses.
Object discovery with perceptual grouping

We propose an iterative method for discovering objects in images. In each iteration, the current estimate of the layout is processed by a sequence of perceptual grouping rules. Perceptual grouping appears to be the basis of visual organization of human. It is concerned with the problem of the formation of wholes from parts. The method does not rely on the mixture of Gaussian model and hence avoids the model selection problem. We use synthetic and real images to demonstrate that the obtained result is better than that obtained by other methods.
Coding of subregions for content-based scalable video

We propose a scheme for coding subregions in video scenes to provide content-based scalable video. For each region, a special color is used to represent the nonobject area, and the resulting frames are coded using conventional video coding algorithms. At the decoder, the region shape is recovered based on chroma keying, and hence, content-based manipulations are made possible. A number of techniques that eliminate boundary artifacts common to region-based coding are presented. In this scheme, no explicit shape coding is needed, and advantages of existing coding algorithms are retained. This scheme was submitted to ISO MPEG-4 and performed very well in the subjective tests.
Real time stereo vision using exponential step cost aggregation on GPU

In this paper, we propose a local cost aggregation approach for real time stereo vision on a graphics processing unit (GPU). Recent research shows that local approaches based on carefully designed cost aggregation strategies can outperform many global approaches. Among those local aggregation approaches, adaptive-weight window produces the best quality disparity map under real-time constraint, but it is slower than other local approaches. We propose a very fast adaptive-weight aggregation method based on exponential step information propagation. The basic idea is to propagate information from long distance pixels within a few iterations. We also discuss important techniques of efficient implementation on GPU platform, which result in 10.5x speed up than a straightforward implementation. Compared to existing real time adaptive-weight approach, our technique reduces the computation time by more than half at improved accuracy. Detailed experimental results show that our technique is Pareto-optimal among existing real time or near real time stereo algorithms in the accuracy-speed trade-off space.
Results on multidimensional nonuniform rational maximally-decimated filter banks with orthonormal filters

The framework of inner product space is introduced and used to derive some known and new filter bank results. It is shown that even the most general filter banks, namely, multidimensional nonuniform filter banks with rational decimation matrices, become a special case of this framework. The subband convolution scheme is obtained by starting from Parseval's relation. Several theoretical wavelet transform results can be derived using this framework. In particular, the wavelet convolution theorem is derived.<<ETX>>
Data Fusion and Cost Minimization for Intrusion Detection

Statistical pattern recognition techniques have recently been shown to provide a finer balance between misdetections and false alarms than the more conventional intrusion detection approaches, namely misuse detection and anomaly detection. A variety of classical machine learning and pattern recognition algorithms has been applied to intrusion detection with varying levels of success. We make two observations about intrusion detection. One is that intrusion detection is significantly more effective by using multiple sources of information in an intelligent way, which is precisely what human experts rely on. Second, different errors in intrusion detection have different costs associated with them-a simplified example being that a false alarm may be more expensive than a misdetection and, hence, the true objective function to be minimized is the cost of errors and not the error rate itself. We present a pattern recognition approach that addresses both of these issues. It utilizes an ensemble of a classifiers approach to intelligently combine information from multiple sources and is explicitly tuned toward minimizing the cost of the errors as opposed to the error rate itself. The information fusion approach dLEARNIN alone is shown to achieve state-of-the-art performances better than those reported in the literature so far, and the cost minimization strategy dCMS further reduces the cost with a significant margin.
Fast, accurate static analysis for fixed-point finite-precision effects in DSP designs

Translating digital signal processing (DSP) software into its finite-precision hardware implementation is often a time-consuming task. We describe a new static analysis technique that can accurately analyze finite-precision effects arising from fixed-point implementations of DSP algorithms. The technique is based on recent interval representation methods from affine arithmetic, and the use of new probabilistic bounds. The resulting numerical error estimates are comparable to detailed statistical simulation, but achieve speedups of four to five orders of magnitude by avoiding actual bit-true simulation. We show error analysis results on both feed forward and feedback DSP kernels.
Clothing cosegmentation for recognizing people

Researches have verified that clothing provides information about the identity of the individual. To extract features from the clothing, the clothing region first must be localized or segmented in the image. At the same time, given multiple images of the same person wearing the same clothing, we expect to improve the effectiveness of clothing segmentation. Therefore, the identity recognition and clothing segmentation problems are inter-twined; a good solution for one aides in the solution for the other. We build on this idea by analyzing the mutual information between pixel locations near the face and the identity of the person to learn a global clothing mask. We segment the clothing region in each image using graph cuts based on a clothing model learned from one or multiple images believed to be the same person wearing the same clothing. We use facial features and clothing features to recognize individuals in other images. The results show that clothing segmentation provides a significant improvement in recognition accuracy for large image collections, and useful clothing masks are simultaneously produced. A further significant contribution is that we introduce a publicly available consumer image collection where each individual is identified. We hope this dataset allows the vision community to more easily compare results for tasks related to recognizing people in consumer image collections.
Object-driven image group annotation

In this paper, we propose a three-stage method for annotating image groups in real-world photo image databases: (1) Image databases are automatically divided into several image groups so that most photos in each group were taken in the same scene using the time information associated with each photo; (2) Objects in each image are recognized using multiclass object recognition; (3) Each image group is categorized into a scene using all object labels from (2) in the image group. Our main contribution is to propose a novel method for annotating image groups using all objects recognized from all images in this image group. We train our method on 6,000 objects in 696 images from the LabelMe dataset and verify the effectiveness of our proposed method on real-world photo databases consists of 4 outdoor scenes.
Space-Time Shapelets for Action Recognition

Recent works in action recognition have begun to treat actions as space-time volumes. This allows actions to be converted into 3-D shapes, thus converting the problem into that of volumetric matching. However, the special nature of the temporal dimension and the lack of intuitive volumetric features makes the problem both challenging and interesting. In a data-driven and bottom-up approach, we propose a dictionary of mid-level features called Space- Time Shapelets. This dictionary tries to characterize the space of local space-time shapes, or equivalently local motion patterns formed by the actions. Representing an action as a bag of these space-time patterns allows us to reduce the combinatorial space of these volumes, become robust to partial occlusions and errors in extracting spatial support. The proposed method is computationally efficient and achieves competitive results on a standard dataset.
Online Modeling and Tracking of Pose-Varying Faces in Video

We propose a face mosaicing approach to model both the facial appearance and geometry from pose-varying videos, and apply it in face tracking and recognition. The basic idea is that by approximating the human head as a 3D ellipsoid, multi-view face images can be back projected onto the surface of the ellipsoid, and the surface texture map is decomposed into an array of local patches. During the online modeling process, the position and pose of the first frame is assumed to be known for a given video sequence. For each frame in the sequence, the algorithm estimates the face-position and pose, and generates a texture map, which is further utilized in updating the mosaic model.
eMeeting: a multimedia application for interactive meeting and seminar

We present a client/server-based un-intrusive collaborative multimedia application - called eMeeting - that extracts and archives salient visual information from seminars, while allowing interaction among presenters and the audience in real time. The application uses APIs of the Lotus Sametime/spl reg/ system such as secure login, directory, meeting and multimedia services, to which we have added our own extensions. These extensions use the binary messaging channel of Sametime and allow us to add stroke-based chat as well as stroke-based annotations of slides. This permits communication among all clients. In parallel with the services just mentioned and in the same Java application, users of eMeeting are presented with slides that the system extracts from the presenter's visual material using an automatic recognition algorithm that classifies video frames as slides. At this time we assume that the system may or may not have access to the visual content in electronic form. Users can follow the seminar in the form of presenter's video, and as a series of extracted images. The server transmits the slides cyclically to each user using IP multicast giving each client the opportunity to select any previous slide. Thus, late joiners can browse slides that have been accumulated by the system.
Estimating age, gender, and identity using first name priors

Recognizing people in images is one of the foremost challenges in computer vision. It is important to remember that consumer photography has a highly social aspect. The photographer captures images not in a random fashion, but rather to remember or document meaningful events in her life. The culture of the society of which the photographer is a part provides a strong context for recognizing the content of the captured images. We demonstrate one aspect of this cultural context by recognizing people from first names. The distribution of first names chosen for newborn babies evolves with time and is gender-specific. As a result, a first name provides a strong prior for describing the individual. Specifically, we use the U.S. Social Security Administration baby name database to learn priors for gender and age for 6693 first names. Most face recognition methods do not even consider the name of the individual of interest, or the name is treated merely as an identifier that provides no information about appearance. In contrast, we combine image-based gender and age classifiers with the cultural context information provided by first names to recognize people with no labeled examples. Our model uses image-based age and gender estimates for assigning first names to people and in turn, the age and gender estimates are improved.
Hand tracking using spatial gesture modeling and visual feedback for a virtual DJ system

The ability to accurately track hand movement provides new opportunities for human computer interaction (HCI). Many of today's commercial hand tracking devices based on gloves can be cumbersome and expensive. An approach that avoids these problems is to use computer vision to capture hand motion. We present a complete real-time hand tracking and 3-D modeling system based on a single camera. In our system, we extract feature points from a video stream of a hand to control a virtual hand model with 2-D global motion and 3-D local motion. The on screen model gives the user instant feedback on the estimated position of the hand. This visual feedback allows a user to compensate for the errors in tracking. The system is used for three example applications. The first application uses hand tracking and gestures to take on the role of the mouse. The second interacts with a 3D virtual environment using the 3D hand model. The last application is a virtual DJ system that is controlled by hand motion tracking and gestures.
Hierarchical Semantics of Objects (hSOs)

We introduce hSOs: hierarchical semantics of objects. An hSO is learnt from a collection of images taken from a particular scene category. The hSO captures the interactions between the objects that tend to co-occur in the scene, and hence are potentially semantically related. Such relationships are typically hierarchical. For example, in a collection of images taken in a living room scene, the TV, DVD player and coffee-table co-occur frequently. The TV and the DVD player are more closely related to each other than the coffee table, and this can be learnt from the fact that the two are located at similar relative locations across images, while the coffee table is somewhat arbitrarily placed. The goal of this paper is to learn this hierarchy that characterizes the scene. The proposed approach, being entirely unsupervised, can detect the parts of the images that belong to the foreground objects, cluster these parts to represent objects, and provide an understanding of the scene by hierarchically clustering these objects in a semantically meaningful way - all from a collection of unlabeled images of a particular scene category. In addition to providing the semantic layout of the scene, learnt hSOs can have several useful applications such as compact scene representation for scene category classification and providing context for enhanced object detection.
Floating-point bit-width optimization for low-power signal processing applications

To enable floating-point (FP) signal processing applications in low-power mobile devices, we propose a lightweight FP design flow that can optimize the bit-width configuration. The optimization considers both the hardware cost and the numerical precision. Variable grouping is used to reduce the complexity of optimization by connecting software description and hardware implementation. The optimization algorithm is able to avoid local optima, and multiple-phase optimization helps to reduce the cost further. We apply the proposed design flow to the design of inverse discrete cosine transform (IDCT), and show that the power consumption of our lightweight FP IDCT is comparable to an optimized fixed-point design. In addition, promising results on some real-world applications such as video coding and speech recognition demonstrate that lightweight FP signal processing will find more and more applications in low-power devices.
Finding commutative multidimensional downsamplers and upsamplers

The commutativity of multidimensional downsamplers and upsamplers has been discussed very intensively for the past few years. This is due to its importance in sampling structure conversion, e.g., the conversion between conventional television signals and high definition television (HDTV) signals. Among many other results, a test for such commutativity was found to be that the two matrices that define the multidimensional downsampling and upsampling should be commutative and coprime. However, the problem of finding multidimensional downsamplers and upsamplers that satisfy these conditions has remained open. We develop a systematic procedure to solve this open problem.
Motion-focusing key frame extraction and video summarization for lane surveillance system

This paper proposes a motion-focusing method to extract key frames and generate summarization synchronously for surveillance videos. Within each pre-segmented video shot, the proposed method focuses on one constant-speed motion and aligns the video frames by fixing this focused motion into a static situation. According to the relative motion theory, the other objects in the video are moving relatively to the selected kind of motion. This method finally generates a summary image containing all moving objects and embedded with spatial and motional information, together with key frames to provide details corresponding to the regions of interest in the summary image. We apply this method to the lane surveillance system and the results provide us a new way to understand the video efficiently.
From Low-Level Features to High-Level Semantics: Are We Bridging the Gap?

The performance of a content-based information retrieval (CBIR) system is very subjective and hence user-dependent. To the user, similarity between objects in the database is often high-level and semantic. However, features extracted from objects directly in their digital representations are often low-level features. The gap between low-level features and high-level semantics has been the major obstacle to better retrieval performance. In this talk, we outline several approaches to bridging the gap between low-level features and high-level semantics, including hidden annotation and relevance feedback. We present a few specific techniques: active learning, annotation propagation, feature space warping, and semantic metric linking, all aiming at propagating the semantics from some objects to the others.
Defending On-Line Web Application Security with User-Behavior Surveillance

With the incoming of information era, web-based service has been developed rapidly and offered more and more business. These "open", and widely "web enabled" applications are subject to greater and greater levels and types of attacks as hackers exploit vulnerabilities within the software like SQL injection and cross site scripts (XSS) attack. In this paper, we proposed a type of novel embedded Markov model (EMM) to detect different web application attacks, monitor the on-line user behavior and defend the malevolent user promptly. Comparing to previous web application attacks detecting approaches, our EMM approach can not only detect user's invalidated input errors but also find out the unreasonable page transition behavior. By detecting unreasonable page transition, we can immediately defend the malevolent or silly user behavior to avoid the further web system failures and sensitive information disclosure. Furthermore, we implement an on-line user behavior surveillance system and use the real web traffic to evaluate the performance of our system. The experiment results show that our proposed EMM method can discover the abnormal behavior of malevolent user and detect the invalidated input attacks like SQL injection, XSS and string buffer overflow attacks.
DISCOV: A Framework for Discovering Objects in Video

This paper presents a probabilistic framework for discovering objects in video. The video can switch between different shots, the unknown objects can leave or enter the scene at multiple times, and the background can be cluttered. The framework consists of an appearance model and a motion model. The appearance model exploits the consistency of object parts in appearance across frames. We use maximally stable extremal regions as observations in the model and hence provide robustness to object variations in scale, lighting and viewpoint. The appearance model provides location and scale estimates of the unknown objects through a compact probabilistic representation. The compact representation contains knowledge of the scene at the object level, thus allowing us to augment it with motion information using a motion model. This framework can be applied to a wide range of different videos and object types, and provides a basis for higher level video content analysis tasks. We present applications of video object discovery to video content analysis problems such as video segmentation and threading, and demonstrate superior performance to methods that exploit global image statistics and frequent itemset data mining techniques.
Background Cutout with Automatic Object Discovery

We present a novel approach to background cutout for image editing. We show how background cutout can be achieved without any user labeling. This is in contrast to current methods, where the user needs to label each image separately. Our method uses automatic object discovery methods to provide location and scale estimates of the object of interest; these estimates then provide seeds for initializing color distributions of a segmentation algorithm. We show that our approach can achieve similar performance as traditional methods that require users to specify for each image a bounding box of the target object.