nils-herrmann/clean_scirepeval huggingface dataset:
The role of integer matrices in multidimensional multirate systems

Various theoretical issues in multidimensional (m-D) multirate signal processing are formulated and solved. In the problems considered, the decimation matrix and the expansion matrix are nondiagonal, so that extensions of 1-D results are nontrivial. The m-D polyphase implementation technique for rational sampling rate alterations, the perfect reconstruction properties for the m-D delay-chain systems, and the periodicity matrices of decimated m-D signals (both deterministic and statistical) are treated. The discussions are based on several key properties of integer matrices, including greatest common divisors and least common multiples. These properties are reviewed.
Example-Based Depth Generation from Single Image for 3D Content

2D plus depth is the most convenient way of representing 3D content. Unlike traditional methods of depth map generation which recovers precise geometry from multiple images, this paper presents an example-based method for automatically creating a layered 3D depth map from a single photograph. In our method, example-based background matching is used to identify the closest background and segment the foreground. Then, we do the foreground objects detection and project these objects into perspective using their location. In the end, we provide a thorough quantitative result of our algorithm on a set of real photos which confirms the benefits of our integrated approach. Therefore, a single image can be the content for 3D world.
Multimedia content classification using motion and audio information

Content-based video segmentation and classification is a key to the success of future multimedia databases. Research in this area in the past several years has focused on the use of speech recognition and image analysis techniques. As a complimentary effort to prior research, we have focused on the use of motion and audio characteristics. Fundamental to both segmentation and classification tasks is the characterization by certain features of a given video segment. In this paper, we describe several audio and motion features that have been found to be effective in distinguishing motion and audio characteristics of different types of scenes.
A mixed bag of emotions: Model, predict, and transfer emotion distributions

This paper explores two new aspects of photos and human emotions. First, we show through psychovisual studies that different people have different emotional reactions to the same image, which is a strong and novel departure from previous work that only records and predicts a single dominant emotion for each image. Our studies also show that the same person may have multiple emotional reactions to one image. Predicting emotions in “distributions” instead of a single dominant emotion is important for many applications. Second, we show not only that we can often change the evoked emotion of an image by adjusting color tone and texture related features but also that we can choose in which “emotional direction” this change occurs by selecting a target image. In addition, we present a new database, Emotion6, containing distributions of emotions.
Joint optimization of background subtraction and object detection for night surveillance

Detecting foreground objects for night surveillance videos remains a challenging problem in scene understanding. Though many efforts have been made for robust background subtraction and robust object detection respectively, the complex illumination condition in night scenes makes it hard to solve each of these tasks individually. In practice, we see these two tasks are coupled and can be combined to help each other. In this work, we apply a recently proposed algorithm — Feedback Enabled Cascaded Classification Models (FECCM) — to combine the background subtraction task and the object detection task into a generic framework. The proposed framework treats each classifier for the respective task as a ‘black-box’, thus allows the usage of most existing algorithms as one of the classifiers. Experiment results show that the proposed method outperforms a state-of-the-art background subtraction method and a state-of-the-art object detection method.
Scribble based interactive 3D reconstruction via scene co-segmentation

In this paper, we present a novel interactive 3D reconstruction algorithm which renders a planar reconstruction of the scene. We consider a scenario where the user has taken a few images of a scene from multiple poses. The goal is to obtain a dense and visually pleasing reconstruction of the scene, including non-planar objects. Using simple user interactions in the form of scribbles indicating the surfaces in the scene, we develop an idea of 3D scribbles to propagate scene geometry across multiple views and perform co-segmentation of all the images into the different surfaces and non-planar objects in the scene. We show that this allows us to render a complete and pleasing reconstruction of the scene along with a volumetric rendering of the non-planar objects. We demonstrate the effectiveness of our algorithm on both outdoor and indoor scenes including the ability to handle featureless surfaces.
Tracking of multiple faces for human-computer interfaces and virtual environments

Describes a real-time face-tracking algorithm. We start with single face tracking based on statistical color modeling and a deformable template. We then expand the algorithm to track multiple faces, possibly with occlusion, by constraining the speed and size changes of the faces. We test the algorithm on sequences with different occlusion patterns and analyze the tracking performance. We also present a tracking software library based on this algorithm. This library can be applied to human-computer interfaces, lip-reading and virtual environments.
Phase linearization of filters in analysis/synthesis filter banks

Analysis/synthesis filter banks with perfect reconstruction property have attracted much attention. In some applications, particularly image coding, it is desirable to design filter banks in which the filters have linear phase. We present a method of linearizing the passband phase response of filters in any given filter bank. While linearizing the phase response, this method preserves the magnitude response of the filters, the perfect reconstruction property, and the paraunitary property (orthonormality). We present the eigen-approach design and the anticausal implementation of this technique. Using this technique, we illustrate the importance of linear phase in subband coding.<<ETX>>
Coding of arbitrarily shaped video objects in MPEG-4

MPEG-4 Visual, that part of the upcoming MPEG-4 standard describing the coding of natural and synthetic video signals, allows the encoding of video objects using motion, texture and shape information. In this paper, the shape coding algorithms and related texture coding algorithms that were thoroughly evaluated within MPEG-4 are reviewed. The evaluation leading to the selection of a block-based shape coder based on a context-based arithmetic encoder/decoder with motion compensation as well as padding and shape adaptive DCT for texture coding are presented.
Joint spatial-temporal alignment of networked cameras

In this paper, we propose a method for aligning networked cameras spatially and temporally. Synchronizing video sequences and recovering spatial information among cameras are crucial steps for applications such as robust tracking and video mosaic. Without prior knowledge of internal and external parameters of cameras, we attempt to automatically estimate their spatial relationship and time offset which is possibly caused by network transmission delay. Our main focus is on cameras with overlapping field of views. Exploiting the fact that spatial and temporal information are related, we use one to boost the other. Initially assuming no time delay, the homography between cameras can be estimated by motion detection. Based on the homography, time difference can thus be recovered by analyzing activities in overlapping regions. We iteratively use spatial and temporal information to boost each other till reaching converging criteria. The algorithm can be extend to finding spatial and temporal relationship in multiple cameras. The experiment is performed in an outdoor parking lot and it is showed that our algorithm can successfully align cameras both in space and time.
Object color categorization in surveillance videos

We address the problem of automatic color categorization of the objects in surveillance videos. This problem is challenging for realistic situations due to the large intra-class variations of the same color and the large portions of noisy areas including the backgrounds and the parts of the objects that do not contribute to color assignments. We develop an integrated color categorization system with algorithms that address these challenges. With the algorithms proposed in this paper, we can improve the average color categorization accuracy by 18% from our previous work .
Audiovisual speech processing

We have reported activities in audiovisual speech processing, with emphasis on lip reading and lip synchronization. These research results have shown that, with lip reading, it is possible to enhance the reliability of audio speech recognition, which may result in a computer that can truly understand the user via hand-free natural spoken language even in a very noisy environments. Similarly, with lip synchronization, it is possible to render realistic talking heads with lip movements synchronized with the voice, which is very useful for human-computer interactions. We envision that in the near future, advancement in audiovisual speech processing will greatly increase the usability of computers. Once that happens, the cameras and the microphone may replace the keyboard and the mouse as better mechanisms for human-computer interaction.
Recent developments in multidimensional multirate systems

The authors review the fundamentals of multidimensional (M-D) multirate signal processing and present recent and new developments in M-D multirate systems. A method is presented for deriving all parallelepiped-shaped filters in M-D multirate applications from an appropriate 1-D filter. It is shown that the generalized-pseudocirculant property is necessary and sufficient for an M-D maximally decimated filter-bank system to be free from aliasing. It is shown how the Smith form, Smith-McMillan form, and the least common multiples of integer matrices can resolve many nonseparable M-D multirate problems. The condition for alias-free decimation and the multistage design of decimation systems are mentioned.
Facial Trait Code

We propose a facial trait code (FTC) to encode human facial images, and apply it to face recognition. Extracted from an exhaustive set of local patches cropped from a large stack of faces, the facial traits and the associated trait patterns can accurately capture the appearance of a given face. The extraction has two phases. The first phase is composed of clustering and boosting upon a training set of faces with neutral expression, even illumination, and frontal pose. The second phase focuses on the extraction of the facial trait patterns from the set of faces with variations in expression, illumination, and poses. To apply the FTC to face recognition, two types of codewords, hard and probabilistic, with different metrics for characterizing the facial trait patterns are proposed. The hard codeword offers a concise representation of a face, while the probabilistic codeword enables matching with better accuracy. Our experiments compare the proposed FTC to other algorithms on several public datasets, all showing promising results.
Principle component analysis and its variants for biometrics

Principle component analysis (PCA) has been widely used for analyzing the statistics of data. While applied to biometrics as a classification scheme, PCA faces certain challenges. We present a number of modifications to PCA in order to meet these challenges. Using face recognition as an example, we show how eigenflow, PCA applied to optimal flow, enables us to measure the difference between two images while allowing expression changes and registration error. We show how PCA can be updated to model time-varying statistics. We also show that PCA can be used to model the surface reflectance of human faces and reduce illumination variation that defeats most existing face recognition algorithms. Finally, we present distinguishing component analysis (DCA) and apply it to multimodal biometric authentication.
Using a Markov Network to Recognize People in Consumer Images

Markov networks are an effective tool for the difficult but important problem of recognizing people in consumer image collections. Given a small set of labeled faces, we seek to recognize the other faces in an image collection. The constraints of the problem are exploited when forming the Markov network edge potentials. Inference is also used to suggest faces for the user to label, minimizing the work on the part of the user. In one test set containing 4 individuals, an 86% recognition rate is achieved with only 3 labeled examples.
CHARQ: Cooperative Hybrid ARQ for wireless video streaming

Forward Error Correction (FEC) coding and Automatic Repeat reQuest (ARQ) are two commonly used techniques to tackle packet erasures in wireless video streaming. To preserve flexibility while reducing end-to-end latency, Hybird ARQ (HARQ) has been proposed as an alternative that combines the advantages of FEC and ARQ. This paper presents Cooperative Hybrid ARQ (CHARQ) that can further boost wireless video streaming quality. CHARQ takes advantage of the fact that, in the wireless environment, broadcast is free (from the sender's point of view) and that errors are location dependent so an intermediate proxy that overhears the source's transmission may transmit on behalf of the source to increase throughput efficiency. Simulation results show improved performance with the aid of cooperative proxies.
In silico vox: Towards speech recognition in silicon

This article consists of a collection of slides from the author's conference presentation on Carnegie Mellon University's In Silico Vox speech recognition in silicon Project. Some of the specific topics discussed include: current speech recognition technologies; the special features and voice commuication capabilities of the In Silico Project; graphics engines; next generation speech recognition applications; how speech recognition works; acoustical capabilities; layered speech models; software design and implementation; pipelining facilities; an FPGA demonstration and experimental results.
It's Not Polite to Point: Describing People with Uncertain Attributes

Visual attributes are powerful features for many different applications in computer vision such as object detection and scene recognition. Visual attributes present another application that has not been examined as rigorously: verbal communication from a computer to a human. Since many attributes are nameable, the computer is able to communicate these concepts through language. However, this is not a trivial task. Given a set of attributes, selecting a subset to be communicated is task dependent. Moreover, because attribute classifiers are noisy, it is important to find ways to deal with this uncertainty. We address the issue of communication by examining the task of composing an automatic description of a person in a group photo that distinguishes him from the others. We introduce an efficient, principled method for choosing which attributes are included in a short description to maximize the likelihood that a third party will correctly guess to which person the description refers. We compare our algorithm to computer baselines and human describers, and show the strength of our method in creating effective descriptions.
QUOTE: “Querying” Users as Oracles in Tag Engines a Semi-Supervised Learning Approach to Personalized Image Tagging

Previous work has correctly identified that many of the tags that users provide on images are not visually relevant, and subsequently assume such user generated tags are noise or irrelevant. Another assumption about user generated tags is that the order of these tags provides no useful information for the prediction of tags on future images. We challenge the aforementioned assumptions, and provide a machine learning approach to the problem of personalized image tagging with the following contributions: 1.) We reformulate the personalized image tagging problem as a search/retrieval ranking problem, 2.) We leverage the order of tags, which does not always reflect visual relevance, provided by the user in the past as a cue to their tag preferences, similar to click data, 3.) We propose a technique to augment sparse user tag data (semi-supervision), and 4.) We demonstrate the efficacy of our method on a subset of Flickr images, showing improvement over our previous methods.
An Empirical Study of Language CNN for Image Captioning

Language models based on recurrent neural networks have dominated recent image caption generation tasks. In this paper, we introduce a language CNN model which is suitable for statistical language modeling tasks and shows competitive performance in image captioning. In contrast to previous models which predict next word based on one previous word and hidden state, our language CNN is fed with all the previous words and can model the long-range dependencies in history words, which are critical for image captioning. The effectiveness of our approach is validated on two datasets: Flickr30K and MS COCO. Our extensive experimental results show that our method outperforms the vanilla recurrent neural network based language models and is competitive with the state-of-the-art methods.
Adaptive joint source-channel coding using rate shaping

We present in this paper an adaptive joint source-channel coding scheme using rate shaping on pre-coded video data. Rate shaping selectively drops portions of the video bitstream before transmitting them in order to satisfy the network bandwidth requirement. In wireless multimedia transport over heterogeneous networks, limited bandwidth is not the only issue. The high error rate of the channel should be considered as well, so channel coding is often applied. We propose a rate shaping method that drops not only the source-coding segments of the video bitstream, but also the channel-coding segments of the video bitstream, adaptively according to the network condition, in order to achieve the optimal rate-distortion performance. The proposed method is based on “discrete rate-distortion combination” to accomplish joint source-channel coding. We consider both the simulcast and multicast scenarios and show promising results.
Video transmission over wireless multihop networks using opportunistic routing

Wireless multihop networks comprise of mobile or stationary stations interconnected via an ad hoc multihop path. The dynamically self-organized and selfconfigured nature of such network provides a flexible alternative in a variety of contexts where the deployment of a fixed infrastructure may be impractical. While this type of network provides new opportunities in multimedia streaming, the high rate requirement coupled with stringent time constraints poses a certain range of challenges in deploying high quality video services. This paper presents an analytical model to study the performance of multihop video streaming using opportunistic routing, an emerging technique that achieves high throughput in the face of volatile wireless links. Specifically, we use a discrete-time Markov chain to model the expected number of transmissions needed for a successful delivery in a multihop network. This model produces a closed-form expression that is later validated by the Monte-Carlo simulation. Using the proposed model, we compare opportunistic routing with the traditional routing method which considers the best predetermined multihop path. We demonstrate performance gains in throughput, latency, as well as decoded video quality from adopting opportunistic routing. The problem of out-of-sequence delivery inherently resulted from opportunistic communication is also discussed.
A visual dictionary attack on Picture Passwords

Microsoft's Picture Password provides a method to authenticate a user without the need of typing a character based password. The password consists of a set of gestures drawn on an image. The position, direction and order of these gestures constitute the password. Besides being more convenient to use on touch screen devices, this authentication method promises improved memorability in addition to improving the password strength against guessing attacks. However, how unpredictable is the picture password? In this paper we exploit the fact that different users are drawn to similar image regions, and therefore these passwords are vulnerable to guessing attacks. More specifically, we show that for portrait pictures users are strongly drawn to use facial features as gesture locations. We collect a set of Picture Passwords and, using computer vision techniques, derive a list of password guesses in decreasing probability order. We show that guessing in this order we are able to improve the likelihood of cracking a password within a limited number of guesses.
Querying Users as Oracles in Tag Engines for Personalized Image Tagging

A common trend in image-tagging research is to focus on visually relevant tags; however, this tends to ignore the personal and social aspects of tags, especially on photo blogging websites such as Flickr. Previous work has correctly identified that many of the tags that users provide on images are not visually relevant (that is, representative of the salient content in the image) and they go on to treat such tags as noise, ignoring the fact that the users chose to provide those tags over others that could have been more visually relevant. Another common assumption about user-generated tags for images is that the order of these tags provides no useful information for the prediction of tags on future images. This assumption also tends to define usefulness in terms of what is visually relevant to the image. For general tagging or labeling applications that focus on providing visual information about image content, these assumptions are reasonable, but when considering personalized image-tagging applications, these assumptions are at best too rigid, ignoring user choice and preferences. This article challenges these assumptions and provides a machine learning approach to the problem of personalized image tagging. The authors reformulate the personalized image-tagging problem as a search/retrieval ranking problem and leverage the order of tags provided by the user in the past as a cue to tag preferences, similar to click data. They propose a technique to augment sparse user tag data (semisupervision) and demonstrate the efficacy of this method on a subset of Flickr images.
Video-Based Rendering using Feature Point Evolution

We propose a novel video-based rendering algorithm with a single moving camera. We reconstruct a dynamic 3D model of the scene with a feature point set that "evolves" over time. As the scene's appearance changes due to camera and object motions, some existing feature points dynamically disappear while some new feature points dynamically appear relative to the camera. The newly generated feature points' 3D positions and motions are initialized using nearby existing feature points' positions and motions. Our feature evolution, when incorporated into standard tracking and 3D reconstruction algorithms, provides for robust and dense 3D meshes, and their corresponding motions. Consequently, the evolution-based, time-dependent 3D meshes, motions, and textures render good-quality images at a virtual viewpoint and at a desired time instance. We also extend the proposed video-based rendering algorithm from using one single moving camera with one reconstructed depth map to using multiple moving cameras with multiple reconstructed depth maps to avoid occlusion and improve the rendering quality.