nils-herrmann/clean_scirepeval huggingface dataset:
Learning class-specific affinities for image labelling

Spectral clustering and eigenvector-based methods have become increasingly popular in segmentation and recognition. Although the choice of the pairwise similarity metric (or affinities) greatly influences the quality of the results, this choice is typically specified outside the learning framework. In this paper, we present an algorithm to learn class-specific similarity functions. Mapping our problem in a conditional random fields (CRF) framework enables us to pose the task of learning affinities as parameter learning in undirected graphical models. There are two significant advances over previous work. First, we learn the affinity between a pair of data-points as a function of a pairwise feature and (in contrast with previous approaches) the classes to which these two data-points were mapped, allowing us to work with a richer class of affinities. Second, our formulation provides a principled probabilistic interpretation for learning all of the parameters that define these affinities. Using ground truth segmentations and labellings for training, we learn the parameters with the greatest discriminative power (in an MLE sense) on the training data. We demonstrate the power of this learning algorithm in the setting of joint segmentation and recognition of object classes. Specifically, even with very simple appearance features, the proposed method achieves state-of-the-art performance on standard datasets.
Joint audio-video processing for multimedia

In this paper, the authors report recent developments in the research of joint audio-visual processing for multimedia applications. These include: bimodality in speech production and perception, automatic lipreading, talking-head animation and lip synchronization. They present in detail the enabling technologies for these applications. A new trend of research is to utilize audio-visual interaction in the coding of talking head video. They show that the marriage of speech analysis and image processing can create a number of new research opportunities.
Networked Intelligent Collaborative Environment (NetICE)

The paper presents NetICE, Networked Intelligent Collaborative Environment, which is a prototype designated to provide an immersive environment to allow people to communicate from anywhere at any time. The NetICE system consists of two components: an intelligent network and smart terminals. The intelligent network allows multiple users to be connected to the same virtual space. The smart terminals provide an immersive environment to the end users by using audio-visual communication and 3D computer graphics animation techniques.
You are Here: Mimicking the Human Thinking Process in Reading Floor-Plans

A human can easily find his or her way in an unfamiliar building, by walking around and reading the floor-plan. We try to mimic and automate this human thinking process. More precisely, we introduce a new and useful task of locating an user in the floor-plan, by using only a camera and a floor-plan without any other prior information. We address the problem with a novel matching-localization algorithm that is inspired by human logic. We demonstrate through experiments that our method outperforms state-of-the-art floor-plan-based localization methods by a large margin, while also being highly efficient for real-time applications.
Immerse whiteboards in a networked collaborative environment

An immersive whiteboard system is presented in a networked collaborative environment where users at multiple locations can communicate with each other. The immersive system consists of five major components: the network, the virtual environment, the avatar representation, the multimedia objects, and the shared whiteboards. The virtual environment is shared among multiple networked users and allows each user to interact independently with the world. The avatar is used to represent each user in the immersive environment and conveys its user identity and facial expressions to the world. The shared whiteboards provide a medium for communication using handwritten inputs, as well as a desirable interface for navigation and mouse control. The goal of this immersive system is to provide a convenient environment for participants to interact with each other. The system is suitable for distance learning and conferencing applications because it uses the whiteboard interface and provides a user avatar representation for any virtual environment.
Least common right/left multiples of integer matrices and applications to multidimensional multirate systems

The basic building blocks in a multidimensional (MD) multirate system are the decimation matrix and the expansion matrix. These matrices are D*D nonsingular integer matrices, where D is the number of dimensions. The authors show that properties of integer matrices, such as greatest common right/left divisors and right/left coprimeness play important roles in MD multirate systems. They also introduce the concept of least common right/left multiple of integer matrices and derive many useful properties of them. They illustrate the importance of these by applying them to several issues in MD multirate signal processing, including interchangeability of decimators and expanders, delay-chain systems, and periodicity matrices.<<ETX>>
Cross-layer features in convolutional neural networks for generic classification tasks

Recent works about convolutional neural networks (CNN) show breakthrough performance on various tasks. However, most of them only use the features extracted from the topmost layer of CNN instead of leveraging the features extracted from different layers. As the first group which explicitly addresses utilizing the features from different layers of CNN, we propose cross-layer CNN features which consist of the features extracted from multiple layers of CNN. Our experimental results show that our proposed cross-layer CNN features outperform not only the state-of-the-art results but also the features commonly used in the traditional CNN framework on three tasks - artistic style, artist, and architectural style classification. As shown by the experimental results, our proposed cross-layer CNN features achieve the best known performance on the three tasks in different domains, which makes our proposed cross-layer CNN features promising solutions for generic tasks.
Who ordered this?: Exploiting implicit user tag order preferences for personalized image tagging

What makes a person pick certain tags over others when tagging an image? Does the order that a person presents tags for a given image follow an implicit bias that is personal? Can these biases be used to improve existing automated image tagging systems? We show that tag ordering, which has been largely overlooked by the image tagging community, is an important cue in understanding user tagging behavior and can be used to improve auto-tagging systems. Inspired by the assumption that people order their tags, we propose a new way of measuring tag preferences, and also propose a new personalized tagging objective function that explicitly considers a user's preferred tag orderings. We also provide a (partially) greedy algorithm that produces good solutions to our new objective and under certain conditions produces an optimal solution. We validate our method on a subset of Flickr images that spans 5000 users, over 5200 tags, and over 90,000 images. Our experiments show that exploiting personalized tag orders improves the average performance of state-of-art approaches both on per-image and per-user bases.
Role of anticausal inverses in multirate filter-banks .II. The FIR case, factorizations, and biorthogonal lapped transforms

For pt. I see ibid., vol.43, no.5, p.1090, 1990. In part I we studied the system-theoretic properties of discrete time transfer matrices in the context of inversion, and classified them according to the types of inverses they had. In particular, we outlined the role of causal FIR matrices with anticausal FIR inverses (abbreviated cafacafi) in the characterization of FIR perfect reconstruction (PR) filter banks. Essentially all FIR PR filter banks can be characterized by causal FIR polyphase matrices having anticausal FIR inverses. In this paper, we introduce the most general degree-one cafacafi building block, and consider the problem of factorizing cafacafi systems into these building blocks. Factorizability conditions are developed. A special class of cafacafi systems called the biorthogonal lapped transform (BOLT) is developed, and shown to be factorizable. This is a generalization of the well-known lapped orthogonal transform (LOT). Examples of unfactorizable cafacafi systems are also demonstrated. Finally it is shown that any causal FIR matrix with FIR inverse can be written as a product of a factorizable cafacafi system and a unimodular matrix.
Feature-based Part Retrieval for Interactive 3D Reassembly

We propose a novel framework for 3D reassembly, the task of assembling a solid object from its broken pieces. The primary challenge in this under-explored problem is to robustly establish compatibility between parts from one object. Feature-based techniques have shown success in domains such as 3D similarity search; unfortunately, the global features typically employed to quantify whole-object similarity are unsuitable for identifying part-level compatibility. Therefore, we propose the use of local features which, in conjunction with robust matching, have become popular for object recognition in 2D images. This paper demonstrates that an analogous framework can be successful for 3D reassembly. Automating part-level compatibility enables the construction of an interactive system for 3D reassembly, where the user can easily assemble a desired object from a large collection of pieces (many of which are irrelevant) by iteratively selecting compatible parts. We evaluate our approach on a simulated database of broken objects and show that it scales well in the presence of noise and extraneous pieces
Reconstructing Dense Light Field From Array of Multifocus Images for Novel View Synthesis

This paper presents a novel method for synthesizing a novel view from two sets of differently focused images taken by an aperture camera array for a scene consisting of two approximately constant depths. The proposed method consists of two steps. The first step is a view interpolation to reconstruct an all-in-focus dense light field of the scene. The second step is to synthesize a novel view by a light-field rendering technique from the reconstructed dense light field. The view interpolation in the first step can be achieved simply by linear filters that are designed to shift different object regions separately, without region segmentation. The proposed method can effectively create a dense array of pin-hole cameras (i.e., all-in-focus images), so that the novel view can be synthesized with better quality
Content-Free Image Retrieval using Bayesian Product Rule

Content-free image retrieval uses accumulated user feedback records to retrieve images without analyzing image pixels. We present a Bayesian-based algorithm to analyze user feedback and show that it outperforms a recent maximum entropy content-free algorithm, according to extensive experiments on trademark logo and 3D model datasets. The proposed algorithm also has the advantage of being applicable to both content-free and traditional content-based image retrieval, thus providing a common framework for these two paradigms
Aesthetic quality assessment of consumer photos with faces

Automatically assessing the subjective quality of a photo is a challenging area in visual computing. Previous works study the aesthetic quality assessment on a general set of photos regardless of the photo's content and mainly use features extracted from the entire image. In this work, we focus on a specific genre of photos: consumer photos with faces. This group of photos constitutes an important part of consumer photo collections. We first conduct an online study on Mechanical Turk to collect ground-truth and subjective opinions for a database of consumer photos with faces. We then extract technical features, perceptual features, and social relationship features to represent the aesthetic quality of a photo, by focusing on face-related regions. Experiments show that our features perform well for categorizing or predicting the aesthetic quality.
Unified eigenfilter approach: with applications to spectral/spatial filtering

The eigenfilter approach is extended to solve general least-squares approximation problems with linear constraints. Such extension unifies previous work in eigenfilters and many other filter design problems, including spectral/spatial filtering, one-dimensional or multidimensional filters, data independent or statistically optimal filtering, etc. With this approach, various filter design problems are transformed into problems of finding an eigenvector of a positive definite matrix that is determined by filter design specifications. This approach has the advantage that many filter design constraints can be incorporated easily. A number of design examples are presented to show the usefulness and flexibility of the proposed approach.<<ETX>>
Prior-based vanishing point estimation through global perspective structure matching

In this paper, we describe a prior-based vanishing point estimation method through global perspective structure matching (GPSM). In contrast to the traditional approaches which require an undistorted image with straight roads for vanishing point estimation, our method first infers vanishing point candidates of an input image from an image database with pre-labeled vanishing points. An image-based retrieval method is used to identify the best candidate images in the database by matching image's perspective structure. The initial estimation of input image's vanishing point is calculated from the pre-labeled vanishing points of the best candidates. Probabilistic refinement (PR) is then used to optimize the vanishing point estimate. Experimental results show that the proposed method works well in a variety of on-road driving environments (e.g., in urban, highway and country-side areas), especially with traffic captured by a fish-eye back-aid camera.
Time-reversed inversion for time-varying filter banks

For an analysis/synthesis filter bank to achieve perfect reconstruction, the synthesis polyphase matrix should be equal to an inverse of the analysis polyphase matrix E(z). Therefore, the problem of perfect reconstruction filter banks is same as the inversion of the multi-input multi-output transfer function E(z). Using state-space notations, it has been shown that the inversion can be achieved by using time-reversed filters given proper initial conditions. In this paper, we extend the idea of time-reversed inversion to the case of time-varying filter banks. Using the state-space framework, we show perfect reconstruction is always guaranteed, no matter how often the filter bank varies with time. This framework covers both maximally-decimated filter banks and under-decimated ones. We also show how the overhead of transmitting initial conditions can be avoided.<<ETX>>
Generalized optimal thresholding for biometric key generation using face images

In this paper, we study biometric key generation using face images. Given a face image, a set of biometric features are extracted. Each feature is compared with a threshold to generate a key-bit. By cascading the key-bits from all the features, we obtain one bio-key that can be used for security applications. The performance of a biometric key generation system, determined by the chosen thresholds, can be evaluated according to reliability and security. A generalized optimal thresholding method is proposed in this paper to improve the reliability by minimizing the authentication error rate and the security by maximizing the entropy of the generated key.
Accurate self-calibration of two cameras by observations of a moving person on a ground plane

A calibration algorithm of two cameras using observations of a moving person is presented. Similar methods have been proposed for self-calibration with a single camera, but internal parameter estimation is only limited to the focal length. Recently it has been demonstrated that principal point supposed in the center of the image causes inaccuracy of all estimated parameters. Our method exploits two cameras, using image points of head and foot locations of a moving person, to determine for both cameras the focal length and the principal point. Moreover with the increasing number of cameras there is a demand of procedures to determine their relative placements. In this paper we also describe a method to find the relative position and orientation of two cameras: the rotation matrix and the translation vector which describe the rigid motion between the coordinate frames fixed in two cameras. Results in synthetic and real scenes are presented to evaluate the performance of the proposed method.
Dense interpolation of 3D points based on surface and color

A laser scan is useful in building the 3D model, and in one run it can capture thousands of 3D points. However these 3D points are sparse compared to a normal image, which can easily have millions of pixels. To achieve a denser 3D map, 3D-interpolation is applied to each pixel in the image. In this work we propose an algorithm to combine the 3D geometry and the color for 3D-interpolation. We segment the 3D points based on their latent surfaces, and combine the surfaces with color through Markov Random Field. We find that the 3D geometry provides rich information for interpolation: 3D points with similar colors can be robustly clustered where not possible in the color space, and the interpolation can be performed on a better fitting surface rather than on the locally linear ones. Our experiments show that the proposed algorithm outperforms the baselines.
Watermarking 2D/3D graphics for copyright protection

In recent years, the importance of 2D maps and 3D models has grown significantly, especially due to the popularity of map data over the Web and computer graphics used in entertainment and e-commerce applications. We review the concept of image watermarking, and extend it to 2D/3D vector data. We present a number of watermarking algorithms for 2D electronic maps and 3D wire-frame models. We show that invisible watermarks can be added to 2D/3D data and detected reliably. The watermarks can be designed to be robust to quantization of the vector data and several other hostile attacks.
VLSI design and implementation fuels the signal-processing revolution

The article provides a comprehensive overview of the history of how signal-processing researchers have been effectively transforming signal-processing algorithms into efficient implementations. Starting from the early days of analog circuits for signal processing, to digital signal processors (DSPs), to application specific DSPs and programmable DSPs, and to the trend of integrating a complete system on a single chip, this article provides a thorough coverage of the past and the present of design and implementation technology for signal processing systems. Moreover, it presents the exciting challenges faced by the study of the design and implementation of current and future signal processing applications. Topics covered include milestones in signal-processing integrated circuits, the past and future of the signal processor, signal processing in consumer applications, and design automation for signal processing.
A journey from signal processing to surveillance

Traditionally, signal processing is considered simply low-level processing. In the past decade, however, signal processing has grown to become the area where a variety of tools are created to solve high-level problems that conventionally would be studied by computer vision or machine learning researchers exclusively. For example, multiresolution analysis created popular image features like SIFT (scale-invariant feature transform), and statistical analysis gave birth to graphical models such as HMM (hidden Markov models) and topic models. In this talk, we will use one application to illustrate this growth of signal processing: object discovery, i.e., extracting the "object of interest" from a set of images in a completely unsupervised manner. Often based on image features like SIFT, and the topic models, object discovery has recently attracted a lot of attention in video content extraction. In this talk, we will outline this approach and extend it from still images to motion videos. We will propose a novel spatial-temporal framework that applies statistical models to both appearance modeling and motion modeling. The spatial and temporal models are integrated so that motion ambiguities can be resolved by appearance, and appearance ambiguities can be resolved by motion. In addition, we can extract hierarchical relationships among objects, completely driven by data without any manual labeling. This framework finds application in video retrieval (e.g., for YouTube or Google Video) and video surveillance.
Virtual view synthesis through linear processing without geometry

This paper presents a new approach for virtual view synthesis that does not require any information of scene geometry. Our approach first generates multiple virtual views at the same position based on multiple depths by the conventional view interpolation method. The interpolated views suffer from blurring and ghosting artifacts due to the pixel mis-correspondence. Secondly, the multiple views are integrated into a novel view where all regions are focused. This integration problem can be formulated as the problem of solving a set of linear equations that relates the multiple views. To solve this set of equations, two methods using projection onto convex sets (POCS) and inverse filtering are presented that effectively integrate the focused regions in each view into a novel view. Experimental results using real images show the validity of our methods.
Audio-visual interaction in multimedia communication

To many people, the word "multimedia" simply means the combination of various forms of information: text, speech, music, images, graphics and video. What is often overlooked is the interaction among these forms. In this paper, we present our results in exploiting the audio-visual interaction that is very significant in multimedia communication. The applications include lip synchronization, joint audio-video coding, and person verification. We present the enabling technologies, including audio-to-visual mapping and facial image analysis, for these applications. Our results show that the joint processing of audio and video provides advantages that are not available when audio and video are studied separately.
Efficient Kernels for identifying unbounded-order spatial features

Higher order spatial features, such as doublets or triplets have been used to incorporate spatial information into the bag-of-local-features model. Due to computational limits, researchers have only been using features up to the 3rd order, i.e., triplets, since the number of features increases exponentially with the order. We propose an algorithm for identifying high-order spatial features efficiently. The algorithm directly evaluates the inner product of the feature vectors from two images to be compared, identifying all high-order features automatically. The algorithm hence serves as a kernel for any kernel-based learning algorithms. The algorithm is based on the idea that if a high-order spatial feature co-occurs in both images, the occurrence of the feature in one image would be a translation from the occurrence of the same feature in the other image. This enables us to compute the kernel in time that is linear to the number of local features in an image (same as the bag of local features approach), regardless of the order. Therefore, our algorithm does not limit the upper bound of the order as in previous work. The experiment results on the object categorization task show that high order features can be calculated efficiently and provide significant improvement in object categorization performance.