OpenRaiser/Intern-Atlas huggingface dataset:
Explaining the efficacy of counterfactually augmented data
Authors: Divyansh Kaushik, Amrith Setlur, Eduard H. Hovy, Zachary Chase Lipton
Venue: Conference ICLR 2021
Year: 2021

In attempts to produce ML models less reliant on spurious patterns in NLP datasets, researchers have recently proposed curating counterfactually augmented data (CAD) via a human-in-the-loop process in which given some documents and their (initial) labels, humans must revise the text to make a counterfactual label applicable. Importantly, edits that are not necessary to flip the applicable label are prohibited. Models trained on the augmented data appear, empirically, to rely less on semantically irrelevant words and to generalize better out of domain. While this work draws loosely on causal thinking, the underlying causal model (even at an abstract level) and the principles underlying the observed out-of-domain improvements remain unclear. In this paper, we introduce a toy analog based on linear Gaussian models, observing interesting relationships between causal models, measurement noise, out-of-domain generalization, and reliance on spurious signals. Our analysis provides some insights that help to explain the efficacy of CAD. Moreover, we develop the hypothesis that while adding noise to causal features should degrade both in-domain and out-of-domain performance, adding noise to non-causal features should lead to relative improvements in out-of-domain performance. This idea inspires a speculative test for determining whether a feature attribution technique has identified the causal spans. If adding noise (e.g., by random word flips) to the highlighted spans degrades both in-domain and out-of-domain performance on a battery of challenge datasets, but adding noise to the complement gives improvements out-of-domain, it suggests we have identified causal spans. We present a large-scale empirical study comparing spans edited to create CAD to those selected by attention and saliency maps. Across numerous domains and models, we find that the hypothesized phenomenon is pronounced for CAD.
Can a deep learning model be a sure bet for tabular prediction?
Authors: Jintai Chen, Jiahuan Yan, Qiyuan Chen 0003, Danny Z. Chen, Jian Wu 0001, Jimeng Sun 0001
Venue: Conference KDD 2024
Year: 2024

Data organized in tabular format is ubiquitous in real-world applications, and users often craft tables with biased feature definitions and flexibly set prediction targets of their interests. Thus, a rapid development of a robust, effective, dataset-versatile, user-friendly tabular prediction approach is highly desired. While Gradient Boosting Decision Trees (GBDTs) and existing deep neural networks (DNNs) have been extensively utilized by professional users, they present several challenges for casual users, particularly: (i) the dilemma of model selection due to their different dataset preferences, and (ii) the need for heavy hyperparameter searching, failing which their performances are deemed inadequate. In this paper, we delve into this question: Can we develop a deep learning model that serves as a sure bet solution for a wide range of tabular prediction tasks, while also being user-friendly for casual users? We delve into three key drawbacks of deep tabular models, encompassing: (P1) lack of rotational variance property, (P2) large data demand, and (P3) over-smooth solution. We propose ExcelFormer, addressing these challenges through a semi-permeable attention module that effectively constrains the influence of less informative features to break the DNNs' rotational invariance property (for P1), data augmentation approaches tailored for tabular data (for P2), and attentive feedforward network to boost the model fitting capability (for P3). These designs collectively make ExcelFormer a sure bet solution for diverse tabular datasets. Extensive and stratified experiments conducted on real-world datasets demonstrate that our model outperforms previous approaches across diverse tabular data prediction tasks, and this framework can be friendly to casual users, offering ease of use without the heavy hyperparameter tuning. The codes are available at https://github.com/whatashot/excelformer.
Scalable Discriminant Feature Selection for Image Retrieval and Recognition
Authors: Nuno Vasconcelos, Manuela Vasconcelos
Venue: Conference CVPR 2004
Year: 2004

Problems such as object recognition or image retrieval require feature selection (FS) algorithms that scale well enough to be applicable to databases containing large numbers of image classes and large amounts of data per class. We exploit recent connections between information theoretic feature selection and minimum Bayes error solutions to derive FS algorithms that are optimal in a discriminant sense without compromising scalability. We start by formalizing the intuition that optimal FS must favor discriminant features while penalizing discriminant features that are redundant. We then rely on this result to derive a new family of FS algorithms that enables an explicit trade-off between complexity and classification optimality. This trade-off is controlled by a parameter that encodes the order of feature redundancies that must be explicitly modeled to achieve the optimal solution. Experimental results on databases of natural images show that this order is usually low, enabling optimal FS with very low complexity.
Visual odometry and map correlation
Authors: Anat Levin, Richard Szeliski
Venue: Conference CVPR 2004
Year: 2004

In this paper we study how estimates of ego-motion based on feature tracking (visual odometry) can be improved using a rough (low accuracy) map of where the observer has been. We call the process of aligning the visual ego-motion with the map locations as map correlation. Since absolute estimates of camera position are unreliable, we use stable local information such as change in orientation to perform the alignment. We also detect when the observer's path has crossed back on itself which helps improve both the visual odometry estimates and the alignment between the video and map sequences. The final alignment is computed using a graphical model whose MAP estimate is inferred using loopy belief propagation. Results are presented on a number of indoor and outdoor sequences.
Probabilistic occlusion boundary detection on spatio-temporal lattices
Authors: Mehmet Emre Sargin, Luca Bertelli, Bangalore S. Manjunath, Kenneth Rose
Venue: Conference ICCV 2009
Year: 2009

In this paper, we present an algorithm for occlusion boundary detection. The main contribution is a probabilistic detection framework defined on spatio-temporal lattices, which enables joint analysis of image frames. For this purpose, we introduce two complementary cost functions for creating the spatio-temporal lattice and for performing global inference of the occlusion boundaries, respectively. In addition, a novel combination of low-level occlusion features is discriminatively learnt in the detection framework. Simulations on the CMU Motion Dataset provide ample evidence that proposed algorithm outperforms the leading existing methods.
Vector quantizing feature space with a regular lattice
Authors: Tinne Tuytelaars, Cordelia Schmid
Venue: Conference ICCV 2007
Year: 2007

Most recent class-level object recognition systems work with visual words, i.e., vector quantized local descriptors. In this paper we examine the feasibility of a data- independent approach to construct such a visual vocabulary, where the feature space is discretized using a regular lattice. Using hashing techniques, only non-empty bins are stored, and fine-grained grids become possible in spite of the high dimensionality of typical feature spaces. Based on this representation, we can explore the structure of the feature space, and obtain state-of-the-art pixelwise classification results. In the case of image classification, we introduce a class-specific feature selection step, which takes the spatial structure of SIFT-like descriptors into account. Results are reported on the Graz02 dataset.
Segmentation of building facades using procedural shape priors
Authors: Olivier Teboul, Loïc Simon, Panagiotis Koutsourakis, Nikos Paragios
Venue: Conference CVPR 2010
Year: 2010

In this paper we propose a novel approach to the perceptual interpretation of building facades that combines shape grammars, supervised classification and random walks. Procedural modeling is used to model the geometric and the photometric variation of buildings. This is fused with visual classification techniques (randomized forests) that provide a crude probabilistic interpretation of the observation space in order to measure the appropriateness of a procedural generation with respect to the image. A random exploration of the grammar space is used to optimize the sequence of derivation rules towards a semantico-geometric interpretation of the observations. Experiments conducted on complex architecture facades with ground truth validate the approach.
Computing rectifying homographies for stereo vision
Authors: Charles T. Loop, Zhengyou Zhang
Venue: Conference CVPR 1999
Year: 1999

Image rectification is the process of applying a pair of 2D projective transforms, or homographies, to a pair of images whose epipolar geometry is known so that epipolar lines in the original images map to horizontally aligned lines in the transformed images. We propose a novel technique for image rectification based on geometrically well defined criteria such that image distortion due to rectification is minimized. This is achieved by decomposing each homography into a specialized projective transform, a similarity transform, followed by a shearing transform. The effect of image distortion at each stage is carefully considered.
High dynamic range image reconstruction from hand-held cameras
Authors: Pei-Ying Lu, Tz-Huan Huang, Meng-Sung Wu, Yi-Ting Cheng, Yung-Yu Chuang
Venue: Conference CVPR 2009
Year: 2009

This paper presents a technique for reconstructing a high-quality high dynamic range (HDR) image from a set of differently exposed and possibly blurred images taken with a hand-held camera. Recovering an HDR image from differently exposed photographs has become very popular. However, it often requires a tripod to keep the camera still when taking photographs of different exposures. To ease the process, it is often preferred to use a hand-held camera. This, however, leads to two problems, misaligned photographs and blurred long-exposed photographs. To overcome these problems, this paper adapts an alignment method and proposes a method for HDR reconstruction from possibly blurred images. We use Bayesian framework to formulate the problem and apply a maximum-likelihood approach to iteratively perform blur kernel estimation, HDR image reconstruction and camera curve recovery. When convergence, we simultaneously obtain an HDR image with rich and clear structures, the camera response curve and blur kernels. To show the effectiveness of our method, we test our method on both synthetic and real photographs. The proposed method compares favorably to two other related methods in the experiments.
Optimal pose for face recognition
Authors: Xiaoming Liu 0002, Jens Rittscher, Tsuhan Chen
Venue: Conference CVPR 2006
Year: 2006

Researchers in psychology have well studied the impact of the pose of a face as perceived by humans, and concluded that the so-called 3/4 view, halfway between the front view and the profile view, is the easiest for face recognition by humans. For face recognition by machines, while much work has been done to create recognition algorithms that are robust to pose variation, little has been done in finding the most representative pose for recognition. In this paper, we use a number of algorithms to evaluate face recognition performance when various poses are used for training. The result, similar to findings in psychology that the 3/4 view is the best, is also justified by the discrimination power of different regions on the face, computed from both the appearance and the geometry of these regions. We believe our study is both scientifically interesting and practically beneficial for many applications.
Robust point set registration using em-icp with information-theoretically optimal outlier handling
Authors: Jeroen Hermans, Dirk Smeets, Dirk Vandermeulen, Paul Suetens
Venue: Conference CVPR 2011
Year: 2011

In this paper the problem of pairwise model-to-scene point set registration is considered. Three contributions are made. Firstly, the relations between correspondence-based and some information-theoretic point cloud registration algorithms are formalized. Starting from the observation that the outlier handling of existing methods relies on heuristically determined models, a second contribution is made exploiting aforementioned relations to derive a new robust point set registration algorithm. Representing model and scene point clouds by mixtures of Gaus-sians, the method minimizes their Kullback-Leibler divergence both w.r.t. the registration transformation parameters and w.r.t. the scene's mixture coefficients. This results in an Expectation-Maximization Iterative Closest Point (EM-ICP) approach with a parameter-free outlier model that is optimal in information-theoretical sense. While the current (CUDA) implementation is limited to the rigid registration case, the underlying theory applies to both rigid and non-rigid point set registration. As a by-product of the registration algorithm's theory, a third contribution is made by suggesting a new point cloud Kernel Density Estimation approach which relies on maximizing the resulting distribution's entropy w.r.t. the kernel weights. The rigid registration algorithm is applied to align different patches of the publicly available Stanford Dragon and Stanford Happy Budha range data. The results show good performance regarding accuracy, robustness and convergence range.
Bayesian autocalibration for surveillance
Authors: Nils Krahnstoever, Paulo R. S. Mendonça
Venue: Conference ICCV 2005
Year: 2005

In the context of visual surveillance of human activity knowledge about a camera's internal and external parameters is useful, as it allows for the establishment of a connection between image and world measurements. Unfortunately, calibration information is rarely available and difficult to obtain after a surveillance system has been installed. In this paper, a method for camera autocalibration based on information gathered by tracking people is developed. It brings two main contributions: first, we show how a foot-to-head plane homology can be used to obtain the calibration parameters and then we show an approach how to efficiently estimate initial parameter estimates from measurements; second, we present a Bayesian solution to the calibration problem that can elegantly handle measurement uncertainties, outliers, as well as prior information. It is shown how the full posterior distribution of calibration parameters given the measurements can be estimated, which allows making statements about the accuracy of both the calibration parameters and the measurements involving them
Retaining privileged information for multi-task learning
Authors: Fengyi Tang, Cao Xiao, Fei Wang, Jiayu Zhou, Li-Wei H. Lehman
Venue: Conference KDD 2019
Year: 2019

Knowledge transfer has been of great interest in current machine learning research, as many have speculated its importance in modeling the human ability to rapidly generalize learned models to new scenarios. Particularly in cases where training samples are limited, knowledge transfer shows improvement on both the learning speed and generalization performance of related tasks. Recently, Learning Using Privileged Information (LUPI) has presented a new direction in knowledge transfer by modeling the transfer of prior knowledge as a Teacher-Student interaction process. Under LUPI, a Teacher model uses Privileged Information (PI) that is only available at training time to improve the sample complexity required to train a Student learner for a given task. In this work, we present a LUPI formulation that allows privileged information to be retained in a multi-task learning setting. We propose a novel feature matching algorithm that projects samples from the original feature space and the privilege information space into a joint latent space in a way that informs similarity between training samples. Our experiments show that useful knowledge from PI is maintained in the latent space and greatly improves the sample efficiency of other related learning tasks. We also provide an analysis of sample complexity of the proposed LUPI method, which under some favorable assumptions can achieve a greater sample efficiency than brute force methods.
A theory of active object localization
Authors: Alexander Andreopoulos, John K. Tsotsos
Venue: Conference ICCV 2009
Year: 2009

We present some theoretical results related to the problem of actively searching for a target in a 3D environment, under the constraint of a maximum search time. We define the object localization problem as the maximization over the search region of the Lebesgue integral of the scene structure probabilities. We study variants of the problem as they relate to actively selecting a finite set of optimal viewpoints of the scene for detecting and localizing an object. We do a complexity-level analysis and show that the problem variants are NP-Complete or NP-Hard. We study the tradeoffs of localizing vs. detecting a target object, using single-view and multiple-view recognition, under imperfect dead-reckoning and an imperfect recognition algorithm. These results motivate a set of properties that efficient and reliable active object localization algorithms should satisfy.
Light field video stabilization
Authors: Brandon M. Smith 0001, Li Zhang 0003, Hailin Jin, Aseem Agarwala
Venue: Conference ICCV 2009
Year: 2009

We describe a method for producing a smooth, stabilized video from the shaky input of a hand-held light field video camera—specifically, a small camera array. Traditional stabilization techniques dampen shake with 2D warps, and thus have limited ability to stabilize a significantly shaky camera motion through a 3D scene. Other recent stabilization techniques synthesize novel views as they would have been seen along a virtual, smooth 3D camera path, but are limited to static scenes. We show that video camera arrays enable much more powerful video stabilization, since they allow changes in viewpoint for a single time instant. Furthermore, we point out that the straightforward approach to light field video stabilization requires computing structure-from-motion, which can be brittle for typical consumer-level video of general dynamic scenes. We present a more robust approach that avoids input camera path reconstruction. Instead, we employ a spacetime optimization that directly computes a sequence of relative poses between the virtual camera and the camera array, while minimizing acceleration of salient visual features in the virtual image plane. We validate our novel method by comparing it to state-of-the-art stabilization software, such as Apple iMovie and 2d3 SteadyMove Pro, on a number of challenging scenes.
Silhouette transformation based on walking speed for gait identification
Authors: Akira Tsuji, Yasushi Makihara, Yasushi Yagi
Venue: Conference CVPR 2010
Year: 2010

We propose a method of gait silhouette transformation from one speed to another to cope with walking speed changes in gait identification. When a person changes his/her walking speed, dynamic features (e.g. stride and joint angle) are changed while static features (e.g. thigh and shin lengths) are unchanged. Based on the fact, firstly, static and dynamic features are separated from gait silhouettes by fitting a human model. Secondly, a factorization-based speed transformation model for the dynamic features is created using a training set for multiple persons on multiple speeds. This model can transform the dynamic features from a reference speed to another arbitrary speed. Finally, silhouettes are restored by combining the unchanged static features and the transformed dynamic features. Evaluation by gait identification using silhouette-based frequency-domain features shows the effectiveness of the proposed method.
Exploiting object hierarchy: Combining models from different category levels
Authors: Alon Zweig, Daphna Weinshall
Venue: Conference ICCV 2007
Year: 2007

We investigated the computational properties of natural object hierarchy in the context of constellation object class models, and its utility for object class recognition. We first observed an interesting computational property of the object hierarchy: comparing the recognition rate when using models of objects at different levels, the higher more inclusive levels (e.g., closed-frame vehicles or vehicles) exhibit higher recall but lower precision when compared with the class specific level (e.g., bus). These inherent differences suggest that combining object classifiers from different hierarchical levels into a single classifier may improve classification, as it appears like these models capture different aspects of the object. We describe a method to combine these classifiers, and analyze the conditions under which improvement can be guaranteed. When given a small sample of a new object class, we describe a method to transfer knowledge across the tree hierarchy, between related objects. Finally, we describe extensive experiments using object hierarchies obtained from publicly available datasets, and show that the combined classifiers significantly improve recognition results.
Robust fusion of dynamic shape and normal capture for high-quality reconstruction of time-varying geometry
Authors: Naveed Ahmed 0001, Christian Theobalt, Petar Dobrev, Hans-Peter Seidel, Sebastian Thrun
Venue: Conference CVPR 2008
Year: 2008

This paper describes a new passive approach to capture time-varying scene geometry in large acquisition volumes from multi-view video. It can be applied to reconstruct complete moving models of human actors that feature even slightest dynamic geometry detail, such as wrinkles and folds in clothing, and that can be viewed from 360deg. Starting from multi-view video streams recorded under calibrated lighting, we first perform marker-less human motion capture based on a smooth template with no high-frequency surface detail. Subsequently, surface reflectance and time-varying normal fields are estimated based on the coarse template shape. The main contribution of this paper is a new statistical approach to solve the non-trivial problem of transforming the captured normal field that is defined over the smooth non-planar 3D template into true 3D displacements. Our spatio-temporal reconstruction method outputs displaced geometry that is accurate at each time step of video and temporally smooth, even if the input data are affected by noise.
Active segmentation with fixation
Authors: Ajay K. Mishra, Yiannis Aloimonos, Loong Fah Cheong
Venue: Conference ICCV 2009
Year: 2009

The human visual system observes and understands a scene/image by making a series of fixations. Every “fixation point” lies inside a particular region of arbitrary shape and size in the scene which can either be an object or just a part of it. We define as a basic segmentation problem the task of segmenting that region containing the “fixation point”. Segmenting this region is equivalent to finding the enclosing contour - a connected set of boundary edge fragments in the edge map of the scene - around the fixation. We present here a novel algorithm that finds this bounding contour and achieves the segmentation of one object, given the fixation. The proposed segmentation framework combines monocular cues (color/intensity/texture) with stereo and/or motion, in a cue independent manner. We evaluate the performance of the proposed algorithm on challenging videos and stereo pairs. Although the proposed algorithm is more suitable for an active observer capable of fixating at different locations in the scene, it applies to a single image as well. In fact, we show that even with monocular cues alone, the introduced algorithm performs as well or better than a number of image segmentation algorithms, when applied to challenging inputs.