OpenRaiser/Intern-Atlas huggingface dataset:
Modeling 3d human poses from uncalibrated monocular images
Authors: Xiaolin K. Wei, Jinxiang Chai
Venue: Conference ICCV 2009
Year: 2009

This paper introduces an efficient algorithm that reconstructs 3D human poses as well as camera parameters from a small number of 2D point correspondences obtained from uncalibrated monocular images. This problem is challenging because 2D image constraints (e.g. 2D point correspondences) are often not sufficient to determine 3D poses of an articulated object. The key idea of this paper is to identify a set of new constraints and use them to eliminate the ambiguity of 3D pose reconstruction. We also develop an optimization process to simultaneously reconstruct both human poses and camera parameters from various forms of reconstruction constraints. We demonstrate the power and effectiveness of our system by evaluating the performance of the algorithm on both real and synthetic data. We show the algorithm can accurately reconstruct 3D poses and camera parameters from a wide variety of real images, including internet photos and key frames extracted from monocular video sequences.
Correcting over-exposure in photographs
Authors: Dong Guo 0001, Yuan Cheng 0005, Shaojie Zhuo, Terence Sim
Venue: Conference CVPR 2010
Year: 2010

This paper introduces a method to correct over-exposure in an existing photograph by recovering the color and lightness separately. First, the dynamic range of well exposed region is slightly compressed to make room for the recovered lightness of the over-exposed region. Then the lightness is recovered based on an over-exposure likelihood. The color of each pixel is corrected via neighborhood propagation and also based on the confidence of the original color. Previous methods make use of ratios between different color channels to recover the over-exposed ones, and thus can not handle regions where all three channels are over-exposed. In contrast, our method does not have this limitation. Our method is fully automatic and requires only one single input photo. We also provide users with the flexibility to control the amount of over-exposure correction. Experiment results demonstrate the effectiveness of the proposed method in correcting over-exposure.
Scalable k -Means Clustering via Lightweight Coresets
Authors: Olivier Bachem, Mario Lucic, Andreas Krause
Venue: Conference KDD 2018
Year: 2018

\emphCoresets are compact representations of data sets such that models trained on a coreset are provably competitive with models trained on the full data set. As such, they have been successfully used to scale up clustering models to massive data sets. While existing approaches generally only allow for multiplicative approximation errors, we propose a novel notion of lightweight coresets that allows for both multiplicative and additive errors. We provide a single algorithm to construct lightweight coresets for k -means clustering as well as soft and hard Bregman clustering. The algorithm is substantially faster than existing constructions, embarrassingly parallel, and the resulting coresets are smaller. We further show that the proposed approach naturally generalizes to statistical k -means clustering and that, compared to existing results, it can be used to compute smaller summaries for empirical risk minimization. In extensive experiments, we demonstrate that the proposed algorithm outperforms existing data summarization strategies in practice.
A new interpretation of average precision
Authors: Stephen Robertson
Venue: Conference SIGIR 2008
Year: 2008

We consider the question of whether Average Precision, as a measure of retrieval effectiveness, can be regarded as deriving from a model of user searching behaviour. It turns out that indeed it can be so regarded, under a very simple stochastic model of user behaviour.
Polarization-based surface reconstruction via patch matching
Authors: Gary A. Atkinson, Edwin R. Hancock
Venue: Conference CVPR 2006
Year: 2006

A new method for multiple viewpoint 3D shape reconstruction is presented that relies on the polarization properties of surface reflection. The method is intended to complement existing stereo techniques by establishing correspondence for surfaces without salient features. The phase and degree of polarization from two views of an object are used to reconstruct surface patches. Local surface properties are then used to both align these patches and to compute a cost for that alignment. This cost is used as a basis to establish correspondence between the two views. The method is tested on an object library comprising shapes of varying complexity and material. An accuracy assessment is also presented where real world data are compared to ground truth.
Efficient visual event detection using volumetric features
Authors: Yan Ke, Rahul Sukthankar, Martial Hebert
Venue: Conference ICCV 2005
Year: 2005

This paper studies the use of volumetric features as an alternative to popular local descriptor approaches for event detection in video sequences. Motivated by the recent success of similar ideas in object detection on static images, we generalize the notion of 2D box features to 3D spatio-temporal volumetric features. This general framework enables us to do real-time video analysis. We construct a realtime event detector for each action of interest by learning a cascade of filters based on volumetric features that efficiently scans video sequences in space and time. This event detector recognizes actions that are traditionally problematic for interest point methods - such as smooth motions where insufficient space-time interest points are available. Our experiments demonstrate that the technique accurately detects actions on real-world sequences and is robust to changes in viewpoint, scale and action speed. We also adapt our technique to the related task of human action classification and confirm that it achieves performance comparable to a current interest point based human activity recognizer on a standard database of human activities.
Scale-dependent 3d geometric features
Authors: John Novatnack, Ko Nishino
Venue: Conference ICCV 2007
Year: 2007

Three-dimensional geometric data play fundamental roles in many computer vision applications. However, their scale-dependent nature, i.e. the relative variation in the spatial extents of local geometric structures, is often overlooked. In this paper we present a comprehensive framework for exploiting this 3D geometric scale variability. Specifically, we focus on detecting scale-dependent geometric features on triangular mesh models of arbitrary topology. The key idea of our approach is to analyze the geometric scale variability of a given 3D model in the scale-space of a dense and regular 2D representation of its surface geometry encoded by the surface normals. We derive novel corner and edge detectors, as well as an automatic scale selection method, that acts upon this representation to detect salient geometric features and determine their intrinsic scales. We evaluate the effectiveness and robustness of our method on a number of models of different topology. The results show that the resulting scale-dependent geometric feature set provides a reliable basis for constructing a rich but concise representation of the geometric structure at hand.
Contrastive syn-to-real generalization
Authors: Wuyang Chen 0001, Zhiding Yu, Shalini De Mello, Sifei Liu, José M. Álvarez 0004, Zhangyang Wang, Anima Anandkumar
Venue: Conference ICLR 2021
Year: 2021

Training on synthetic data can be beneficial for label or data-scarce scenarios. However, synthetically trained models often suffer from poor generalization in real domains due to domain gaps. In this work, we make a key observation that the diversity of the learned feature embeddings plays an important role in the generalization performance. To this end, we propose contrastive synthetic-to-real generalization (CSG), a novel framework that leverages the pre-trained ImageNet knowledge to prevent overfitting to the synthetic domain, while promoting the diversity of feature embeddings as an inductive bias to improve generalization. In addition, we enhance the proposed CSG framework with attentional pooling (A-pool) to let the model focus on semantically important regions and further improve its generalization. We demonstrate the effectiveness of CSG on various synthetic training tasks, exhibiting state-of-the-art performance on zero-shot domain generalization.
Visual persuasion in covid-19 social media content: A multi-modal characterization
Authors: Mesut Erhan Unal, Adriana Kovashka, Wen-Ting Chung, Yu-Ru Lin
Venue: Conference WWW 2022
Year: 2022

Social media content routinely incorporates multi-modal design to covey information and shape meanings, and sway interpretations toward desirable implications, but the choices and impacts of using both texts and visual images have not been sufficiently studied. This work proposes a computational approach to analyze the impacts of persuasive multi-modal content on popularity and reliability, in COVID-19-related news articles shared on Twitter. The two aspects are intertwined in the spread of misinformation: for example, an unreliable article that aims to misinform has to attain some popularity. This work has several contributions. First, we propose a multi-modal (image and text) approach to effectively identify popularity and reliability of information sources simultaneously. Second, we identify textual and visual elements that are predictive to information popularity and reliability. Third, by modeling cross-modal relations and similarity, we are able to uncover how unreliable articles construct multi-modal meaning in a distorted, biased fashion. Our work demonstrates how to use multi-modal analysis for understanding influential content and has implications to social media literacy and engagement.
Generic intent representation in web search
Authors: Hongfei Zhang, Xia Song, Chenyan Xiong, Corby Rosset, Paul N. Bennett, Nick Craswell, Saurabh Tiwary
Venue: Conference SIGIR 2019
Year: 2019

This paper presents GEneric iNtent Encoder (GEN Encoder) which learns a distributed representation space for user intent in search. Leveraging large scale user clicks from Bing search logs as weak supervision of user intent, GEN Encoder learns to map queries with shared clicks into similar embeddings end-to-end and then fine-tunes on multiple paraphrase tasks. Experimental results on an intrinsic evaluation task - query intent similarity modeling - demonstrate GEN Encoder's robust and significant advantages over previous representation methods. Ablation studies reveal the crucial role of learning from implicit user feedback in representing user intent and the contributions of multi-task learning in representation generality. We also demonstrate that GEN Encoder alleviates the sparsity of tail search traffic and cuts down half of the unseen queries by using an efficient approximate nearest neighbor search to effectively identify previous queries with the same search intent. Finally, we demonstrate distances between GEN encodings reflect certain information seeking behaviors in search sessions.
The chains model for detecting parts by their context
Authors: Leonid Karlinsky, Michael Dinerstein, Daniel Harari, Shimon Ullman
Venue: Conference CVPR 2010
Year: 2010

Detecting an object part relies on two sources of information - the appearance of the part itself and the context supplied by surrounding parts. In this paper we consider problems in which a target part cannot be recognized reliably using its own appearance, such as detecting low-resolution hands, and must be recognized using the context of surrounding parts. We develop the `chains model' which can locate parts of interest in a robust and precise manner, even when the surrounding context is highly variable and deformable. In the proposed model, the relation between context features and the target part is modeled in a non-parametric manner using an ensemble of feature chains leading from parts in the context to the detection target. The method uses the configuration of the features in the image directly rather than through fitting an articulated 3-D model of the object. In addition, the chains are composable, meaning that new chains observed in the test image can be composed of sub-chains seen during training. Consequently, the model is capable of handling object poses which are infrequent, even non-existent, during training. We test the approach in different settings, including object parts detection, as well as complete object detection. The results show the advantages of the chains model for detecting and localizing parts of complex deformable objects.
Bilattice-based logical reasoning for human detection
Authors: Vinay D. Shet, Jan Neumann, Visvanathan Ramesh, Larry S. Davis
Venue: Conference CVPR 2007
Year: 2007

The capacity to robustly detect humans in video is a critical component of automated visual surveillance systems. This paper describes a bilattice based logical reasoning approach that exploits contextual information and knowledge about interactions between humans, and augments it with the output of different low level detectors for human detection. Detections from low level parts-based detectors are treated as logical facts and used to reason explicitly about the presence or absence of humans in the scene. Positive and negative information from different sources, as well as uncertainties from detections and logical rules, are integrated within the bilattice framework. This approach also generates proofs or justifications for each hypothesis it proposes. These justifications (or lack thereof) are further employed by the system to explain and validate, or reject potential hypotheses. This allows the system to explicitly reason about complex interactions between humans and handle occlusions. These proofs are also available to the end user as an explanation of why the system thinks a particular hypothesis is actually a human. We employ a boosted cascade of gradient histograms based detector to detect individual body parts. We have applied this framework to analyze the presence of humans in static images from different datasets.
Social interactions: A first-person perspective
Authors: Alireza Fathi, Jessica K. Hodgins, James M. Rehg
Venue: Conference CVPR 2012
Year: 2012

This paper presents a method for the detection and recognition of social interactions in a day-long first-person video of u social event, like a trip to an amusement park. The location and orientation of faces are estimated and used to compute the line of sight for each face. The context provided by all the faces in a frame is used to convert the lines of sight into locations in space to which individuals attend. Further, individuals are assigned roles based on their patterns of attention. The rotes and locations of individuals are analyzed over time to detect and recognize the types of social interactions. In addition to patterns of face locations and attention, the head movements of the first-person can provide additional useful cues as to their attentional focus. We demonstrate encouraging results on detection and recognition of social interactions in first-person videos captured from multiple days of experience in amusement parks.
A semi-automatic method for resolving occlusion in augmented reality
Authors: Vincent Lepetit, Marie-Odile Berger
Venue: Conference CVPR 2000
Year: 2000

Realistic merging of virtual and real objects requires that the augmented patterns be correctly occluded by foreground objects. In this paper we propose a semi-automatic method for resolving occlusion in augmented reality which makes use of key-views. Once the user has outlined the occluding objects in the key-views, our system detects automatically these occluding objects in the intermediate views. A region of interest that contains the occluding objects is first computed from the outlined silhouettes. One of the main contribution of this paper is that this region takes into account the uncertainty on the computed interframe motion. Then a deformable region-based approach is used to recover the actual occluding boundary within the region of interest from this prediction.
What is that building? an end-to-end system for building recognition from streetside images
Authors: Chiqun Zhang, Dragomir Yankov, Chun-Ting Wu, Simon Shapiro, Jason Hong, Wei Wu 0014
Venue: Conference KDD 2020
Year: 2020

The paper describes Streetside Building Search-Retrieve System (SBSRS) - a system for recognizing buildings from steetside images. SBSRS powers several distinct applications: 1) it improves map-search by enriching its streetview service with semantic information, such as location, business name, open hours, etc.; 2) it enables search by image and location - a novel form of visual image search where both visual and location signals are used to identify the most relevant result to a query image of a building.
From accurate range imaging sensor calibration to accurate model-based 3d object localization
Authors: Guillaume Champleboux, Stéphane Lavallée, Richard Szeliski, Lionel Brunie
Venue: Conference CVPR 1992
Year: 1992

The registration of multiple 3D data sets obtained with a laser range finder is examined. A sensor calibration technique based on the conjunction of a mathematical camera model, the N-planes B-spline (NPBS), with an accurate mechanical calibration setup, is proposed. An algorithm for recovering the rigid transformation (rotation and translation) between the two sets of 3D coordinates obtained by the calibrated imaging range sensor is developed. Its input is sets of points lying on the surface of the object. This input is converted into an octree-spline that allows point-to-surface distances to be computed quickly. A robust nonlinear least-squares minimization technique then finds the optimal pose by minimizing the sum of square distances between the two sets of 3D coordinates. The algorithm has been applied to matching human faces with highly accurate results.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Accurate camera calibration from multi-view stereo and bundle adjustment
Authors: Yasutaka Furukawa, Jean Ponce
Venue: Conference CVPR 2008
Year: 2008

The advent of high-resolution digital cameras and sophisticated multi-view stereo algorithms offers the promises of unprecedented geometric fidelity in image-based modeling tasks, but it also puts unprecedented demands on camera calibration to fulfill these promises. This paper presents a novel approach to camera calibration where top-down information from rough camera parameter estimates and the output of a publicly available multiview-stereo system [6] on scaled-down input images are used to effectively guide the search for additional image correspondences and significantly improve camera calibration parameters using a standard bundle adjustment algorithm [14]. The proposed method has been tested on several real datasets—including objects without salient features for which image correspondences cannot be found in a purely bottom-up fashion, and image-based modeling tasks-including the construction of visual hulls where thin structures are lost without our calibration procedure.
Cracking white-box dnn watermarks via invariant neuron transforms
Authors: Xudong Pan, Mi Zhang 0001, Yifan Yan, Yining Wang, Min Yang 0002
Venue: Conference KDD 2023
Year: 2023

Recently, how to protect the Intellectual Property (IP) of deep neural networks (DNN) becomes a major concern for the AI industry. To combat potential model piracy, recent works explore various watermarking strategies to embed secret identity messages into the prediction behaviors or the internals (e.g., weights and neuron activation) of the target model. Sacrificing less functionality and involving more knowledge about the target model, the latter branch of watermarking schemes (i.e., white-box model watermarking) is claimed to be accurate, credible and secure against most known watermark removal attacks, with emerging research efforts and applications in the industry.
Learning Stable Classifiers by Transferring Unstable Features
Authors: Yujia Bao, Shiyu Chang, Regina Barzilay
Venue: Conference ICML 2022
Year: 2022

While unbiased machine learning models are essential for many applications, bias is a human-defined concept that can vary across tasks. Given only input-label pairs, algorithms may lack sufficient information to distinguish stable (causal) features from unstable (spurious) features. However, related tasks often share similar biases -- an observation we may leverage to develop stable classifiers in the transfer setting. In this work, we explicitly inform the target classifier about unstable features in the source tasks. Specifically, we derive a representation that encodes the unstable features by contrasting different data environments in the source task. We achieve robustness by clustering data of the target task according to this representation and minimizing the worst-case risk across these clusters. We evaluate our method on both text and image classifications. Empirical results demonstrate that our algorithm is able to maintain robustness on the target task for both synthetically generated environments and real-world environments. Our code is available at https://github.com/YujiaBao/Tofu.
Biased normalized cuts
Authors: Subhransu Maji, Nisheeth K. Vishnoi, Jitendra Malik
Venue: Conference CVPR 2011
Year: 2011

We present a modification of “Normalized Cuts” to incorporate priors which can be used for constrained image segmentation. Compared to previous generalizations of “Normalized Cuts” which incorporate constraints, our technique has two advantages. First, we seek solutions which are sufficiently “correlated” with priors which allows us to use noisy top-down information, for example from an object detector. Second, given the spectral solution of the unconstrained problem, the solution of the constrained one can be computed in small additional time, which allows us to run the algorithm in an interactive mode. We compare our algorithm to other graph cut based algorithms and highlight the advantages.
Hierarchical semantics of objects (hsos
Authors: Devi Parikh, Tsuhan Chen
Venue: Conference ICCV 2007
Year: 2007

We introduce hSOs: hierarchical semantics of objects. An hSO is learnt from a collection of images taken from a particular scene category. The hSO captures the interactions between the objects that tend to co-occur in the scene, and hence are potentially semantically related. Such relationships are typically hierarchical. For example, in a collection of images taken in a living room scene, the TV, DVD player and coffee-table co-occur frequently. The TV and the DVD player are more closely related to each other than the coffee table, and this can be learnt from the fact that the two are located at similar relative locations across images, while the coffee table is somewhat arbitrarily placed. The goal of this paper is to learn this hierarchy that characterizes the scene. The proposed approach, being entirely unsupervised, can detect the parts of the images that belong to the foreground objects, cluster these parts to represent objects, and provide an understanding of the scene by hierarchically clustering these objects in a semantically meaningful way - all from a collection of unlabeled images of a particular scene category. In addition to providing the semantic layout of the scene, learnt hSOs can have several useful applications such as compact scene representation for scene category classification and providing context for enhanced object detection.