OpenRaiser/Intern-Atlas huggingface dataset:
Expectation grammars: Leveraging high-level expectations for activity recognition
Authors: David Minnen, Irfan A. Essa, Thad Starner
Venue: Conference CVPR 2003
Year: 2003

Video-based recognition and prediction of a temporally extended activity can benefit from a detailed description of high-level expectations about the activity. Stochastic grammars allow for an efficient representation of such expectations and are well-suited for the specification of temporally well-ordered activities. In this paper, we extend stochastic grammars by adding event parameters, state checks, and sensitivity to an internal scene model. We present an implemented system that uses human-specified grammars to recognize a person performing the Towers of Hanoi task from a video sequence by analyzing object interaction events. Experimental results from several videos show robust recognition of the full task and its constituent sub-tasks even though no appearance models of the objects in the video are provided. These experiments include videos of the task performed with different shaped objects and with distracting and extraneous interactions.
The space of all stereo images
Authors: Steven M. Seitz
Venue: Conference ICCV 2001
Year: 2001

A theory of stereo image formation is presented that enables a complete classification of all possible stereo views, including non-perspective varieties. Towards this end, the notion of epipolar geometry is generalized to apply to multiperspective images. It is shown that any stereo pair must consist of rays lying on one of three varieties of quadric surfaces. A unified representation is developed to model all classes of stereo views, based on the concept of a quadric view. The benefits include a unified treatment of projection and triangulation operations for all stereo views. The framework is applied to derive new types of stereo image representations with unusual and useful properties.
Unilp: Unified topology-aware generative framework for link prediction in knowledge graph
Authors: Ben Liu, Miao Peng, Wenjie Xu, Xu Jia, Min Peng
Venue: Conference WWW 2024
Year: 2024

Link prediction (LP) in knowledge graph (KG) is a crucial task that has received increasing attention recently. Due to the heterogeneous structures of KGs, various application scenarios, and demand-specific downstream objectives, there exist multiple subtasks in LP. Most studies only focus on designing a dedicated architecture for a specific subtask, which results in various complicated LP models. The isolated architectures and chaotic situations make it significant to construct a unified model that can handle multiple LP subtasks simultaneously. However, unifying all subtasks in LP presents numerous challenges, including unified input forms, task-specific context modeling, and topological information encoding. To address these challenges, we propose a topology-aware generative framework, namely UniLP, which utilizes a generative pre-trained language model to accomplish different LP subtasks universally. Specifically, we introduce a context demonstration template to convert task-specific context into a unified generative formulation. Based on the unified formulation, to address the limitation of transformer architecture that may overlook important structural signals in KGs, we design novel topology-aware soft prompts to deeply couple topology and text information in a contextualized manner. Extensive experiment results demonstrate that our framework achieves substantial performance gain and provides a real unified end-to-end solution for the whole LP subtasks. We also perform comprehensive ablation studies to support in-depth analysis of each component in UniLP.
Shrink boost for selecting multi-lbp histogram features in object detection
Authors: Cher Keng Heng, Sumio Yokomitsu, Yuichi Matsumoto, Hajime Tamura
Venue: Conference CVPR 2012
Year: 2012

Feature selection from sparse and high dimension features using conventional greedy based boosting gives classifiers of poor generalization. We propose a novel “shrink boost” method to address this problem. It solves a sparse regularization problem with two iterative steps. First, a “boosting” step uses weighted training samples to learn a full high dimensional classifier on all features. This avoids over fitting to few features and improves generalization. Next, a “shrinkage” step shrinks least discriminative classifier dimension to zero to remove the redundant features. In our object detection system, we use “shrink boost” to select sparse features from histograms of local binary pattern (LBP) of multiple quantization and image channels to learn classifier of additive lookup tables (LUT). Our evaluation shows that our classifier has much better generalization than those from greedy based boosting and those from SVM methods, even under limited number of train samples. On public dataset of human detection and pedestrian detection, we achieve better performance than state of the arts. On our more challenging dataset of bird detection, we show promising results.
Automatic removal of chromatic aberration from a single image
Authors: Sing Bing Kang
Venue: Conference CVPR 2007
Year: 2007

Many high resolution images exhibit chromatic aberration (CA), where the color channels appear shifted. Unfortunately, merely compensating for these shifts is sometimes inadequate, because the intensities are modified by other effects such as spatially-varying defocus and (surprisingly) in-camera sharpening. In this paper, we start from the basic principles of image formation to characterize CA, and show how its effects can be substantially reduced. We also show results of CA correction on a number of high-resolution images taken with different cameras.
Smoothing NDCG metrics using tied scores
Authors: Andrey Kustarev, Yury Ustinovsky, Yury Logachev, Evgeny Grechnikov, Ilya Segalovich, Pavel Serdyukov
Venue: Conference CIKM 2011
Year: 2011

One of promising directions in research on learning to rank concerns the problem of appropriate choice of the objective function to maximize by means of machine learning algorithms. We describe a novel technique of smoothing an arbitrary ranking metric and demonstrate how to utilize it to maximize the retrieval quality in terms of the $NDCG$ metric. The idea behind our listwise ranking model called TieRank is artificial probabilistic tying of predicted relevance scores at each iteration of learning process, which defines a distribution on the set of all permutations of retrieved documents. Such distribution provides a desired smoothed version of the target retrieval quality metric. This smooth function is possible to maximize using a gradient descent method. Experiments on LETOR collections show that TieRank outperforms most of the existing learning to rank algorithms.
Prediction error as a quality metric for motion and stereo
Authors: Richard Szeliski
Venue: Conference ICCV 1999
Year: 1999

This paper presents a new methodology for evaluating the quality of motion estimation and stereo correspondence algorithms. Motivated by applications such as novel view generation and motion-compensated compression, we suggest that the ability to predict new views or frames is a natural metric for evaluating such algorithms. Our new metric has several advantages over comparing algorithm outputs to true motions or depths. First of all, it does not require the knowledge of ground truth data, which may be difficult or laborious to obtain. Second, it more closely matches the ultimate requirements of the application, which are typically tolerant of errors in uniform color regions, but very sensitive to isolated pixel errors or disocclusion errors. In the paper we develop a number of error metrics based on this paradigm, including forward and inverse prediction errors, residual motion error and local motion-compensated prediction error. We show results on a number of widely used motion and stereo sequences, many of which do not have associated ground truth data.
High-arity interactions, polyhedral relaxations, and cutting plane algorithm for soft constraint optimisation (map-mrf
Authors: Tomás Werner
Venue: Conference CVPR 2008
Year: 2008

LP relaxation approach to soft constraint optimisation (i.e. MAP-MRF) has been mostly considered only for binary problems. We present its generalisation to n-ary problems, including a simple algorithm to optimise the LP bound, n-ary max-sum diffusion. As applications, we show that a hierarchy of gradually tighter polyhedral relaxations of MAP-MRF is obtained by adding zero interactions. We propose a cutting plane algorithm, where cuts correspond to adding zero interactions and the separation problem to finding an unsatisfiable constraint satisfaction subproblem. Next, we show that certain high-arity interactions, e.g. certain global constraints, can be included into the framework in a principled way. Finally, we prove that n-ary max-sum diffusion finds global optimum for n-ary supermodular problems.
Kissing cuisines: Exploring worldwide culinary habits on the web
Authors: Sina Sajadmanesh, Sina Jafarzadeh, Seyed Ali Ossia, Hamid R. Rabiee, Hamed Haddadi, Yelena Mejova, Mirco Musolesi, Emiliano De Cristofaro, Gianluca Stringhini
Venue: Conference WWW 2017
Year: 2017

As food and nutrition occupy an increasingly prevalent space on the web, dishes and recipes shared online provide an invaluable mirror into culinary cultures and attitudes around the world. More specifically, ingredients, flavors, and nutrition information become strong signals of the taste preferences of individuals and civilizations. However, there is little understanding of these palate varieties. In this paper, we present a large-scale study of recipes published on the web and their content, aiming to understand cuisines and culinary habits around the world. Using a database of more than 157K recipes from over 200 different cuisines, we analyze ingredients, flavors, and nutritional values which distinguish dishes from different regions, and use this knowledge to assess the predictability of recipes from different cuisines. We then use country health statistics to understand the relation between these factors and health indicators of different nations, such as obesity, diabetes, migration, and health expenditure. Our results confirm the strong effects of geographical and cultural similarities on recipes, health indicators, and culinary preferences between countries around the world.
Color constancy using kl-divergence
Authors: Charles R. Rosenberg, Martial Hebert, Sebastian Thrun
Venue: Conference ICCV 2001
Year: 2001

Color is a useful feature for machine vision tasks. However its effectiveness is often limited by the fact that the measured pixel values in a scene are influenced by both object surface reflectance properties and incident illumination. Color constancy algorithms attempt to compute color features which are invariant of the incident illumination by estimating the parameters of the global scene illumination and factoring out its effect. A number of recently developed algorithms utilize statistical methods to estimate the maximum likelihood values of the illumination parameters. This paper details the use of KL-divergence as a means of selecting estimated illumination parameter values. We provide experimental results demonstrating the usefulness of the KL-divergence technique for accurately estimating the global illumination parameters of real world images.
Deep domain adaptation hashing with adversarial learning
Authors: Fuchen Long, Ting Yao, Qi Dai, Xinmei Tian, Jiebo Luo, Tao Mei
Venue: Conference SIGIR 2018
Year: 2018

The recent advances in deep neural networks have demonstrated high capability in a wide variety of scenarios. Nevertheless, fine-tuning deep models in a new domain still requires a significant amount of labeled data despite expensive labeling efforts. A valid question is how to leverage the source knowledge plus unlabeled or only sparsely labeled target data for learning a new model in target domain. The core problem is to bring the source and target distributions closer in the feature space. In the paper, we facilitate this issue in an adversarial learning framework, in which a domain discriminator is devised to handle domain shift. Particularly, we explore the learning in the context of hashing problem, which has been studied extensively due to its great efficiency in gigantic data. Specifically, a novel Deep Domain Adaptation Hashing with Adversarial learning (DeDAHA) architecture is presented, which mainly consists of three components: a deep convolutional neural networks (CNN) for learning basic image/frame representation followed by an adversary stream on one hand to optimize the domain discriminator, and on the other, to interact with each domain-specific hashing stream for encoding image representation to hash codes. The whole architecture is trained end-to-end by jointly optimizing two types of losses, i.e., triplet ranking loss to preserve the relative similarity ordering in the input triplets and adversarial loss to maximally fool the domain discriminator with the learnt source and target feature distributions. Extensive experiments are conducted on three domain transfer tasks, including cross-domain digits retrieval, image to image and image to video transfers, on several benchmarks. Our DeDAHA framework achieves superior results when compared to the state-of-the-art techniques.
Image denoising: Can plain neural networks compete with BM3D?
Authors: Harold Christopher Burger, Christian J. Schuler, Stefan Harmeling
Venue: Conference CVPR 2012
Year: 2012

Image denoising can be described as the problem of mapping from a noisy image to a noise-free image. The best currently available denoising methods approximate this mapping with cleverly engineered algorithms. In this work we attempt to learn this mapping directly with a plain multi layer perceptron (MLP) applied to image patches. While this has been done before, we will show that by training on large image databases we are able to compete with the current state-of-the-art image denoising methods. Furthermore, our approach is easily adapted to less extensively studied types of noise (by merely exchanging the training data), for which we achieve excellent results as well.
Randomized trees for human pose detection
Authors: Grégory Rogez, Jonathan Rihan, Srikumar Ramalingam, Carlos Orrite, Philip H. S. Torr
Venue: Conference CVPR 2008
Year: 2008

This paper addresses human pose recognition from video sequences by formulating it as a classification problem. Unlike much previous work we do not make any assumptions on the availability of clean segmentation. The first step of this work consists in a novel method of aligning the training images using 3D Mocap data. Next we define classes by discretizing a 2D manifold whose two dimensions are camera viewpoint and actions. Our main contribution is a pose detection algorithm based on random forests. A bottom-up approach is followed to build a decision tree by recursively clustering and merging the classes at each level. For each node of the decision tree we build a list of potentially discriminative features using the alignment of training images; in this paper we consider Histograms of Orientated Gradient (HOG). We finally grow an ensemble of trees by randomly sampling one of the selected HOG blocks at each node. Our proposed approach gives promising results with both fixed and moving cameras.
Multilinear pose and body shape estimation of dressed subjects from image sets
Authors: Nils Hasler, Hanno Ackermann, Bodo Rosenhahn, Thorsten Thormählen, Hans-Peter Seidel
Venue: Conference CVPR 2010
Year: 2010

In this paper we propose a multilinear model of human pose and body shape which is estimated from a database of registered 3D body scans in different poses. The model is generated by factorizing the measurements into pose and shape dependent components. By combining it with an ICP based registration method, we are able to estimate pose and body shape of dressed subjects from single images. If several images of the subject are available, shape and poses can be optimized simultaneously for all input images. Additionally, while estimating pose and shape, we use the model as a virtual calibration pattern and also recover the parameters of the perspective camera model the images were created with.
Articulated soft objects for video-based body modeling
Authors: Ralf Plänkers, Pascal Fua
Venue: Conference ICCV 2001
Year: 2001

We develop a framework for 3-D shape and motion recovery of articulated deformable objects. We propose a formalism that incorporates the use of implicit surfaces into earlier robotics approaches that were designed to handle articulated structures. We demonstrate its effectiveness for human body modeling from video sequences. Our method is both robust and generic. It could easily be applied to other shape and motion recovery problems.
Subspace clustering using log-determinant rank approximation
Authors: Chong Peng, Zhao Kang, Huiqing Li, Qiang Cheng
Venue: Conference KDD 2015
Year: 2015

A number of machine learning and computer vision problems, such as matrix completion and subspace clustering, require a matrix to be of low-rank. To meet this requirement, most existing methods use the nuclear norm as a convex proxy of the rank function and minimize it. However, the nuclear norm simply adds all nonzero singular values together instead of treating them equally as the rank function does, which may not be a good rank approximation when some singular values are very large. To reduce this undesirable weighting effect, we use a log-determinant function as a non-convex rank approximation which reduces the contributions of large singular values while keeping those of small singular values close to zero. We apply the method of augmented Lagrangian multipliers to optimize this non-convex rank approximation-based objective function and obtain closed-form solutions for all subproblems of minimizing different variables alternatively. The log-determinant low-rank optimization method is used to solve subspace clustering problem, for which we construct an affinity matrix based on the angular information of the low-rank representation to enhance its separability property. Extensive experimental results on face clustering and motion segmentation data demonstrate the effectiveness of the proposed method.
Photometric self-calibration of a projector-camera system
Authors: Ray Juang, Aditi Majumder
Venue: Conference CVPR 2007
Year: 2007

In this paper, we present a method for photometric self- calibration of a projector-camera system. In addition to the input transfer functions (commonly called gamma functions), we also reconstruct the spatial intensity fall-off from the center to fringe (commonly called the vignetting effect) for both the projector and camera. Projector-camera systems are becoming more popular in a large number of applications like scene capture, 3D reconstruction, and calibrating multi-projector displays. Our method enables the use of photometrically uncalibrated projectors and cameras in all such applications.
Enforcing integrability for surface reconstruction algorithms using belief propagation in graphical models
Authors: Nemanja Petrovic, Ira Cohen, Brendan J. Frey, Ralf Koetter, Thomas S. Huang
Venue: Conference CVPR 2001
Year: 2001

Accurate calculation of the three dimensional shape of an object is one of the classic research areas of computer vision. Many of the existing methods are based on surface normal estimation, and subsequent integration of surface gradients. In general, these methods do not produce valid surfaces due to violation of surface integrability. We introduce a new method for shape reconstruction by integration of valid surface gradient maps. The essence of the new approach is in the strict enforcement of the surface integrability via belief propagation across graphical models. The graphical model is selected in such a way as to extract information from underlying, possibly noisy, surface gradient estimators, utilize the surface integrability constraint, and produce the maximum a-posteriori estimate of a valid surface. We demonstrate the algorithm for two classic shape reconstruction techniques; shape-from-shading and photometric stereo. On a set of real and synthetic examples, the new approach is shown to be fast and accurate, in the sense that shape can be rendered even in the presence of high levels of noise and sharp occlusion boundaries.
Edge-preserving photometric stereo via depth fusion
Authors: Qing Zhang 0017, Mao Ye 0005, Ruigang Yang, Yasuyuki Matsushita, Bennett Wilburn, Huimin Yu
Venue: Conference CVPR 2012
Year: 2012

We present a sensor fusion scheme that combines active stereo with photometric stereo. Aiming at capturing full-frame depth for dynamic scenes at a minimum of three lighting conditions, we formulate an iterative optimization scheme that (1) adaptively adjusts the contribution from photometric stereo so that discontinuity can be preserved; (2) detects shadow areas by checking the visibility of the estimated point with respect to the light source, instead of using image-based heuristics; and (3) behaves well for ill-conditioned pixels that are under shadow, which are inevitable in almost any scene. Furthermore, we decompose our non-linear cost function into subproblems that can be optimized efficiently using linear techniques. Experiments show significantly improved results over the previous state-of-the-art in sensor fusion.
Least-squares congealing for large numbers of images
Authors: Mark Cox, Sridha Sridharan, Simon Lucey, Jeffrey F. Cohn
Venue: Conference ICCV 2009
Year: 2009

In this paper we pursue the task of aligning an ensemble of images in an unsupervised manner. This task has been commonly referred to as “congealing” in literature. A form of congealing, using a least-squares criteria, has been recently demonstrated to have desirable properties over conventional congealing. Least-squares congealing can be viewed as an extension of the Lucas & Kanade (LK) image alignment algorithm. It is well understood that the alignment performance for the LK algorithm, when aligning a single image with another, is theoretically and empirically equivalent for additive and compositional warps. In this paper we: (i) demonstrate that this equivalence does not hold for the extended case of congealing, (ii) characterize the inherent drawbacks associated with least-squares congealing when dealing with large numbers of images, and (iii) propose a novel method for circumventing these limitations through the application of an inverse-compositional strategy that maintains the attractive properties of the original method while being able to handle very large numbers of images.
Auto-directed video stabilization with robust l1 optimal camera paths
Authors: Matthias Grundmann, Vivek Kwatra, Irfan A. Essa
Venue: Conference CVPR 2011
Year: 2011

We present a novel algorithm for automatically applying constrainable, L1-optimal camera paths to generate stabilized videos by removing undesired motions. Our goal is to compute camera paths that are composed of constant, linear and parabolic segments mimicking the camera motions employed by professional cinematographers. To this end, our algorithm is based on a linear programming framework to minimize the first, second, and third derivatives of the resulting camera path. Our method allows for video stabilization beyond the conventional filtering of camera paths that only suppresses high frequency jitter. We incorporate additional constraints on the path of the camera directly in our algorithm, allowing for stabilized and retargeted videos. Our approach accomplishes this without the need of user interaction or costly 3D reconstruction of the scene, and works as a post-process for videos from any camera or from an online source.