OpenRaiser/Intern-Atlas huggingface dataset:
Physics-Based 3D Position Analysis of a Soccer Ball from Monocular Image Sequences
Authors: Taeone Kim, Yongduek Seo, Ki-Sang Hong
Venue: Conference ICCV 1998
Year: 1998

In this paper, we propose a method for locating 3D position of a soccer ball from monocular image sequence of soccer games. Toward this goal, we adopted ground-model-to-image transformation together with physics-based approach, that a ball follows the parabolic trajectory in the air. By using the transformation the heights of a ball can be easily calculated using simple triangular geometric relations given the start and the end position of the ball on the ground. Here the heights of a ball are determined in terms of a player's height. Even if the end position of a ball is not given on the ground due to kicking or heading of a falling ball before it touches the ground, the most probable trajectory can be determined by searching based on the physical fact that the ball follows a parabolic trajectory in the air. We have tested and experimented with a real image sequence the results of which seem promising.
Bayesian inference for transductive learning of kernel matrix using the tanner-wong data augmentation algorithm
Authors: Zhihua Zhang 0004, Dit-Yan Yeung, James T. Kwok
Venue: Conference ICML 2004
Year: 2004

In kernel methods, an interesting recent development seeks to learn a good kernel from empirical data automatically. In this paper, by regarding the transductive learning of the kernel matrix as a missing data problem, we propose a Bayesian hierarchical model for the problem and devise the Tanner-Wong data augmentation algorithm for making inference on the model. The Tanner-Wong algorithm is closely related to Gibbs sampling, and it also bears a strong resemblance to the expectation-maximization (EM) algorithm. For an efficient implementation, we propose a simplified Bayesian hierarchical model and the corresponding Tanner-Wong algorithm. We express the relationship between the kernel on the input space and the kernel on the output space as a symmetric-definite generalized eigenproblem. Based on this eigenproblem, an efficient approach to choosing the base kernel matrices is presented. The effectiveness of our Bayesian model with the Tanner-Wong algorithm is demonstrated through some classification experiments showing promising results.
Modified MMI/MPE: A direct evaluation of the margin in speech recognition
Authors: Georg Heigold, Thomas Deselaers, Ralf Schlüter, Hermann Ney
Venue: Conference ICML 2008
Year: 2008

In this paper we show how common speech recognition training criteria such as the Minimum Phone Error criterion or the Maximum Mutual Information criterion can be extended to incorporate a margin term. Different margin-based training algorithms have been proposed to refine existing training algorithms for general machine learning problems. However, for speech recognition, some special problems have to be addressed and all approaches proposed either lack practical applicability or the inclusion of a margin term enforces significant changes to the underlying model, e.g. the optimization algorithm, the loss function, or the parameterization of the model. In our approach, the conventional training criteria are modified to incorporate a margin term. This allows us to do large-margin training in speech recognition using the same efficient algorithms for accumulation and optimization and to use the same software as for conventional discriminative training. We show that the proposed criteria are equivalent to Support Vector Machines with suitable smooth loss functions, approximating the non-smooth hinge loss function or the hard error (e.g. phone error). Experimental results are given for two different tasks: the rather simple digit string recognition task Sietill which severely suffers from overfitting and the large vocabulary European Parliament Plenary Sessions English task which is supposed to be dominated by the risk and the generalization does not seem to be such an issue.
Extracting structured information from user queries with semi-supervised conditional random fields
Authors: Xiao Li, Ye-Yi Wang, Alex Acero
Venue: Conference SIGIR 2009
Year: 2009

When search is against structured documents, it is beneficial to extract information from user queries in a format that is consistent with the backend data structure. As one step toward this goal, we study the problem of query tagging which is to assign each query term to a pre-defined category. Our problem could be approached by learning a conditional random field (CRF) model (or other statistical models) in a supervised fashion, but this would require substantial human-annotation effort. In this work, we focus on a semi-supervised learning method for CRFs that utilizes two data sources: (1) a small amount of manually-labeled queries, and (2) a large amount of queries in which some word tokens have derived labels, i.e., label information automatically obtained from additional resources. We present two principled ways of encoding derived label information in a CRF model. Such information is viewed as hard evidence in one setting and as soft evidence in the other. In addition to the general methodology of how to use derived labels in semi-supervised CRFs, we also present a practical method on how to obtain them by leveraging user click data and an in-domain database that contains structured documents. Evaluation on product search queries shows the effectiveness of our approach in improving tagging accuracies.
Equivalence of non-iterative algorithms for simultaneous low rank approximations of matrices
Authors: Kohei Inoue, Kiichi Urahama
Venue: Conference CVPR 2006
Year: 2006

Recently four non-iterative algorithms for simultaneous low rank approximations of matrices (SLRAM) have been presented by several researchers. In this paper, we show that those algorithms are equivalent to each other because they are reduced to the eigenvalue problems of row-row and column-column covariance matrices of given matrices. Also, we show a relationship between the non-iterative algorithms and another algorithm which is claimed to be an analytical algorithm for the SLRAM. Experimental results show that the analytical algorithm does not necessarily give the optimal solution of the SLRAM.
Gbase: a scalable and general graph management system
Authors: U Kang, Hanghang Tong, Jimeng Sun, Ching-Yung Lin, Christos Faloutsos
Venue: Conference KDD 2011
Year: 2011

Graphs appear in numerous applications including cyber-security, the Internet, social networks, protein networks, recommendation systems, and many more. Graphs with millions or even billions of nodes and edges are common-place. How to store such large graphs efficiently? What are the core operations/queries on those graph? How to answer the graph queries quickly? We propose GBASE, a scalable and general graph management and mining system. The key novelties lie in 1) our storage and compression scheme for a parallel setting and 2) the carefully chosen graph operations and their efficient implementation. We designed and implemented an instance of GBASE using MapReduce/Hadoop. GBASE provides a parallel indexing mechanism for graph mining operations that both saves storage space, as well as accelerates queries. We ran numerous experiments on real graphs, spanning billions of nodes and edges, and we show that our proposed GBASE is indeed fast, scalable and nimble, with significant savings in space and time.
Query likelihood with negative query generation
Authors: Yuanhua Lv, ChengXiang Zhai
Venue: Conference CIKM 2012
Year: 2012

The query likelihood retrieval function has proven to be empirically effective for many retrieval tasks. From theoretical perspective, however, the justification of the standard query likelihood retrieval function requires an unrealistic assumption that ignores the generation of a "negative query" from a document. This suggests that it is a potentially non-optimal retrieval function.
Regularized latent semantic indexing
Authors: Quan Wang, Jun Xu, Hang Li, Nick Craswell
Venue: Conference SIGIR 2011
Year: 2011

Topic modeling can boost the performance of information retrieval, but its real-world application is limited due to scalability issues. Scaling to larger document collections via parallelization is an active area of research, but most solutions require drastic steps such as vastly reducing input vocabulary. We introduce Regularized Latent Semantic Indexing (RLSI), a new method which is designed for parallelization. It is as effective as existing topic models, and scales to larger datasets without reducing input vocabulary. RLSI formalizes topic modeling as a problem of minimizing a quadratic loss function regularized by l₂ and/or l₁ norm. This formulation allows the learning process to be decomposed into multiple sub-optimization problems which can be optimized in parallel, for example via MapReduce. We particularly propose adopting l₂ norm on topics and l₁ norm on document representations, to create a model with compact and readable topics and useful for retrieval. Relevance ranking experiments on three TREC datasets show that RLSI performs better than LSI, PLSI, and LDA, and the improvements are sometimes statistically significant. Experiments on a web dataset, containing about 1.6 million documents and 7 million terms, demonstrate a similar boost in performance on a larger corpus and vocabulary than in previous studies.
Contextual weighting for vocabulary tree based image retrieval
Authors: Xiaoyu Wang 0002, Ming Yang 0007, Timothée Cour, Shenghuo Zhu, Kai Yu 0001, Tony X. Han
Venue: Conference ICCV 2011
Year: 2011

In this paper we address the problem of image retrieval from millions of database images. We improve the vocabulary tree based approach by introducing contextual weighting of local features in both descriptor and spatial domains. Specifically, we propose to incorporate efficient statistics of neighbor descriptors both on the vocabulary tree and in the image spatial domain into the retrieval. These contextual cues substantially enhance the discriminative power of individual local features with very small computational overhead. We have conducted extensive experiments on benchmark datasets, i.e., the UKbench, Holidays, and our new Mobile dataset, which show that our method reaches state-of-the-art performance with much less computation. Furthermore, the proposed method demonstrates excellent scalability in terms of both retrieval accuracy and efficiency on large-scale experiments using 1.26 million images from the ImageNet database as distractors.
Using 3D scene structure to improve tracking
Authors: Jan Prokaj, Gérard G. Medioni
Venue: Conference CVPR 2011
Year: 2011

In this work we consider the problem of tracking objects from a moving airborne platform in wide area surveillance through long occlusions and/or when their motion is unpredictable. The main idea is to take advantage of the known 3D scene structure to estimate a dynamic occlusion map, and to use the occlusion map to determine traffic entry and exit into these zones, which we call sources and sinks. Then the track linking problem is formulated as an alignment of sequences of tracks entering a sink and leaving a source. The sequence alignment problem is solved optimally and efficiently using dynamic programming. We have evaluated our algorithm on a vehicle tracking task in wide area motion imagery and have shown that track fragmentation is significantly decreased and outperforms the Hungarian algorithm.
Vehicle detection and tracking in wide field-of-view aerial video
Authors: Jiangjian Xiao, Hui Cheng, Harpreet S. Sawhney, Feng Han 0002
Venue: Conference CVPR 2010
Year: 2010

This paper presents a joint probabilistic relation graph approach to simultaneously detect and track a large number of vehicles in low frame rate aerial videos. Due to low frame rate, low spatial resolution and sheer number of moving objects, detection and tracking in wide area video poses unique challenges. In this paper, we explore vehicle behavior model from road structure and generate a set of constraints to regulate both object based vertex matching and pairwise edge matching schemes. The proposed relation graph approach then unifies these two matching schemes into a single cost minimization framework to produce a quadratic optimized association result. The experiments on hours of real videos demonstrate the graph matching framework with vehicle behavior model effectively improves tracking performance in large scale dense traffic scenarios.
Online decoding of markov models under latency constraints
Authors: Mukund Narasimhan, Paul A. Viola, Michael Shilman
Venue: Conference ICML 2006
Year: 2006

The Viterbi algorithm is an efficient and optimal method for decoding linear-chain Markov Models. However, the entire input sequence must be observed before the labels for any time step can be generated, and therefore Viterbi cannot be directly applied to online/interactive/streaming scenarios without incurring significant (possibly unbounded) latency. A widely used approach is to break the input stream into fixed-size windows, and apply Viterbi to each window. Larger windows lead to higher accuracy, but result in higher latency.We propose several alternative algorithms to the fixed-sized window decoding approach. These approaches compute a certainty measure on predicted labels that allows us to trade off latency for expected accuracy dynamically, without having to choose a fixed window size up front. Not surprisingly, this more principled approach gives us a substantial improvement over choosing a fixed window. We show the effectiveness of the approach for the task of spotting semi-structured information in large documents. When compared to full Viterbi, the approach suffers a 0.1 percent error degradation with a average latency of 2.6 time steps (versus the potentially infinite latency of Viterbi). When compared to fixed windows Viterbi, we achieve a 40x reduction in error and 6x reduction in latency.
Rectifying transformations that minimize resampling effects
Authors: Joshua Gluckman, Shree K. Nayar
Venue: Conference CVPR 2001
Year: 2001

Image rectification is the process of warping a pair of stereo images in order to align the epipolar lines with the scan-lines of the images. Once a pair of images is rectified, stereo matching can be implemented in an efficient manner. Given the epipolar geometry, it is straightforward to define a rectifying transformation, however, many transformations will lead to unwanted image distortions. In this paper, we present a novel method for stereo rectification that determines the transformation that minimizes the effects of resampling that can impede stereo matching. The effects we seek to minimize are the loss of pixels due to under-sampling and the creation of new pixels due to over-sampling. To minimize these effects we parameterize the family of rectification transformations and solve for the one that minimizes the change in local area integrated over the area of the images.
Certified defenses against adversarial examples
Authors: Aditi Raghunathan, Jacob Steinhardt, Percy Liang
Venue: Conference ICLR 2018
Year: 2018

While neural networks have achieved high accuracy on standard image classification benchmarks, their accuracy drops to nearly zero in the presence of small adversarial perturbations to test inputs. Defenses based on regularization and adversarial training have been proposed, but often followed by new, stronger attacks that defeat these defenses. Can we somehow end this arms race? In this work, we study this problem for neural networks with one hidden layer. We first propose a method based on a semidefinite relaxation that outputs a certificate that for a given network and test input, no attack can force the error to exceed a certain value. Second, as this certificate is differentiable, we jointly optimize it with the network parameters, providing an adaptive regularizer that encourages robustness against all attacks. On MNIST, our approach produces a network and a certificate that no attack that perturbs each pixel by at most ε= 0.1 can cause more than 35% test error.
On detecting adversarial perturbations
Authors: Jan Hendrik Metzen, Tim Genewein, Volker Fischer 0003, Bastian Bischoff
Venue: Conference ICLR 2017
Year: 2017

Machine learning and deep learning in particular has advanced tremendously on perceptual tasks in recent years. However, it remains vulnerable against adversarial perturbations of the input that have been crafted specifically to fool the system while being quasi-imperceptible to a human. In this work, we propose to augment deep neural networks with a small "detector" subnetwork which is trained on the binary classification task of distinguishing genuine data from data containing adversarial perturbations. Our method is orthogonal to prior work on addressing adversarial perturbations, which has mostly focused on making the classification network itself more robust. We show empirically that adversarial perturbations can be detected surprisingly well even though they are quasi-imperceptible to humans. Moreover, while the detectors have been trained to detect only a specific adversary, they generalize to similar and weaker adversaries. In addition, we propose an adversarial attack that fools both the classifier and the detector and a novel training procedure for the detector that counteracts this attack.
Disentangling user conversations with voice assistants for online shopping
Authors: Nikhita Vedula, Marcus D. Collins, Oleg Rokhlenko
Venue: Conference SIGIR 2023
Year: 2023

Conversation disentanglement aims to identify and group utterances from a conversation into separate threads. Existing methods primarily focus on disentangling multi-party conversations with three or more speakers, explicitly or implicitly incorporating speaker-related feature signals to disentangle. Most existing models require a large amount of human annotated data for model training, and often focus on pairwise relations between utterances, not accounting much for the conversational context. In this work, we propose a multi-task learning approach with a contrastive learning objective, DiSC, to disentangle conversations between two speakers -- a user and a virtual speech assistant, for a novel domain of e-commerce. We analyze multiple ways and granularities to define conversation "threads''. DiSC jointly learns the relation between pairs of utterances, as well as between utterances and their respective thread context. We train and evaluate our models on multiple multi-threaded conversation datasets that were automatically created, without any human labeling effort. Experimental results on public datasets as well as real-world shopping conversations from a commercial speech assistant show that DiSC outperforms state-of-the-art baselines by at least 3%, across both automatic and human evaluation metrics. We also demonstrate how DiSC improves downstream dialog response generation in the shopping domain.
A Framework for Recognizing a Facial Image from A Police Sketch
Authors: Robert G. Uhl Jr., Niels da Vitoria Lobo
Venue: Conference CVPR 1996
Year: 1996

We present a theory and practical computations for automatically matching a police artist sketch to a set of true photographs. We locate facial features in both the sketch as well as the set of photograph images. Then, the sketch is photometrically standardized to facilitate comparison with a photo and then both the sketch and the photos are geometrically standardized. Finally, for matching, eigenanalysis is employed. Results using real police sketches and arrest photos are presented.
A complete two-plane camera calibration method and experimental comparisons
Authors: Guo-Qing Wei, Song De Ma
Venue: Conference ICCV 1993
Year: 1993

For both 3-D reconstruction and prediction of image coordinates, cameras can be calibrated implicitly without involving their physical parameters. The authors present a two-plane method for such a complete calibration, which models all kinds of lens distortions. First, the modeling is done in a general case without imposing the pinhole constraint. Epipolar curves considering lens distortions are introduced and are found in a closed form. Then, a set of constraints of perspectivity is derived to constrain the modeling process. With these constraints, the camera physical parameters can be related directly to the modeling parameters. Extensive experimental comparisons of the methods with the classic photogrammetric method and Tsai's method relating to the aspects of 3-D measurement, the effect of the number of calibration points, and the prediction of image coordinates, are made using real images from 15 different depth values.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Netizen-Style Commenting on Fashion Photos: Dataset and Diversity Measures
Authors: Wen Hua Lin, Kuan-Ting Chen, HungYueh Chiang, Winston H. Hsu
Venue: Conference WWW 2018
Year: 2018

Recently, deep neural network models have achieved promising results in image captioning task. Yet, "vanilla'' sentences, only describing shallow appearances (e.g., types, colors), generated by current works are not satisfied netizen style resulting in lacking engagements, contexts, and user intentions. To tackle this problem, we propose Netizen Style Commenting (NSC), to automatically generate characteristic comments to a user-contributed fashion photo. We are devoted to modulating the comments in a vivid "netizen'' style which reflects the culture in a designated social community and hopes to facilitate more engagement with users. In this work, we design a novel framework that consists of three major components: (1) We construct a large-scale clothing dataset named NetiLook, which contains 300K posts (photos) with 5M comments to discover netizen-style comments. (2) We propose three unique measures to estimate the diversity of comments. (3) We bring diversity by marrying topic models with neural networks to make up the insufficiency of conventional image captioning works. Experimenting over Flickr30k and our NetiLook datasets, we demonstrate our proposed approaches benefit fashion photo commenting and improve image captioning tasks both in accuracy and diversity.
Photometric stereo and weather estimation using internet images
Authors: Li Shen 0003, Ping Tan 0002
Venue: Conference CVPR 2009
Year: 2009

We extend photometric stereo to make it work with internet images, which are typically associated with different viewpoints and significant noise. For popular tourism sites, thousands of images can be obtained from internet search engines. With these images, our method computes the global illumination for each image and the surface orientation at some scene points. The illumination information can then be used to estimate the weather conditions (such as sunny or cloudy) for each image, since there is a strong correlation between weather and scene illumination. We demonstrate our method on several challenging examples.
Sensing increased image resolution using aperture masks
Authors: Ankit Mohan, Xiang Huang 0006, Jack Tumblin, Ramesh Raskar
Venue: Conference CVPR 2008
Year: 2008

We present a technique to construct increased-resolution images from multiple photos taken without moving the camera or the sensor. Like other super-resolution techniques, we capture and merge multiple images, but instead of moving the camera sensor by sub-pixel distances for each image, we change masks in the lens aperture and slightly defocus the lens. The resulting capture system is simpler, and tolerates modest mask registration errors well. We present a theoretical analysis of the camera and image merging method, show both simulated results and actual results from a crudely modified consumer camera, and compare its results to robust ‘blind’ methods that rely on uncontrolled camera displacements.