One-Shot Adaptation of Supervised Deep Convolutional Models
Authors: Trevor Darrell, Eric Tzeng, Yangqing Jia, Judy Hoffman, Kate Saenko, Jeff Donahue
ICLR 2014

Abstract: Dataset bias remains a significant barrier towards solving real world computer vision tasks. Though deep convolutional networks have proven to be a competitive approach for image classification, a question remains: have these models have solved the dataset bias problem? In general, training or fine-tuning a state-of-the-art deep model on a new domain requires a significant amount of data, which for many applications is simply not available. Transfer of models directly to new domains without adaptation has historically led to poor recognition performance. In this paper, we pose the following question: is a single image dataset, much larger than previously explored for adaptation, comprehensive enough to learn general deep models that may be effectively applied to new image domains? In other words, are deep CNNs trained on large amounts of labeled data as susceptible to dataset bias as previous methods have been shown to be? We show that a generic supervised deep CNN model trained on a large dataset reduces, but does not remove, dataset bias. Furthermore, we propose several methods for adaptation with deep models that are able to operate with little (one example per category) or no labeled domain specific data. Our experiments show that adaptation of deep models on benchmark visual domain adaptation datasets can provide a significant performance boost.


Peer review:
[Review]:
This paper studies dataset bias problem. It investigates whether using large source datasets can eliminate dataset bias. The paper shows that such datasets reduce but still do not remove completely dataset bias.  It also shows that deep learning features are useful in helping domain adaptation. 

This is a largely empirical study of the important issues.  The observation made the paper is important and thought-inspiring and worth reporting.

It might be interesting to report all layers (instead of just DeCAF6 and DeCAF7)'s performance on adaptation --- is it always the case that high-level layers are better at adaptation? 

GFK was used in the paper as an unsupervised domain adaptation method. However, it can be used easily as a semi-supervised or supervised method. For example, once GFK is learnt on unlabeled data, one can learn a classifier by revealing the labels of the target data. The benefit is a better metric is used to measure distances.