We present an image retrieval system driven by free-hand sketched queries depicting shape. We introduce Gradient Field HoG (GF-HOG) as a depiction invariant image descriptor, encapsulating local spatial structure in the sketch and facilitating efficient codebook based retrieval. We show improved retrieval accuracy over 3 leading descriptors (Self Similarity, SIFT, HoG) across two datasets (Flickr160, ETHZ extended objects), and explain how GF-HOG can be combined with RANSAC to localize sketched objects within relevant images. We also demonstrate a prototype sketch driven photo montage application based on our system.
Hu et al. REF introduced Gradient Field HoG (GF-HOG) for describing images and incorporated the GF-HOG descriptors into a BoW system to improve the retrieval performance.
Gradient field descriptor for sketch based retrieval and localization
2010 IEEE International Conference on Image Processing
2010 IEEE International Conference on Image Processing
Computer Science