  A robot that can operate in novel and unstructured environments must be
capable of recognizing new, previously unseen, objects. In this work, a visual
cue is used to specify a novel object of interest which must be localized in
new environments. An end-to-end neural network equipped with a Siamese network
is used to learn the cue, infer the object of interest, and then to localize it
in new environments. We show that a simulated robot can pick-and-place novel
objects pointed to by a laser pointer. We also evaluate the performance of the
proposed approach on a dataset derived from the Omniglot handwritten character
dataset and on a small dataset of toys.
