We present and evaluate an approach for human-in-the-loop specification of shape reconstruction with annotations for basic robot-object interactions. Our method is based on the idea of model annotation: the addition of simple cues to an underlying object model to specify shape and delineate a simple task. The goal is to explore reducing the complexity of CAD-like interfaces so that novice users can quickly recover an object's shape and describe a manipulation task that is then carried out by a robot. The object modeling and interaction annotation capabilities are tested with a user study and compared against results obtained using existing approaches. The approach has been analyzed using a variety of shape comparison, grasping, and manipulation metrics, and tested with the PR2 robot platform, where it was shown to be successful.
Annotations that help solve particular instances of problems can be gathered into a knowledge-base that can, in turn, be studied to gain insights into the solving of more general situations. One way to gather a large amount of usable annotations is to take advantage of the growing infrastructure for tasks, like Amazon's Mechanical Turk @cite_25 .
@cite_25
2141708418
Human computation can address complex computational problems by tapping into large resource pools for relatively little cost. Two prominent human-computation techniques - games with a purpose (GWAP) and microtask crowdsourcing - can help resolve semantic-technology-related tasks, including knowledge representation, ontology alignment, and semantic annotation. To evaluate which approach is better with respect to costs and benefits, the authors employ categorization challenges in Wikipedia to ultimately create a large, general-purpose ontology. They first use the OntoPronto GWAP, then replicate its problem-solving setting in Amazon Mechanical Turk, using a similar task-design structure, evaluation mechanisms, and input data. In this work we present results from using Amazon's Mechanical Turk (MTurk) to annotate translation lexicons between English and a large set of less commonly used languages. We generate candidate translations for 100 English words in each of 42 foreign languages using Wikipedia and a lexicon induction framework. We evaluate the MTurk annotations by using positive and negative control candidate translations. Additionally, we evaluate the annotations by adding pairs to our seed dictionaries, providing a feedback loop into the induction system. MTurk workers are more successful in annotating some languages than others and are not evenly distributed around the world or among the world's languages. However, in general, we find that MTurk is a valuable resource for gathering cheap and simple annotations for most of the languages that we explored, and these annotations provide useful feedback in building a larger, more accurate lexicon. Recent efforts in computer vision tackle the problem of human activity understanding in video sequences. Traditionally, these algorithms require annotated video data to learn models. In this paper, we introduce a novel data collection framework, to take advantage of the large amount of video data available on the web. We use this new framework to retrieve videos of human activities in order to build datasets for training and evaluating computer vision algorithms. We rely on Amazon Mechanical Turk workers to obtain high accuracy annotations. An agglomerative clustering technique brings the possibility to achieve reliable and consistent annotations for temporal localization of human activities in videos. Using two different datasets, Olympics Sports and our novel Daily Human Activities dataset, we show that our collection annotation framework achieves robust annotations for human activities in large amount of video data. We show how to outsource data annotation to Amazon Mechanical Turk. Doing so has produced annotations in quite large numbers relatively cheaply. The quality is good, and can be checked and controlled. Annotations are produced quickly. We describe results for several different annotation problems. We describe some strategies for determining when the task is well specified and properly priced.