This paper introduces Click'n'Cut, a novel web tool for interactive object segmentation designed for crowdsourcing tasks. Click'n'Cut combines bounding boxes and clicks generated by workers to obtain accurate object segmentations. These segmentations are created by combining precomputed object candidates in a light computational fashion that allows an immediate response from the interface. Click'n'Cut has been tested with a crowdsourcing campaign to annotate images from publicly available datasets. Results are competitive with state-of-the-art approaches, especially in terms of time needed to converge to a high quality segmentation.  Categories and Subject Descriptors Computer vision is an active research area, which has became increasingly relevant and pervasive, thanks to the growing amount of ubiquitous cameras that generate large amounts of visual data. A classic problem in computer vision systems is object detection, a task in which machines are expected to locate and recognize the objects present in a scene with an accuracy similar or better than a human would achieve. The mapping between the quantized pixels analyzed by computers and the concepts with which the human mind seems to operate has been referred as the semantic gap. Despite continuous advances in the field, such a gap has not been completely bridged yet. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. This paper addresses the problem of how to leverage the human power of the crowd into solving the problem of pixelwise object segmentation. The goal of an object segmentation process is to label all pixels of an image depending on whether they belong to a certain object or not. Typically, the output of such process is a binary mask whose white pixels represent the object and the black pixels correspond to the background. Our work explores how crowdsourcing can be used as a valid strategy for obtaining a large amount of high quality object segmentation results (masks), as long as the appropriate tools and data collection strategies are adopted. The main contribution of this paper is the design and test of an online web interface (Click'n'Cut) that is responsive to the users' interactions and guides them into collecting highly informative data. Our results indicate that this tool is competitive with other state-of-the-art interactive segmentation techniques, especially in terms of responsiveness. Additionally, we present a technique to filter out noisy user interactions, specifically designed for the object segmentation task. Interactive Object Segmentation. The segmentation of objects by combining human interaction and image processing algorithms has been extensively explored in the literature. In such interactive setup, the graphical user interface responds to some sort of weak annotation (e.g., bounding box, scribbles, or clicks) from the user by generating and displaying a complete segmentation of the object. The typical workflow expects the user to interact with the proposed solution either by accepting it or by providing more traces that may allow the segmentation algorithm to converge to a satisfactory result. Most interactive segmentation techniques normally propagate the user-generated labels of foreground and background pixels through a graph-based representation of the image. The foundational proposal for interactive foreground segmentation was based on graph cuts [5, 16] . The algorithm considers every pixel as a node in a graph, connected to their spatial neighbors by an edge whose weight depends on the visual similarity between pixels. Segmenting an object is equivalent to minimizing an energy function defined over the graph. Partition-based solutions [15, 13] avoid the computational load of a pixel-by-pixel segmentation by working with unsupervised image segmentations performed offline. The adjacency information between regions coded in a partition can be further enriched by iteratively merging pairs 53
Our Click'n'Cut system has been introduced in REF and uses this combination of object candidates.
Click'n'Cut: Crowdsourced Interactive Segmentation with Object Candidates
CrowdMM '14
Computer Science