Interactive segmentation•

1. Deep Extreme Cut: From Extreme Points to Object Segmentation•

2. Interactive Image Segmentation with Latent Diversity•

3. Fast Interactive Object Annotation with Curve-GCN•

4. PhraseClick: Toward Achieving Flexible Interactive Segmentation by Phrase and Click•

5. Deep Interactive Thin Object Selection•

6. Efficient Full Image Interactive Segmentation by Leveraging Within-image Appearance
Similarity

Deep Extreme Cut: From Extreme Points to ObjectSegmentation•This paper use of extreme points in an object (left-most, right-most, top, bottom pixels) asinput to obtain precise object segmentation for images and videos.

••

1. Input: a 4-channel input (RGB+ heatmap). We center a 2D Gaussian around each of thepoints, in order to create a single heatmap. The input is cropped by the bounding box,
formed from the extreme point annotations.

pyramid scene parsing
1×1
2×2
3×3
6×6

•
•

•

1. Input: a 4-channel input (RGB+ heatmap).
2. Network: Deeplab-v2 model. choose ResNet-101 as the backbone of our architecture.
remove the fully connected layers as well as the max pooling layers in the last two stages.
user pyramid scene parsing (PSP) module instead of Atrous spatial pyramid (ASPP).
3. Loss:•••

1. Input: a 4-channel input (RGB+ heatmap).
2. Network: Deeplab-v2 model.
3. Loss: class-balanced cross entropy loss , where the loss for each class in the batch isweighted by its inverse frequency.

Interactive Image Segmentation with Latent Diversity•

When the user clicks on a door, do they intend to select the door or the whole house?

••The first is trained to synthesize a diverse set of plausible segmentations that conform to theuser’s input.
The second is trained to select among these.

•••Input: The total number of channels in the input is 1,477 (the image X, clicks Sp and Sn,distance maps, feature map )feature maps : Use VGG-19 network pretrained on the ImageNet to extract the featuremaps from the following layers: ‘conv1 2’, ‘conv2 2’, ‘conv3 2’, ‘conv4 2’, and ‘conv5 2’.
clicks & distance maps:••

The output layer has M channels, one for each synthesized segmentation mask.
The final nonlinearity is a sigmoid that maps each pixel to the range [0, 1].
Loss:

•

Loss:

•
•

Jaccard (IoU) distance：
m: mask number (m=6)

•

Loss:

•

soft constraints

•

Input: The the image X, clicks Sp and Sn, distance maps and M channels of SegmentationMasks••

The output : M-vector
Loss：cross-entropy loss•testing: The first click is positive and each subsequent click is placed on apixel that is still misclassified .

Fast Interactive Object Annotation with Curve-GCN••

Predict all the vertices of a polygon using a Graph Convolutional Network simultaneously .

Extracting Features•

1. CNN serving as an image feature extractor.
– additional branches are trained to predict the probability of existence of an object
edge/vertex on a 28 × 28 grid. We train these two branches with the binary
cross entropy loss.

initialize the GCN nodes••

1.CNN serving as an image feature extractor.
2. GCN:––––We initialize the nodes of the GCN to be at a static initial central position.
Represent object using N control points, which are connected to form a cycle, with straight lines (thus forming apolygon), or higher order curves (forming a spline).
GCN predicts a location offset for each node, aiming to move the node correctly onto the object’s boundary .
We define the graph to be G = (V; E)

•

GCN Model : multi-layer GCN

Input feature

Graph-ResNet ：On top of the last GCN layer, apply a single FC layer to predict a relative location shift:•••

1.CNN serving as an image feature extractor.
2. GCN
3. Spline Parametrization

••
•
•

1.CNN serving as an image feature extractor.
2. GCN
3. Spline Parametrization
4. Loss Point Matching Loss:

Differentiable Accuracy Loss:PhraseClick: Toward Achieving Flexible Interactive
Segmentation by Phrase and Click•propose to employ phrase expressions as another interaction input to infer the attributes oftarget object.

PhraseClick: Toward Achieving Flexible Interactive
Segmentation by Phrase and Click•propose to employ phrase expressions as another interaction input to infer the attributes oftarget object.

••••

Vision part Network: ResNet-101 based DeepLabv3+
distance maps and concatenated with original image to form a 5-channel input .
language part : word-to-vector model and bi-directional LSTM
post-processing : graph cut

Vision part NetworkASPPlanguage part••Loss:
the binary cross-entropy loss for the segmentation•

the attribute learning :Deep Interactive Thin Object Selection•

1. generate rough object segmentation••

1. generate rough object segmentation
2. edge stream•••

1. generate rough object segmentation
2. edge stream
3. fusionEfficient Full Image Interactive Segmentation byLeveraging Within-image Appearance Similarity•

interactive full-image semantic segmentation•

we represent the output of the global pixel similarity computation between the labeled andunlabeled pixels in the form of distance maps.