  Within the field of instance segmentation, most of the state-of-the-art deep
learning networks rely nowadays on cascade architectures, where multiple object
detectors are trained sequentially, re-sampling the ground truth at each step.
This offers a solution to the problem of exponentially vanishing positive
samples. However, it also translates into an increase in network complexity in
terms of the number of parameters. To address this issue, we propose
Recursively Refined R-CNN ($R^3$-CNN) which avoids duplicates by introducing a
loop mechanism instead. At the same time, it achieves a quality boost using a
recursive re-sampling technique, where a specific IoU quality is utilized in
each recursion to eventually equally cover the positive spectrum. Our
experiments highlight the specific encoding of the loop mechanism in the
weights, requiring its usage at inference time. The $R^3$-CNN architecture is
able to surpass the recently proposed HTC model, while reducing the number of
parameters significantly. Experiments on COCO minival 2017 dataset show
performance boost independently from the utilized baseline model. The code is
available online at https://github.com/IMPLabUniPr/mmdetection/tree/r3_cnn.
