CARIS: Context-Aware Referring Image Segmentation

This repository is for the ACM MM 2023 paper "CARIS: Context-Aware Referring Image Segmentation". Code is available here.

Pretrained CARIS Models

We have released pretrained CARIS models including:

• standard: 4 models trained on the training split of RefCOCO/RefCOCO+/RefCOCOg separately,
• combined: 1 model trained on the combination of the training splits of RefCOCO/RefCOCO+/RefCOCOg(umd split).

Performance (standard)

Dataset	P@0.5	P@0.7	P@0.9	Overall IoU	Mean IoU
RefCOCO val	87.92	80.99	39.54	75.13	77.41
RefCOCO testA	91.13	84.55	40.06	78.19	79.71
RefCOCO testB	83.75	84.82	40.47	71.75	74.51
RefCOCO+ val	79.97	73.54	36.15	66.63	70.66
RefCOCO+ testA	85.33	79.17	36.87	72.17	74.82
RefCOCO+ testB	71.81	63.51	33.48	58.54	63.97
RefCOCOg val(U)	77.39	68.57	32.62	64.32	68.61
RefCOCOg test(U)	78.11	69.06	32.27	65.82	69.02
RefCOCOg val(G)	76.72	68.72	34.03	63.84	67.80

Performance (combined)

Dataset	P@0.5	P@0.7	P@0.9	Overall IoU	Mean IoU
RefCOCO val	88.94	81.93	40.75	76.63	78.49
RefCOCO testA	91.50	84.80	40.14	79.40	80.21
RefCOCO testB	84.51	76.25	41.92	73.52	75.42
RefCOCO+ val	81.07	74.56	37.00	68.02	71.81
RefCOCO+ testA	86.10	80.00	37.98	73.69	75.75
RefCOCO+ testB	72.76	65.13	35.51	60.41	65.15
RefCOCOg val(U)	80.84	71.73	35.07	67.95	71.53
RefCOCOg test(U)	81.91	73.22	34.95	69.75	72.18