Package: davisinteractive
DAVIS Interactive Evaluation FrameworkThis is a framework to evaluate interactive segmentation models over theDAVIS 2017dataset. The code aims to provide an easy-to-use interface to test and validate interactive segmentation models.This tool is also used to evaluate theInteractive Trackof the DAVIS Challenges on Video Object Segmentation. More information about the latest challenge edition in theDAVIS website.You can find an example of how to use the package in the following repository:Scribble-OSVOSDAVIS ScribblesIn the classical DAVIS Semi-supervised Challenge track, the task is to segment an object in asemi-supervisedmanner, i.e. the given input is the ground truth mask of the first frame. In theDAVIS Interactive Challenge, in contrast, the user input is scribbles, which can be drawn much faster by humans and thus are a more realistic type of input.The interactive annotation and segmentation consist in an iterative loop which is evaluated as follows:In the first interaction, a human-annotated scribble for each object in the video sequence is provided to the segmentation model. As a result, the model has to predict a segmentation mask containing all the objects for all the frames.Note: all the scribbles are annotated in a single frame, but this does not have to be the first frame in the sequence, as the annotators were instructed to annotate the most relevant and meaningful frame. This is in contrast to the semi-supervised track, where - only and strictly - the first frame is annotated.Then, the predicted masks are submitted to a server that returns human-simulated scribbles. These scribbles are always annotated in a single frame. The frame is selected as the one with the worst evaluation result among a list of frames specified by the user. By default, this list contains all the frames in the sequence.During the following steps, the segmentation model keeps iterating between predicting the masks using the new scribbles and submitting the masks to obatain new scribbles.Evaluation: The evaluation metric is the mean of the Region similarity $\mathcal{J}$ and the Contour Accuracy $\mathcal{F}$. More information of the metricshere. The evaluation for thetrainandvalsubsets can be done offline at any time, whereas the evaluation for thetest-devhas to be done against a server that is only available during the challanges period.More information: Please check theInstallationguide to install the package and dowload the scribbles. Moreover, refer to theUsageguide to learn how to interface your code with the server.Contributions: If you would like to add new features to the package, please do not hesitate to send apull request.CitationPlease cite both papers in your publications if DAVIS or this code helps your research.@article{Caelles_arXiv_2018,
  author ={Sergi Caelles and Alberto Montes and Kevis-Kokitsi Maninis and Yuhua Chen and Luc{Van Gool}and Federico Perazzi and Jordi Pont-Tuset},
  title ={The 2018 DAVIS Challenge on Video Object Segmentation},
  journal ={arXiv:1803.00557},
  year ={2018}}@article{Pont-Tuset_arXiv_2017,
  author ={Jordi Pont-Tuset and Federico Perazzi and Sergi Caelles and Pablo Arbel\'aez and Alexander Sorkine-Hornung and Luc{Van Gool}},
  title ={The 2017 DAVIS Challenge on Video Object Segmentation},
  journal ={arXiv:1704.00675},
  year ={2017}}