Package: faster-coco-eval
Faster-COCO-EvalDisclaimerI often use this project, but I saw it abandoned and without a public repository on github.
Also, part of the project remained unfinished for a long time. I implemented some of the author's ideas and decided to make the results publicly available.Faster-COCO-Eval baseThis package wraps a facebook C++ implementation of COCO-eval operations found in thepycocotoolspackage.
This implementation greatly speeds up the evaluation time
for coco's AP metrics, especially when dealing with a high number of instances in an image.ComparisonFor our use case with a test dataset of 5000 images from the coco val dataset.
Testing was carried out using the mmdetection framework and the eval_metric.py script. The indicators are presented below.Visualization of testingcomparison.ipynbavailable in directoryexamples/comparisonTested with yolo3 model (bbox eval) and yoloact model (segm eval)TypeCOCOevalCOCOeval_fasterProfitbbox18.477 sec.7.345 sec.2.5x fastersegm29.819 sec.15.840 sec.2x fasterUsageThis package contains a faster implementation of thepycocotoolsCOCOEvalclass.To import and useCOCOeval_fastertype:fromfaster_coco_evalimportCOCO,COCOeval_faster....For usage, look at the originalCOCOEvalclass documentation.Usage plot curvesfromfaster_coco_evalimportCOCOfromfaster_coco_eval.extraimportCurvescocoGt=COCO(....)cocoDt=cocoGt.loadRes(....)cur=Curves(cocoGt,cocoDt,iou_tresh=0.5,iouType='segm')cur.plot_pre_rec()Usage pnly GT previewcocoGt=COCO(....)preview=PreviewResults(cocoGt,iouType="segm")preview.display_tp_fp_fn(data_folder=.....,image_ids=list(cocoGt.imgs.keys())[:2],display_gt=True,)Setup dependenciesnumpyplotly (optional if extra.Curve usage)historyv1.4.2append Auto-formattersappend py36 supportappend pandas to requirements for plotly[express]update mask api with pycootoolsv1.4.1append Plotly fig returnappend preview GT only func. Without eval.cocoGt=COCO(...)preview=PreviewResults(cocoGt,iouType='segm')preview.display_tp_fp_fn(data_folder=...,image_ids=...,display_gt=True)v1.4.0fix issuehttps://github.com/MiXaiLL76/faster_coco_eval/issues/12Updated pre-rec calculation methodUpdated required librariesMoved all matplotlib dependencies to plotlyAppend new examples & mmeval test filev1.3.3fix by ViTroxhttps://github.com/vitrox-technologies/faster_coco_evalmissing file issueissue discovered by torchmetricfstring for python3.7Windows compilationv1.3.2rework math_matches function. moved to faster_eval_apiMoved calculations from python to c++Separated extra classesAdded new sample dataappend mIoU based on TP pred.append mAUC based on Coco pre/rec.v1.3.1rework mask codechange np.float to float (numpy deprecations)v1.3.0remove pycocotools dependenciesclean c/c++ codev1.2.3Implemented of mean IoU for TPset FP-red FN-bluev1.2.2Removed own implementation of pre-recSwitched to the implementation of pre-rec calculation from COCO evalLost backward compatibilityImplemented output fp/fn/tp + gt to picturesv1.2.1bug fix with pre-rec curverework error calc (tp/fp/fn)change image plot to plotlyappend docker auto builderappend native iou calc (slow but accurate)rework auc calc withlinkv1.1.3-v1.1.4rebuild plotly backendSegm bug-fixv1.1.2Append fp fn error analysis via curvesAppend confusion matrixAppend plotly backend support for ROC / AUCv1.1.1Redesigned curvesReworked data preloadAppend csrc to setupBuild sdist Packagev1.1.0Wrap c++ codeGet it to compileAdd COCOEval class wraperRemove detectron2 dependenciesRemove torch dependenciesAppend unittestAppend ROC / AUC curvesCheck if it works on windowsLicenseThe original module was licensed with apache 2, I will continue with the same license.
Distributed under the apache version 2.0 license, seelicensefor more information.