## panoptic [09/05 15:23:13] d2.data.dataset_mapper INFO: [DatasetMapper] Augmentations used in inference: [ResizeShortestEdge(short_edge_length=(800, 800), max_size=1333, sample_style='choice')] [09/05 15:23:13] d2.data.common INFO: Serializing the dataset using: [09/05 15:23:13] d2.data.common INFO: Serializing 1000 elements to byte tensors and concatenating them all ... [09/05 15:23:13] d2.data.common INFO: Serialized dataset takes 0.90 MiB [09/05 15:23:13] d2.evaluation.evaluator INFO: Start inference on 250 batches [09/05 15:23:29] d2.evaluation.evaluator INFO: Inference done 1/250. Dataloading: 6.3985 s/iter. Inference: 8.7794 s/iter. Eval: 0.1248 s/iter. Total: 15.3067 s/iter. ETA=1:03:31 [09/05 15:23:34] d2.evaluation.evaluator INFO: Inference done 17/250. Dataloading: 0.0016 s/iter. Inference: 0.2801 s/iter. Eval: 0.0415 s/iter. Total: 0.3233 s/iter. ETA=0:01:15 [09/05 15:23:39] d2.evaluation.evaluator INFO: Inference done 33/250. Dataloading: 0.0017 s/iter. Inference: 0.2795 s/iter. Eval: 0.0412 s/iter. Total: 0.3224 s/iter. ETA=0:01:09 [09/05 15:23:44] d2.evaluation.evaluator INFO: Inference done 49/250. Dataloading: 0.0017 s/iter. Inference: 0.2796 s/iter. Eval: 0.0414 s/iter. Total: 0.3227 s/iter. ETA=0:01:04 [09/05 15:23:49] d2.evaluation.evaluator INFO: Inference done 65/250. Dataloading: 0.0017 s/iter. Inference: 0.2797 s/iter. Eval: 0.0412 s/iter. Total: 0.3226 s/iter. ETA=0:00:59 [09/05 15:23:55] d2.evaluation.evaluator INFO: Inference done 81/250. Dataloading: 0.0017 s/iter. Inference: 0.2799 s/iter. Eval: 0.0415 s/iter. Total: 0.3232 s/iter. ETA=0:00:54 [09/05 15:24:00] d2.evaluation.evaluator INFO: Inference done 97/250. Dataloading: 0.0017 s/iter. Inference: 0.2799 s/iter. Eval: 0.0417 s/iter. Total: 0.3233 s/iter. ETA=0:00:49 [09/05 15:24:05] d2.evaluation.evaluator INFO: Inference done 113/250. Dataloading: 0.0017 s/iter. Inference: 0.2798 s/iter. Eval: 0.0417 s/iter. Total: 0.3232 s/iter. ETA=0:00:44 [09/05 15:24:10] d2.evaluation.evaluator INFO: Inference done 129/250. Dataloading: 0.0017 s/iter. Inference: 0.2798 s/iter. Eval: 0.0417 s/iter. Total: 0.3232 s/iter. ETA=0:00:39 [09/05 15:24:15] d2.evaluation.evaluator INFO: Inference done 145/250. Dataloading: 0.0017 s/iter. Inference: 0.2797 s/iter. Eval: 0.0416 s/iter. Total: 0.3230 s/iter. ETA=0:00:33 [09/05 15:24:20] d2.evaluation.evaluator INFO: Inference done 161/250. Dataloading: 0.0017 s/iter. Inference: 0.2798 s/iter. Eval: 0.0416 s/iter. Total: 0.3231 s/iter. ETA=0:00:28 [09/05 15:24:26] d2.evaluation.evaluator INFO: Inference done 177/250. Dataloading: 0.0017 s/iter. Inference: 0.2798 s/iter. Eval: 0.0415 s/iter. Total: 0.3230 s/iter. ETA=0:00:23 [09/05 15:24:31] d2.evaluation.evaluator INFO: Inference done 193/250. Dataloading: 0.0017 s/iter. Inference: 0.2797 s/iter. Eval: 0.0415 s/iter. Total: 0.3230 s/iter. ETA=0:00:18 [09/05 15:24:36] d2.evaluation.evaluator INFO: Inference done 209/250. Dataloading: 0.0017 s/iter. Inference: 0.2798 s/iter. Eval: 0.0416 s/iter. Total: 0.3231 s/iter. ETA=0:00:13 [09/05 15:24:41] d2.evaluation.evaluator INFO: Inference done 225/250. Dataloading: 0.0017 s/iter. Inference: 0.2798 s/iter. Eval: 0.0415 s/iter. Total: 0.3230 s/iter. ETA=0:00:08 [09/05 15:24:46] d2.evaluation.evaluator INFO: Inference done 241/250. Dataloading: 0.0017 s/iter. Inference: 0.2798 s/iter. Eval: 0.0415 s/iter. Total: 0.3230 s/iter. ETA=0:00:02 [09/05 15:24:51] d2.evaluation.evaluator INFO: Total inference time: 0:01:20.646216 (0.329168 s / iter per device, on 4 devices) [09/05 15:24:51] d2.evaluation.evaluator INFO: Total inference pure compute time: 0:01:08 (0.279827 s / iter per device, on 4 devices) [09/05 15:24:51] d2.evaluation.panoptic_evaluation INFO: Writing all panoptic predictions to /tmp/panoptic_evalkbm4i2_s ... [09/05 15:25:22] d2.evaluation.panoptic_evaluation INFO: Panoptic Evaluation Results: | | PQ | SQ | RQ | #categories | |:------:|:------:|:------:|:------:|:-------------:| | All | 19.267 | 43.738 | 23.496 | 40 | | Things | 29.925 | 66.721 | 36.406 | 10 | | Stuff | 15.714 | 36.077 | 19.192 | 30 | [09/05 15:25:22] d2.engine.defaults INFO: Evaluation results for bdd10k_40_panoptic_val in csv format: [09/05 15:25:22] d2.evaluation.testing INFO: copypaste: Task: panoptic_seg [09/05 15:25:22] d2.evaluation.testing INFO: copypaste: PQ,SQ,RQ,PQ_th,SQ_th,RQ_th,PQ_st,SQ_st,RQ_st [09/05 15:25:22] d2.evaluation.testing INFO: copypaste: 19.2670,43.7382,23.4955,29.9245,66.7207,36.4057,15.7145,36.0774,19.1921 ## semantic [05/13 20:58:58] d2.data.dataset_mapper INFO: [DatasetMapper] Augmentations used in inference: [ResizeShortestEdge(short_edge_length=(800, 800), max_size=1333, sample_style='choice')] [05/13 20:58:58] d2.data.common INFO: Serializing the dataset using: [05/13 20:58:58] d2.data.common INFO: Serializing 1000 elements to byte tensors and concatenating them all ... [05/13 20:58:58] d2.data.common INFO: Serialized dataset takes 0.22 MiB [05/13 20:58:58] d2.evaluation.evaluator INFO: Start inference on 500 batches [05/13 20:59:07] d2.evaluation.evaluator INFO: Inference done 11/500. Dataloading: 0.0019 s/iter. Inference: 0.2794 s/iter. Eval: 0.0359 s/iter. Total: 0.3172 s/iter. ETA=0:02:35 [05/13 20:59:12] d2.evaluation.evaluator INFO: Inference done 26/500. Dataloading: 0.0026 s/iter. Inference: 0.2958 s/iter. Eval: 0.0368 s/iter. Total: 0.3352 s/iter. ETA=0:02:38 [05/13 20:59:17] d2.evaluation.evaluator INFO: Inference done 42/500. Dataloading: 0.0026 s/iter. Inference: 0.2891 s/iter. Eval: 0.0361 s/iter. Total: 0.3279 s/iter. ETA=0:02:30 [05/13 20:59:22] d2.evaluation.evaluator INFO: Inference done 55/500. Dataloading: 0.0062 s/iter. Inference: 0.2865 s/iter. Eval: 0.0509 s/iter. Total: 0.3437 s/iter. ETA=0:02:32 [05/13 20:59:27] d2.evaluation.evaluator INFO: Inference done 71/500. Dataloading: 0.0053 s/iter. Inference: 0.2847 s/iter. Eval: 0.0476 s/iter. Total: 0.3377 s/iter. ETA=0:02:24 [05/13 20:59:32] d2.evaluation.evaluator INFO: Inference done 87/500. Dataloading: 0.0048 s/iter. Inference: 0.2836 s/iter. Eval: 0.0455 s/iter. Total: 0.3340 s/iter. ETA=0:02:17 [05/13 20:59:37] d2.evaluation.evaluator INFO: Inference done 103/500. Dataloading: 0.0045 s/iter. Inference: 0.2828 s/iter. Eval: 0.0438 s/iter. Total: 0.3312 s/iter. ETA=0:02:11 [05/13 20:59:42] d2.evaluation.evaluator INFO: Inference done 119/500. Dataloading: 0.0042 s/iter. Inference: 0.2823 s/iter. Eval: 0.0428 s/iter. Total: 0.3294 s/iter. ETA=0:02:05 [05/13 20:59:47] d2.evaluation.evaluator INFO: Inference done 135/500. Dataloading: 0.0040 s/iter. Inference: 0.2819 s/iter. Eval: 0.0419 s/iter. Total: 0.3279 s/iter. ETA=0:01:59 [05/13 20:59:53] d2.evaluation.evaluator INFO: Inference done 151/500. Dataloading: 0.0039 s/iter. Inference: 0.2816 s/iter. Eval: 0.0413 s/iter. Total: 0.3268 s/iter. ETA=0:01:54 [05/13 20:59:58] d2.evaluation.evaluator INFO: Inference done 167/500. Dataloading: 0.0037 s/iter. Inference: 0.2813 s/iter. Eval: 0.0408 s/iter. Total: 0.3260 s/iter. ETA=0:01:48 [05/13 21:00:03] d2.evaluation.evaluator INFO: Inference done 183/500. Dataloading: 0.0037 s/iter. Inference: 0.2811 s/iter. Eval: 0.0404 s/iter. Total: 0.3252 s/iter. ETA=0:01:43 [05/13 21:00:08] d2.evaluation.evaluator INFO: Inference done 199/500. Dataloading: 0.0036 s/iter. Inference: 0.2809 s/iter. Eval: 0.0401 s/iter. Total: 0.3246 s/iter. ETA=0:01:37 [05/13 21:00:13] d2.evaluation.evaluator INFO: Inference done 215/500. Dataloading: 0.0035 s/iter. Inference: 0.2808 s/iter. Eval: 0.0398 s/iter. Total: 0.3241 s/iter. ETA=0:01:32 [05/13 21:00:18] d2.evaluation.evaluator INFO: Inference done 231/500. Dataloading: 0.0034 s/iter. Inference: 0.2807 s/iter. Eval: 0.0396 s/iter. Total: 0.3237 s/iter. ETA=0:01:27 [05/13 21:00:23] d2.evaluation.evaluator INFO: Inference done 247/500. Dataloading: 0.0034 s/iter. Inference: 0.2806 s/iter. Eval: 0.0393 s/iter. Total: 0.3234 s/iter. ETA=0:01:21 [05/13 21:00:28] d2.evaluation.evaluator INFO: Inference done 263/500. Dataloading: 0.0033 s/iter. Inference: 0.2805 s/iter. Eval: 0.0391 s/iter. Total: 0.3230 s/iter. ETA=0:01:16 [05/13 21:00:33] d2.evaluation.evaluator INFO: Inference done 279/500. Dataloading: 0.0033 s/iter. Inference: 0.2804 s/iter. Eval: 0.0389 s/iter. Total: 0.3227 s/iter. ETA=0:01:11 [05/13 21:00:38] d2.evaluation.evaluator INFO: Inference done 295/500. Dataloading: 0.0033 s/iter. Inference: 0.2803 s/iter. Eval: 0.0388 s/iter. Total: 0.3225 s/iter. ETA=0:01:06 [05/13 21:00:43] d2.evaluation.evaluator INFO: Inference done 311/500. Dataloading: 0.0032 s/iter. Inference: 0.2803 s/iter. Eval: 0.0387 s/iter. Total: 0.3223 s/iter. ETA=0:01:00 [05/13 21:00:49] d2.evaluation.evaluator INFO: Inference done 327/500. Dataloading: 0.0032 s/iter. Inference: 0.2802 s/iter. Eval: 0.0385 s/iter. Total: 0.3221 s/iter. ETA=0:00:55 [05/13 21:00:55] d2.evaluation.evaluator INFO: Inference done 339/500. Dataloading: 0.0033 s/iter. Inference: 0.2871 s/iter. Eval: 0.0399 s/iter. Total: 0.3303 s/iter. ETA=0:00:53 [05/13 21:01:00] d2.evaluation.evaluator INFO: Inference done 351/500. Dataloading: 0.0049 s/iter. Inference: 0.2879 s/iter. Eval: 0.0411 s/iter. Total: 0.3342 s/iter. ETA=0:00:49 [05/13 21:01:06] d2.evaluation.evaluator INFO: Inference done 367/500. Dataloading: 0.0048 s/iter. Inference: 0.2876 s/iter. Eval: 0.0409 s/iter. Total: 0.3336 s/iter. ETA=0:00:44 [05/13 21:01:11] d2.evaluation.evaluator INFO: Inference done 383/500. Dataloading: 0.0047 s/iter. Inference: 0.2872 s/iter. Eval: 0.0408 s/iter. Total: 0.3329 s/iter. ETA=0:00:38 [05/13 21:01:16] d2.evaluation.evaluator INFO: Inference done 399/500. Dataloading: 0.0046 s/iter. Inference: 0.2869 s/iter. Eval: 0.0406 s/iter. Total: 0.3323 s/iter. ETA=0:00:33 [05/13 21:01:21] d2.evaluation.evaluator INFO: Inference done 415/500. Dataloading: 0.0045 s/iter. Inference: 0.2865 s/iter. Eval: 0.0404 s/iter. Total: 0.3317 s/iter. ETA=0:00:28 [05/13 21:01:26] d2.evaluation.evaluator INFO: Inference done 431/500. Dataloading: 0.0045 s/iter. Inference: 0.2863 s/iter. Eval: 0.0403 s/iter. Total: 0.3312 s/iter. ETA=0:00:22 [05/13 21:01:31] d2.evaluation.evaluator INFO: Inference done 447/500. Dataloading: 0.0044 s/iter. Inference: 0.2860 s/iter. Eval: 0.0401 s/iter. Total: 0.3307 s/iter. ETA=0:00:17 [05/13 21:01:36] d2.evaluation.evaluator INFO: Inference done 463/500. Dataloading: 0.0043 s/iter. Inference: 0.2857 s/iter. Eval: 0.0400 s/iter. Total: 0.3303 s/iter. ETA=0:00:12 [05/13 21:01:41] d2.evaluation.evaluator INFO: Inference done 479/500. Dataloading: 0.0043 s/iter. Inference: 0.2855 s/iter. Eval: 0.0398 s/iter. Total: 0.3299 s/iter. ETA=0:00:06 [05/13 21:01:46] d2.evaluation.evaluator INFO: Inference done 495/500. Dataloading: 0.0042 s/iter. Inference: 0.2853 s/iter. Eval: 0.0397 s/iter. Total: 0.3294 s/iter. ETA=0:00:01 [05/13 21:02:04] d2.evaluation.evaluator INFO: Total inference time: 0:02:58.901348 (0.361417 s / iter per device, on 2 devices) [05/13 21:02:04] d2.evaluation.evaluator INFO: Total inference pure compute time: 0:02:21 (0.285237 s / iter per device, on 2 devices) [05/13 21:02:05] d2.evaluation.sem_seg_evaluation INFO: OrderedDict([('sem_seg', {'mIoU': 52.28159952897864, 'S_mIoU': 62.4033284436156, 'U_mIoU': 30.351186880598497, 'fwIoU': 78.81153316177783, 'IoU-road': 92.70506509223631, 'BoundaryIoU-road': 82.46904118889213, 'min(IoU, B-Iou)-road': 82.46904118889213, 'IoU-sidewalk': 51.9319400477429, 'BoundaryIoU-sidewalk': 15.115879683222818, 'min(IoU, B-Iou)-sidewalk': 15.115879683222818, 'IoU-building': 80.66494852792113, 'BoundaryIoU-building': 48.985035523295636, 'min(IoU, B-Iou)-building': 48.985035523295636, 'IoU-wall': 27.635162028612747, 'BoundaryIoU-wall': 2.660408531801321, 'min(IoU, B-Iou)-wall': 2.660408531801321, 'IoU-fence': 44.28298167444838, 'BoundaryIoU-fence': 12.001754372579496, 'min(IoU, B-Iou)-fence': 12.001754372579496, 'IoU-pole': 2.235102345182876, 'BoundaryIoU-pole': 8.632448756996535, 'min(IoU, B-Iou)-pole': 2.235102345182876, 'IoU-traffic light': 34.23828289550927, 'BoundaryIoU-traffic light': 33.81387597833923, 'min(IoU, B-Iou)-traffic light': 33.81387597833923, 'IoU-traffic sign': 5.043381298525499, 'BoundaryIoU-traffic sign': 15.129719648084702, 'min(IoU, B-Iou)-traffic sign': 5.043381298525499, 'IoU-vegetation': 59.23882495952436, 'BoundaryIoU-vegetation': 54.41298220087815, 'min(IoU, B-Iou)-vegetation': 54.41298220087815, 'IoU-terrain': 10.474563221091863, 'BoundaryIoU-terrain': 9.281163402624182, 'min(IoU, B-Iou)-terrain': 9.281163402624182, 'IoU-sky': 91.55364903685151, 'BoundaryIoU-sky': 85.27205920541334, 'min(IoU, B-Iou)-sky': 85.27205920541334, 'IoU-person': 70.75621400651572, 'BoundaryIoU-person': 49.867620452611035, 'min(IoU, B-Iou)-person': 49.867620452611035, 'IoU-rider': 53.18330941152348, 'BoundaryIoU-rider': 29.059447211936103, 'min(IoU, B-Iou)-rider': 29.059447211936103, 'IoU-car': 83.22141593747642, 'BoundaryIoU-car': 0.0, 'min(IoU, B-Iou)-car': 0.0, 'IoU-truck': 50.552088707539426, 'BoundaryIoU-truck': 0.0, 'min(IoU, B-Iou)-truck': 0.0, 'IoU-bus': 83.29964337873285, 'BoundaryIoU-bus': 0.0, 'min(IoU, B-Iou)-bus': 0.0, 'IoU-train': 2.885102075248861, 'BoundaryIoU-train': 0.0, 'min(IoU, B-Iou)-train': 0.0, 'IoU-motorcycle': 79.75637258251896, 'BoundaryIoU-motorcycle': 0.0, 'min(IoU, B-Iou)-motorcycle': 0.0, 'IoU-bicycle': 69.69234382339127, 'BoundaryIoU-bicycle': 0.0, 'min(IoU, B-Iou)-bicycle': 0.0, 'mACC': 66.79151642978614, 'pACC': 86.11552324870566, 'Recall': 86.11552324870566, 'S_Recall': 94.28912437363685, 'U_Recall': 69.6486403646182, 'ACC-road': 96.30089349914222, 'ACC-sidewalk': 80.74366752348936, 'ACC-building': 92.25210807552885, 'ACC-wall': 65.0651605448796, 'ACC-fence': 65.54861565278581, 'ACC-pole': 7.993593459682023, 'ACC-traffic light': 81.54222288837208, 'ACC-traffic sign': 14.598979801344573, 'ACC-vegetation': 62.08768939109001, 'ACC-terrain': 11.445465527188594, 'ACC-sky': 99.05081752793117, 'ACC-person': 77.65844371733924, 'ACC-rider': 68.48109481743874, 'ACC-car': 88.70899458967976, 'ACC-truck': 90.27905300439592, 'ACC-bus': 90.83941110697238, 'ACC-train': 4.3707666185894505, 'ACC-motorcycle': 86.04606154441649, 'ACC-bicycle': 86.02577287567038})]) [05/13 21:02:05] d2.engine.defaults INFO: Evaluation results for bdd10k_val_sem_seg in csv format: [05/13 21:02:05] d2.evaluation.testing INFO: copypaste: Task: sem_seg [05/13 21:02:05] d2.evaluation.testing INFO: copypaste: mIoU,S_mIoU,U_mIoU,fwIoU,mACC,pACC,Recall,S_Recall,U_Recall [05/13 21:02:05] d2.evaluation.testing INFO: copypaste: 52.2816,62.4033,30.3512,78.8115,66.7915,86.1155,86.1155,94.2891,69.6486