[02/12 13:47:35] d2.evaluation.evaluator INFO: Inference done 500/500. Dataloading: 0.0030 s/iter. Inference: 9.5918 s/iter. Eval: 2.8583 s/iter. Total: 12.4564 s/iter. ETA=0:00:00
[02/12 13:47:36] d2.evaluation.evaluator INFO: Total inference time: 1:42:46.454001 (12.457483 s / iter per device, on 4 devices)
[02/12 13:47:36] d2.evaluation.evaluator INFO: Total inference pure compute time: 1:19:07 (9.591839 s / iter per device, on 4 devices)
[02/12 13:47:36] d2.evaluation.panoptic_evaluation INFO: Writing all panoptic predictions to /tmp/panoptic_eval0fajeh5q ...
[02/12 13:50:11] d2.evaluation.panoptic_evaluation INFO: Panoptic Evaluation Results:
|        |   PQ   |   SQ   |   RQ   |  #categories  |
|:------:|:------:|:------:|:------:|:-------------:|
|  All   | 18.504 | 57.632 | 23.182 |      65       |
| Things | 14.386 | 57.548 | 17.903 |      37       |
| Stuff  | 23.946 | 57.744 | 30.157 |      28       |
[02/12 13:50:11] d2.evaluation.sem_seg_evaluation INFO: OrderedDict([('sem_seg', {'mIoU': 27.306496930347734, 'fwIoU': 66.40574505470411, 'IoU-bird,birds': 30.105847221264053, 'BoundaryIoU-bird,birds': 79.15040018801348, 'min(IoU, B-Iou)-bird,birds': 30.105847221264053, 'IoU-ground animal,ground animate being,dog,cat,horse,cow,sheep,zebra,giraffe': 45.71480399134828, 'BoundaryIoU-ground animal,ground animate being,dog,cat,horse,cow,sheep,zebra,giraffe': 1.5629648696130982, 'min(IoU, B-Iou)-ground animal,ground animate being,dog,cat,horse,cow,sheep,zebra,giraffe': 1.5629648696130982, 'IoU-curb,curbs': 10.735548292591549, 'BoundaryIoU-curb,curbs': 4.287710525187304, 'min(IoU, B-Iou)-curb,curbs': 4.287710525187304, 'IoU-fence,fences': 37.40954848594667, 'BoundaryIoU-fence,fences': 18.643631237014745, 'min(IoU, B-Iou)-fence,fences': 18.643631237014745, 'IoU-guard rail': 23.667593744980294, 'BoundaryIoU-guard rail': 0.0, 'min(IoU, B-Iou)-guard rail': 0.0, 'IoU-barrier': 3.613037879484995, 'BoundaryIoU-barrier': 0.0, 'min(IoU, B-Iou)-barrier': 0.0, 'IoU-wall,walls,brick wall,stone wall,interior wall': 32.581514194323205, 'BoundaryIoU-wall,walls,brick wall,stone wall,interior wall': 0.0, 'min(IoU, B-Iou)-wall,walls,brick wall,stone wall,interior wall': 0.0, 'IoU-bike Lane': 9.425416760501195, 'BoundaryIoU-bike Lane': 0.0, 'min(IoU, B-Iou)-bike Lane': 0.0, 'IoU-crosswalk': 0.02291875063729632, 'BoundaryIoU-crosswalk': 0.0, 'min(IoU, B-Iou)-crosswalk': 0.0, 'IoU-curb cut': 0.12097694267062392, 'BoundaryIoU-curb cut': 0.0, 'min(IoU, B-Iou)-curb cut': 0.0, 'IoU-parking': 2.7176931767545263, 'BoundaryIoU-parking': 0.0, 'min(IoU, B-Iou)-parking': 0.0, 'IoU-pedestrian area': 0.6118292337626263, 'BoundaryIoU-pedestrian area': 0.0, 'min(IoU, B-Iou)-pedestrian area': 0.0, 'IoU-rail track': 31.360448588229957, 'BoundaryIoU-rail track': 0.0, 'min(IoU, B-Iou)-rail track': 0.0, 'IoU-road': 56.448630997076165, 'BoundaryIoU-road': 0.0, 'min(IoU, B-Iou)-road': 0.0, 'IoU-service lane': 1.8910609773843852, 'BoundaryIoU-service lane': 0.0, 'min(IoU, B-Iou)-service lane': 0.0, 'IoU-sidewalk,pavement': 44.24482239579092, 'BoundaryIoU-sidewalk,pavement': 0.0, 'min(IoU, B-Iou)-sidewalk,pavement': 0.0, 'IoU-bridge': 61.07304798825782, 'BoundaryIoU-bridge': 0.0, 'min(IoU, B-Iou)-bridge': 0.0, 'IoU-building,buildings': 78.32560362294723, 'BoundaryIoU-building,buildings': 0.0, 'min(IoU, B-Iou)-building,buildings': 0.0, 'IoU-tunnel': 3.024164184605879, 'BoundaryIoU-tunnel': 0.0, 'min(IoU, B-Iou)-tunnel': 0.0, 'IoU-person,child,girl,boy,woman,man,people,children,girls,boys,women,men,lady,guy,ladies,guys,clothes': 73.30093199955847, 'BoundaryIoU-person,child,girl,boy,woman,man,people,children,girls,boys,women,men,lady,guy,ladies,guys,clothes': 0.0, 'min(IoU, B-Iou)-person,child,girl,boy,woman,man,people,children,girls,boys,women,men,lady,guy,ladies,guys,clothes': 0.0, 'IoU-bicyclist,bicyclists': 35.176415643867045, 'BoundaryIoU-bicyclist,bicyclists': 0.0, 'min(IoU, B-Iou)-bicyclist,bicyclists': 0.0, 'IoU-motorcyclist,motorcyclists': 13.103934141928274, 'BoundaryIoU-motorcyclist,motorcyclists': 0.0, 'min(IoU, B-Iou)-motorcyclist,motorcyclists': 0.0, 'IoU-other rider,rider': 0.5208639118984022, 'BoundaryIoU-other rider,rider': 0.0, 'min(IoU, B-Iou)-other rider,rider': 0.0, 'IoU-lane marking of crosswalk': 17.65292427314915, 'BoundaryIoU-lane marking of crosswalk': 0.0, 'min(IoU, B-Iou)-lane marking of crosswalk': 0.0, 'IoU-lane marking': 8.502994593879837, 'BoundaryIoU-lane marking': 0.0, 'min(IoU, B-Iou)-lane marking': 0.0, 'IoU-mountain,mountains': 40.248408681281035, 'BoundaryIoU-mountain,mountains': 0.0, 'min(IoU, B-Iou)-mountain,mountains': 0.0, 'IoU-sand': 44.66028746301561, 'BoundaryIoU-sand': 0.0, 'min(IoU, B-Iou)-sand': 0.0, 'IoU-sky,clouds': 88.42910320690378, 'BoundaryIoU-sky,clouds': 0.0, 'min(IoU, B-Iou)-sky,clouds': 0.0, 'IoU-snow': 76.14714170504587, 'BoundaryIoU-snow': 0.0, 'min(IoU, B-Iou)-snow': 0.0, 'IoU-terrain,river,sea,grass,dirt,rock': 53.705849128742635, 'BoundaryIoU-terrain,river,sea,grass,dirt,rock': 0.0, 'min(IoU, B-Iou)-terrain,river,sea,grass,dirt,rock': 0.0, 'IoU-vegetation,tree,trees,palm tree,bushes': 82.52501912096926, 'BoundaryIoU-vegetation,tree,trees,palm tree,bushes': 0.0, 'min(IoU, B-Iou)-vegetation,tree,trees,palm tree,bushes': 0.0, 'IoU-water': 19.794066323254878, 'BoundaryIoU-water': 0.0, 'min(IoU, B-Iou)-water': 0.0, 'IoU-banner,streamer': 27.55368909340848, 'BoundaryIoU-banner,streamer': 0.0, 'min(IoU, B-Iou)-banner,streamer': 0.0, 'IoU-bench,benches': 41.13808778121765, 'BoundaryIoU-bench,benches': 0.0, 'min(IoU, B-Iou)-bench,benches': 0.0, 'IoU-bike rack': 9.70617380972136, 'BoundaryIoU-bike rack': 0.0, 'min(IoU, B-Iou)-bike rack': 0.0, 'IoU-billboard,hoarding': 19.05493929898331, 'BoundaryIoU-billboard,hoarding': 0.0, 'min(IoU, B-Iou)-billboard,hoarding': 0.0, 'IoU-catch basin': 3.517507721903504, 'BoundaryIoU-catch basin': 0.0, 'min(IoU, B-Iou)-catch basin': 0.0, 'IoU-cctv camera,cctv': 2.144296625144209, 'BoundaryIoU-cctv camera,cctv': 0.0, 'min(IoU, B-Iou)-cctv camera,cctv': 0.0, 'IoU-fire hydrant,fireplug,plug': 60.21442718095086, 'BoundaryIoU-fire hydrant,fireplug,plug': 0.0, 'min(IoU, B-Iou)-fire hydrant,fireplug,plug': 0.0, 'IoU-junction box': 0.008316835183482994, 'BoundaryIoU-junction box': 0.0, 'min(IoU, B-Iou)-junction box': 0.0, 'IoU-mailbox,postbox,mailbox,letter box': 6.381307303601377, 'BoundaryIoU-mailbox,postbox,mailbox,letter box': 0.0, 'min(IoU, B-Iou)-mailbox,postbox,mailbox,letter box': 0.0, 'IoU-manhole': 4.652205761655418, 'BoundaryIoU-manhole': 0.0, 'min(IoU, B-Iou)-manhole': 0.0, 'IoU-phone booth,telephone booth,call box,telephone box,telephone kiosk': 8.16601193339899, 'BoundaryIoU-phone booth,telephone booth,call box,telephone box,telephone kiosk': 0.0, 'min(IoU, B-Iou)-phone booth,telephone booth,call box,telephone box,telephone kiosk': 0.0, 'IoU-pothole': 2.7603861498307443, 'BoundaryIoU-pothole': 0.0, 'min(IoU, B-Iou)-pothole': 0.0, 'IoU-street light': 1.3160395917567984, 'BoundaryIoU-street light': 0.0, 'min(IoU, B-Iou)-street light': 0.0, 'IoU-pole': 2.7118203312878695, 'BoundaryIoU-pole': 0.0, 'min(IoU, B-Iou)-pole': 0.0, 'IoU-traffic sign frame': 2.9817725353830618, 'BoundaryIoU-traffic sign frame': 0.0, 'min(IoU, B-Iou)-traffic sign frame': 0.0, 'IoU-utility pole': 2.8624966336892994, 'BoundaryIoU-utility pole': 0.0, 'min(IoU, B-Iou)-utility pole': 0.0, 'IoU-traffic light,traffic signal,traffic lights': 64.42694780884612, 'BoundaryIoU-traffic light,traffic signal,traffic lights': 0.0, 'min(IoU, B-Iou)-traffic light,traffic signal,traffic lights': 0.0, 'IoU-traffic sign (back),back of traffic sign,traffic sign back': 1.5553462291751563, 'BoundaryIoU-traffic sign (back),back of traffic sign,traffic sign back': 0.0, 'min(IoU, B-Iou)-traffic sign (back),back of traffic sign,traffic sign back': 0.0, 'IoU-traffic sign (front),front of traffic sign,traffic sign front': 13.628979872383894, 'BoundaryIoU-traffic sign (front),front of traffic sign,traffic sign front': 0.0, 'min(IoU, B-Iou)-traffic sign (front),front of traffic sign,traffic sign front': 0.0, 'IoU-trash can,ashcan,garbage can,wastebin,ash bin,ash-bin,ashbin,dustbin,trash barrel,trash bin': 16.78600034975869, 'BoundaryIoU-trash can,ashcan,garbage can,wastebin,ash bin,ash-bin,ashbin,dustbin,trash barrel,trash bin': 0.0, 'min(IoU, B-Iou)-trash can,ashcan,garbage can,wastebin,ash bin,ash-bin,ashbin,dustbin,trash barrel,trash bin': 0.0, 'IoU-bicycle,bike': 65.11402719637516, 'BoundaryIoU-bicycle,bike': 0.0, 'min(IoU, B-Iou)-bicycle,bike': 0.0, 'IoU-boat': 66.04059168984107, 'BoundaryIoU-boat': 0.0, 'min(IoU, B-Iou)-boat': 0.0, 'IoU-bus,autobus,double-decker,jitney,motorbus,motorcoach,omnibus,passenger vehicle': 77.9907586888077, 'BoundaryIoU-bus,autobus,double-decker,jitney,motorbus,motorcoach,omnibus,passenger vehicle': 0.0, 'min(IoU, B-Iou)-bus,autobus,double-decker,jitney,motorbus,motorcoach,omnibus,passenger vehicle': 0.0, 'IoU-car,automobile,cars': 53.23464350861962, 'BoundaryIoU-car,automobile,cars': 0.0, 'min(IoU, B-Iou)-car,automobile,cars': 0.0, 'IoU-caravan': 52.27131279941462, 'BoundaryIoU-caravan': 0.0, 'min(IoU, B-Iou)-caravan': 0.0, 'IoU-motorcycle,motorcycles': 66.24990278161323, 'BoundaryIoU-motorcycle,motorcycles': 0.0, 'min(IoU, B-Iou)-motorcycle,motorcycles': 0.0, 'IoU-on rails': 4.015564410913884, 'BoundaryIoU-on rails': 0.0, 'min(IoU, B-Iou)-on rails': 0.0, 'IoU-other vehicle,vehicle': 0.07979092112850243, 'BoundaryIoU-other vehicle,vehicle': 0.0, 'min(IoU, B-Iou)-other vehicle,vehicle': 0.0, 'IoU-trailer': 11.647211386152833, 'BoundaryIoU-trailer': 0.0, 'min(IoU, B-Iou)-trailer': 0.0, 'IoU-truck,motortruck': 58.05579821787688, 'BoundaryIoU-truck,motortruck': 0.0, 'min(IoU, B-Iou)-truck,motortruck': 0.0, 'IoU-wheeled slow': 0.0024462206285840093, 'BoundaryIoU-wheeled slow': 0.0, 'min(IoU, B-Iou)-wheeled slow': 0.0, 'IoU-car mount': 1.7910521818984602, 'BoundaryIoU-car mount': 0.0, 'min(IoU, B-Iou)-car mount': 0.0, 'IoU-ego vehicle': 0.0, 'BoundaryIoU-ego vehicle': 0.0, 'min(IoU, B-Iou)-ego vehicle': 0.0, 'mACC': 40.70725629601299, 'pACC': 75.10560880802616, 'ACC-bird,birds': 34.788773296434854, 'ACC-ground animal,ground animate being,dog,cat,horse,cow,sheep,zebra,giraffe': 63.76343601850005, 'ACC-curb,curbs': 25.016641237342373, 'ACC-fence,fences': 42.01684203013513, 'ACC-guard rail': 86.53740112733537, 'ACC-barrier': 4.570862941282186, 'ACC-wall,walls,brick wall,stone wall,interior wall': 78.3475023585315, 'ACC-bike Lane': 27.54934554280794, 'ACC-crosswalk': 0.05599313653004884, 'ACC-curb cut': 0.1545105093886352, 'ACC-parking': 3.4439361206444183, 'ACC-pedestrian area': 0.624520142000639, 'ACC-rail track': 38.500327814201576, 'ACC-road': 61.037673935018674, 'ACC-service lane': 8.69855665327384, 'ACC-sidewalk,pavement': 87.54650030750643, 'ACC-bridge': 72.0022291235648, 'ACC-building,buildings': 90.02854829914598, 'ACC-tunnel': 3.6125711762424877, 'ACC-person,child,girl,boy,woman,man,people,children,girls,boys,women,men,lady,guy,ladies,guys,clothes': 92.39964033547572, 'ACC-bicyclist,bicyclists': 48.05147161516413, 'ACC-motorcyclist,motorcyclists': 31.675338879419524, 'ACC-other rider,rider': 9.77667251804174, 'ACC-lane marking of crosswalk': 51.64228066767696, 'ACC-lane marking': 20.47291500455367, 'ACC-mountain,mountains': 65.61730491486747, 'ACC-sand': 51.75807055394219, 'ACC-sky,clouds': 91.20829174491836, 'ACC-snow': 87.98796621128572, 'ACC-terrain,river,sea,grass,dirt,rock': 84.49720240431003, 'ACC-vegetation,tree,trees,palm tree,bushes': 87.99271392462053, 'ACC-water': 21.387821538743427, 'ACC-banner,streamer': 48.10294328608588, 'ACC-bench,benches': 82.05224065674307, 'ACC-bike rack': 24.80337640516862, 'ACC-billboard,hoarding': 22.913637706357257, 'ACC-catch basin': 4.155969568201579, 'ACC-cctv camera,cctv': 7.71982860597776, 'ACC-fire hydrant,fireplug,plug': 79.80045261615, 'ACC-junction box': 0.008553075103179989, 'ACC-mailbox,postbox,mailbox,letter box': 9.083891338457759, 'ACC-manhole': 9.501522472590489, 'ACC-phone booth,telephone booth,call box,telephone box,telephone kiosk': 35.59567717565293, 'ACC-pothole': 89.41846181244057, 'ACC-street light': 17.44120532943479, 'ACC-pole': 5.15702229006818, 'ACC-traffic sign frame': 3.081453681635692, 'ACC-utility pole': 11.54638783089541, 'ACC-traffic light,traffic signal,traffic lights': 79.36993247433422, 'ACC-traffic sign (back),back of traffic sign,traffic sign back': 3.54689667285814, 'ACC-traffic sign (front),front of traffic sign,traffic sign front': 15.83866691848106, 'ACC-trash can,ashcan,garbage can,wastebin,ash bin,ash-bin,ashbin,dustbin,trash barrel,trash bin': 22.501553408997943, 'ACC-bicycle,bike': 74.54538124607501, 'ACC-boat': 80.53104374669026, 'ACC-bus,autobus,double-decker,jitney,motorbus,motorcoach,omnibus,passenger vehicle': 94.32173711099655, 'ACC-car,automobile,cars': 83.95747131144132, 'ACC-caravan': 72.68690802922399, 'ACC-motorcycle,motorcycles': 80.10022847849861, 'ACC-on rails': 4.324742749268254, 'ACC-other vehicle,vehicle': 0.5438086976407699, 'ACC-trailer': 16.56425938963111, 'ACC-truck,motortruck': 91.22835508204331, 'ACC-wheeled slow': 0.002723490988319738, 'ACC-car mount': 2.761464499806239, 'ACC-ego vehicle': 0.0})])
[02/12 13:50:11] d2.engine.defaults INFO: Evaluation results for openvocab_mapillary_vistas_panoptic_val in csv format:
[02/12 13:50:11] d2.evaluation.testing INFO: copypaste: Task: panoptic_seg
[02/12 13:50:11] d2.evaluation.testing INFO: copypaste: PQ,SQ,RQ,PQ_th,SQ_th,RQ_th,PQ_st,SQ_st,RQ_st
[02/12 13:50:11] d2.evaluation.testing INFO: copypaste: 18.5044,57.6322,23.1817,14.3862,57.5477,17.9029,23.9463,57.7438,30.1573
[02/12 13:50:11] d2.evaluation.testing INFO: copypaste: Task: sem_seg
[02/12 13:50:11] d2.evaluation.testing INFO: copypaste: mIoU,fwIoU,mACC,pACC
[02/12 13:50:11] d2.evaluation.testing INFO: copypaste: 27.3065,66.4057,40.7073,75.1056