Package: opencoodx
OpenCOODXOverviewOpenCOOD is anOpenCOOperativeDetection framework for autonomous driving. It is also the official implementation of theICRA 2022[Website] [Paper: OPV2V] [Documents] [OpenCOOD]opencoodxis a ready-to-go package of OpenCOOD. You can install it easily by using pip.Installationpipinstallopencoodx# to upgradepipinstall--upgradeopencoodxFeaturesProvide easy data API for the Vehicle-to-Vehicle (V2V) multi-modal perception datasetOPV2VIt currently provides easy API to load LiDAR data from multiple agents simultaneously in a structured format and
convert to PyTorch Tesnor directly for model use.Provide multiple SOTA 3D detection backboneIt supports state-of-the-art LiDAR detector includingPointPillar,Pixor,VoxelNet, andSECOND.Support most common fusion strategiesIt includes 3 most common fusion strategies: early fusion, late fusion, and intermediate fusion across different agents.Support several SOTA multi-agent visual fusion modelIt supports the most recent multi-agent perception algorithms (currently up to Sep. 2021) includingAttentive Fusion,Cooper (early fusion),F-Cooper,V2VNetetc. We will keep updating
the newest algorithms.Provide a convenient log replay toolbox for OPV2V dataset (coming soon)It also provides an easy tool to replay the original OPV2V dataset. More importantly, it allows users to enrich the original dataset by
attaching new sensors or define additional tasks (e.g. tracking, prediction)
without changing the events in the initial dataset (e.g. positions and number of all vehicles, traffic speed).Prerequisite - Dependency1. Pytorch Installation (>=1.10)Go tohttps://pytorch.org/to install pytorch cuda version. Pytorch 1.11 version is recommended.2. Spconv (2.x)Install spconv 2.x based on your cuda version. For more details, please check:https://pypi.org/project/spconv/3. Bbx IOU cuda version compileInstall bbx nms calculation cuda version using following command:opencoodx--bbxPrerequisite - Data files1. Download trained model filesTo download these models, you can run the following command in your terminal:# download all modelsopencoodx--modelall# download one modelopencoodx--model${model_name}Arguments Explanation:all: To download all modelsmodel_name: We have 11 trained models that are ready to use, you can choose from the following:pointpillar_attentive_fusionpointpillar_early_fusionpointpillar_fcooperpointpillar_late_fusionv2vnetvoxelnet_early_fusionvoxelnet_attentive_fusionsecond_early_fusionsecond_attentive_fusionsecond_late_fusionpixor_early_fusion2. Offline data download (optional)To download offline dataset, you can simply use the command:opencoodx--data${dataset_name}Arguments Explanation:dataset_name: str type. There are 4 different dataset_name. You can choose from 'test_culver_city', 'test', 'validate' or 'train'Quick StartData sequence visualizationTo quickly visualize the LiDAR stream in the OPV2V dataset, you need to download offline data to your current working directory and the run the following command:opencoodx--vis_data${dataset_name}--vis_color${color_mode}Arguments Explanation:dataset_name: str type, including dataset you've download.  You can choose from 'test_culver_city', 'test', 'validate' or 'train'color_mode: str type, indicating the lidar color rendering mode. You can choose from 'constant', 'intensity' or 'z-value'.Benchmark and model zooResults on OPV2V dataset (AP@0.7 for no-compression/ compression)BackboneFusion StrategyBandwidth (Megabit),before/after compressionDefault TownsCulver CityDownloadNaive LatePointPillarLate0.024/0.0240.781/0.7810.668/0.668urlCooperPointPillarEarly7.68/7.680.800/x0.696/xurlAttentive FusionPointPillarIntermediate126.8/1.980.815/0.8100.735/0.731urlF-CooperPointPillarIntermediate72.08/1.120.790/0.7880.728/0.726urlV2VNetPointPillarIntermediate72.08/1.120.822/0.8140.734/0.729urlNaive LateVoxelNetLate0.024/0.0240.738/0.7380.588/0.588urlCooperVoxelNetEarly7.68/7.680.758/x0.677/xurlAttentive FusionVoxelNetIntermediate576.71/1.120.864/0.8520.775/0.746urlNaive LateSECONDLate0.024/0.0240.775/0.7750.682/0.682urlCooperSECONDEarly7.68/7.680.813/x0.738/xurlAttentiveSECONDIntermediate63.4/0.990.826/0.7830.760/0.760urlNaive LatePIXORLate0.024/0.0240.578/0.5780.360/0.360urlCooperPIXOREarly7.68/7.680.678/x0.558/xurlAttentivePIXORIntermediate313.75/1.220.687/0.6120.546/0.492urlNote:We suggest usingPointPillaras the backbone when you are creating your method and try to compare with
our benchmark, as we implement most of the SOTA methods with this backbone only.We assume the transimssion rate is 27Mbp/s. Considering the frequency of LiDAR is 10Hz, the
bandwidth requirement should be less than2.7Mbpto avoid severe delay.A 'x' in the benchmark table represents the bandwidth requirement is too large, which
can not be considered to employ in practice.TutorialsWe have a series of tutorials to help you understand OpenCOOD more. Please check the series of ourtutorials.CitationIf you are using our OpenCOOD framework or OPV2V dataset for your research, please cite the following paper:@inproceedings{xu2022opencood,author={Runsheng Xu, Hao Xiang, Xin Xia, Xu Han, Jinlong Li, Jiaqi Ma},title={OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication},booktitle={2022 IEEE International Conference on Robotics and Automation (ICRA)},year={2022}}Also, under this LICENSE, OpenCOOD is for non-commercial research only. Researchers can modify the source code for their own research only. Contracted work that generates corporate revenues and other general commercial use are prohibited under this LICENSE. See the LICENSE file for details and possible opportunities for commercial use.Future PlansProvide camera APIs for OPV2VProvide the log replay toolboxImplement F-CooperImplement V2VNetImplement DiscoNetContributorsOpenCOOD is supported by theUCLA Mobility Lab. We also appreciate the great work fromOpenPCDet, as
part of our works use their framework.Lab Principal Investigator:Dr. Jiaqi Ma (linkedin,UCLA Samueli)Project Lead:Runsheng Xu (linkedin,github)Hao Xiang (linkedin,github)