Package: wf-pytorch-yolo-v4
Pytorch-YOLOv4A minimal PyTorch implementation of YOLOv4.Paper Yolo v4:https://arxiv.org/abs/2004.10934Source code:https://github.com/AlexeyAB/darknetMore details:http://pjreddie.com/darknet/yolo/InferenceTrainMosaic├── README.md
├── dataset.py            dataset
├── demo.py               demo to run pytorch --> tool/darknet2pytorch
├── demo_darknet2onnx.py  tool to convert into onnx --> tool/darknet2pytorch
├── demo_pytorch2onnx.py  tool to convert into onnx
├── models.py             model for pytorch
├── train.py              train models.py
├── cfg.py                cfg.py for train
├── cfg                   cfg --> darknet2pytorch
├── data            
├── weight                --> darknet2pytorch
├── tool
│   ├── camera.py           a demo camera
│   ├── coco_annotation.py       coco dataset generator
│   ├── config.py
│   ├── darknet2pytorch.py
│   ├── region_loss.py
│   ├── utils.py
│   └── yolo_layer.pyWildflower Forked VersionChangesRun using a CLI (yolov4)Automatically download weightsPyPi ready so project can be used as a library0. Weights Download0.1 darknetbaidu(https://pan.baidu.com/s/1dAGEW8cm-dqK14TbhhVetAExtraction code:dm5b)google(https://drive.google.com/open?id=1cewMfusmPjYWbrnuJRuKhPMwRe_b9PaT)0.2 pytorchyou can use darknet2pytorch to convert it yourself, or download my converted model.baiduyolov4.pth(https://pan.baidu.com/s/1ZroDvoGScDgtE1ja_QqJVwExtraction code:xrq9)yolov4.conv.137.pth(https://pan.baidu.com/s/1ovBie4YyVQQoUrC3AY0joAExtraction code:kcel)googleyolov4.pth(https://drive.google.com/open?id=1wv_LiFeCRYwtpkqREPeI13-gPELBDwuJ)yolov4.conv.137.pth(https://drive.google.com/open?id=1fcbR0bWzYfIEdLJPzOsn4R5mlvR6IQyA)1. Trainuse yolov4 to train your own dataDownload weightTransform dataFor coco dataset,you can use tool/coco_annotation.py.# train.txt
image_path1 x1,y1,x2,y2,id x1,y1,x2,y2,id x1,y1,x2,y2,id ...
image_path2 x1,y1,x2,y2,id x1,y1,x2,y2,id x1,y1,x2,y2,id ...
...
...Trainyou can set parameters in cfg.py.python train.py -g [GPU_ID] -dir [Dataset direction] ...2. Inference2.1 Performance on MS COCO dataset (using pretrained DarknetWeights fromhttps://github.com/AlexeyAB/darknet)ONNX and TensorRT models are converted from Pytorch (TianXiaomo): Pytorch->ONNX->TensorRT.See following sections for more details of conversions.val2017 dataset (input size: 416x416)Model typeAPAP50AP75APSAPMAPLDarkNet (YOLOv4 paper)0.4710.7100.5100.2780.5250.636Pytorch (TianXiaomo)0.4660.7040.5050.2670.5240.629TensorRT FP32 + BatchedNMSPlugin0.4720.7080.5110.2730.5300.637TensorRT FP16 + BatchedNMSPlugin0.4720.7080.5110.2730.5300.636testdev2017 dataset (input size: 416x416)Model typeAPAP50AP75APSAPMAPLDarkNet (YOLOv4 paper)0.4120.6280.4430.2040.4440.560Pytorch (TianXiaomo)0.4040.6150.4360.1960.4380.552TensorRT FP32 + BatchedNMSPlugin0.4120.6250.4450.2000.4460.564TensorRT FP16 + BatchedNMSPlugin0.4120.6250.4450.2000.4460.5632.2 Image input size for inferenceImage input size is NOT restricted in320 * 320,416 * 416,512 * 512and608 * 608.
You can adjust your input sizes for a different input ratio, for example:320 * 608.
Larger input size could help detect smaller targets, but may be slower and GPU memory exhausting.height=320+96*n,nin{0,1,2,3,...}width=320+96*m,min{0,1,2,3,...}2.3Different inference optionsLoad the pretrained darknet model and darknet weights to do the inference (image size is configured in cfg file already)pythondemo.py-cfgfile<cfgFile>-weightfile<weightFile>-imgfile<imgFile>Load pytorch weights (pth file) to do the inferencepythonmodels.py<num_classes><weightfile><imgfile><IN_IMAGE_H><IN_IMAGE_W><namefile(optional)>Load converted ONNX file to do inference (See section 3 and 4)Load converted TensorRT engine file to do inference (See section 5)2.4 Inference outputThere are 2 inference outputs.One is locations of bounding boxes, its shape is[batch, num_boxes, 1, 4]which represents x1, y1, x2, y2 of each bounding box.The other one is scores of bounding boxes which is of shape[batch, num_boxes, num_classes]indicating scores of all classes for each bounding box.Until now, still a small piece of post-processing including NMS is required. We are trying to minimize time and complexity of post-processing.3. Darknet2ONNXThis script is to convert the official pretrained darknet model into ONNXPytorch version Recommended:Pytorch 1.4.0 for TensorRT 7.0 and higherPytorch 1.5.0 and 1.6.0 for TensorRT 7.1.2 and higherInstall onnxruntimepipinstallonnxruntimeRun python script to generate ONNX model and run the demopythondemo_darknet2onnx.py<cfgFile><namesFile><weightFile><imageFile><batchSize>3.1 Dynamic or static batch sizePositive batch size will generate ONNX model of static batch size, otherwise, batch size will be dynamicDynamic batch size will generate only one ONNX modelStatic batch size will generate 2 ONNX models, one is for running the demo (batch_size=1)4. Pytorch2ONNXYou can convert your trained pytorch model into ONNX using this scriptPytorch version Recommended:Pytorch 1.4.0 for TensorRT 7.0 and higherPytorch 1.5.0 and 1.6.0 for TensorRT 7.1.2 and higherInstall onnxruntimepipinstallonnxruntimeRun python script to generate ONNX model and run the demopythondemo_pytorch2onnx.py<weight_file><image_path><batch_size><n_classes><IN_IMAGE_H><IN_IMAGE_W>For example:pythondemo_pytorch2onnx.pyyolov4.pthdog.jpg8804164164.1 Dynamic or static batch sizePositive batch size will generate ONNX model of static batch size, otherwise, batch size will be dynamicDynamic batch size will generate only one ONNX modelStatic batch size will generate 2 ONNX models, one is for running the demo (batch_size=1)5. ONNX2TensorRTTensorRT version Recommended: 7.0, 7.15.1 Convert from ONNX of static Batch sizeRun the following command to convert YOLOv4 ONNX model into TensorRT enginetrtexec--onnx=<onnx_file>--explicitBatch--saveEngine=<tensorRT_engine_file>--workspace=<size_in_megabytes>--fp16Note: If you want to use int8 mode in conversion, extra int8 calibration is needed.5.2 Convert from ONNX of dynamic Batch sizeRun the following command to convert YOLOv4 ONNX model into TensorRT enginetrtexec--onnx=<onnx_file>\--minShapes=input:<shape_of_min_batch>--optShapes=input:<shape_of_opt_batch>--maxShapes=input:<shape_of_max_batch>\--workspace=<size_in_megabytes>--saveEngine=<engine_file>--fp16For example:trtexec--onnx=yolov4_-1_3_320_512_dynamic.onnx\--minShapes=input:1x3x320x512--optShapes=input:4x3x320x512--maxShapes=input:8x3x320x512\--workspace=2048--saveEngine=yolov4_-1_3_320_512_dynamic.engine--fp165.3 Run the demopythondemo_trt.py<tensorRT_engine_file><input_image><input_H><input_W>This demo here only works when batchSize is dynamic (1 should be within dynamic range) or batchSize=1, but you can update this demo a little for other dynamic or static batch sizes.Note1: input_H and input_W should agree with the input size in the original ONNX file.Note2: extra NMS operations are needed for the tensorRT output. This demo uses python NMS code fromtool/utils.py.6. ONNX2TensorflowFirst:Conversion to ONNXtensorflow >=2.01: Thanks:github:https://github.com/onnx/onnx-tensorflow2: Run git clonehttps://github.com/onnx/onnx-tensorflow.git&& cd onnx-tensorflow
Run pip install -e .Note:Errors will occur when using "pip install onnx-tf", at least for me,it is recommended to use source code installation7. ONNX2TensorRT and DeepStream InferenceCompile the DeepStream Nvinfer Plugincd DeepStream
    makeBuild a TRT Engine.For single batch,trtexec --onnx=<onnx_file> --explicitBatch --saveEngine=<tensorRT_engine_file> --workspace=<size_in_megabytes> --fp16For multi-batch,trtexec --onnx=<onnx_file> --explicitBatch --shapes=input:Xx3xHxW --optShapes=input:Xx3xHxW --maxShapes=input:Xx3xHxW --minShape=input:1x3xHxW --saveEngine=<tensorRT_engine_file> --fp16Note :The maxShapes could not be larger than model original shape.Write the deepstream config file for the TRT Engine.Reference:https://github.com/eriklindernoren/PyTorch-YOLOv3https://github.com/marvis/pytorch-caffe-darknet-converthttps://github.com/marvis/pytorch-yolo3@article{yolov4,
  title={YOLOv4: YOLOv4: Optimal Speed and Accuracy of Object Detection},
  author={Alexey Bochkovskiy, Chien-Yao Wang, Hong-Yuan Mark Liao},
  journal = {arXiv},
  year={2020}
}