Package: netspresso
☀️ NetsPresso Model Zoo ☀️YOLO Fastest|YOLOX|YOLOv8|YOLOv7|YOLOv5|PIDNet|PyTorch-CIFAR-Models🌟 STMicro x NetsPresso 🌟STM32 model zooUseNetsPressofor a seamless model optimization process.
NetsPresso resolves AI-related constraints in business use cases and enables cost-efficiency and enhanced performance by removing the requirement for high-spec servers and network connectivity and preventing high latency and personal data breaches.Easily compress various models with our resources. Please browse theDocsfor details, and join ourDiscussion Forumfor providing feedback or sharing your use cases.To get started with NetsPresso, you'll need to sign uphere.We offer a comprehensive guide to walk you through the process of optimizing an AI model using NetPresso. A full tutorial can be foundGoogle Colab.StepTypeDescriptionTrainnp.trainerBuild and train a model.Model ZooImage ClassificationPyTorch-CIFAR-ModelsObject DetectionYOLO FastestYOLOXYOLOv5YOLOv7Semantic SegmentationPIDNetPose EstimationYOLOv8Compressnp.compressorCompress and optimize the user’s model.Convertnp.converterConvert and quantize the user’s model to run efficiently on device.Benchmarknp.benchmarkerBenchmark the user's model to measure model inference speed on diverse device.InstallationPrerequisitesPython3.8|3.9|3.10PyTorch1.13.0(recommended) (compatible with:1.11.x-1.13.x)TensorFlow2.8.0(recommended) (compatible with:2.3.x-2.8.x)Install with PyPI (stable)pipinstallnetspressoTo useeditable modeordocker, seeINSTALLATION.md.Getting startedLoginLog-in to your netspresso account. Please sign-uphereif you need one.fromnetspressoimportNetsPressonetspresso=NetsPresso(email="YOUR_EMAIL",password="YOUR_PASSWORD")TrainerTrainTo start training a model, first select a task.Then configure the dataset, model, augmentation, and hyperparameters.Once setup is finished, enter the GPU number and project name for training.fromnetspresso.enumsimportTaskfromnetspresso.trainer.optimizersimportAdamWfromnetspresso.trainer.schedulersimportCosineAnnealingWarmRestartsWithCustomWarmUpfromnetspresso.trainer.augmentationsimportResize# 1. Declare trainertrainer=netspresso.trainer(task=Task.OBJECT_DETECTION)# IMAGE_CLASSIFICATION, OBJECT_DETECTION, SEMANTIC_SEGMENTATION# 2. Set config for training# 2-1. Datatrainer.set_dataset_config(name="traffic_sign_config_example",root_path="/root/traffic-sign",train_image="images/train",train_label="labels/train",valid_image="images/valid",valid_label="labels/valid",id_mapping=["prohibitory","danger","mandatory","other"],)# 2-2. Modelprint(trainer.available_models)# ['EfficientFormer', 'YOLOX-S', 'ResNet', 'MobileNetV3', 'MixNetL', 'MixNetM', 'MixNetS']trainer.set_model_config(model_name="YOLOX-S",img_size=512)# 2-3. Augmentationtrainer.set_augmentation_config(train_transforms=[Resize()],inference_transforms=[Resize()],)# 2-4. Trainingoptimizer=AdamW(lr=6e-3)scheduler=CosineAnnealingWarmRestartsWithCustomWarmUp(warmup_epochs=10)trainer.set_training_config(epochs=40,batch_size=16,optimizer=optimizer,scheduler=scheduler,)# 3. Traintraining_result=trainer.train(gpus="0, 1",project_name="PROJECT_TRAIN_SAMPLE")RetrainTo start retraining a model, usehparams.yamlfile which is one of the artifacts generated during the training of the original model.Then, enter the compressed model path, which is an artifact of the compressor in fx_model_path.Adjust the training hyperparameters as needed. (See 2-2. for detailed code.)fromnetspresso.trainer.optimizersimportAdamW# 1. Declare trainertrainer=netspresso.trainer(yaml_path="./temp/hparams.yaml")# 2. Set config for retraining# 2-1. FX Modeltrainer.set_fx_model(fx_model_path="./temp/FX_MODEL_PATH.pt")# 2-2. Trainingoptimizer=AdamW(lr=6e-3)trainer.set_training_config(epochs=30,batch_size=16,optimizer=optimizer,)# 3. Trainretraining_result=trainer.train(gpus="0, 1",project_name="PROJECT_RETRAIN_SAMPLE")CompressorCompress (Automatic compression)To start compressing a model, enter the model path to compress and the appropriate compression ratio.The compressed model will be saved in the specified output directory (output_dir).# 1. Declare compressorcompressor=netspresso.compressor()# 2. Run automatic compressioncompression_result=compressor.automatic_compression(input_shapes=[{"batch":1,"channel":3,"dimension":[224,224]}],input_model_path="./examples/sample_models/graphmodule.pt",output_dir="./outputs/compressed/pytorch_automatic_compression",compression_ratio=0.5,)ConverterConvertTo start converting a model, enter the model path to convert and the target framework and device name.For NVIDIA GPUs and Jetson devices, enter the software version additionally due to the jetpack version.The converted model will be saved in the specified output directory (output_dir).fromnetspresso.enumsimportDeviceName,Framework,SoftwareVersion# 1. Declare converterconverter=netspresso.converter()# 2. Run convertconversion_result=converter.convert_model(input_model_path="./examples/sample_models/test.onnx",output_dir="./outputs/converted/TENSORRT_JETSON_AGX_ORIN_JETPACK_5_0_1",target_framework=Framework.TENSORRT,target_device_name=DeviceName.JETSON_AGX_ORIN,target_software_version=SoftwareVersion.JETPACK_5_0_1,)BenchmarkerBenchmarkTo start benchmarking a model, enter the model path to benchmark and the target device name.For NVIDIA GPUs and Jetson devices, device name and software version have to be matched with the target device of the conversion.TensorRT Model has strong dependency with the device type and its jetpack version.fromnetspresso.enumsimportDeviceName,SoftwareVersion# 1. Declare benchmarkerbenchmarker=netspresso.benchmarker()# 2. Run benchmarkbenchmark_result=benchmarker.benchmark_model(input_model_path="./outputs/converted/TENSORRT_JETSON_AGX_ORIN_JETPACK_5_0_1/TENSORRT_JETSON_AGX_ORIN_JETPACK_5_0_1.trt",target_device_name=DeviceName.JETSON_AGX_ORIN,target_software_version=SoftwareVersion.JETPACK_5_0_1,)print(f"model inference latency:{benchmark_result['result']['latency']}ms")print(f"model gpu memory footprint:{benchmark_result['result']['memory_footprint_gpu']}MB")print(f"model cpu memory footprint:{benchmark_result['result']['memory_footprint_cpu']}MB")Supported options for Converter & BenchmarkerFrameworks that support conversion for model's frameworkTarget / Source FrameworkONNXTENSORFLOW_KERASTENSORFLOWTENSORRT✔️DRPAI✔️OPENVINO✔️TENSORFLOW_LITE✔️✔️✔️Devices that support benchmarks for model's frameworkDevice / FrameworkONNXTENSORRTTENSORFLOW_LITEDRPAIOPENVINORASPBERRY_PI_5✔️✔️RASPBERRY_PI_4B✔️✔️RASPBERRY_PI_3B_PLUS✔️✔️RASPBERRY_PI_ZERO_W✔️✔️RASPBERRY_PI_ZERO_2W✔️✔️ARM_ETHOS_U_SERIES✔️(only INT8)ALIF_ENSEMBLE_E7_DEVKIT_GEN2✔️(only INT8)RENESAS_RA8D1✔️(only INT8)RENESAS_RZ_V2L✔️✔️RENESAS_RZ_V2M✔️✔️JETSON_NANO✔️✔️JETSON_TX2✔️✔️JETSON_XAVIER✔️✔️JETSON_NX✔️✔️JETSON_AGX_ORIN✔️✔️JETSON_ORIN_NANO✔️✔️AWS_T4✔️✔️INTEL_XEON_W_2233✔️Software versions that support conversions and benchmarks for specific devicesSoftware Versions requires only Jetson Device. If you are using a different device, you do not need to enter it.Software Version / DeviceJETSON_NANOJETSON_TX2JETSON_XAVIERJETSON_NXJETSON_AGX_ORINJETSON_ORIN_NANOJETPACK_4_4_1✔️JETPACK_4_6✔️✔️✔️✔️JETPACK_5_0_1✔️JETPACK_5_0_2✔️JETPACK_6_0✔️The code below is an example of using software version.conversion_result=converter.convert_model(input_model_path=INPUT_MODEL_PATH,output_dir=OUTPUT_DIR,target_framework=Framework.TENSORRT,target_device_name=DeviceName.JETSON_AGX_ORIN,target_software_version=SoftwareVersion.JETPACK_5_0_1,)benchmark_result=benchmarker.benchmark_model(input_model_path=CONVERTED_MODEL_PATH,target_device_name=DeviceName.JETSON_AGX_ORIN,target_software_version=SoftwareVersion.JETPACK_5_0_1,)Hardware type that support benchmarks for specific devicesBenchmark and compare models with and without Arm Helium.RENESAS_RA8D1andALIF_ENSEMBLE_E7_DEVKIT_GEN2are available for use.The benchmark results with Helium can be up to twice as fast as without Helium.The code below is an example of using hardware type.benchmark_result=benchmarker.benchmark_model(input_model_path=CONVERTED_MODEL_PATH,target_device_name=DeviceName.RENESAS_RA8D1,target_data_type=DataType.INT8,target_hardware_type=HardwareType.HELIUM)Guide to Credit Consumption by ModuleModuleFeatureCreditCompressorAutomatic compression25Advanced compression50ConverterConvert50BenchmarkerBenchmark25ContactJoin ourDiscussion Forumfor providing feedback or sharing your use cases, and if you want to talk more with Nota, please contact ushere.Or you can also do it via email(netspresso@nota.ai) or phone(+82 2-555-8659)!