Package: openvino2tensorflow
openvino2tensorflowThis script converts the ONNX/OpenVINO IR model to Tensorflow's saved_model, tflite, h5, tfjs, tftrt(TensorRT), CoreML, EdgeTPU, ONNX and pb. PyTorch (NCHW) -> ONNX (NCHW) -> OpenVINO (NCHW) -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> TFLite (NHWC/NCHW). And the conversion from .pb to saved_model and from saved_model to .pb and from .pb to .tflite and saved_model to .tflite and saved_model to onnx. Support for building environments with Docker. It is possible to directly access the host PC GUI and the camera to verify the operation. NVIDIA GPU (dGPU) support. Intel iHD GPU (iGPU) support.Special custom TensorFlow binariesandspecial custom TensorFLow Lite binariesare used.Work in progress now.1. EnvironmentPython 3.8+TensorFlow v2.10.0+PyTorch v1.12.1+TorchVisionTorchAudioOpenVINO 2022.1.0TensorRT 8.4.0+trtexecpycuda 2022.1tensorflowjscoremltoolspaddle2onnxonnxonnxruntime-gpu (CUDA, TensorRT, OpenVINO)onnxruntime-extensionsonnx_graphsurgeononnx-simplifieronnxconverter-commononnxmltoolsonnx-tensorrttf2onnxtorch2trtonnx-tftensorflow-datasetstf_slimedgetpu_compilertflite2tensorflowopenvino2tensorflowsimple-onnx-processing-toolsgdownpandasmatplotlibpaddlepaddlepaddle2onnxpycocotoolsscipyblobconverterIntel-Media-SDKIntel iHD GPU (iGPU) supportOpenCLgluoncvLLVMNNPACKWSL2 OpenCL↥ Back to top2. Use casePyTorch (NCHW) -> ONNX (NCHW) -> OpenVINO (NCHW) ->->openvino2tensorflow-> Tensorflow/Keras (NHWC/NCHW) -> TFLite (NHWC/NCHW)->openvino2tensorflow-> Tensorflow/Keras (NHWC/NCHW) -> TFJS (NHWC/NCHW)->openvino2tensorflow-> Tensorflow/Keras (NHWC/NCHW) -> TF-TRT (NHWC/NCHW)->openvino2tensorflow-> Tensorflow/Keras (NHWC) -> EdgeTPU (NHWC)->openvino2tensorflow-> Tensorflow/Keras (NHWC/NCHW) -> CoreML (NHWC/NCHW)->openvino2tensorflow-> Tensorflow/Keras (NHWC/NCHW) -> ONNX (NHWC/NCHW)->openvino2tensorflow-> Myriad Inference Engine Blob (NCHW)Caffe (NCHW) -> OpenVINO (NCHW) ->->openvino2tensorflow-> Tensorflow/Keras (NHWC/NCHW) -> TFLite (NHWC/NCHW)->openvino2tensorflow-> Tensorflow/Keras (NHWC/NCHW) -> TFJS (NHWC/NCHW)->openvino2tensorflow-> Tensorflow/Keras (NHWC/NCHW) -> TF-TRT (NHWC/NCHW)->openvino2tensorflow-> Tensorflow/Keras (NHWC) -> EdgeTPU (NHWC)->openvino2tensorflow-> Tensorflow/Keras (NHWC/NCHW) -> CoreML (NHWC/NCHW)->openvino2tensorflow-> Tensorflow/Keras (NHWC/NCHW) -> ONNX (NHWC/NCHW)->openvino2tensorflow-> Myriad Inference Engine Blob (NCHW)MXNet (NCHW) -> OpenVINO (NCHW) ->->openvino2tensorflow-> Tensorflow/Keras (NHWC/NCHW) -> TFLite (NHWC/NCHW)->openvino2tensorflow-> Tensorflow/Keras (NHWC/NCHW) -> TFJS (NHWC/NCHW)->openvino2tensorflow-> Tensorflow/Keras (NHWC/NCHW) -> TF-TRT (NHWC/NCHW)->openvino2tensorflow-> Tensorflow/Keras (NHWC) -> EdgeTPU (NHWC)->openvino2tensorflow-> Tensorflow/Keras (NHWC/NCHW) -> CoreML (NHWC/NCHW)->openvino2tensorflow-> Tensorflow/Keras (NHWC/NCHW) -> ONNX (NHWC/NCHW)->openvino2tensorflow-> Myriad Inference Engine Blob (NCHW)Keras (NHWC) -> OpenVINO (NCHW・Optimized) ->->openvino2tensorflow-> Tensorflow/Keras (NHWC/NCHW) -> TFLite (NHWC/NCHW)->openvino2tensorflow-> Tensorflow/Keras (NHWC/NCHW) -> TFJS (NHWC/NCHW)->openvino2tensorflow-> Tensorflow/Keras (NHWC/NCHW) -> TF-TRT (NHWC/NCHW)->openvino2tensorflow-> Tensorflow/Keras (NHWC) -> EdgeTPU (NHWC)->openvino2tensorflow-> Tensorflow/Keras (NHWC/NCHW) -> CoreML (NHWC/NCHW)->openvino2tensorflow-> Tensorflow/Keras (NHWC/NCHW) -> ONNX (NHWC/NCHW)->openvino2tensorflow-> Myriad Inference Engine Blob (NCHW)saved_model ->saved_model_to_pb-> pbsaved_model ->->saved_model_to_tflite-> TFLite->saved_model_to_tflite-> TFJS->saved_model_to_tflite-> TF-TRT->saved_model_to_tflite-> EdgeTPU->saved_model_to_tflite-> CoreML->saved_model_to_tflite-> ONNXpb ->pb_to_tflite-> TFLitepb ->pb_to_saved_model-> saved_model↥ Back to top3. Supported LayersCurrently, there are problems with theReshapeandTransposeoperation of 2D,3D,5D Tensor. Since it is difficult to accurately predict the shape of a simple shape change, I have added support for forced replacement of transposition parameters using JSON files.#6-7-replace-weights-or-constant-values-in-const-op-and-add-transpose-or-reshape-or-cast-or-squeeze-or-unsqueeze-or-add-or-multiply-just-beforeafter-the-operation-specified-by-layer_idSupported LayersNo.OpenVINO LayerTF LayerRemarks1ParameterInputConvert to NHWC (Default) or NCHW2ConstConstant, Bias3ConvolutionConv1D, Conv2D, Conv3DConv3D has limited support4AddAdd5ReLUReLU6PReLUPReLUMaximum(0.0,x)+Minimum(0.0,alpha*x)7MaxPoolMaxPool2D8AvgPoolAveragePooling1D, AveragePooling2D, AveragePooling3D9GroupConvolutionDepthwiseConv2D, Conv2D/Split/Concat10ConvolutionBackpropDataConv2DTranspose, Conv3DTransposeConv3DTranspose has limited support11ConcatConcat12MultiplyMultiply13TanTan14TanhTanh15EluElu16SigmoidSigmoid17HardSigmoidhard_sigmoid18SoftPlusSoftPlus19SwishSwishYou can replace swish and hard-swish with each other by using the "--replace_swish_and_hardswish" option20InterpolateResizeNearestNeighbor, ResizeBilinear4D [N,H,W,C] or 5D [N,D,H,W,C]21ShapeOfShape22ConvertCast23StridedSliceStrided_Slice24PadPad, MirrorPad25ClampReLU6, Clip26TopKArgMax, top_k27TransposeTranspose28SqueezeSqueeze29UnsqueezeIdentity, expand_dimsWIP30ReduceMeanreduce_mean31ReduceMaxreduce_max32ReduceMinreduce_min33ReduceSumreduce_sum34ReduceProdreduce_prod35SubtractSubtract36MatMulMatMul37ReshapeReshape38RangeRangeWIP39ExpExp40AbsAbs41SoftMaxSoftMax42NegativeNegative43MaximumMaximumNo broadcast44MinimumMinimumNo broadcast45AcosAcos46AcoshAcosh47AsinAsin48AsinhAsinh49AtanAtan50AtanhAtanh51CeilingCeil52CosCos53CoshCosh54SinSin55SinhSinh56GatherGather57DivideDivide, FloorDiv58ErfErf59FloorFloor60FloorModFloorMod61HSwishHardSwishx*ReLU6(x+3)*0.16666667, You can replace swish and hard-swish with each other by using the "--replace_swish_and_hardswish" option62LogLog63PowerPowNo broadcast64MishMishx*Tanh(softplus(x))65SeluSelu66Equalequal67NotEqualnot_equal68Greatergreater69GreaterEqualgreater_equal70Lessless71LessEqualless_equal72SelectSelectNo broadcast73LogicalAndlogical_and74LogicalNotlogical_not75LogicalOrlogical_or76LogicalXorlogical_xor77Broadcastbroadcast_to, ones, Multiplynumpy / bidirectional mode, WIP78SplitSplit79VariadicSplitSplit, Slice, SplitV80MVNreduce_mean, sqrt, reduce_variance(x - reduce_mean(x)) / sqrt(reduce_variance(x) + eps)81NonZeronot_equal, boolean_mask82ReduceL2square, reduce_sum, sqrt83SpaceToDepthSpaceToDepth84DepthToSpaceDepthToSpace85Sqrtsqrt86SquaredDifferencesquared_difference87FakeQuantizesubtract, multiply, round, greater, where, less_equal, add88Tiletile89GatherNDgather_nd, reshape, cumprod, multiply, reduce_sum, gather, concat90NonMaxSuppressionnon_max_suppressionWIP. Only available for batch size 1.91Gelugelu92NormalizeL2tf.math.add, tf.math.l2_normalizex/sqrt(max(sum(x**2), eps)) or x/sqrt(add(sum(x**2), eps))93ScatterElementsUpdateshape, rank, floormod, add, cast, range, expand_dims, meshgrid, concat, reshape, tensor_scatter_nd_update94ROIAligncrop_and_resize, avg_pool, max_pool95ScatterNDUpdatetensor_scatter_nd_update96GatherElementsrank, add, shape, cast, floormod, range, tensor_scatter_nd_update, constant, transpose, meshgrid, expand_dims, concat, gather_ndWIP97ConvertLikeCast98ReduceL1Abs, ReduceSum99ShuffleChannelsreshape, transpose100PriorBoxClusteredConstant101CumSumcumsum102PriorBoxConstant103ReverseSequencereverse104ExtractImagePatchesextract_patches105LogSoftmaxreduce_max, log, reduce_sum, exp106Einsumeinsum107ScatterUpdatescatter_update108ResultIdentityOutput↥ Back to top4. Setup4-1.[Environment construction pattern 1]Execution by Docker (strongly recommended)You do not need to install any packages other than Docker. It consumes 23.4GB of storage.$dockerpullghcr.io/pinto0309/openvino2tensorflow:latest
or# $ mv .dockerignore d# $ docker build \# -t ghcr.io/pinto0309/openvino2tensorflow:base.11.7.1-cudnn8-tf2.10.0-trt8.4.3-openvino2022.1.0 \# -f Dockerfile.base .# $ mv d .dockerignore$dockerbuild--no-cache-tghcr.io/pinto0309/openvino2tensorflow:latest.# If you don't need to access the GUI of the HostPC and the USB camera.$dockerrun-it--rm\-v`pwd`:/home/user/workdir\ghcr.io/pinto0309/openvino2tensorflow:latest# If conversion to TF-TRT is not required. And if you need to access the HostPC GUI and USB camera.$xhost+local:&&\dockerrun-it--rm\-v`pwd`:/home/user/workdir\-v/tmp/.X11-unix/:/tmp/.X11-unix:rw\--device/dev/video0:/dev/video0:mwr\--net=host\-eXDG_RUNTIME_DIR=$XDG_RUNTIME_DIR\-eDISPLAY=$DISPLAY\--privileged\ghcr.io/pinto0309/openvino2tensorflow:latest# If you need to convert to TF-TRT. And if you need to access the HostPC GUI and USB camera.$xhost+local:&&\dockerrun--gpusall-it--rm\-v`pwd`:/home/user/workdir\-v/tmp/.X11-unix/:/tmp/.X11-unix:rw\--device/dev/video0:/dev/video0:mwr\--net=host\-eXDG_RUNTIME_DIR=$XDG_RUNTIME_DIR\-eDISPLAY=$DISPLAY\--privileged\ghcr.io/pinto0309/openvino2tensorflow:latest# If you are using iGPU (OpenCL). And if you need to access the HostPC GUI and USB camera.$xhost+local:&&\dockerrun-it--rm\-v`pwd`:/home/user/workdir\-v/tmp/.X11-unix/:/tmp/.X11-unix:rw\--device/dev/video0:/dev/video0:mwr\--net=host\-eLIBVA_DRIVER_NAME=iHD\-eXDG_RUNTIME_DIR=$XDG_RUNTIME_DIR\-eDISPLAY=$DISPLAY\--privileged\ghcr.io/pinto0309/openvino2tensorflow:latest↥ Back to top4-2.[Environment construction pattern 2]Execution by Host machineTo install using thePython Package Index (PyPI), use the following command.$pip3install--user--upgradeopenvino2tensorflowTo install with the latest source code of the main branch, use the following command.$pip3install--user--upgradegit+https://github.com/PINTO0309/openvino2tensorflow↥ Back to top5. Usage5-1. openvino to tensorflow convertusage:openvino2tensorflow[-h]--model_pathMODEL_PATH[--model_output_pathMODEL_OUTPUT_PATH][--output_saved_model][--output_h5][--output_weight_and_json][--output_pb][--output_no_quant_float32_tflite][--output_dynamic_range_quant_tflite][--output_weight_quant_tflite][--output_float16_quant_tflite][--output_integer_quant_tflite][--output_full_integer_quant_tflite][--output_integer_quant_typeOUTPUT_INTEGER_QUANT_TYPE][--string_formulas_for_normalizationSTRING_FORMULAS_FOR_NORMALIZATION][--calib_ds_typeCALIB_DS_TYPE][--ds_name_for_tfds_for_calibrationDS_NAME_FOR_TFDS_FOR_CALIBRATION][--split_name_for_tfds_for_calibrationSPLIT_NAME_FOR_TFDS_FOR_CALIBRATION][--download_dest_folder_path_for_the_calib_tfdsDOWNLOAD_DEST_FOLDER_PATH_FOR_THE_CALIB_TFDS][--tfds_download_flg][--load_dest_file_path_for_the_calib_npyLOAD_DEST_FILE_PATH_FOR_THE_CALIB_NPY][--output_tfjs][--output_tftrt_float32][--output_tftrt_float16][--tftrt_maximum_cached_enginesTFTRT_MAXIMUM_CACHED_ENGINES][--output_coreml][--output_edgetpu][--edgetpu_compiler_timeoutEDGETPU_COMPILER_TIMEOUT][--edgetpu_num_segmentsEDGETPU_NUM_SEGMENTS][--output_onnx][--onnx_opsetONNX_OPSET][--onnx_extra_opsetONNX_EXTRA_OPSET][--disable_onnx_nchw_conversion][--disable_onnx_optimization][--output_myriad][--vpu_number_of_shavesVPU_NUMBER_OF_SHAVES][--vpu_number_of_cmx_slicesVPU_NUMBER_OF_CMX_SLICES][--replace_swish_and_hardswish][--optimizing_hardswish_for_edgetpu][--replace_prelu_and_minmax][--replace_argmax][--replace_argmax_indices_to_float32][--restricted_resize_image_mode][--weight_replacement_configWEIGHT_REPLACEMENT_CONFIG][--disable_experimental_new_quantizer][--disable_per_channel][--optimizing_barracuda][--layerids_of_the_terminating_outputLAYERIDS_OF_THE_TERMINATING_OUTPUT][--keep_input_tensor_in_nchw][--input_as_ncdhw][--non_verbose]optionalarguments:-h,--helpshowthishelpmessageandexit--model_pathMODEL_PATHinputIRmodelpath(.xml)--model_output_pathMODEL_OUTPUT_PATHTheoutputfolderpathoftheconvertedmodelfile--output_saved_modelsaved_modeloutputswitch--output_h5.h5outputswitch--output_weight_and_jsonweightofh5andjsonoutputswitch--output_pb.pboutputswitch--output_no_quant_float32_tflitefloat32tfliteoutputswitch--output_dynamic_range_quant_tflitedynamicrangequanttfliteoutputswitch--output_weight_quant_tfliteweightquanttfliteoutputswitch--output_float16_quant_tflitefloat16quanttfliteoutputswitch--output_integer_quant_tfliteintegerquanttfliteoutputswitch--output_full_integer_quant_tflitefullintegerquanttfliteoutputswitch--output_integer_quant_typeOUTPUT_INTEGER_QUANT_TYPEInputandoutputtypeswhendoingIntegerQuantization('int8 (default)'or'uint8')--string_formulas_for_normalizationSTRING_FORMULAS_FOR_NORMALIZATIONStringformulasfornormalization.ItisevaluatedbyPythonseval()function.Default:'(data - [127.5,127.5,127.5]) / [127.5,127.5,127.5]'--calib_ds_typeCALIB_DS_TYPETypesofdatasetsforcalibration.tfdsornumpyDefault:numpy--ds_name_for_tfds_for_calibrationDS_NAME_FOR_TFDS_FOR_CALIBRATIONDatasetnameforTensorFlowDatasetsforcalibration.https://www.tensorflow.org/datasets/catalog/overview--split_name_for_tfds_for_calibrationSPLIT_NAME_FOR_TFDS_FOR_CALIBRATIONSplitnameforTensorFlowDatasetsforcalibration.https://www.tensorflow.org/datasets/catalog/overview--download_dest_folder_path_for_the_calib_tfdsDOWNLOAD_DEST_FOLDER_PATH_FOR_THE_CALIB_TFDSDownloaddestinationfolderpathforthecalibrationdataset.Default:$HOME/TFDS--tfds_download_flgTruetoautomaticallydownloaddatasetsfromTensorFlowDatasets.TrueorFalse--load_dest_file_path_for_the_calib_npyLOAD_DEST_FILE_PATH_FOR_THE_CALIB_NPYThepathfromwhichtoloadthe.npyfilecontainingthenumpybinaryversionofthecalibrationdata.Default:sample_npy/calibration_data_img_sample.npy--output_tfjstfjsmodeloutputswitch--output_tftrt_float32tftrtfloat32modeloutputswitch--output_tftrt_float16tftrtfloat16modeloutputswitch--tftrt_maximum_cached_enginesSpecifiesthequantityoftftrt_maximum_cached_enginesforTFTRT.Default:10000--output_coremlcoremlmodeloutputswitch--output_edgetpuedgetpumodeloutputswitch--edgetpu_compiler_timeoutedgetpu_compilertimeoutforonecompilationprocessinseconds.Default:3600--edgetpu_num_segmentsPartitionthemodelinto'num_segments'segments.Default:1(nopartition)--output_onnxonnxmodeloutputswitch--onnx_opsetONNX_OPSETonnxopsetversionnumber--onnx_extra_opsetONNX_EXTRA_OPSETThenameoftheonnx'extra_opset'toenable.Default:'''com.microsoft:1'or'ai.onnx.contrib:1'or'ai.onnx.ml:1'--disable_onnx_nchw_conversionDisableNCHWconversion--disable_onnx_optimizationDisableonnxoptimization--output_myriadmyriadinferenceenginebloboutputswitch--vpu_number_of_shavesVPU_NUMBER_OF_SHAVESvpunumberofshaves.Default:4--vpu_number_of_cmx_slicesVPU_NUMBER_OF_CMX_SLICESvpunumberofcmxslices.Default:4--replace_swish_and_hardswishReplaceswishandhard-swishwitheachother--optimizing_hardswish_for_edgetpuOptimizinghardswishforedgetpu--replace_prelu_and_minmaxReplacepreluandminimum/maximumwitheachother--replace_argmaxReplace'ArgMax (TopK)'withaprimitiveoperation.Optimizes'ArgMax'toEdgeTPU.Ifyouhave'ArgMax'attheendofyourmodel,usethe'--replace_argmax_indices_to_float32'optiontogether.--replace_argmax_indices_to_float32EnablingthisoptionmayallowfullmappingtoEdgeTPUwhen'ArgMax (TopK)'isattheendofthemodelfortaskssuchasSemanticSegmentation.Ifyouapplyitto'ArgMax (TopK)',whichislocatedinthemiddleofthemodel,themodeltransformationismorelikelytofail.--restricted_resize_image_modeSpecifythisiftheupsamplingcontainsOPsthatarenotscaledbyintegermultiples.OptimizationforEdgeTPUwillbedisabled.--weight_replacement_configWEIGHT_REPLACEMENT_CONFIGReplacesthevalueofConstforeachlayer_iddefinedinjson.Specifythepathtothejsonfile.'weight_replacement_config.json'--disable_experimental_new_quantizerDisableMLIRsnewquantizationfeatureduringINT8quantizationinTensorFlowLite.--disable_per_channelDisableper-channelquantizationfortflite.--optimizing_barracudaGeneratesONNXbyreplacingBarracudaunsupportedlayerswithstandardlayers.Forexample,GatherND.--layerids_of_the_terminating_outputLAYERIDS_OF_THE_TERMINATING_OUTPUTAcomma-separatedlistoflayerIDstobeusedasoutputlayers.e.g.--layerids_of_the_terminating_output100,201,560Default:''--keep_input_tensor_in_nchwDoesnotconverttheinputtoNHWC,butkeepstheNCHWformat.Transposeisinsertedrightaftertheinputlayer,andthemodelinternalsarehandledbyNHWC.Only4Dinputissupported.--input_as_ncdhwSpecifywhentheshapeofINPUTisthe5DtensorofNCDHW.WhenconvertingtoTensorFlow,theinputgeometryisautomaticallyconvertedtoNDHWCformat.--non_verboseDonotshowalltheweightinformationofeachlayerintheconversionlog.↥ Back to top5-2. saved_model to tflite convertusage:saved_model_to_tflite[-h]--saved_model_dir_pathSAVED_MODEL_DIR_PATH[--signature_defSIGNATURE_DEF][--input_shapesINPUT_SHAPES][--model_output_dir_pathMODEL_OUTPUT_DIR_PATH][--output_no_quant_float32_tflite][--output_dynamic_range_quant_tflite][--output_weight_quant_tflite][--output_float16_quant_tflite][--output_integer_quant_tflite][--output_full_integer_quant_tflite][--output_integer_quant_typeOUTPUT_INTEGER_QUANT_TYPE][--string_formulas_for_normalizationSTRING_FORMULAS_FOR_NORMALIZATION][--calib_ds_typeCALIB_DS_TYPE][--ds_name_for_tfds_for_calibrationDS_NAME_FOR_TFDS_FOR_CALIBRATION][--split_name_for_tfds_for_calibrationSPLIT_NAME_FOR_TFDS_FOR_CALIBRATION][--download_dest_folder_path_for_the_calib_tfdsDOWNLOAD_DEST_FOLDER_PATH_FOR_THE_CALIB_TFDS][--tfds_download_flg][--load_dest_file_path_for_the_calib_npyLOAD_DEST_FILE_PATH_FOR_THE_CALIB_NPY][--output_tfjs][--output_tftrt_float32][--output_tftrt_float16][--tftrt_maximum_cached_enginesTFTRT_MAXIMUM_CACHED_ENGINES][--output_coreml][--output_edgetpu][--edgetpu_compiler_timeoutEDGETPU_COMPILER_TIMEOUT][--edgetpu_num_segmentsEDGETPU_NUM_SEGMENTS][--output_onnx][--onnx_opsetONNX_OPSET][--onnx_extra_opsetONNX_EXTRA_OPSET][--disable_onnx_nchw_conversion][--disable_onnx_optimization][--disable_experimental_new_quantizer][--disable_per_channel]optionalarguments:-h,--helpshowthishelpmessageandexit--saved_model_dir_pathSAVED_MODEL_DIR_PATHInputsaved_modeldirpath--signature_defSIGNATURE_DEFSpecifiesthesignaturenametoloadfromsaved_model--input_shapesINPUT_SHAPESOverwritesanundefinedinputdimension(Noneor-1).Specifytheinputshapein[n,h,w,c]format.Fornon-4Dtensors,specify[a,b,c,d,e],[a,b],etc.Acomma-separatedlistiftherearemultipleinputs.(e.g.)--input_shapes[1,256,256,3],[1,64,64,3],[1,2,16,16,3]--model_output_dir_pathMODEL_OUTPUT_DIR_PATHTheoutputfolderpathoftheconvertedmodelfile--output_no_quant_float32_tflitefloat32tfliteoutputswitch--output_dynamic_range_quant_tflitedynamicrangequanttfliteoutputswitch--output_weight_quant_tfliteweightquanttfliteoutputswitch--output_float16_quant_tflitefloat16quanttfliteoutputswitch--output_integer_quant_tfliteintegerquanttfliteoutputswitch--output_full_integer_quant_tflitefullintegerquanttfliteoutputswitch--output_integer_quant_typeOUTPUT_INTEGER_QUANT_TYPEInputandoutputtypeswhendoingIntegerQuantization('int8 (default)'or'uint8')--string_formulas_for_normalizationSTRING_FORMULAS_FOR_NORMALIZATIONStringformulasfornormalization.ItisevaluatedbyPythonseval()function.Default:'(data - [127.5,127.5,127.5]) / [127.5,127.5,127.5]'--calib_ds_typeCALIB_DS_TYPETypesofdatasetsforcalibration.tfdsornumpyDefault:numpy--ds_name_for_tfds_for_calibrationDS_NAME_FOR_TFDS_FOR_CALIBRATIONDatasetnameforTensorFlowDatasetsforcalibration.https://www.tensorflow.org/datasets/catalog/overview--split_name_for_tfds_for_calibrationSPLIT_NAME_FOR_TFDS_FOR_CALIBRATIONSplitnameforTensorFlowDatasetsforcalibration.https://www.tensorflow.org/datasets/catalog/overview--download_dest_folder_path_for_the_calib_tfdsDOWNLOAD_DEST_FOLDER_PATH_FOR_THE_CALIB_TFDSDownloaddestinationfolderpathforthecalibrationdataset.Default:$HOME/TFDS--tfds_download_flgTruetoautomaticallydownloaddatasetsfromTensorFlowDatasets.TrueorFalse--load_dest_file_path_for_the_calib_npyLOAD_DEST_FILE_PATH_FOR_THE_CALIB_NPYThepathfromwhichtoloadthe.npyfilecontainingthenumpybinaryversionofthecalibrationdata.Default:sample_npy/calibration_data_img_sample.npy--output_tfjstfjsmodeloutputswitch--output_tftrt_float32tftrtfloat32modeloutputswitch--output_tftrt_float16tftrtfloat16modeloutputswitch--tftrt_maximum_cached_enginesSpecifiesthequantityoftftrt_maximum_cached_enginesforTFTRT.Default:10000--output_coremlcoremlmodeloutputswitch--output_edgetpuedgetpumodeloutputswitch--edgetpu_compiler_timeoutedgetpu_compilertimeoutforonecompilationprocessinseconds.Default:3600--edgetpu_num_segmentsPartitionthemodelinto'num_segments'segments.Default:1(nopartition)--output_onnxonnxmodeloutputswitch--onnx_opsetONNX_OPSETonnxopsetversionnumber--onnx_extra_opsetONNX_EXTRA_OPSETThenameoftheonnx'extra_opset'toenable.Default:'''com.microsoft:1'or'ai.onnx.contrib:1'or'ai.onnx.ml:1'--disable_onnx_nchw_conversionDisableNCHWconversion--disable_onnx_optimizationDisableonnxoptimization--disable_experimental_new_quantizerDisableMLIRsnewquantizationfeatureduringINT8quantizationinTensorFlowLite.--disable_per_channelDisableper-channelquantizationfortflite.↥ Back to top5-3. pb to saved_model convertusage:pb_to_saved_model[-h]--pb_file_pathPB_FILE_PATH--inputsINPUTS--outputsOUTPUTS[--model_output_pathMODEL_OUTPUT_PATH]optionalarguments:-h,--helpshowthishelpmessageandexit--pb_file_pathPB_FILE_PATHInput.pbfilepath(.pb)--inputsINPUTS(e.g.1)input:0,input:1,input:2(e.g.2)images:0,input:0,param:0--outputsOUTPUTS(e.g.1)output:0,output:1,output:2(e.g.2)Identity:0,Identity:1,output:0--model_output_pathMODEL_OUTPUT_PATHTheoutputfolderpathoftheconvertedmodelfile↥ Back to top5-4. pb to tflite convertusage:pb_to_tflite[-h]--pb_file_pathPB_FILE_PATH--inputsINPUTS--outputsOUTPUTS[--model_output_pathMODEL_OUTPUT_PATH]optionalarguments:-h,--helpshowthishelpmessageandexit--pb_file_pathPB_FILE_PATHInput.pbfilepath(.pb)--inputsINPUTS(e.g.1)input,input_1,input_2(e.g.2)images,input,param--outputsOUTPUTS(e.g.1)output,output_1,output_2(e.g.2)Identity,Identity_1,output--model_output_pathMODEL_OUTPUT_PATHTheoutputfolderpathoftheconvertedmodelfile↥ Back to top5-5. saved_model to pb convertusage:saved_model_to_pb[-h]--saved_model_dir_pathSAVED_MODEL_DIR_PATH[--model_output_dir_pathMODEL_OUTPUT_DIR_PATH][--signature_nameSIGNATURE_NAME]optionalarguments:-h,--helpshowthishelpmessageandexit--saved_model_dir_pathSAVED_MODEL_DIR_PATHInputsaved_modeldirpath--model_output_dir_pathMODEL_OUTPUT_DIR_PATHTheoutputfolderpathoftheconvertedmodelfile(.pb)--signature_nameSIGNATURE_NAMESignaturenametobeextractedfromsaved_model↥ Back to top5-6. Extraction of IR weightusage:ir_weight_extractor[-h]-mMODEL-oOUTPUT_PATH

optionalarguments:-h,--helpshowthishelpmessageandexit-mMODEL,--modelMODELinputIRmodelpath-oOUTPUT_PATH,--output_pathOUTPUT_PATHweightsoutputfolderpath↥ Back to top6. Execution sample6-1. Conversion of OpenVINO IR to Tensorflow modelsOutOfMemory may occur when converting to saved_model or h5 when the file size of the original model is large, please try the conversion to a pb file alone.$ openvino2tensorflow \
  --model_path openvino/448x448/FP32/Resnet34_3inputs_448x448_20200609.xml \
  --output_saved_model \
  --output_pb \
  --output_weight_quant_tflite \
  --output_float16_quant_tflite \
  --output_no_quant_float32_tflite↥ Back to top6-2. Convert Protocol Buffer (.pb) to saved_modelThis tool is useful if you want to check the internal structure of pb files, tflite files, .h5 files, coreml files and IR (.xml) files.https://lutzroeder.github.io/netron/$ pb_to_saved_model \
  --pb_file_path model_float32.pb \
  --inputs inputs:0 \
  --outputs Identity:0↥ Back to top6-3. Convert Protocol Buffer (.pb) to tflite$ pb_to_tflite \
  --pb_file_path model_float32.pb \
  --inputs inputs \
  --outputs Identity,Identity_1,Identity_2↥ Back to top6-4. Convert saved_model to Protocol Buffer (.pb)$ saved_model_to_pb \
  --saved_model_dir_path saved_model \
  --model_output_dir_path pb_from_saved_model \
  --signature_name serving_default↥ Back to top6-5. Converts saved_model to OpenVINO IR$ python3 ${INTEL_OPENVINO_DIR}/deployment_tools/model_optimizer/mo_tf.py \
  --saved_model_dir saved_model \
  --output_dir openvino/reverse↥ Back to top6-6. Checking the structure of saved_model$ saved_model_cli show \
  --dir saved_model \
  --tag_set serve \
  --signature_def serving_default↥ Back to top6-7. Replace weights or constant values inConstOP, and addTransposeorReshapeorCastorSqueezeorUnsqueezeorAddorMultiplyjust before/after the operation specified by layer_id6-7-1. OverviewIf the transformation behavior ofReshape,Transpose, etc. does not go as expected, you can force theConstcontent to change by defining weights and constant values in a JSON file and having it read in. Alternatively,TransposeorReshapeorCastorSqueezeorUnsqueezeorAddorMultiplycan be added just before the operation specified by layer_id. After changing the structure, you need to carefully check the consistency ofReshape,TransposeandInterpolatebefore and after. Even if the model is successfully transformed, there is a possibility that the dimension that should be changed is transformed incorrectly. In particular,ReshapeandInterpolateare often able to transform the model even if the number of elements in the dimension is messed up.$ openvino2tensorflow \
  --model_path xxx.xml \
  --output_saved_model \
  --output_pb \
  --output_weight_quant_tflite \
  --output_float16_quant_tflite \
  --output_no_quant_float32_tflite \
  --weight_replacement_config weight_replacement_config_sample.jsonStructure of JSON sample{"format_version":2,"layers":[{"layer_id":"659","type":"Const","replace_mode":"direct","values":[0,1,2]},{"layer_id":"660","type":"Reshape","replace_mode":"insert_after","values":[2100,85]},{"layer_id":"680","type":"Cast","replace_mode":"insert_after","values":"i64"},{"layer_id":"442","type":"Concat","replace_mode":"change_axis","values":4},{"layer_id":"450","type":"SoftMax","replace_mode":"change_axis","values":2},{"layer_id":"500","type":"StridedSlice","replace_mode":"change_attributes","values":[0,0,0,0,0]},{"layer_id":"550","type":"StridedSlice","replace_mode":"replace","values":[[0,0,0,8],[2,7,11,16],[1,1,1,1],0,0,0,0,0]},{"layer_id":"600","type":"MaxPool","replace_mode":"change_padding_mode","values":"REFLECT"},{"layer_id":"720","type":"PReLU","replace_mode":"change_shared_axes","values":[1,2]},{"layer_id":"800","type":"ReverseSequence","replace_mode":"change_seq_axis","values":2},{"layer_id":"850","type":"Squeeze","replace_mode":"insert_after","values":1},{"layer_id":"900","type":"Unsqueeze","replace_mode":"insert_before","values":2},{"layer_id":"1000","type":"Einsum","replace_mode":"change_equation","values":"vu,nctu->nctv"},{"layer_id":"1005","type":"Add","replace_mode":"insert_after","values":[0,0,0,2]},{"layer_id":"1010","type":"Multiply","replace_mode":"insert_after","values":[1.0,1.0,-0.5,1.0]}]}No.ElementsDescription1format_versionFormat version of weight_replacement_config. Values less than or equal to 2.2layersA list of layers. Enclose it with "[ ]" to define multiple layers to child elements.2-1layer_idID of the Const layer whose weight/constant parameter is to be swapped. The important thing to note is that you cannot create multiple settings for a single layer_id. There should always be a single setting for a single layer_id. For example, specify "1123" for layer id="1123" for type="Const" in .xml.2-2typeFixed value replacement or type of operation to be added. "Const" or "Transpose" or "Reshape" or "Cast" or "Concat" or "SoftMax" or "StridedSlice" or "MaxPool" or "PReLU" or "ReverseSequence" or "Squeeze" or "Unsqueeze" or "LogSoftmax" or "Einsum" or "Add" or "Multiply"2-3replace_mode"direct" or "npy" or "insert_before" or "insert_after" or "change_axis" or "change_attributes"."direct": Specify the values of the Numpy matrix directly in the "values" attribute. Ignores the values recorded in the .bin file and replaces them with the values specified in "values"."npy": Load a Numpy binary file with the matrix output bynp.save('xyz', a). The "values" attribute specifies the path to the Numpy binary file."insert_before": AddTransposeorReshapeorCastorSqueezeorUnsqueezeorAddorMultiplyjust before the operation specified by layer_id. Note that whenSqueezeandUnsqueezeare specified, the value to set for "values" is the axis of the dimension operation target."insert_after": AddTransposeorReshapeorCastorSqueezeorUnsqueezeorAddorMultiplyjust after the operation specified by layer_id. Note that whenSqueezeandUnsqueezeare specified, the value to set for "values" is the axis of the dimension operation target."change_axis": Changes theaxisof theConcatorSoftMaxorShuffleChannelsorLogSoftmaxattribute value."change_attributes": Changes theATTRIBUTESof theStridedSliceattribute value. Specify five values in numerical list format in the order ofbegin_mask,end_mask,ellipsis_mask,new_axis_mask,shrink_axis_mask."replace": Replaces OP by specifying parameters directly in TensorFlowStrided_Slicespecification.begin,end,strides,begin_mask,end_mask,ellipsis_mask,new_axis_mask,shrink_axis_maskhttps://www.tensorflow.org/api_docs/python/tf/strided_slice"change_padding_mode": Change the padding mode ofMaxPool."change_shared_axes": Changed shared_axes inPReLU."change_batch_axis","change_seq_axis": Changed axis inReverseSequence."change_equation": Changed equation inEinsum.2-4valuesSpecify the value or the path to the Numpy binary file to replace the weight/constant value recorded in .bin. The way to specify is as described in the description of 'replace_mode'. The table below shows the correspondence between the strings that can be specified for the "Cast" operation and the TensorFlow types. In most cases, you will probably only use "i32", "i64", "f32", and "f16".change_padding_mode:"ZERO"or"SYMMETRIC"or"REFLECT".https://www.tensorflow.org/api_docs/python/tf/padchange_shared_axes:https://www.tensorflow.org/api_docs/python/tf/keras/layers/PReLUchange_batch_axis, change_seq_axis:https://docs.openvino.ai/2021.4/openvino_docs_ops_movement_ReverseSequence_1.html"change_equation":https://numpy.org/doc/stable/reference/generated/numpy.einsum.html↥ Back to top6-7-2. ExampleYOLOX Nano 320x320 (NCHW format)yolox_nano_320x320.xmlyolox_nano_320x320.binLet's assume that you don't needTransposein the final layer of the model. Here you have[1, 85, 2100]as input, and the original OpenVINO model transposes[0, 2, 1]in that order to obtain the tensor[1, 2100, 85]. This figure shows the visualization of ayolox_nano_320x320.xmlfile usingNetron. The number shown in theOUTPUTS-output-name:is the layer ID ofTranspose. The layer ID 660 is the number in the part before the colon. The number in the part after the colon is called the port number 2. However, what you are trying to change is the transposition parameter of theINPUTS-custom-name:part. The name of the parameter you are trying to change is625. Note that625is not a layer ID, just a name.Check the model structure as recorded in .xml. First, openyolox_nano_320x320.xmlin your favorite IDE.Search forto-layer="660"(Transpose) in the IDE. In the figure below, Layer ID658and Layer ID659are represented as input values connected to Layer ID660.In the figure below, one of them is658and one of them is659. It is difficult to determine exactly what it is from the image alone. You must again note that658:3in the image is only a name, not a layer ID. It is worth noting here that the type of value you want to replace isConst.Now you will search for layer ID"658"in the IDE. The type is"Concat", so the desired layer was not this one. What you are looking for is"Const".Now, search for layer ID659in the IDE. The type is"Const". Now you can finally identify that the layer ID of the layer you want to replace is659.Create a JSON file to replace the constants[0, 2, 1]with[0, 1, 2], and you can use any name for the JSON file. Suppose you save the file with the namereplace.json. If you want to replace it with a numpy matrix, specify"npy"for"replace_mode":and the path to the.npyfile for"values":.{"format_version":2,"layers":[{"layer_id":"659","type":"Const","replace_mode":"direct","values":[0,1,2]}]}{"format_version":2,"layers":[{"layer_id":"659","type":"Const","replace_mode":"npy","values":"path/to/your/xxx.npy"}]}Specify the created JSON file as the argument of the--weight_replacement_configparameter of the conversion command and execute it. This is the end of the explanation of how to replace weights and constants.$ openvino2tensorflow \
--model_path yolox_nano_320x320.xml \
--output_saved_model \
--output_pb \
--output_no_quant_float32_tflite \
--weight_replacement_config replace.json↥ Back to top6-8. Check the contents of the .npy file, which is a binary version of the image file$ view_npy --npy_file_path sample_npy/calibration_data_img_sample.npyPress theQbutton to display the next image.calibration_data_img_sample.npycontains 20 images extracted from the MS-COCO data set.↥ Back to top6-9. Sample image of a conversion error messageSince it is very difficult to mechanically predict the correct behavior ofTransposeandReshape, errors like the one shown below may occur. Using the information in the figure below, try several times to force the replacement of constants and weights using the--weight_replacement_configoption#6-7-replace-weights-or-constant-values-in-const-op-and-add-transpose-or-reshape-or-cast-or-squeeze-or-unsqueeze-or-add-or-multiply-just-beforeafter-the-operation-specified-by-layer_id. This is a very patient process, but if you take the time, you should be able to convert it correctly.↥ Back to top6-10. Ability to specify an output layer for debugging the output values of the modelIf you want to debug the output values of each layer, specify multiple layer IDs separated by commas in the--layerids_of_the_terminating_outputoption to check the output values. For example, if you want to debug the output values of two layers,LayerID=1007 (Add)andLayerID=1214 (Sigmoid), as shown in the figure below, specify as--layerids_of_the_terminating_output 1007,1214.When you convert a model, the output will be censored at the two specified layer IDs, and the model will be generated with the output of the model available for review. Note that if you specify a layer ID for an operation that has multiple outputs, such asSplit,VariadicSplit,TopK, orNonMaxSuppression, all output values will be used as outputs.↥ Back to top7. Output sample↥ Back to top8. Model Structurehttps://digital-standard.com/threedpose/models/Resnet34_3inputs_448x448_20200609.onnxONNX (NCHW)OpenVINO (NCHW)TFLite (NHWC)↥ Back to top9. My article[English] Converting PyTorch, ONNX, Caffe, and OpenVINO (NCHW) models to Tensorflow / TensorflowLite (NHWC) in a snapPyTorch, ONNX, Caffe, OpenVINO (NCHW) のモデルをTensorflow / TensorflowLite (NHWC) へお手軽に変換するtf.image.resizeを含むFull Integer Quantization (.tflite)モデルのEdgeTPUモデルへの変換後の推論時に発生する "main.ERROR - Only float32 and uint8 are supported currently, got -xxx.Node number n (op name) failed to invoke" エラーの回避方法↥ Back to top10. Conversion Confirmed Modelsu-2-netmobilenet-v2-pytorchmidasnetfootprintsefficientnet-b0-pytorchefficientdet-d0dense_depthdeeplabv3colorization-v2-norebalage-gender-recognition-retail-0013resnetarcfaceemotion-ferplusmosaicretinanetshufflenet-v2squeezenetversion-RFB-320yolov4yolov4x-mishThreeDPoseUnityBarracuda - Resnet34_3inputs_448x448efficientnet-lite4nanodetyolov4-tinyyolov5syolactMiDaS v2MODNetPerson ReidentificationDeepSortDINO (Transformer)↥ Back to top