Package: rtmlib
rtmlibrtmlib is a super lightweight library to conduct pose estimation based onRTMPosemodelsWITHOUTany dependencies like mmcv, mmpose, mmdet, etc.Basically, rtmlib only requires these dependencies:numpyopencv-pythonopencv-contrib-pythononnxruntimeOptionally, you can use other common backends like opencv, onnxruntime, openvino, tensorrt to accelerate the inference process.For openvino users, please add the path<your python path>\envs\<your env name>\Lib\site-packages\openvino\libsinto your environment path.Installationinstall from pypi:pipinstallrtmlib-ihttps://pypi.org/simpleinstall from source code:gitclonehttps://github.com/Tau-J/rtmlib.gitcdrtmlib

pipinstall-rrequirements.txt

pipinstall-e.# [optional]# pip install onnxruntime-gpu# pip install openvinoQuick StartHere is a simple demo to show how to use rtmlib to conduct pose estimation on a single image.importcv2fromrtmlibimportWholebody,draw_skeletondevice='cpu'# cpu, cudabackend='onnxruntime'# opencv, onnxruntime, openvinoimg=cv2.imread('./demo.jpg')openpose_skeleton=False# True for openpose-style, False for mmpose-stylewholebody=Wholebody(to_openpose=openpose_skeleton,mode='balanced',# 'performance', 'lightweight', 'balanced'. Default: 'balanced'backend=backend,device=device)keypoints,scores=wholebody(img)# visualize# if you want to use black background instead of original image,# img_show = np.zeros(img_show.shape, dtype=np.uint8)img_show=draw_skeleton(img_show,keypoints,scores,kpt_thr=0.5)cv2.imshow('img',img_show)cv2.waitKey()WebUIRunwebui.py:# Please make sure you have installed gradio# pip install gradiopythonwebui.pyAPIsSolutions (High-level APIs)WholebodyBodyHandPoseTrackerModels (Low-level APIs)YOLOXRTMDetRTMPoseRTMPose for 17 keypointsRTMW for 133 keypointsRTMO for one-stage pose estimation (17 keypoints)Visualizationdraw_bboxdraw_skeletonFor high-level APIs (Solution), you can choose to passmodeordet+posearguments to specify the detector and pose estimator you want to use.# By modewholebody=Wholebody(mode='performance',# 'performance', 'lightweight', 'balanced'. Default: 'balanced'backend=backend,device=device)# By det and posebody=Body(det='https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/onnx_sdk/yolox_x_8xb8-300e_humanart-a39d44ed.zip',det_input_size=(640,640),pose='https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/onnx_sdk/rtmpose-x_simcc-body7_pt-body7_700e-384x288-71d7b7e9_20230629.zip',pose_input_size=(288,384),backend=backend,device=device)For low-level APIs (Model), you can specify the model you want to use by passing theonnx_modelargument.# By onnx_model (.onnx)pose_model=RTMPose(onnx_model='/path/to/your_model.onnx',# download link or local pathbackend=backend,device=device)# By onnx_model (.zip)pose_model=RTMPose(onnx_model='https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/onnx_sdk/rtmpose-m_simcc-body7_pt-body7_420e-256x192-e48f03d0_20230504.zip',# download link or local pathbackend=backend,device=device)Model ZooBy defaults, rtmlib will automatically download and apply models with the best performance.More models can be found inRTMPose Model Zoo.DetectorsPersonNotes:Models trained on HumanArt can detect both real human and cartoon characters.Models trained on COCO can only detect real human.ONNX ModelInput SizeDescriptionYOLOX-l640x640trained on COCO val2017YOLOX-nano416x416trained on HumanArtYOLOX-tiny416x416trained on HumanArtYOLOX-s640x640trained on HumanArtYOLOX-m640x640trained on HumanArtYOLOX-l640x640trained on HumanArtYOLOX-x640x640trained on HumanArtPose EstimatorsBodyONNX ModelInput SizeDescriptionRTMPose-t256x192Body 17 KeypointsRTMPose-s256x192Body 17 KeypointsRTMPose-m256x192Body 17 KeypointsRTMPose-l384x288Body 17 KeypointsRTMPose-x384x288Body 17 KeypointsRTMO-s640x640Body 17 KeypointsRTMO-m640x640Body 17 KeypointsRTMO-l640x640Body 17 KeypointsWholeBodyONNX ModelInput SizeDescriptionRTMW-m256x192Wholebody 133 KeypointsRTMW-l256x192Wholebody 133 KeypointsRTMW-l384x288Wholebody 133 KeypointsRTMW-x384x288Wholebody 133 KeypointsVisualizationMMPose-styleOpenPose-styleCitation@misc{rtmlib,
  title={rtmlib},
  author={Jiang, Tao},
  year={2023},
  howpublished = {\url{https://github.com/Tau-J/rtmlib}},
}

@misc{jiang2023,
  doi = {10.48550/ARXIV.2303.07399},
  url = {https://arxiv.org/abs/2303.07399},
  author = {Jiang, Tao and Lu, Peng and Zhang, Li and Ma, Ningsheng and Han, Rui and Lyu, Chengqi and Li, Yining and Chen, Kai},
  keywords = {Computer Vision and Pattern Recognition (cs.CV), FOS: Computer and information sciences, FOS: Computer and information sciences},
  title = {RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose},
  publisher = {arXiv},
  year = {2023},
  copyright = {Creative Commons Attribution 4.0 International}
}

@misc{lu2023rtmo,
      title={{RTMO}: Towards High-Performance One-Stage Real-Time Multi-Person Pose Estimation},
      author={Peng Lu and Tao Jiang and Yining Li and Xiangtai Li and Kai Chen and Wenming Yang},
      year={2023},
      eprint={2312.07526},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}AcknowledgementOur code is based on these repos:MMPoseRTMPoseDWPose