Package: nosmpl
NoSMPLAn enchanced and accelerated SMPL operation which commonly used in 3D human mesh generation. It takes a poses, shapes, cam_trans as inputs, outputs a high-dimensional 3D mesh verts.However, SMPL codes and models are so messy out there, they have a lot of codes do calculation, some of them can not be easily deployed or accerlarated. So we havenosmplhere, it provides:build on smplx, but with onnx support;can be inference via onnx;we also demantrated some using scenarios infer withnosmplbut without any model, only onnx.This packages provides:Highly optimized pytorch acceleration with FP16 infer enabled;Supported ONNX export and infer via ort, so that it might able used into TensorRT or OpenVINO on cpu;Support STAR, next generation of SMPL.Provide commonly used geoemtry built-in support without torchgeometry or kornia.STAR model download from:https://star.is.tue.mpg.deSMPL ONNX Model DownloadsI have exported 2 models, includeSMPL-HandSMPL, which can cover most using scenarios:smpl:linksmpl-h:linkThey can also be found at github release.For usage, you can take examples likeexamples/demo_smplh_onnx.py.Quick StartNow you can usingnosmplto visualize smpl with just few line of codes withoutdownload any SMPL file:fromnosmpl.smpl_onnximportSMPLOnnxRuntimeimportnumpyasnpsmpl=SMPLOnnxRuntime()body=np.random.randn(1,23,3).astype(np.float32)global_orient=np.random.randn(1,1,3).astype(np.float32)outputs=smpl.forward(body,global_orient)print(outputs)# you can visualize the verts with Open3D now.So your predicted 3d pose such as SPIN, HMR, PARE etc, grap your model ouput, and through thisnosmplfunc, you will get your SMPL vertices!Updates2023.02.28: An SMPL-H ONNX model released! Now You can using ONNXRuntime to get a 3D SMPL Mesh from a pose!2022.05.16: Now addedhuman_priorinsidenosmpl, you don't need install that lib anymore, or install torchgeometry either:fromnosmpl.vpose.tools.model_loaderimportload_vposerself.vposer,_=load_vposer(VPOSER_PATH,vp_model="snapshot")then you can load vpose to use.2022.05.10: Add BHV reader, you can now read and write bvh file:fromnosmpl.parsersimportbvh_ioimportsysanimation=bvh_io.load(sys.argv[1])print(animation.names)print(animation.frametime)print(animation.parent)print(animation.offsets)print(animation.shape)2022.05.07: Added a visualization for Human36m GT, you can using like this to visualize h36m data now:import nosmpl.datasets.h36m_data_utils as data_utils
from nosmpl.datasets.h36m_vis import h36m_vis_on_gt_file
import sys

if __name__ == "__main__":
    h36m_vis_on_gt_file(sys.argv[1])Just send a h36m txt annotation file, and you can see the animation result. Also, you can usingfrom nosmpl.datasets.h36m_vis import h36m_load_gt_3d_datato load 3d data in 3D space.2022.03.03: I add somebox_transformcode intonosmpl, no we can get box_scale info when recover cropped img predicted 3d vertices back to original image. This is helpful when you project 3d vertices back to original image when usingrealrender.
the usage like:from nosmpl.box_trans import get_box_scale_info, convert_vertices_to_ori_img
box_scale_o2n, box_topleft, _ = get_box_scale_info(img, bboxes)
frame_verts = convert_vertices_to_ori_img(
          frame_verts, s, t, box_scale_o2n, box_topleft
      )2022.03.05: More to go.FeaturesThe most exciting feature innosmplisyou don't need download any SMPL files anymore, you just need to download my exportedSMPL.onnxorSMPLH.onnx, then you can using numpy to generate a Mesh!!!nosmplalso provided a script to visualize it~!importonnxruntimeasrtimporttorchimportnumpyasnpfromnosmpl.vis.vis_o3dimportvis_mesh_o3ddefgen():sess=rt.InferenceSession("smplh_sim.onnx")foriinrange(5):body_pose=(torch.randn([1,63],dtype=torch.float32).clamp(0,0.4).cpu().numpy())left_hand_pose=(torch.randn([1,6],dtype=torch.float32).clamp(0,0.4).cpu().numpy())right_hand_pose=(torch.randn([1,6],dtype=torch.float32).clamp(0,0.4).cpu().numpy())outputs=sess.run(None,{"body":body_pose,"lhand":left_hand_pose,"rhand":right_hand_pose})vertices,joints,faces=outputsvertices=vertices[0].squeeze()joints=joints[0].squeeze()faces=faces.astype(np.int32)vis_mesh_o3d(vertices,faces)if__name__=="__main__":gen()You will see a mesh with your pose, generated:As you can see, we are using a single ONNX model, by some randome poses, you can generated a visualized mesh.this is useful when you wanna test your predict pose is right or not!If you using this in your project, your code will be decrease 190%, if it helps, consider citenosmplin your project!More details you can join our Metaverse Wechat group for discussion! QQ join link:Examplesan example to callnosmlp:fromnosmpl.smplimportSMPLsmpl=SMPL(smplModelPath,extra_regressor='extra_data/body_module/data_from_spin/J_regressor_extra.npy').to(device)# get your betas and rotmatpred_vertices,pred_joints_3d,faces=smpl(pred_betas,pred_rotmat)# note that we returned faces in SMPL model, you can use for visualization# joints3d will add extra joints if you have extra_regressor like in SPIN or VIBEThe output shape of onnx model like:basicModel_neutral_lbs_10_207_0_v1.0.0.onnx Detail
╭───────────────┬────────────────────────────┬──────────────────────────┬────────────────╮
│ Name          │ Shape                      │ Input/Output             │ Dtype          │
├───────────────┼────────────────────────────┼──────────────────────────┼────────────────┤
│ 0             │ [1, 10]                    │ input                    │ float32        │
│ 1             │ [1, 24, 3, 3]              │ input                    │ float32        │
│ verts         │ [-1, -1, -1]               │ output                   │ float32        │
│ joints        │ [-1, -1, -1]               │ output                   │ float32        │
│ faces         │ [13776, 3]                 │ output                   │ int32          │
╰───────────────┴────────────────────────────┴──────────────────────────┴────────────────╯
                             Table generated by onnxexplorerNotesAbout quaterniontheaa2quatfunction, will converts quaternion inwxyzas default order. This isdifferentfrom scipy. It's consistent as mostly 3d software such as Blender or UE.ResultsSome pipelines build withnosmplsupport.CopyrightsCopyrights belongs to Copyright (C) 2020 Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V. (MPG) and Lucas Jin