Package: sixdrepnet
6D Rotation Representation for Unconstrained Head Pose Estimation (Pytorch)CitingIf you find our work useful, please cite the paper:@INPROCEEDINGS{9897219,author={Hempel, Thorsten and Abdelrahman, Ahmed A. and Al-Hamadi, Ayoub},booktitle={2022 IEEE International Conference on Image Processing (ICIP)},title={6d Rotation Representation For Unconstrained Head Pose Estimation},year={2022},volume={},number={},pages={2496-2500},doi={10.1109/ICIP46576.2022.9897219}}Updates13.09.20226DRepNet is now avaiable as pip package for even more accessable usage:pip3 install SixDRepNet20.06.20226DRepNet has been accepted to ICIP 2022.29.05.2022Simplified training scriptUpdated default training configuration for more robust resultsPaperThorsten Hempel and Ahmed A. Abdelrahman and Ayoub Al-Hamadi, "6D Rotation Representation for Unconstrained Head Pose Estimation",accepted to ICIP 2022.[ResearchGate][Arxiv]AbstractIn this paper, we present a method for unconstrained end-to-end head pose estimation. We address the problem of ambiguous rotation labels by introducing the rotation matrix formalism for our ground truth data and propose a continuous 6D rotation matrix representation for efficient and robust direct regression. This way, our method can learn the full rotation appearance which is contrary to previous approaches that restrict the pose prediction to a narrow-angle for satisfactory results. In addition, we propose a geodesic distance-based loss to penalize our network with respect to themanifold geometry. Experiments on the public AFLW2000 and BIWI datasets demonstrate that our proposed method significantly outperforms other state-of-the-art methods by up to 20%.Trained on 300W-LP, Test on AFLW2000 and BIWIFull RangeYawPitchRollMAEYawPitchRollMAEHopeNet (=2)N6.476.565.446.165.176.983.395.18HopeNet  (=1)N6.926.645.676.414.816.613.274.90FSA-NetN4.506.084.645.074.274.962.764.00HPEN4.806.184.875.283.125.184.574.29QuatNetN3.975.623.924.502.945.494.014.15WHENet-VN4.445.754.314.833.604.102.733.48WHENetY/N5.116.244.925.423.994.393.063.81TriNetY4.045.774.204.674.114.763.053.97FDNN3.785.613.884.424.524.702.563.936DRepNetY3.634.913.373.973.244.482.683.47BIWI 70/30YawPitchRollMAEHopeNet (=1)3.293.393.003.23FSA-Net2.894.293.603.60TriNet2.933.042.442.80FDN3.003.982.883.296DRepNet2.692.922.362.66Fine-tuned ModelsFine-tuned models can be download from here:https://drive.google.com/drive/folders/1V1pCV0BEW3mD-B9MogGrz_P91UhTtuE_?usp=sharingQuick Start:Pip install:pip3installsixdrepnetExample usage:# Import SixDRepNetfromsixdrepnetimportSixDRepNetimportcv2# Create model# Weights are automatically downloadedmodel=SixDRepNet()img=cv2.imread('/path/to/image.jpg')pitch,yaw,roll=model.predict(img)model.draw_axis(img,yaw,pitch,roll)cv2.imshow("test_window",img)cv2.waitKey(0)Setting it up your own:gitclonehttps://github.com/thohemp/6DRepNetcd6DRepNetSet up a virtual environment:python3-mvenvvenvsourcevenv/bin/activate
pipinstall-rrequirements.txt# Install required packagesIn order to run the demo scripts you need to install the face detectorpipinstallgit+https://github.com/elliottzheng/face-detection.git@masterCamera Demo:python./sixdrepnet/demo.py--snapshot6DRepNet_300W_LP_AFLW2000.pth\--cam0Test/Train 3DRepNetPreparing datasetsDownload datasets:300W-LP,AFLW2000fromhere.BIWI(Biwi Kinect Head Pose Database) fromhereStore them in thedatasetsdirectory.For 300W-LP and AFLW2000 we need to create afilenamelist.python create_filename_list.py --root_dir datasets/300W_LPThe BIWI datasets needs be preprocessed by a face detector to cut out the faces from the images. You can use the script providedhere. For 7:3 splitting of the BIWI dataset you can use the equivalent scripthere. We set the cropped image size to256.Testing:pythontest.py--batch_size64\--datasetAFLW2000\--data_dirdatasets/AFLW2000\--filename_listdatasets/AFLW2000/files.txt\--snapshotoutput/snapshots/1.pth\--show_vizFalseTrainingDownload pre-trained RepVGG model 'RepVGG-B1g2-train.pth' fromhereand save it in the root directory.pythonsixdrepnet/train.pyDeploy modelsFor reparameterization the trained models into inference-models use the convert script.python convert.py input-model.tar output-model.pthInference-models are loaded with the flagdeploy=True.model=SixDRepNet(backbone_name='RepVGG-B1g2',backbone_file='',deploy=True,pretrained=False)