Package: h3ds
H3DS DatasetThis repository contains the code for using the H3DS dataset introduced inH3D-Net: Few-Shot High-Fidelity 3D Head Reconstruction (ICCV 2021)and expanded inSIRA++: Implicit Shape and Appearance Priors for Few-Shot
Full Head Reconstruction.AccessThe H3DS dataset is only available for non-commercial research purposes. To request access, please fill in thecontact formwith your academic email. Your application will be reviewed and, after acceptance, you will recieve aH3DS_ACCESS_TOKEN. Each version of the dataset has a different acces token.SetupThe simplest way to use the H3DS dataset is by installing it as a pip package:pipinstallh3dsAccessing H3DS dataThe dataset has grown over a series of releases. Version 1.0 (config_v1.toml), introduced in H3D-Net, included 23 subjects. Version 2.0 (config_v2.toml), introduced in SIRA++, includes 59 subjects. Although version 2.0 contains all the subjects of version 1.0, we have decided to maintain both as the mesh alignment could be slightly different.You can start using H3DS in your project with a few lines of codefromh3ds.datasetimportH3DSh3ds=H3DS(path='local/path/to/h3ds',config_id='config_v2')h3ds.download(token=H3DS_ACCESS_TOKEN)# Python zip decryption is very slow. You can invoke an external program.mesh,images,masks,cameras=h3ds.load_scene(scene_id='1b2a8613401e42a8')It is also possible to download and unzip the dataset manually (v1 link,v2 link). Then, unzip the file using theH3DS_ACCESS_TOKENas password.unzip-PH3DS_ACCESS_TOKENlocal/path/to/h3ds.zip-dlocal/path/to/h3dsTo list the available scenes, simply use:scenes=h3ds.scenes()# returns all the scenes ['1b2a8613401e42a8', ...]scenes=h3ds.scenes(tags={'h3d-net'})# returns the scenes used in H3D-Net paperscenes=h3ds.scenes(tags={'sira++'})# returns the scenes used in SIRA++ paperIn order to reproduce the results from H3D-Net and SIRA++, we provide default views configurations for each scene:views_configs=h3ds.default_views_configs(scene_id='1b2a8613401e42a8')# '1', '3', '4', '6', '8', '16' and '32' viewsmesh,images,masks,cameras=h3ds.load_scene(scene_id='1b2a8613401e42a8',views_config_id='3')This will load a scene with a mesh, 3 images, 3 masks and 3 cameras.EvaluationWe provide a method for evaluating your reconstructions with a single line of codemesh_pred,landmarks_pred=my_rec_method(images,masks,cameras)chamfer,_,_,_=h3ds.evaluate_scene('1b2a8613401e42a8',mesh_pred,landmarks_pred)Thelandmarks_predis an optional dictionary containing landmarks used for a coarse alignment between the predicted mesh and the ground truth mesh. Please, checkthis descriptionof the landmarks positions.For more insights, check the examples provided.Comparison against H3D-Net and SIRA++The results reported in the H3D-Net and SIRA++ papers slightly differ from the ones obtained using the evaluation code provided in this repository. This is due to minor implementation changes in the alignment process and in the cutting of the regions. In the following table we provide the results obtained using the evaluation code from this repository. We encourage everyone to use theevaluate_scenemethod provided in this repository to report comparable results accross different works.Evaluation in H3DS version 1.0 (config_v1):Method \ Views3481632IDR2.79 / 14.581.88 / 8.991.83 / 8.341.31 / 6.371.25 / 5.71H3D-Net1.33 / 10.521.35 / 7.711.18 / 6.451.05 / 5.361.03 / 5.25SIRA++1.13 / 10.841.18 / 8.071.15 / 7.081.00 / 6.340.99 / 5.79Evaluation in H3DS version 2.0 (config_v2):Method \ Views134681632SIRA++1.56 / 13.661.18 / 9.701.21 / 6.091.05 / 5.591.18 / 5.281.08 / 4.961.03 / 4.71The numbers from the table can be obtained by running theevaluation scriptfrom the examples folder, which usesthe 3D reconstructions from the paperto compute the metrics.Terms of useThe H3DS dataset is available for non-commercial or research use only. By using or downloading the files included in the dataset you agree to the terms of thislicense agreementand accept to comply with its terms. If you do not agree with these terms, you may not accesss, download or use the files.Data acquisitionThe data acquisition process for each scene in the dataset involves several steps. Initially, the camera of an iPad Pro is calibrated to be aligned to an attached structured light sensor, specifically the Occipital Structure Sensor Pro. This calibration process allows us to obtain paired RGB images and camera parameters, along with a low-resolution mesh scan. Simultaneously, a high-end Artec Eva scanner is utilized to capture high-quality 3D scans. Subsequently, we align the low and high-resolution meshes, along with the paired cameras, by employing six manually annotated 3D landmarks and utilizing the iterative closest point (ICP) refinement technique. Furthermore, for each image within the dataset, we manually annotate a foreground mask, providing additional information for foreground-background separation.As of data acquisition in v2, the subjects remains static for images at +-90° degrees from the front. This yields a consistent lighting and background, as well as better cameras. This cannot be assumed for subjects from v1.Projects using H3DSHere are some works that we have noticed using the dataset. Please reach out if we missed you!Neural Haircut: Prior-Guided Strand-Based Hair Reconstruction (ICCV 2023)Sklyarova, Vanessa and Chelishev, Jenya and Dogaru, Andreea and Medvedev, Igor and Lempitsky, Victor and Zakharov, EgorNeRF-Art: Text-Driven Neural Radiance Fields Stylization (TVCG2023)Wang, Can and Jiang, Ruixiang and Chai, Menglei and He, Mingming and Chen, Dongdong and Liao, JingReal-Time Radiance Fields for Single-Image Portrait View Synthesis (SIGGRAPH2023)Alex Trevithick and Matthew Chan and Michael Stengel and Eric R. Chan and Chao Liu and Zhiding Yu and Sameh Khamis and Manmohan Chandraker and Ravi Ramamoorthi and Koki NaganoMulti-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions (2023)Burkov, Egor and Rakhimov, Ruslan and Safin, Aleksandr and Burnaev, Evgeny and Lempitsky, VictorSIRA: Relightable Avatars From a Single Image (WACV2023)Caselles, Pol and Ramon, Eduard and Garcia, Jaime and Giro-i-Nieto, Xavier and Moreno-Noguer, Francesc and Triginer, GilNPBG++: Accelerating Neural Point-Based Graphics (CVPR2022)Rakhimov, Ruslan and Ardelean, Andrei-Timotei and Lempitsky, Victor and Burnaev, EvgenyLearning an Isometric Surface Parameterization for Texture Unwrapping (ECCV 2022)Das, Sagnik and Ma, Ke and Shu, Zhixin and Samaras, DimitrisVoRF: Volumetric Relightable Faces (BMVC 2022)Rao, Pramod and {B R}, Mallikarjun and Fox, Gereon and Weyrich, Tim and Bickel, Bernd and Seidel, Hans-Peter and Pfister, Hanspeter and Matusik, Wojciech and Tewari, Ayush and Theobalt, Christian and  Elgharib, MohamedContact InfoPol Caselles:pol.caselles@crisalix.comCitationIf you find this project helpful to your research, please consider citing the following:@inproceedings{ramon2021h3d,
  title={H3D-Net: Few-Shot High-Fidelity 3D Head Reconstruction},
  author={Ramon, Eduard and Triginer, Gil and Escur, Janna and Pumarola, Albert and Garcia, Jaime and Giro-i-Nieto, Xavier and Moreno-Noguer, Francesc},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={5620--5629},
  year={2021}
}@article{caselles2023implicit,
  title={Implicit Shape and Appearance Priors for Few-Shot Full Head Reconstruction},
  author={Caselles, Pol and Ramon, Eduard and Garcia, Jaime and Triginer, Gil and Moreno-Noguer, Francesc},
  journal={arXiv preprint arXiv:2310.08784},
  year={2023}
}