Package: threefiner
ThreefinerAn interface for text-guided mesh refinement.https://github.com/3DTopia/threefiner/assets/25863658/a4abe725-b542-4a4a-a6d4-e4c4821f7d96FeaturesMesh in, mesh out: we supportplywith vertex colors,obj, and single objectglb/gltfwith textures!Easy to use: both a CLI and a GUI is available.Performant: Refine your texture in 1 minute with Deepfloyd-IF-II.InstallWe rely ontorchand several CUDA extensions, please make sure you install them correctly first!# tiny-cuda-nnpipinstallgit+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch# nvdiffrastpipinstallgit+https://github.com/NVlabs/nvdiffrast# [optional, will use pysdf if unavailable] cubvh:pipinstallgit+https://github.com/ashawkey/cubvhTo useDeepfloyd-IFmodels, please log in to your huggingface and accept thelicense.To install this package:# install from pypipipinstallthreefiner# install from githubpipinstallgit+https://github.com/3DTopia/threefiner# local installgitclonehttps://github.com/3DTopia/threefinercdthreefiner
pipinstall.Usage### command line interfacethreefiner--help# this is short forpython-mthreefiner.cli--help### refine a coarse mesh ('input.obj') using Stable-diffusion and save to 'logs/hamburger.glb'threefinersd--meshinput.obj--prompt'a hamburger'--outdirlogs--savehamburger.glb### if the initial texture is good, we recommend using IF2 for refinement.# by default, it will save to './name_fine.glb'threefinerif2--meshname.glb--prompt'description'### if the initial texture is not good, we recommend using SD or IF first.threefinersd--meshname.glb--prompt'description'threefinerif--meshname.glb--prompt'description'### if the initial geometry is good, you can fix the geometry.threefinersd_fixgeo--meshname.glb--prompt'description'threefinerif_fixgeo--meshname.glb--prompt'description'threefinerif2_fixgeo--meshname.glb--prompt'description'### advanced# directional text prompt (append front/side/back view in text prompt)# you need to know the mesh's front facing direction and specify it by '--front_dir'# we use the OpenGL coordinate system, i.e., +x is right, +y is up, +z is front (more details: https://kit.kiui.moe/camera/)# clock-wise rotation can be specified per 90 degree, e.g., +z1, -y2threefinerif2--meshinput.glb--prompt'description'--text_dir--front_dir='+z'# adjust training iterationsthreefinerif2--meshinput.glb--prompt'description'--iters1000# explicitly fix the geometry and only refine texturethreefinerif2--fix-geo--geom_modemesh--meshinput.glb--prompt'description'# equals if2_fixgeo# open a GUI to visualize the training progress (needs a desktop)threefinerif2--meshinput.glb--prompt'description'--guiGradio demo:# requires gradio 4pythongradio_app.pyif2For more examples, please seescripts.Q&AHow to make sure--front_dirfor your model?You may first visualize it in a 3D viewer that follows OpenGL coordinate system:The chair is facing down the Y axis (Green), so we can use `--front_dir="-y"` to rectify it to face +Z axis (Blue).fatal error: EGL/egl.h: No such file or directoryBy default, we use the OpenGL rasterizer. This error means there is no OpenGL installation, which is often the case for headless servers.
It's recommended to install OpenGL (along with NVIDIA driver) as it brings better performance.
Otherwise, you can append--force_cuda_rastto use the CUDA rasterizer instead.AcknowledgementThis work is built on many amazing research works and open-source projects, thanks a lot to all the authors for sharing!SDSguidanceclasses are based ondiffusers.diffmcgeometry is based ondiso.meshgeometry is based onnerf2mesh.Texture encoding is based ontinycudann.Mesh renderer is based onnvdiffrast.GUI is based ondearpygui.The coarse models used in demo are generated byGenieand3DTopia.