DiffusionPipeline.from_pretrained raises ImportError
First of all, this diffuser looks on paper, amazing. Do you have an article/academic paper on this model benchmarks?
Second, loading your pipeline in this way:
pipe = DiffusionPipeline.from_pretrained("gvecchio/MatForger",trust_remote_code=True,)
Raises ImportError on "FromSingleFileMixin" from pipeline.py line 10.
I guess it's a problem in HF library and not in your model.
After looking up, I tried with StableDiffusionPipeline, seems to work.
But raises these warning for all layers:
While I have successfully used the model to generate PBR with the example prompt "terracotta brick wall".
This is the basecolor.
Thanks in advance,
Gil.
P.S. Is there a combined form of your texture maps, like in https://ambientcg.com/ website?
Hi, thanks for pointing it out. Something must have changed with the latest release of diffusers. I will look into it.
Second, there's no paper for this model in particular. The architecture is based on the MatFuse paper. The main difference from that architecture is the use of a VAE instead of a vector quantized VAE.
About the training dataset (if I got your question correctly) it is trained on the MatSynth dataset. The version of MatFuse trained on MatSynth presented in that paper is roughly the same as this one.
Thanks!