xi0v/Stable-Video-Diffusion-Img2Vid · Suggest the DragNUWA model to the user

Fabrice-TIERCELIN

Jul 8

I have found this model in Civitai. Let's see if it's good 🙂 And we will rollback if it fails.

I have also set the FPS to 25 by default as it is the most realistic option.

Suggest the DragNUWA model to the user6c473b45

xi0v changed pull request status to merged Jul 8

xi0v

Owner Jul 8

•

edited Jul 8

Here is the error:

Runtime error
_blocks.3.resnets.0.spatial_res_block.flow_cond_norm.weight, up_blocks.3.resnets.0.spatial_res_block.flow_gamma_spatial.bias, up_blocks.3.resnets.0.spatial_res_block.flow_gamma_spatial.weight, up_blocks.3.resnets.0.spatial_res_block.flow_gamma_temporal.bias, up_blocks.3.resnets.0.spatial_res_block.flow_gamma_temporal.weight']
The config attributes {'use_legacy': False} were passed to AutoencoderKLTemporalDecoder, but are not expected and will be ignored. Please verify your config.json configuration file.

Loading pipeline components...:  80%|████████  | 4/5 [00:00<00:00,  5.09it/s]
Traceback (most recent call last):
  File "/home/user/app/app.py", line 26, in <module>
    dragnuwaPipe = StableVideoDiffusionPipeline.from_pretrained(
  File "/usr/local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/diffusers/pipelines/pipeline_utils.py", line 881, in from_pretrained
    loaded_sub_model = load_sub_model(
  File "/usr/local/lib/python3.10/site-packages/diffusers/pipelines/pipeline_loading_utils.py", line 703, in load_sub_model
    loaded_sub_model = load_method(os.path.join(cached_folder, name), **loading_kwargs)
  File "/usr/local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/diffusers/models/modeling_utils.py", line 750, in from_pretrained
    raise ValueError(
ValueError: Cannot load <class 'diffusers.models.autoencoders.autoencoder_kl_temporal_decoder.AutoencoderKLTemporalDecoder'> from /home/user/.cache/huggingface/hub/models--a-r-r-o-w--dragnuwa-svd/snapshots/bd4a863ea6a4c299733e9c0ccbf958fb405155ae/vae because the following keys are missing: 
 quant_conv.weight, quant_conv.bias. 
 Please make sure to pass `low_cpu_mem_usage=False` and `device_map=None` if you want to randomly initialize those weights or else make sure your checkpoint file is correct.

xi0v

Owner Jul 8

I'll rollback temporarily