Repository: Mikubill/sd-webui-controlnet (language: python)

Issue:
[Bug]: Validation Error in Latest Update Using Deforum
### Is there an existing issue for this?

- [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

### What happened?

After installing the latest update when using Controlnet in Deforum I get the following error (user firendly version)

Error: 1 validation error for ControlNetUnit
resize_mode
  value is not a valid enumeration member; permitted: 'Just Resize', 'Crop and Resize', 'Resize and Fill' (type=type_error.enum; enum_values=[<ResizeMode.RESIZE: 'Just Resize'>, <ResizeMode.INNER_FIT: 'Crop and Resize'>, <ResizeMode.OUTER_FIT: 'Resize and Fill'>]). Please, check your schedules/ init values.

### Steps to reproduce the problem

1. Add controlnets to a deforum project and generate


### What should have happened?

No errors

### Commit where the problem happens

webui: latest as of 7th May 2024
controlnet:  latest as of 7th May 2024


### What browsers do you use to access the UI ?

Google Chrome

### Command Line Arguments

```Shell
@echo off

set PYTHON=
set GIT=
set VENV_DIR=
set COMMANDLINE_ARGS= --api --ckpt-dir 'D:\AI_Work\Models_Archive' --opt-sdp-attention --xformers 

call webui.bat --medvram --opt-split-attention
```


### List of enabled extensions

Deforum
Controlnet
Default extensions installed with A1111

### Console logs

```Shell
To create a public link, set `share=True` in `launch()`.
Startup time: 75.9s (prepare environment: 60.8s, import torch: 6.0s, import gradio: 1.1s, setup paths: 2.2s, initialize shared: 0.3s, other imports: 0.7s, list SD models: 0.1s, load scripts: 2.9s, create ui: 0.9s, gradio launch: 0.4s, add APIs: 0.4s).
Applying attention optimization: xformers... done.
Model loaded in 6.7s (load weights from disk: 1.2s, create model: 1.5s, apply weights to model: 2.3s, apply half(): 0.7s, calculate empty prompt: 0.9s).
Reusing loaded model absolutereality_v10.safetensors [9483a64615] to load edgeOfRealism_eorV20Fp16BakedVAE.safetensors
Calculating sha256 for D:\AI_Work\Models_Archive\edgeOfRealism_eorV20Fp16BakedVAE.safetensors: reading custom settings from D:\AI_Work\Projects\Video_Work\Low Point High\Silver Spirits 2.txt
7f6146b8a9ad2a92b739954c1d5ef34e26fb0b65eef2d70ffd128e6046a3d0ab
Loading weights [7f6146b8a9] from D:\AI_Work\Models_Archive\edgeOfRealism_eorV20Fp16BakedVAE.safetensors
Applying attention optimization: xformers... done.
Weights loaded in 13.7s (send model to cpu: 0.1s, calculate hash: 13.0s, apply weights to model: 0.5s).
started run_deforum
times_to_run: 1
Deforum extension for auto1111 webui
Git commit: 32242685
Starting job task(0oiwgx1tx4arjt7)-0...
Video to extract: D:\AI_Work\Video_Source\Silver Water Bugs.mp4
Extracting video (1 every 1) frames to D:\AI_Work\A1111\outputs\img2img-images\Low_Point_High_Silver_Spirits_Two\inputframes...
Trying to extract frames from video with input FPS of 24.0. Please wait patiently.
Extracted 139 frames from video in 0.61 seconds!
Using 139 input frames from D:\AI_Work\A1111\outputs\img2img-images\Low_Point_High_Silver_Spirits_Two\inputframes...
Using init_image from video: D:\AI_Work\A1111\outputs\img2img-images\Low_Point_High_Silver_Spirits_Two\inputframes\Silver Water Bugs000000000.jpg
Unpacking ControlNet 2 base video
Exporting Video Frames to D:\AI_Work\A1111\outputs\img2img-images\Low_Point_High_Silver_Spirits_Two\controlnet_2_inputframes...
Trying to extract frames from video with input FPS of 24.0. Please wait patiently.
Extracted 139 frames from video in 0.48 seconds!
Loading 139 input frames from D:\AI_Work\A1111\outputs\img2img-images\Low_Point_High_Silver_Spirits_Two\controlnet_2_inputframes and saving video frames to D:\AI_Work\A1111\outputs\img2img-images\Low_Point_High_Silver_Spirits_Two
ControlNet 2 base video unpacked!
Unpacking ControlNet 3 base video
Exporting Video Frames to D:\AI_Work\A1111\outputs\img2img-images\Low_Point_High_Silver_Spirits_Two\controlnet_3_inputframes...
Trying to extract frames from video with input FPS of 24.0. Please wait patiently.
Extracted 139 frames from video in 0.58 seconds!
Loading 139 input frames from D:\AI_Work\A1111\outputs\img2img-images\Low_Point_High_Silver_Spirits_Two\controlnet_3_inputframes and saving video frames to D:\AI_Work\A1111\outputs\img2img-images\Low_Point_High_Silver_Spirits_Two
ControlNet 3 base video unpacked!
Unpacking ControlNet 5 base video
Exporting Video Frames to D:\AI_Work\A1111\outputs\img2img-images\Low_Point_High_Silver_Spirits_Two\controlnet_5_inputframes...
Trying to extract frames from video with input FPS of 24.0. Please wait patiently.
Extracted 139 frames from video in 0.43 seconds!
Loading 139 input frames from D:\AI_Work\A1111\outputs\img2img-images\Low_Point_High_Silver_Spirits_Two\controlnet_5_inputframes and saving video frames to D:\AI_Work\A1111\outputs\img2img-images\Low_Point_High_Silver_Spirits_Two
ControlNet 5 base video unpacked!
Saving animation frames to:
D:\AI_Work\A1111\outputs\img2img-images\Low_Point_High_Silver_Spirits_Two
Animation frame: 0/139
Seed: 2012698371
Prompt: photographic, tiny mannequins floating in the sky, cinematic, highly detailed, bokeh, xt-3
Neg Prompt: illustration, drawn, digital art
╭─────┬────┬───────┬─────┬────┬──────┬──────┬────┬────╮
│Steps│CFG │Denoise│Angle│Zoom│Tr C X│Tr C Y│Tr X│Tr Y│
├─────┼────┼───────┼─────┼────┼──────┼──────┼────┼────┤
│ 20  │10.0│  0.3  │  0  │ 0  │ 0.5  │ 0.5  │ 0  │ 0  │
╰─────┴────┴───────┴─────┴────┴──────┴──────┴────┴────╯
Reading ControlNet 2 base frame #0 at D:\AI_Work\A1111\outputs\img2img-images\Low_Point_High_Silver_Spirits_Two\controlnet_2_inputframes\000000000.jpg
Reading ControlNet 3 base frame #0 at D:\AI_Work\A1111\outputs\img2img-images\Low_Point_High_Silver_Spirits_Two\controlnet_3_inputframes\000000000.jpg
Reading ControlNet 5 base frame #0 at D:\AI_Work\A1111\outputs\img2img-images\Low_Point_High_Silver_Spirits_Two\controlnet_5_inputframes\000000000.jpg

*START OF TRACEBACK*
Traceback (most recent call last):
  File "D:\AI_Work\A1111\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\run_deforum.py", line 116, in run_deforum
    render_animation(args, anim_args, video_args, parseq_args, loop_args, controlnet_args, root)
  File "D:\AI_Work\A1111\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\render.py", line 577, in render_animation
    image = generate(args, keys, anim_args, loop_args, controlnet_args, root, parseq_adapter, frame_idx, sampler_name=scheduled_sampler_name)
  File "D:\AI_Work\A1111\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\generate.py", line 80, in generate
    image, caught_vae_exception = generate_with_nans_check(args, keys, anim_args, loop_args, controlnet_args, root, parseq_adapter, frame, sampler_name)
  File "D:\AI_Work\A1111\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\generate.py", line 114, in generate_with_nans_check
    raise e
  File "D:\AI_Work\A1111\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\generate.py", line 108, in generate_with_nans_check
    image = generate_inner(args, keys, anim_args, loop_args, controlnet_args, root, parseq_adapter, frame, sampler_name)
  File "D:\AI_Work\A1111\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\generate.py", line 280, in generate_inner
    process_with_controlnet(p, args, anim_args, controlnet_args, root, parseq_adapter, is_img2img=True, frame_idx=frame)
  File "D:\AI_Work\A1111\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\deforum_controlnet.py", line 315, in process_with_controlnet
    cn_units = [cnet.ControlNetUnit(**create_cnu_dict(controlnet_args, f"cn_{i + 1}", img_np, mask_np, frame_idx, CnSchKeys))
  File "D:\AI_Work\A1111\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\deforum_controlnet.py", line 315, in <listcomp>
    cn_units = [cnet.ControlNetUnit(**create_cnu_dict(controlnet_args, f"cn_{i + 1}", img_np, mask_np, frame_idx, CnSchKeys))
  File "pydantic\main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for ControlNetUnit
resize_mode
  value is not a valid enumeration member; permitted: 'Just Resize', 'Crop and Resize', 'Resize and Fill' (type=type_error.enum; enum_values=[<ResizeMode.RESIZE: 'Just Resize'>, <ResizeMode.INNER_FIT: 'Crop and Resize'>, <ResizeMode.OUTER_FIT: 'Resize and Fill'>])
*END OF TRACEBACK*

User friendly error message:
Error: 1 validation error for ControlNetUnit
resize_mode
  value is not a valid enumeration member; permitted: 'Just Resize', 'Crop and Resize', 'Resize and Fill' (type=type_error.enum; enum_values=[<ResizeMode.RESIZE: 'Just Resize'>, <ResizeMode.INNER_FIT: 'Crop and Resize'>, <ResizeMode.OUTER_FIT: 'Resize and Fill'>]). Please, check your schedules/ init values.
```


### Additional information

_No response_


Discussion / hints:
im having the exactly same issue with deforum any help would be appreciated 

Fix patch:
diff --git a/internal_controlnet/args.py b/internal_controlnet/args.py
index a32ec5218..f36448ef1 100644
--- a/internal_controlnet/args.py
+++ b/internal_controlnet/args.py
@@ -95,6 +95,20 @@ def check_model(cls, value: str) -> str:
     image: Optional[Any] = None
 
     resize_mode: ResizeMode = ResizeMode.INNER_FIT
+
+    @validator("resize_mode", always=True, pre=True)
+    def check_resize_mode(cls, value) -> ResizeMode:
+        resize_mode_aliases = {
+            "Inner Fit (Scale to Fit)": "Crop and Resize",
+            "Outer Fit (Shrink to Fit)": "Resize and Fill",
+            "Scale to Fit (Inner Fit)": "Crop and Resize",
+            "Envelope (Outer Fit)": "Resize and Fill",
+        }
+        if isinstance(value, str):
+            return ResizeMode(resize_mode_aliases.get(value, value))
+        assert isinstance(value, ResizeMode)
+        return value
+
     low_vram: bool = False
     processor_res: int = -1
     threshold_a: float = -1
@@ -378,7 +392,9 @@ def get_input_images_rgba(self) -> Optional[List[np.ndarray]]:
 
             np_image = self.parse_image(image)
             np_mask = self.parse_image(mask) if mask is not None else None
-            np_images.append(self.combine_image_and_mask(np_image, np_mask))  # [H, W, 4]
+            np_images.append(
+                self.combine_image_and_mask(np_image, np_mask)
+            )  # [H, W, 4]
 
         return np_images
 


Test patch:
diff --git a/unit_tests/args_test.py b/unit_tests/args_test.py
index af2529788..0c053f73f 100644
--- a/unit_tests/args_test.py
+++ b/unit_tests/args_test.py
@@ -151,6 +151,9 @@ def test_mask_alias_conflict():
 
 def test_resize_mode():
     ControlNetUnit(resize_mode="Just Resize")
+    # Alias should also work. For deforum
+    # See https://github.com/deforum-art/sd-webui-deforum/blob/322426851408ebca2cd49492bfeb1ec86e1dc869/scripts/deforum_helpers/deforum_controlnet.py#L150
+    ControlNetUnit(resize_mode="Inner Fit (Scale to Fit)")
 
 
 def test_weight():