Repository: huggingface/diffusers

Issue:
EulerDiscreteScheduler not have get_velocity function and add_noise get error for prediction_type == "v_prediction"?
### Describe the bug

Finetune stable video diffusion but found two problems:

1. EulerDiscreteScheduler not have get_velocity function ?
2. EulerDiscreteScheduler  add_noise get error for prediction_type == "v_prediction"?

### Reproduction

1. EulerDiscreteScheduler not have get_velocity function ?
https://github.com/huggingface/diffusers/blob/main/examples/research_projects/controlnet/train_controlnet_webdataset.py#L1376

2. EulerDiscreteScheduler  add_noise get error for prediction_type == "v_prediction"?
https://github.com/huggingface/diffusers/blob/645a62bf3bcba47f97bf2a36a8689f40bd6b04fd/examples/research_projects/controlnet/train_controlnet_webdataset.py#L1341

### Logs

_No response_

### System Info

latest master branch

### Who can help?

@yiyixuxu @patrickvonplaten


Discussion:
No body ...
I meet same err. Have you solve this?
I got `RuntimeError: a Tensor with 0 elements cannot be converted to Scalar.` when calling `add_noise` in stable video diffusion. DDIM scheduler also met same problem. Have you met this too?
Hey everybody,

We would need a reproducible code snippet here - it's too time-consuming trying to guess what's going on. Could anybody try to provide a reproducible code snippet? 

Resolving patch:
diff --git a/src/diffusers/schedulers/scheduling_euler_discrete.py b/src/diffusers/schedulers/scheduling_euler_discrete.py
--- a/src/diffusers/schedulers/scheduling_euler_discrete.py
+++ b/src/diffusers/schedulers/scheduling_euler_discrete.py
@@ -576,5 +576,44 @@ def add_noise(
         noisy_samples = original_samples + noise * sigma
         return noisy_samples
 
+    def get_velocity(
+        self, sample: torch.FloatTensor, noise: torch.FloatTensor, timesteps: torch.FloatTensor
+    ) -> torch.FloatTensor:
+        if (
+            isinstance(timesteps, int)
+            or isinstance(timesteps, torch.IntTensor)
+            or isinstance(timesteps, torch.LongTensor)
+        ):
+            raise ValueError(
+                (
+                    "Passing integer indices (e.g. from `enumerate(timesteps)`) as timesteps to"
+                    " `EulerDiscreteScheduler.get_velocity()` is not supported. Make sure to pass"
+                    " one of the `scheduler.timesteps` as a timestep."
+                ),
+            )
+
+        if sample.device.type == "mps" and torch.is_floating_point(timesteps):
+            # mps does not support float64
+            schedule_timesteps = self.timesteps.to(sample.device, dtype=torch.float32)
+            timesteps = timesteps.to(sample.device, dtype=torch.float32)
+        else:
+            schedule_timesteps = self.timesteps.to(sample.device)
+            timesteps = timesteps.to(sample.device)
+
+        step_indices = [self.index_for_timestep(t, schedule_timesteps) for t in timesteps]
+        alphas_cumprod = self.alphas_cumprod.to(sample)
+        sqrt_alpha_prod = alphas_cumprod[step_indices] ** 0.5
+        sqrt_alpha_prod = sqrt_alpha_prod.flatten()
+        while len(sqrt_alpha_prod.shape) < len(sample.shape):
+            sqrt_alpha_prod = sqrt_alpha_prod.unsqueeze(-1)
+
+        sqrt_one_minus_alpha_prod = (1 - alphas_cumprod[step_indices]) ** 0.5
+        sqrt_one_minus_alpha_prod = sqrt_one_minus_alpha_prod.flatten()
+        while len(sqrt_one_minus_alpha_prod.shape) < len(sample.shape):
+            sqrt_one_minus_alpha_prod = sqrt_one_minus_alpha_prod.unsqueeze(-1)
+
+        velocity = sqrt_alpha_prod * noise - sqrt_one_minus_alpha_prod * sample
+        return velocity
+
     def __len__(self):
         return self.config.num_train_timesteps


Test patch: