File size: 7,661 Bytes
72f4497 57ab61c 7912fa9 72f4497 fd88134 52e9ccd 32b2f3e 52e9ccd 4f5d0f9 9ad8cec d6d6a6f 9ad8cec 32b2f3e 1c3d9fb 254197d 27fdf10 32b2f3e 52e9ccd ec50837 7c0da50 52e9ccd 2d6478e 52e9ccd 4f5d0f9 52e9ccd 2d6478e 52e9ccd 4f5d0f9 9ad8cec e8ada34 20f32fb 8e53e19 9ad8cec 4f5d0f9 9ad8cec 7caa7c0 88478cd d6d6a6f 9ad8cec 88478cd 32b2f3e e85f1e2 1c3d9fb e85f1e2 254197d 5cc5c6a 60b4908 d6d6a6f 254197d 32b2f3e 27fdf10 32b2f3e 88478cd 32b2f3e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 |
---
tags:
- stable diffusion
- animatediff
- motion lora
language:
- en
---
- [Animatediff V2 motion LORAs](#animatediff-v2-motion-loras)
- [File summary](#file-summary)
- [General recommendation](#general-recommendation)
- [Examples](#examples)
- [v2 Drone orbit right](#v2-drone-orbit-right)
- [v2 Drone orbit left](#v2-drone-orbit-left)
- [v2 Drone backing](#v2-drone-backing)
- [v2 Flying backwards](#v2-flying-backwards)
- [v2 Drone forward](#v2-drone-forward)
- [v2 Drone walking back-view](#v2-drone-walking-back-view)
- [v2 walking front-view](#v2-walking-front-view)
- [v2 casual walking front-view](#v2-casual-walking-front-view)
- [v2 running front-view](#v2-running-front-view)
- [Animatediff V3 motion LORAs](#animatediff-v3-motion-loras)
- [File summary](#file-summary-1)
- [General recommendations](#general-recommendations)
- [Drone LORAs](#drone-loras)
- [Ballerina spin LORAs](#ballerina-spin-loras)
- [Walking back-view LORAs](#walking-back-view-loras)
# Animatediff V2 motion LORAs
## File summary
| Name | Compatible Motion Model | Resolution (w/h) | Frames | input videos |
| --------------------------------------------------------------------------------- | ----------------------- | ---------------- | ------ | ------------ |
| 280_cseti_2906282_drone-orbitright-mv2_r64_w576_h384_fr16.safetensors | v2, animateLCM | 576x384 (3:2) | 16 | 1 |
| 280_cseti_6933991_drone-orbitleft-mv2_r64_w576_h384_fr16.safetensors | v2, animateLCM | 576x384 (3:2) | 16 | 1 |
| 280_cseti_6160305_drone-backing-mv2_r64_w576_h384_fr16.safetensors | v2, animateLCM | 576x384 (3:2) | 16 | 1 |
| 800_cseti_3730960_flying_backwards-3vid_mv2_0.00031_56_w768_h512_fr16.safetensors | v2, animateLCM | 576x384 (3:2) | 16 | 3 |
| 260_cseti_8890531_drone-forward-mv2_r64_w576_h384_fr16.safetensors | v2, animateLCM | 576x384 (3:2) | 16 | 1 |
| 200_cseti_3413444_drone-walking-mv2_r64_w576_h384_fr16.safetensors | v2, animateLCM | 576x384 (3:2) | 16 | 1 |
| 500_cseti_8315653_drone-orbitright-3vid_mv2_r64_w576_h384_fr16.safetensors | v2, animateLCM | 576x384 (3:2) | 16 | 3 |
| 600_cseti_4941259_walking-frontview-6vid_mv2_r64_w576_h384_fr16.safetensors | v2, animateLCM | 576x384 (3:2) | 16 | 6 |
| 500_cseti_3734954_casualmove-frontview-3vid_mv2_r64_w576_h384_fr16.safetensors | v2, animateLCM | 576x384 (3:2) | 16 | 3 |
| 1800_cseti_1118902_running-frontview-8vid_mv2_r64_w768_h512_fr16.safetensors | v2, animateLCM | 576x384 (3:2) | 16 | 8 |
## General recommendation
- These are finetuned on a v2 model. They work also with AnimateLCM but don't work with v3 models
- For the drone LORAs, keyword "drone" or "drone shot" might help with the motion
- Animatediff v3 adapter lora is recommended regardless they are v2 models
- If you want more motion try incrasing the scale multival (e.g. to 1.2-1.5)
- I recommend using the above resolutions and upscale the animation or keep at least the aspect ratios
- The drone LORAs are trained on landscape footage so might don't work very well on portraits however can give you interesting results
## Examples
### v2 Drone orbit right
<video controls src="https://huggingface.co/Cseti/AD_Motion_LORAs/resolve/main/nolora-candidate_00001.mp4" title="Orbit right"></video>
### v2 Drone orbit left
<video controls src="https://huggingface.co/Cseti/AD_Motion_LORAs/resolve/main/orbitleft-candidate_00004.mp4" title="Orbit left"></video>
### v2 Drone backing
<video controls src="https://huggingface.co/Cseti/AD_Motion_LORAs/resolve/main/dronebacking-candidate_00019.mp4" title="Drone backing"></video>
### v2 Flying backwards
- The difference compared to the Drone LoRA is that this LoRA was trained on a dataset containing also flying footage.
<video controls src="https://huggingface.co/Cseti/AD_Motion_LORAs/resolve/main/800_cseti_3730960_flying_backwards-3vid_mv2_0.00031_56_w768_h512_fr16.mp4" title="Drone backing"></video>
### v2 Drone forward
<video controls src="https://huggingface.co/Cseti/AD_Motion_LORAs/resolve/main/droneforward-comparison_00002.mp4" title="Drone backing"></video>
### v2 Drone walking back-view
- recommended to use words like "back-view", "from behind" in your prompts
<video controls src="https://huggingface.co/Cseti/AD_Motion_LORAs/resolve/main/candidate-comparison_00003.mp4" title="Drone walking back-view"></video>
### v2 walking front-view
- recommended to use words like "front-view"
<video width="576" height="384" controls src="https://huggingface.co/Cseti/AD_Motion_LORAs/resolve/main/walking-frontview-candidate.mp4" title="walking front-view"></video>
### v2 casual walking front-view
- recommended to use words like "front-view"
- play with the strength: 0.6-0.8 can already give good results
<video width="576" height="384" controls src="https://huggingface.co/Cseti/AD_Motion_LORAs/resolve/main/AD-IPA-testing-upscale_00002.mp4" title="casual walking front-view"></video>
### v2 running front-view
- recommended to use words like "front-view" and "running"
<video width="576" height="384" controls src="https://huggingface.co/Cseti/AD_Motion_LORAs/resolve/main/running-8vid-frontview-candidate.mp4" title="running front view"></video>
# Animatediff V3 motion LORAs
## File summary
| Name | Compatible Motion Model | Resolution (w/h) | Frames | Comments |
| ---------------------------------------------------- | ----------------------- | ---------------- | ------ | ----------------- |
| 250_cseti_Drone-Rotating_3D_footage_fr16 | v3 | 384x384 (1:1) | 16 | Drone 3d rotation |
| 400_cseti_9222750_drone-rot_w576_h384_fr32_r64 | v3 | 576x384 (3:2) | 32 | Drone 3d rotation |
| 200_3723723_cseti_ballerina-spin_w512_h768_r64 | v3 | 512x768 (2:3) | 16 | Ballerina spin |
| 250_3723723_cseti_ballerina-spin_w512_h768_r64 | v3 | 512x768 (2:3) | 16 | Ballerina spin |
| 400_cseti_5294370_walk_backview-1_w576_h384_fr32_r56 | v3 | 576x384 (3:2) | 32 | Walking back view |
| 450_cseti_5294370_walk_backview-1_w576_h384_fr32_r56 | v3 | 576x384 (3:2) | 32 | Walking back view |
## <b>General recommendations</b>
- These are finetuned on v3 model. Does not work with v2 or animateLCM motion models
- Animatediff v3 adapter lora is recommended
- If you want more motion try incrasing the scale multival (e.g. to 1.2)
- I recommend using the above resolutions and upscale the animation or keep at least the aspect ratios
## <b>Drone LORAs</b>
- For more motion, try incrasing the LORA strength or try adding the below trigger words to your prompt
- Try trigger words: dronearound, flying around
## <b>Ballerina spin LORAs</b>
- Sometimes a white veil / skirt appears during rotation. Unfortunately, this is from the original video. In such cases, try a new generation or reduce the LORA strength.
- I recommend using LORA strength of 0.8-0.9
## <b>Walking back-view LORAs</b>
- I recommend using LORA strength of 0.8-0.9
- Recommended prompts:
- "back view"
- "drone shot" - for an upper view
|