---
language:
 - "en"
tags:
 - video
 - onlyflow
license: apache-2.0
datasets:
- TempoFunk/webvid-10M
base_model:
  - stable-diffusion-v1-5/stable-diffusion-v1-5
pipeline_tag: text-to-video
library_name: diffusers
---
# OnlyFlow - Obvious Research

![](assets/obvious_research.png)

## Model for [OnlyFlow: Optical Flow based Motion Conditioning for Video Diffusion Models](https://arxiv.org/pdf/2411.10501).

![](assets/onlyflow.png)


## Project [website](https://obvious-research.github.io/onlyflow/)

## Code, available on [Github](https://huggingface.co/obvious-research/OnlyFlow)


## Model weight release, on [HuggingFace](https://huggingface.co/obvious-research/OnlyFlow)

We release the model weights of our best training in fp32, fp16 ckpt and safetensors format.
The model is trained on the Webvid-10M dataset on a 8 A100 node for 20h.

Here are examples of videos created by the model, with from left to right:
- input
- optical flow extracted from input
- output with onylflow influence (gamma parameter set to 0.)
- output with gamma = 0.5
- output with gamma = 0.75
- output with gamma = 1.

![](assets/onlyflow_example_1.gif)
![](assets/onlyflow_example_2.gif)


## Usage for inference

You can use the gradio interface on our [HuggingFace Space](https://huggingface.co/spaces/obvious-research/OnlyFlow).
You can also use the inference script in our [Github repository](https://github.com/obvious-research/onlyflow) to test the released model.

Please note that the model is really sensitive to the input video framerate. You should try downsampling the video to 8fps before using it if you don't get the desired results.

## Next steps

We are working on improving OnlyFlow and train it for other base model than AnimateDiff.

We appreciate any help, feel free to reach out! You can contact us:

- On Twitter: [@obv_research](https://x.com/obv_research)
- By mail: research.obvious@gmail.com

## About Obvious Research

Obvious Research is an Artificial Intelligence research laboratory dedicated to creating new AI artistic tools, initiated by the artists’ trio [Obvious](https://obvious-art.com/), in partnership with La Sorbonne Université.