zgzaacm commited on
Commit
d680f74
1 Parent(s): 6a75cc0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +50 -3
README.md CHANGED
@@ -1,3 +1,50 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ ---
4
+ ---
5
+ license: apache-2.0
6
+ ---
7
+ # VFIMamba: Video Frame Interpolation with State Space Models
8
+ This is the official checkpoint library for [VFIMamba: Video Frame Interpolation with State Space Models](https://arxiv.org/abs/2407.02315).
9
+ Please refer to [this repository](https://github.com/MCG-NJU/VFIMamba) for our code.
10
+
11
+ ## Model Description
12
+ VFIMamba is the first approach to adapt the SSM model to the video frame interpolation task.
13
+ 1. We devise the Mixed-SSM Block (MSB) for efficient inter-frame modeling using S6.
14
+ 2. We explore various rearrangement methods to convert two frames into a sequence, discovering that interleaved rearrangement is more suitable for VFI tasks.
15
+ 3. We propose a curriculum learning strategy to further leverage the potential of the S6 model.
16
+
17
+ Experimental results demonstrate that VFIMamba achieves the state-of-the-art performance across various datasets, in particular highlighting the potential of the SSM model for VFI tasks with high resolution.
18
+
19
+ ## Usage
20
+ We provide two models, an efficient version (VFIMamba-S) and a stronger one (VFIMamba). You can choose what you need by specifying the parameter model.
21
+
22
+ ### Manually Load
23
+ Please refer to [the instruction here](https://github.com/MCG-NJU/VFIMamba/tree/main?tab=readme-ov-file#sunglassesplay-with-demos) for manually loading the checkpoints and a more customized experience.
24
+ ```bash
25
+ python demo_2x.py --model **model[VFIMamba_S/VFIMamba]** # for 2x interpolation
26
+ python demo_Nx.py --n 8 --model **model[VFIMamba_S/VFIMamba]** # for 8x interpolation
27
+ ```
28
+
29
+
30
+ ### Hugging Face Demo
31
+ For Hugging Face demo, please refer to [the code here](https://github.com/MCG-NJU/VFIMamba/blob/main/hf_demo_2x.py).
32
+ ```bash
33
+ python hf_demo_2x.py --model **model[VFIMamba_S/VFIMamba]** # for 2x interpolation
34
+ ```
35
+
36
+
37
+ ## Citation
38
+ If you think this project is helpful in your research or for application, please feel free to leave a star⭐️ and cite our paper:
39
+ ```
40
+ @misc{zhang2024vfimambavideoframeinterpolation,
41
+ title={VFIMamba: Video Frame Interpolation with State Space Models},
42
+ author={Guozhen Zhang and Chunxu Liu and Yutao Cui and Xiaotong Zhao and Kai Ma and Limin Wang},
43
+ year={2024},
44
+ eprint={2407.02315},
45
+ archivePrefix={arXiv},
46
+ primaryClass={cs.CV},
47
+ url={https://arxiv.org/abs/2407.02315},
48
+ }
49
+ ```
50
+