Spaces:
Running
on
Zero
Running
on
Zero
File size: 1,539 Bytes
2d9a728 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
# Single-modality
## Installation
Please follow the installation instructions in [INSTALL](./INSTALL.md).
## Datasets
You can find the dataset instructions in [DATASET](./DATASET.md).
## Model ZOO
You can find all the models and the scripts in [MODEL_ZOO](./MODEL_ZOO.md).
## Pre-Training
We use [InternVL](https://github.com/OpenGVLab/InternVL/) and [VideoMAEv2](https://github.com/OpenGVLab/VideoMAEv2) pretrained models as teachers by default
For training, you can simply run the pretraining scripts in `scripts/pretraining` as follows:
```shell
bash ./scripts/pretraining/1B_pt.sh
```
:warning: **Notes:**
1. Chage `DATA_PATH` to your data path before running the scripts.
2. `--sampling_rate` is set to 1 for **sprase sampling**.
3. The latest checkpoint will be automatically saved while training, thus we use a large `--save_ckpt_freq`.
4. For InternVideo2-1B and 6B, we use InternVL-C-13B and VideoMAEv2-g.
## Finetuning
For finetuning, you can simply run the pretraining scripts in `scripts/finetuning` as follows:
```shell
bash ./scripts/finetuning/full_tuning/k400/1B_ft_k710_ft_k400_f8.sh
```
:warning: **Notes:**
1. Chage `DATA_PATH` And `PREFIX` to your data path before running the scripts.
2. Chage `MODEL_PATH` to your model path.
3. Set `--use_checkpoint` and `--checkpoint_num` to save GPU memory.
4. The best checkpoint will be automatically evaluated with `--test_best`.
5. Set `--test_num_segment` and `--test_num_crop` for different evaluation strategies.
6. To only run evaluation, just set `--eval`.
|