Ligeng-Zhu/panda70m-download

How to use?

huggingface-cli download Ligeng-Zhu/panda70m \
  --local-dir panda70m --repo-type dataset --local-dir-use-symlinks False

Then install dependencies

pip install fire yt_dlp pandas

Next pull the videos

python main.py --csv=<your csv files>

or split by shards to accelerate downloading

python main.py --csv=<your csv files> --shards=0 --total=10
python main.py --csv=<your csv files> --shards=1 --total=10
...
python main.py --csv=<your csv files> --shards=9 --total=10