Pre-training code

#47

by bilibraker - opened May 13

Discussion

bilibraker

May 13

Thank you for the awesome model!
Is there a plan to release a pre-training script similar to LLaVA?

HugoLaurencon

HuggingFaceM4 org May 13

Thanks!
We will not open the codebase as it is now complex and consistently changing, but it follows closely the implementation in Transformers, and we use DeepSpeed Zero 3 for the model parallelization.

HugoLaurencon changed discussion status to closed May 13

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment