gabrielmbmb
/

Upcycled-Qwen1.5-MoE2.7B-LoRA-merged-32

Text Generation

Inference Endpoints

Model card Files Files and versions Community

No model card

New: Create and edit this model card directly on the website!

Contribute a Model Card

Downloads last month: 3

Safetensors

Model size

14B params

Tensor type

BF16

·

Collection including gabrielmbmb/Upcycled-Qwen1.5-MoE2.7B-LoRA-merged-32

Upcycling Experiments

Models I pre-trained initialising SMoE models using dense model weights and the upcycling process used for Qwen1.5-MoE2.7BA (or something similar) • 6 items • Updated Apr 1