Edit model card

This model is based on the fusion strategy offered by Fanqi Wan(https://github.com/fanqiwan/FuseLLM).

Three models are fused together.

Base model: TinyLlama/TinyLlama-1.1B-Chat-v1.0

Blending model 1: HanNayeoniee/LHK_DPO_v1

Blending model 2: yunconglong/Truthful_DPO_TomGrc_FusionNet_7Bx2_MoE_13B

This model will be optimized by Laser and DPO later.

This project is to make the on-device sLM. We are doing experiments on the models.

Downloads last month
71
Safetensors
Model size
1.1B params
Tensor type
BF16
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.