Text Generation
Transformers
PyTorch
English
llama
Inference Endpoints
text-generation-inference

Merging technique

#8
by khoantap - opened

Hi, Thank you for sharing such a great model.
Could you elaborate on how the 2 models were merge? Was it merged by combining the lora weight to the base model? Was any further finetuning conduct?

Thank you

Reading the paper it looks like yeah they just apply the lora adapter to merge.

Sign up or log in to comment