Edit model card

Dromedary Model Card

NOTE: This "delta model" cannot be used directly.
Users have to apply it on top of the original LLaMA weights to get actual Dromedary weights.
See https://github.com/IBM/Dromedary#model-weights for instructions.

Model details

Dromedary Logo

Model type: Dromedary is an open-source self-aligned language model trained with minimal human supervision. The base language model is LLaMA-65b, based on the transformer architecture.

Model date: Dromedary was trained between April 2023 and May 2023, but its knowledge only goes up until Sept-2021.

Organizations developing the model: The Dromedary team as a joint effort between CMU and IBM.

Paper or resources for more information: https://mitibmdemos.draco.res.ibm.com/dromedary

License: LLaMA's Non-commercial bespoke license

Where to send questions or comments about the model: https://github.com/IBM/Dromedary/issues

Intended use

Primary intended uses: The primary use of Dromedary is research on the alignment of large language models.

Primary intended users: The primary intended users of the model are researchers in artificial intelligence.

Delta weights

We use the following configuration for the LoRA weights:

--lora_target_modules='[q_proj,k_proj,v_proj,o_proj]' \
--lora_r=16 \

Training dataset

Fewer than 300 lines of human annotations (including < 200 seed prompts, 16 generic principles, and 5 exemplars for in-context learning),

Evaluation dataset

We evaluate Dromedary on TruthfulQA and HHH Eval, as well as Vicuna benchmark questions.

Downloads last month
Inference API (serverless) has been turned off for this model.