I finetuned TinyLlama/TinyLlama-1.1B-Chat-v1.0 on the following datasets:

• dumb-dev/cpp-10k
• dumb-dev/Encoding-Detection-w-cChardet-DB
• Neloy262/rust_instruction_dataset
• m-a-p/CodeFeedback-Filtered-Instruction
• sahil2801/CodeAlpaca-20k
• vicgalle/alpaca-gpt4

Their LORAs can be found here

In the final model only the 1e-4 LORAs have been used! Everything was trained a total of 2 epochs.

probably the reason why it works this bad:

Following 3 are fp16, the other ones are fp32:

1. this
2. this
3. this

If someone knows how to improve, please let me know.

Instagram: dev2care