---
library_name: peft
tags:
- Llama2
- code
- llama
- opensource
pipeline_tag: text-generation
inference: true
base_model: codellama/CodeLlama-34b-Instruct-hf
---

## Aria Code is based on Code LLAMA 34B Instruct finetuned on French

Aria code is a model built for educational purposes in French speaking countries on coding.

The model is built to help students to learn coding in their native language while having the base qualities of LLAMA CODE which has been
trained over 500 billions token of coding content. We belive coding skills are very valuable to reduce youth unemployment rates around the world 
and gives more scientific skills to students. LLAMA 2 base models have enough safeguards and censorship to ensure a safe use by kids and within academic environments.

GPU used for training: Nvidia A100

Timing: Less than 24 hours

Number of finetuning tokens : Over 10.000 high quality french language tokens.

## Training procedure


The following `bitsandbytes` quantization config was used during training:
- quant_method: bitsandbytes
- load_in_8bit: True
- load_in_4bit: False
- llm_int8_threshold: 6.0
- llm_int8_skip_modules: None
- llm_int8_enable_fp32_cpu_offload: False
- llm_int8_has_fp16_weight: False
- bnb_4bit_quant_type: fp4
- bnb_4bit_use_double_quant: False
- bnb_4bit_compute_dtype: float32
### Framework versions


- PEFT 0.6.0.dev0