---
license: apache-2.0
language:
- en
pipeline_tag: text-generation
---

# Falcon-40b-oasst1-en
Falcon-40b-oasst1-en is finedtuned on [Falcon-40B](https://huggingface.co/tiiuae/falcon-40b) using [OpenAssistant/oasst1](https://huggingface.co/datasets/OpenAssistant/oasst1) dataset. Model was only trained on 'en' language dataset which took about 03:46:23 for one Epoch.

# Approach
Model was finetuned using [QLoRA](https://arxiv.org/pdf/2305.14314.pdf). Here is the [Github](https://github.com/artidoro/qlora).

# CUDA
- CUDA Version 12.0
- Hardware: 1x NVIDIA A100-SXM 
- Memory: {0: '37GB'}