hopefully improve the README (#419)
Browse files* hopefully improve the README
* exitcode -9 help
* table of contents
* formatting
README.md
CHANGED
@@ -1,10 +1,39 @@
|
|
1 |
# Axolotl
|
2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
3 |
<div align="center">
|
4 |
<img src="image/axolotl.png" alt="axolotl" width="160">
|
5 |
<div>
|
6 |
<p>
|
7 |
-
<b>
|
8 |
</p>
|
9 |
<p>
|
10 |
Go ahead and axolotl questions!!
|
@@ -14,6 +43,10 @@
|
|
14 |
</div>
|
15 |
</div>
|
16 |
|
|
|
|
|
|
|
|
|
17 |
## Axolotl supports
|
18 |
|
19 |
| | fp16/fp32 | lora | qlora | gptq | gptq w/ lora | gptq w/flash attn | flash attn | xformers attn |
|
@@ -29,6 +62,8 @@
|
|
29 |
|
30 |
## Quickstart ⚡
|
31 |
|
|
|
|
|
32 |
**Requirements**: Python >=3.9 and Pytorch >=2.0.
|
33 |
|
34 |
```bash
|
@@ -130,6 +165,7 @@ accelerate launch scripts/finetune.py examples/openllama-3b/lora.yml \
|
|
130 |
|
131 |
### Dataset
|
132 |
|
|
|
133 |
Have dataset(s) in one of the following format (JSONL recommended):
|
134 |
|
135 |
- `alpaca`: instruction; input(optional)
|
@@ -622,7 +658,7 @@ CUDA_VISIBLE_DEVICES="" python3 scripts/finetune.py ...
|
|
622 |
|
623 |
## Common Errors 🧰
|
624 |
|
625 |
-
> Cuda out of memory
|
626 |
|
627 |
Please reduce any below
|
628 |
- `micro_batch_size`
|
@@ -630,6 +666,10 @@ Please reduce any below
|
|
630 |
- `gradient_accumulation_steps`
|
631 |
- `sequence_len`
|
632 |
|
|
|
|
|
|
|
|
|
633 |
> RuntimeError: expected scalar type Float but found Half
|
634 |
|
635 |
Try set `fp16: true`
|
@@ -658,6 +698,8 @@ Building something cool with Axolotl? Consider adding a badge to your model card
|
|
658 |
|
659 |
## Community Showcase
|
660 |
|
|
|
|
|
661 |
Open Access AI Collective
|
662 |
- [Minotaur 13b](https://huggingface.co/openaccess-ai-collective/minotaur-13b)
|
663 |
- [Manticore 13b](https://huggingface.co/openaccess-ai-collective/manticore-13b)
|
|
|
1 |
# Axolotl
|
2 |
|
3 |
+
Axolotl is a tool designed to streamline the fine-tuning of various AI models, offering support for multiple configurations and architectures.
|
4 |
+
|
5 |
+
<table>
|
6 |
+
<tr>
|
7 |
+
<td>
|
8 |
+
|
9 |
+
## Table of Contents
|
10 |
+
- [Introduction](#axolotl)
|
11 |
+
- [Supported Features](#axolotl-supports)
|
12 |
+
- [Quickstart](#quickstart-)
|
13 |
+
- [Installation](#installation)
|
14 |
+
- [Docker Installation](#environment)
|
15 |
+
- [Conda/Pip venv Installation](#condapip-venv)
|
16 |
+
- [LambdaLabs Installation](#lambdalabs)
|
17 |
+
- [Dataset](#dataset)
|
18 |
+
- [How to Add Custom Prompts](#how-to-add-custom-prompts)
|
19 |
+
- [Config](#config)
|
20 |
+
- [Train](#train)
|
21 |
+
- [Inference](#inference)
|
22 |
+
- [Merge LORA to Base](#merge-lora-to-base)
|
23 |
+
- [Common Errors](#common-errors-)
|
24 |
+
- [Need Help?](#need-help-)
|
25 |
+
- [Badge](#badge-)
|
26 |
+
- [Community Showcase](#community-showcase)
|
27 |
+
- [Contributing](#contributing-)
|
28 |
+
|
29 |
+
</td>
|
30 |
+
<td>
|
31 |
+
|
32 |
<div align="center">
|
33 |
<img src="image/axolotl.png" alt="axolotl" width="160">
|
34 |
<div>
|
35 |
<p>
|
36 |
+
<b>Axolotl provides a unified repository for fine-tuning <br />a variety of AI models with ease</b>
|
37 |
</p>
|
38 |
<p>
|
39 |
Go ahead and axolotl questions!!
|
|
|
43 |
</div>
|
44 |
</div>
|
45 |
|
46 |
+
</td>
|
47 |
+
</tr>
|
48 |
+
</table>
|
49 |
+
|
50 |
## Axolotl supports
|
51 |
|
52 |
| | fp16/fp32 | lora | qlora | gptq | gptq w/ lora | gptq w/flash attn | flash attn | xformers attn |
|
|
|
62 |
|
63 |
## Quickstart ⚡
|
64 |
|
65 |
+
Get started with Axolotl in just a few steps! This quickstart guide will walk you through setting up and running a basic fine-tuning task.
|
66 |
+
|
67 |
**Requirements**: Python >=3.9 and Pytorch >=2.0.
|
68 |
|
69 |
```bash
|
|
|
165 |
|
166 |
### Dataset
|
167 |
|
168 |
+
Axolotl supports a variety of dataset formats. Below are some of the formats you can use.
|
169 |
Have dataset(s) in one of the following format (JSONL recommended):
|
170 |
|
171 |
- `alpaca`: instruction; input(optional)
|
|
|
658 |
|
659 |
## Common Errors 🧰
|
660 |
|
661 |
+
> If you encounter a 'Cuda out of memory' error, it means your GPU ran out of memory during the training process. Here's how to resolve it:
|
662 |
|
663 |
Please reduce any below
|
664 |
- `micro_batch_size`
|
|
|
666 |
- `gradient_accumulation_steps`
|
667 |
- `sequence_len`
|
668 |
|
669 |
+
> `failed (exitcode: -9)` usually means your system has run out of system memory.
|
670 |
+
Similarly, you should consider reducing the same settings as when you run out of VRAM.
|
671 |
+
Additionally, look into upgrading your system RAM which should be simpler than GPU upgrades.
|
672 |
+
|
673 |
> RuntimeError: expected scalar type Float but found Half
|
674 |
|
675 |
Try set `fp16: true`
|
|
|
698 |
|
699 |
## Community Showcase
|
700 |
|
701 |
+
Check out some of the projects and models that have been built using Axolotl! Have a model you'd like to add to our Community Showcase? Open a PR with your model.
|
702 |
+
|
703 |
Open Access AI Collective
|
704 |
- [Minotaur 13b](https://huggingface.co/openaccess-ai-collective/minotaur-13b)
|
705 |
- [Manticore 13b](https://huggingface.co/openaccess-ai-collective/manticore-13b)
|