yanghaojin (Haojin Yang)

Posts 2

Post

665

Full parameter fine-tuning of the LLaMA-3 8B model using a single GTX 3090 GPU with 24GB of graphics memory?

Please check out our tool for fine-tuning, inferencing, and evaluating GreenBitAI's low-bit LLMs:
https://github.com/GreenBitAI/green-bit-llm
Model Zoo:
https://huggingface.co/GreenBitAI

Post

769

Dear all,

We are happy to share that we have just open-sourced over 200 low-bit LLMs. For the MLX community, we have prepared 2-4 bit versions of mainstream LLMs. You can visit the following collection to access them: GreenBitAI/greenbitai-mlx-llm-6614eb6ceb8da657c2b4ed58.

These low-bit models can be conveniently used through our open-source tool at https://github.com/GreenBitAI/gbx-lm.

Compared to other open-source quantization algorithms, these models provide better accuracy retention. We have provided some model evaluation results here:
https://github.com/GreenBitAI/green-bit-llm/blob/main/green_bit_llm/evaluation/README.md.

You can also evaluate the models yourself using the evaluation script we provided.

Papers 1

arxiv:2309.06759

models

None public yet

datasets

None public yet

Haojin Yang

AI & ML interests

Organizations

Posts 2

Papers 1

models

datasets