README.md

---
title: README
emoji: 🏢
colorFrom: pink
colorTo: blue
sdk: static
pinned: false
---


**LLM360 is an initiative to open-source large language models (LLMs) that foster transparency, trust, and collaborative research.**

LLM360 releases complete training details including all of the following artifacts associated with each model:

* Frequent Intermediate Model Checkpoints: During training, model parameters and optimizer states are collected regularly. These artifacts can offer valuable insights for studying LLM training dynamics and how it scales with data.
* Training Data with Full Data Sequence: The entire training dataset is fully disclosed and made publicly available. In addition, the full sequence of data examples used at each training step.
* Source Code: All the code used, including data processing, training, evaluation, and analysis.
* Logs and Metrics: All the training logs and evaluations done during training and using the final model are publicly disclosed.​

LLM360 is a collaboration between Petuum, MBZUAI, and Cerebras.