File size: 2,555 Bytes
89831d5 a519f10 5034d89 89831d5 a519f10 e7ae5bd a519f10 0a2b9c6 1f9a0a0 a519f10 0a2b9c6 a519f10 d32df80 a519f10 e7ae5bd a519f10 0a2b9c6 a519f10 0a2b9c6 a519f10 0a2b9c6 a519f10 0a2b9c6 e7ae5bd |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 |
---
license: apache-2.0
tags:
- snowflake
- arctic
- moe
---
## Model Details
Arctic is a dense-MoE Hybrid transformer architecture pre-trained from scratch by the Snowflake AI
Research Team. We are releasing model checkpoints for both the base and instruct-tuned versions of
Arctic under an Apache-2.0 license. This means you can use them freely in your own research,
prototypes, and products. Please see our blog
[Snowflake Arctic: The Best LLM for Enterprise AI — Efficiently Intelligent, Truly Open](https://www.snowflake.com/blog/arctic-open-and-efficient-foundation-language-models-snowflake)
for more information on Arctic and links to other relevant resources such as our series of cookbooks
covering topics around training your own custom MoE models, how to produce high-quality training data,
and much more.
* [Arctic-Base](https://huggingface.co/Snowflake/snowflake-arctic-base/)
* [Acrtic-Instruct](https://huggingface.co/Snowflake/snowflake-arctic-instruct/)
For the latest details about Snowflake Arctic including tutorials, etc. please refer to our github repo:
* https://github.com/Snowflake-Labs/snowflake-arctic
**Model developers** Snowflake AI Research Team
**License** Apache-2.0
**Input** Models input text only.
**Output** Models generate text and code only.
**Model Release Date** April, 24th 2024.
## Model Architecture
Arctic combines a 10B dense transformer model with a residual 128x3.66B MoE MLP resulting in 480B
total and 17B active parameters chosen using a top-2 gating. For more details about Arctic's model
Architecture, training process, data, etc. [see our series of cookbooks](https://www.snowflake.com/en/data-cloud/arctic/cookbook/).
## Usage
As of 4/24/2024 we are actively working with the maintainers of `transformers` to include the Arctic
model implementation. Until this support is released please follow these instructions to get the
required dependencies for using Arctic:
```python
pip install git+https://github.com/Snowflake-Labs/transformers.git@arctic
```
Arctic leverages several features from [DeepSpeed](https://github.com/microsoft/DeepSpeed), you will need to
install the latest version of DeepSpeed to get all of these required features:
```python
pip install "deepspeed>=0.14.2"
```
### Inference
The Arctic github page has several resources around running inference:
* Example with pure-HF: https://github.com/Snowflake-Labs/snowflake-arctic/blob/main/inference
* Tutorial using vLLM: https://github.com/Snowflake-Labs/snowflake-arctic/tree/main/inference/vllm |