Hugging Face's logo

leafspark
/
IridiumLlama-72B-v0.1

Text Generation
Transformers
Safetensors
English
Chinese
llama
mergekit
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
leafspark commited on
Commit
ea1bcbd
·
verified ·
1 Parent(s): 02e49cd

docs: update model card

Browse files
Files changed (1) hide show
  1. README.md +9 -8
README.md CHANGED
@@ -12,16 +12,16 @@ tags:
12
  - llama
13
  ---
14
 
15
- # FeatherLlama-72B-v0.1
16
 
17
  ## Model Description
18
  FeatherLlama is a 72B parameter language model created through a merge of Qwen2-72B-Instruct, calme2.1-72b, and magnum-72b-v1 using `model_stock`.
19
 
20
- This is converted from [leafspark/FeatherQwen2-72B-v0.1](https://huggingface.co/leafspark/FeatherQwen2-72B-v0.1)
21
 
22
  ## Features
23
  - 72 billion parameters
24
- - Sharded in 31 files (unlike FeatherQwen2, which has 1,043 shards due to the merging process)
25
  - Combines Magnum prose with Calam smarts
26
  - Llamaified for easy use
27
 
@@ -32,6 +32,7 @@ This is converted from [leafspark/FeatherQwen2-72B-v0.1](https://huggingface.co/
32
  - Models: Qwen2-72B-Instruct (base), calme2.1-72b, magnum-72b-v1
33
  - Merged layers: 80
34
  - Total tensors: 1,043
 
35
 
36
  ### Tensor Distribution
37
  - Attention layers: 560 files
@@ -49,15 +50,15 @@ Custom script utilizing safetensors library.
49
  from transformers import AutoModelForCausalLM, AutoTokenizer
50
  import torch
51
 
52
- model = AutoModelForCausalLM.from_pretrained("leafspark/FeatherLlama-72B-v0.1",
53
  device_map="auto",
54
  torch_dtype=torch.float16)
55
- tokenizer = AutoTokenizer.from_pretrained("leafspark/FeatherLlama-72B-v0.1")
56
  ```
57
  ### GGUFs
58
 
59
- Find them here: [leafspark/FeatherLlama-72B-v0.1-GGUF](https://huggingface.co/leafspark/FeatherLlama-72B-v0.1-GGUF)
60
 
61
  ### Hardware Requirements
62
- - Minimum ~140GB of storage
63
- - ~140GB VRAM
 
12
  - llama
13
  ---
14
 
15
+ # IridiumLlama-72B-v0.1
16
 
17
  ## Model Description
18
  FeatherLlama is a 72B parameter language model created through a merge of Qwen2-72B-Instruct, calme2.1-72b, and magnum-72b-v1 using `model_stock`.
19
 
20
+ This is converted from [leafspark/Iridium-72B-v0.1](https://huggingface.co/leafspark/Iridium-72B-v0.1)
21
 
22
  ## Features
23
  - 72 billion parameters
24
+ - Sharded in 31 files (unlike Iridium, which has 963 shards due to the merging process)
25
  - Combines Magnum prose with Calam smarts
26
  - Llamaified for easy use
27
 
 
32
  - Models: Qwen2-72B-Instruct (base), calme2.1-72b, magnum-72b-v1
33
  - Merged layers: 80
34
  - Total tensors: 1,043
35
+ - Context length: 32k
36
 
37
  ### Tensor Distribution
38
  - Attention layers: 560 files
 
50
  from transformers import AutoModelForCausalLM, AutoTokenizer
51
  import torch
52
 
53
+ model = AutoModelForCausalLM.from_pretrained("leafspark/IridiumLlama-72B-v0.1",
54
  device_map="auto",
55
  torch_dtype=torch.float16)
56
+ tokenizer = AutoTokenizer.from_pretrained("leafspark/IridiumLlama-72B-v0.1")
57
  ```
58
  ### GGUFs
59
 
60
+ Find them here: [leafspark/IridiumLlama-72B-v0.1-GGUF](https://huggingface.co/leafspark/IridiumLlama-72B-v0.1-GGUF)
61
 
62
  ### Hardware Requirements
63
+ - At least ~150GB of free space
64
+ - ~150GB VRAM