3 contributors

History: 25 commits

Faisal AlKhateeb

added support for position interpolation

2f32550 7 months ago

.gitattributes

1.52 kB

initial commit 10 months ago
README.md

12.6 kB

added support for position interpolation 7 months ago
config.json

1.24 kB

change mup param names 10 months ago
configuration_btlm.py

9.67 kB

added support for position interpolation 7 months ago
figure_1_memory_footprint.png

301 kB

fix three images (#4) 10 months ago
figure_2_half_the_size_twice_the_speed.png

342 kB

upload images from blog (#2) 10 months ago
figure_3_performance_vs_3b_models.png

238 kB

upload images from blog (#2) 10 months ago
figure_4_performance_vs_7b_models.jpg

102 kB

upload images from blog (#2) 10 months ago
figure_5_xentropy_with_sequence_lengths.png

52.1 kB

upload images from blog (#2) 10 months ago
figure_5_xentropy_with_sequence_lengths.svg

520 kB

upload svg for long msl xentropy 10 months ago
generation_config.json

119 Bytes

add bfloat16 checkpoint 10 months ago
merges.txt

456 kB

add the tokenizer 10 months ago
modeling_btlm.py

72 kB

added support for position interpolation 7 months ago
pytorch_model.bin
Detected Pickle imports (3)
- "torch._utils._rebuild_tensor_v2",
- "torch.BFloat16Storage",
- "collections.OrderedDict"
What is a pickle import?
5.29 GB
LFS

add bfloat16 checkpoint 10 months ago
special_tokens_map.json

99 Bytes

add the tokenizer 10 months ago
table_1_downstream_performance_3b.png

590 kB

fix three images (#4) 10 months ago
table_2_downstream_performance_7b.png

243 kB

upload images from blog (#2) 10 months ago
tokenizer_config.json

234 Bytes

add the tokenizer 10 months ago
vocab.json

1.04 MB

add the tokenizer 10 months ago

Detected Pickle imports (3)