Hugging Face's logo

backyardai
/
Meta-Llama-3.1-70B-Instruct-GGUF

Text Generation
Transformers
GGUF
PyTorch
facebook
meta
llama
llama-3
Inference Endpoints
conversational
Model card Files Community
brooketh commited on
Commit
c164c7f
•
1 Parent(s): 4bbc39e

Upload Meta-Llama-3.1-70B-Instruct-GGUF

Browse files
Files changed (23) hide show
  1. Meta-Llama-3.1-70B-Instruct.F32-split-00001-of-00006.gguf +1 -1
  2. Meta-Llama-3.1-70B-Instruct.IQ1_M.gguf +1 -1
  3. Meta-Llama-3.1-70B-Instruct.IQ1_S.gguf +1 -1
  4. Meta-Llama-3.1-70B-Instruct.IQ2_M.gguf +1 -1
  5. Meta-Llama-3.1-70B-Instruct.IQ2_S.gguf +1 -1
  6. Meta-Llama-3.1-70B-Instruct.IQ2_XS.gguf +1 -1
  7. Meta-Llama-3.1-70B-Instruct.IQ2_XXS.gguf +1 -1
  8. Meta-Llama-3.1-70B-Instruct.IQ3_M.gguf +1 -1
  9. Meta-Llama-3.1-70B-Instruct.IQ3_S.gguf +1 -1
  10. Meta-Llama-3.1-70B-Instruct.IQ3_XS.gguf +1 -1
  11. Meta-Llama-3.1-70B-Instruct.IQ3_XXS.gguf +1 -1
  12. Meta-Llama-3.1-70B-Instruct.IQ4_XS.gguf +1 -1
  13. Meta-Llama-3.1-70B-Instruct.Q3_K_L.gguf +1 -1
  14. Meta-Llama-3.1-70B-Instruct.Q3_K_M.gguf +1 -1
  15. Meta-Llama-3.1-70B-Instruct.Q3_K_S.gguf +1 -1
  16. Meta-Llama-3.1-70B-Instruct.Q4_K_M.gguf +1 -1
  17. Meta-Llama-3.1-70B-Instruct.Q4_K_S.gguf +1 -1
  18. Meta-Llama-3.1-70B-Instruct.Q5_K_M.gguf +1 -1
  19. Meta-Llama-3.1-70B-Instruct.Q5_K_S.gguf +1 -1
  20. Meta-Llama-3.1-70B-Instruct.Q6_K-split-00001-of-00002.gguf +1 -1
  21. Meta-Llama-3.1-70B-Instruct.Q8_0-split-00001-of-00002.gguf +1 -1
  22. Meta-Llama-3.1-70B-Instruct.imatrix +1 -1
  23. README.md +1 -1
Meta-Llama-3.1-70B-Instruct.F32-split-00001-of-00006.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d785432d4bc3788cbb071febe5db498c43f45e423e85c2806880c21200041dae
3
  size 49644104480
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:39238c982bbdc268b2f79ef271166b9dc802f7936a5899ed2f98017fe4f459d0
3
  size 49644104480
Meta-Llama-3.1-70B-Instruct.IQ1_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:84ca14d89b7f3dfd3d0469a389707027d984783fe67bd2758899f6044c159fe5
3
  size 16751196320
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6a2ca2cb7bd674d7ea9f2de30408127f0383d1113abe332bc02936aab37ef4f3
3
  size 16751196320
Meta-Llama-3.1-70B-Instruct.IQ1_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:50b116b9ae8729c7ad9bd2e9826b49b412d82a1ed31c3f109baad9a01438dfc0
3
  size 15343483040
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f674250af5fc2f52c70b3db5a811be1d0d32cb84ca0246504548d2ff3e891f8a
3
  size 15343483040
Meta-Llama-3.1-70B-Instruct.IQ2_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c816ed627e4f5a3c1ffca87d2f1e608c2ecb004908140adb34ffa9b40d0330a3
3
  size 24119294112
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b6e6b5dabef7efd5b8fcaf71900ca591099dca44c76a567c9b863bb3f26674ee
3
  size 24119294112
Meta-Llama-3.1-70B-Instruct.IQ2_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:17beee0e2da1ad45e8f65506adc3f662d052e4471a762c3caac1728ecffe485f
3
  size 22242343072
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0618492f8fb8d3319e03fd227e3f9c8edfe1cb3158c373c0e56f07827516fa09
3
  size 22242343072
Meta-Llama-3.1-70B-Instruct.IQ2_XS.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3c57058e930e6f7d2d4527911bc8464b6f6aa95193729aa0e5002a446e7c33dd
3
  size 21142108320
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:834de08299c89298abcd29d07b5e35fd84db74a17de08b5c32bb41a8be4e07f4
3
  size 21142108320
Meta-Llama-3.1-70B-Instruct.IQ2_XXS.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b2fe691d337a76cf85517b963ba439d506cdd5110925244d06d73d7d5ae9f01e
3
  size 19097385120
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3a26b790be37614b56f59b765bc68e188cacbd22d03c8ab96e18eeb95f85958b
3
  size 19097385120
Meta-Llama-3.1-70B-Instruct.IQ3_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a17a089dec6f5bee5d8426c5a9f5c293abe376ee4be5b39a827fdd51dc598e41
3
  size 31937034400
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:73c26c38eef1f5f82923dd62814918e2e2b2c85d4f7fcbc123b4d00367c578ac
3
  size 31937034400
Meta-Llama-3.1-70B-Instruct.IQ3_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6d169018c9eecd67340ec442d1e422a2a6b744d2db00664cb8530ebd9757f118
3
  size 30912051360
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:808603ccfa66d894ff9a5041c9d6c551103a1b439c132385b961a7f014994658
3
  size 30912051360
Meta-Llama-3.1-70B-Instruct.IQ3_XS.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:51b89e8f056d448b596a03bb0813ea1d879fb6c090f118e826ec29ee21ce62e1
3
  size 29307730080
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9b7318b59b5b4aa5e4af5bcd29ea2a18f1c9070302124be0cbb181bdd0d4b270
3
  size 29307730080
Meta-Llama-3.1-70B-Instruct.IQ3_XXS.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7318937cc8952782085ffcf91940969f5560da8b09b13a97d7ce3138d0edcdc4
3
  size 27469494432
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5d1812bb2f2039c1d196d22620f1c6af27d635ef210b3f165b2b6c1ad00d1509
3
  size 27469494432
Meta-Llama-3.1-70B-Instruct.IQ4_XS.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ce959b8c91b43d9f85419d0f40b4aba3874bf6339d699f57f1af229f45956b9a
3
  size 37902661792
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:045014faab8482c9ef25360f745e0d7546910155253971597417d3c941fffb30
3
  size 37902661792
Meta-Llama-3.1-70B-Instruct.Q3_K_L.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:16d73f9f260fe102083601e53d5cc3f6eb474188df62db1dc3752dac86ef0832
3
  size 37140592512
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:997135a6f9bf4272ceb77d46ce8848d99f821ed1b69f4288a98b4564a53febb3
3
  size 37140592512
Meta-Llama-3.1-70B-Instruct.Q3_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:73553b90c401d42e9477e0b648a80ad2951119ef44fb503084e4fa815f27191a
3
  size 34267494272
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:787e6a78501d2b008cf7a225534003c9f038440da591f185a5591d043196a3f1
3
  size 34267494272
Meta-Llama-3.1-70B-Instruct.Q3_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b0bbb5fa5f6b62f53a2168daab66be578ca81167d6c2212f7543817b291f288d
3
  size 30912051072
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e9f3e12a5873073c517cb037db5f085492b3498649be9c81476465b3cc2f0f99
3
  size 30912051072
Meta-Llama-3.1-70B-Instruct.Q4_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3b94ace1121e815b0e217db0421ea2a030d5277ed68f765a0b748879ed2c3cbe
3
  size 42520393600
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f5dd23a6ca09b42dab9a29fface1a190c772f5f6d9d3f93672ff43c0ecb390c5
3
  size 42520393600
Meta-Llama-3.1-70B-Instruct.Q4_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:660bc27ae6f7662f98ee3cd7883fdb75489557fcc2a81ca5567d2d2bb028d434
3
  size 40347219840
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8925a41ac10ab0367cb71356f6df3599e420e7b13cdb09a0eb294eef8ca5aaed
3
  size 40347219840
Meta-Llama-3.1-70B-Instruct.Q5_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e45df6bf4c8899d3bdf2d4fdef8efa9af86ea994d2a5e9d9ded864ae259f33fe
3
  size 49949816704
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:23c527c2ba205af9dcc67a61a936233371e85754577ebb3cdfc5edb3b64af7db
3
  size 49949816704
Meta-Llama-3.1-70B-Instruct.Q5_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:dfc5aeb2db88ae72b64ee3c3329bf3e00d8b6a503ff5b5a57dcc1e5a96236958
3
  size 48657446784
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4e5000d9d5619afb01411f40feee3c7a85a178cb6b51ddff5b075b43fc2ea45a
3
  size 48657446784
Meta-Llama-3.1-70B-Instruct.Q6_K-split-00001-of-00002.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f2faa8f13cddba3bbe95f985a378e2152b3914ea0d72e22aeb18c1b1a11a7b89
3
  size 49999782432
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f95a6d62f065278e64b17ca64427c9d5d6d1f7996478cbd15acc1a7b91421ccf
3
  size 49999782432
Meta-Llama-3.1-70B-Instruct.Q8_0-split-00001-of-00002.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c43e1df15e98e6005ca776b842b1e0eca80e111fb9a6602b45da9dc6524ce90c
3
  size 49809932128
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:88a021fc711f7c4c2c44c231af28641f32a380bd20955fe0da4b1e2760f41954
3
  size 49809932128
Meta-Llama-3.1-70B-Instruct.imatrix CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:04a4fb9c02f4397a7f477fe03372f3741091b7563466c0c3e57877e635ccde91
3
  size 24922344
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:52f0b6745bd0847caa0f8971d282afe41fc8587f308dc7824645cbb2dbe46990
3
  size 24922344
README.md CHANGED
@@ -206,7 +206,7 @@ parameter_count: 70553706496
206
  - **Original:** [Meta Llama 3.1 70B Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct)
207
  - **Date Created:** 2024-07-16
208
  - **Trained Context:** 131072 tokens
209
- - **Description:** Version 3.1 of the Meta Llama base model. This is the instruct variant with 70 billion parameters. Not finetuned for roleplay.
210
  ***
211
  ## What is a GGUF?
212
  GGUF is a large language model (LLM) format that can be split between CPU and GPU. GGUFs are compatible with applications based on llama.cpp, such as Backyard AI. Where other model formats require higher end GPUs with ample VRAM, GGUFs can be efficiently run on a wider variety of hardware.
 
206
  - **Original:** [Meta Llama 3.1 70B Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct)
207
  - **Date Created:** 2024-07-16
208
  - **Trained Context:** 131072 tokens
209
+ - **Description:** Version 3.1 of the Meta Llama base model. This is the instruct variant with 8 billion parameters. Not finetuned for roleplay.
210
  ***
211
  ## What is a GGUF?
212
  GGUF is a large language model (LLM) format that can be split between CPU and GPU. GGUFs are compatible with applications based on llama.cpp, such as Backyard AI. Where other model formats require higher end GPUs with ample VRAM, GGUFs can be efficiently run on a wider variety of hardware.