afrideva's picture
Update README.md
36a38f4
|
raw
history blame
1.54 kB
metadata
base_model: niallturbitt/mpt-3b-8k-instruct
inference: false
model_creator: niallturbitt
model_name: mpt-3b-8k-instruct
pipeline_tag: text-generation
quantized_by: afrideva
tags:
  - gguf
  - ggml
  - quantized
  - q2_k
  - q3_k_m
  - q4_k_m
  - q5_k_m
  - q6_k
  - q8_0

TESTING, MODEL INFERENCE CURRENTLY BROKEN.

niallturbitt/mpt-3b-8k-instruct-GGUF

Quantized GGUF model files for mpt-3b-8k-instruct from niallturbitt

Name Quant method Size
mpt-3b-8k-instruct.q2_k.gguf q2_k 611.20 MB
mpt-3b-8k-instruct.q3_k_m.gguf q3_k_m 763.49 MB
mpt-3b-8k-instruct.q4_k_m.gguf q4_k_m 916.13 MB
mpt-3b-8k-instruct.q5_k_m.gguf q5_k_m 1.06 GB
mpt-3b-8k-instruct.q6_k.gguf q6_k 1.16 GB
mpt-3b-8k-instruct.q8_0.gguf q8_0 1.51 GB

Original Model Card: