RichardErkhov/fhai50032_-_Mistral-4B-8bits

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Mistral-4B - bnb 8bits

• Model creator: https://huggingface.co/fhai50032/
• Original model: https://huggingface.co/fhai50032/Mistral-4B/

Original model description:

tags: - merge - mergekit - mistral - fhai50032/RolePlayLake-7B - fhai50032/RolePlayLake-7B - fhai50032/RolePlayLake-7B - fhai50032/RolePlayLake-7B - fhai50032/RolePlayLake-7B - fhai50032/RolePlayLake-7B base_model: - fhai50032/RolePlayLake-7B - fhai50032/RolePlayLake-7B - fhai50032/RolePlayLake-7B - fhai50032/RolePlayLake-7B - fhai50032/RolePlayLake-7B - fhai50032/RolePlayLake-7B

Mistral-4B

🧩 Configuration

dtype: bfloat16
merge_method: passthrough
slices:
- sources:
  - layer_range: [0, 8]
    model: fhai50032/RolePlayLake-7B
- sources:
  - layer_range: [11, 12]
    model: fhai50032/RolePlayLake-7B
- sources:
  - layer_range: [15, 16]
    model: fhai50032/RolePlayLake-7B
- sources:
  - layer_range: [19, 20]
    model: fhai50032/RolePlayLake-7B
- sources:
  - layer_range: [24, 25]
    model: fhai50032/RolePlayLake-7B
- sources:
  - layer_range: [28, 32]
    model: fhai50032/RolePlayLake-7B