dranger003
/

gemma-7b-it-iMat.GGUF

Text Generation

Inference Endpoints

Model card Files Files and versions Community

dranger003 commited on Feb 24, 2024

Commit

cf722c1

·

verified ·

1 Parent(s): 3e5d768

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -16,7 +16,7 @@ The importance matrix was trained for ~50K tokens (105 batches of 512 tokens) us
 **NOTE**: This model [doesn't like repeat penalty](https://huggingface.co/google/gemma-7b-it/discussions/38#65d7b14adb51f7c160769fa1) for some reason, so it seems I need to set it to `1.0` (i.e. disable it):
 ```
-main -p "<start_of_turn>user\nWrite an essay about AI.<end_of_turn>\n<start_of_turn>model\n" -e --repeat-penalty 1.0 -ngl 29 -m ggml-gemma-7b-it-iq3_xxs.gguf
 ```
 The response below was generated using the new IQ3_M quant:
 ## The Dawn of Tomorrow: Artificial Intelligence and the Human Future

 **NOTE**: This model [doesn't like repeat penalty](https://huggingface.co/google/gemma-7b-it/discussions/38#65d7b14adb51f7c160769fa1) for some reason, so it seems I need to set it to `1.0` (i.e. disable it):
 ```
+main -p "<start_of_turn>user\nWrite an essay about AI.<end_of_turn>\n<start_of_turn>model\n" -e --repeat-penalty 1.0 -ngl 29 -m ggml-gemma-7b-it-iq3_m.gguf
 ```
 The response below was generated using the new IQ3_M quant:
 ## The Dawn of Tomorrow: Artificial Intelligence and the Human Future