michaelfeil commited on
Commit
bc4c196
1 Parent(s): e005212

Upload bigcode/starcoderbase ctranslate fp16 weights

Browse files
Files changed (1) hide show
  1. README.md +4 -3
README.md CHANGED
@@ -254,9 +254,9 @@ Speedup inference while reducing memory by 2x-4x using int8 inference in C++ on
254
 
255
  quantized version of [bigcode/starcoderbase](https://huggingface.co/bigcode/starcoderbase)
256
  ```bash
257
- pip install hf-hub-ctranslate2>=2.0.8
258
  ```
259
- Converted on 2023-05-30 using
260
  ```
261
  ct2-transformers-converter --model bigcode/starcoderbase --output_dir /home/michael/tmp-ct2fast-starcoderbase --force --copy_files merges.txt tokenizer.json README.md tokenizer_config.json vocab.json generation_config.json special_tokens_map.json .gitattributes --quantization float16 --trust_remote_code
262
  ```
@@ -280,7 +280,8 @@ model = GeneratorCT2fromHfHub(
280
  )
281
  outputs = model.generate(
282
  text=["How do you call a fast Flan-ingo?", "User: How are you doing? Bot:"],
283
- max_length=64
 
284
  )
285
  print(outputs)
286
  ```
 
254
 
255
  quantized version of [bigcode/starcoderbase](https://huggingface.co/bigcode/starcoderbase)
256
  ```bash
257
+ pip install hf-hub-ctranslate2>=2.0.8 ctranslate2>=3.14.0
258
  ```
259
+ Converted on 2023-05-31 using
260
  ```
261
  ct2-transformers-converter --model bigcode/starcoderbase --output_dir /home/michael/tmp-ct2fast-starcoderbase --force --copy_files merges.txt tokenizer.json README.md tokenizer_config.json vocab.json generation_config.json special_tokens_map.json .gitattributes --quantization float16 --trust_remote_code
262
  ```
 
280
  )
281
  outputs = model.generate(
282
  text=["How do you call a fast Flan-ingo?", "User: How are you doing? Bot:"],
283
+ max_length=64,
284
+ include_prompt_in_result=False
285
  )
286
  print(outputs)
287
  ```