DotIN13 commited on
Commit
971e0b5
1 Parent(s): 2a637bc

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -2
README.md CHANGED
@@ -12,11 +12,17 @@ tags:
12
 
13
  # MOSS Fix Autotune
14
 
15
- This model repository serves as a temporary drop-in fix for an error that occurs during inference on int4 family of MOSS models, specifically, TypeError: `'<' not supported between instances of 'tuple' and 'float'` in [#159](https://github.com/OpenLMLab/MOSS/issues/159).
16
 
17
  When loading the model with `model = AutoModelForCausalLM.from_pretrained("fnlp/moss-moon-003-sft-int4", trust_remote_code=True).half().cuda()`, the custom_autotune.py from the huggingface model files will be used. As such, it is necessary to update the file in the huggingface model repo.
18
 
19
- Before the official maintainers update and fix the issue, I have applied the patch and re-uploaded the model files in this repo for convenience. The patched model files can be used with `model = AutoModelForCausalLM.from_pretrained("DotIN13/moss-moon-003-sft-int4-fix-autotune", trust_remote_code=True).half().cuda()`.
 
 
 
 
 
 
20
 
21
  ## Table of Contents
22
 
 
12
 
13
  # MOSS Fix Autotune
14
 
15
+ This model repository serves as a temporary drop-in fix for an error that occurs during inference on int4 family of MOSS models, specifically, `TypeError: '<' not supported between instances of 'tuple' and 'float'` in [#159](https://github.com/OpenLMLab/MOSS/issues/159).
16
 
17
  When loading the model with `model = AutoModelForCausalLM.from_pretrained("fnlp/moss-moon-003-sft-int4", trust_remote_code=True).half().cuda()`, the custom_autotune.py from the huggingface model files will be used. As such, it is necessary to update the file in the huggingface model repo.
18
 
19
+ Before the official maintainers update and fix the issue, I have applied the patch and re-uploaded the model files in this repo for convenience.
20
+
21
+ The patched model files can be used with:
22
+
23
+ ```python
24
+ model = AutoModelForCausalLM.from_pretrained("DotIN13/moss-moon-003-sft-int4-fix-autotune", trust_remote_code=True).half().cuda()
25
+ ```
26
 
27
  ## Table of Contents
28