Fix markdown code spans in README.md
Browse files
README.md
CHANGED
@@ -12,11 +12,11 @@ tags:
|
|
12 |
|
13 |
# MOSS Fix Autotune
|
14 |
|
15 |
-
This model repository serves as a temporary drop-in fix for an error that occurs during inference on int4 family of MOSS models, specifically, TypeError: '<' not supported between instances of 'tuple' and 'float' in [#159](https://github.com/OpenLMLab/MOSS/issues/159).
|
16 |
|
17 |
-
When loading the model with model = AutoModelForCausalLM.from_pretrained("fnlp/moss-moon-003-sft-int4", trust_remote_code=True).half().cuda()
|
18 |
|
19 |
-
Before the official maintainers update and fix the issue, I have applied the patch and re-uploaded the model files in this repo for convenience. The patched model files can be used with model = AutoModelForCausalLM.from_pretrained("DotIN13/moss-moon-003-sft-int4-fix-autotune", trust_remote_code=True).half().cuda()
|
20 |
|
21 |
## Table of Contents
|
22 |
|
|
|
12 |
|
13 |
# MOSS Fix Autotune
|
14 |
|
15 |
+
This model repository serves as a temporary drop-in fix for an error that occurs during inference on int4 family of MOSS models, specifically, TypeError: `'<' not supported between instances of 'tuple' and 'float'` in [#159](https://github.com/OpenLMLab/MOSS/issues/159).
|
16 |
|
17 |
+
When loading the model with `model = AutoModelForCausalLM.from_pretrained("fnlp/moss-moon-003-sft-int4", trust_remote_code=True).half().cuda()`, the custom_autotune.py from the huggingface model files will be used. As such, it is necessary to update the file in the huggingface model repo.
|
18 |
|
19 |
+
Before the official maintainers update and fix the issue, I have applied the patch and re-uploaded the model files in this repo for convenience. The patched model files can be used with `model = AutoModelForCausalLM.from_pretrained("DotIN13/moss-moon-003-sft-int4-fix-autotune", trust_remote_code=True).half().cuda()`.
|
20 |
|
21 |
## Table of Contents
|
22 |
|