Upload folder using huggingface_hub
Browse files
README.md
CHANGED
|
@@ -78,7 +78,12 @@ vLLM aslo supports OpenAI-compatible serving. See the [documentation](https://do
|
|
| 78 |
|
| 79 |
<details>
|
| 80 |
<summary>Creation details</summary>
|
| 81 |
-
This model was created with [llm-compressor](https://github.com/vllm-project/llm-compressor) by running the code snippet below
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 82 |
|
| 83 |
|
| 84 |
```python
|
|
|
|
| 78 |
|
| 79 |
<details>
|
| 80 |
<summary>Creation details</summary>
|
| 81 |
+
This model was created with [llm-compressor](https://github.com/vllm-project/llm-compressor) by running the code snippet below with:
|
| 82 |
+
|
| 83 |
+
```bash
|
| 84 |
+
python int4.py --model_path HuggingFaceTB/SmolLM3-3B --calib_size 1024 --dampening_frac 0.1 --observer minmax --actorder group --sym false
|
| 85 |
+
```
|
| 86 |
+
where `int4.py` is as follows:
|
| 87 |
|
| 88 |
|
| 89 |
```python
|