Repository: huggingface/optimum-amd (language: python)

Issue:
Fix: handle models without BOS/EOS tokens defined (e.g., Qwen1.5)
Some models don't have BOS/EOS tokens defined (e.g., Qwen1.5), we should handle this case.


Discussion / hints:

Fix patch:
diff --git a/examples/quantization/brevitas/README.md b/examples/quantization/brevitas/README.md
index 98de56a6..1f662459 100644
--- a/examples/quantization/brevitas/README.md
+++ b/examples/quantization/brevitas/README.md
@@ -6,17 +6,24 @@ The example shows an example on how to quantize a decoder-class LLM model throug
 - Optional conversion of the LLM model to an FX representation, leveraging Hugging Face transformers' tracer.
 - Support of executing post-training quantization (PTQ) algorithms and validation (SmoothQuant, GPTQ), while leveraging CPU offload from Hugging Face accelerate.
 - Validation of the quantized model.
-- Export of the quantized model as ONNX, QDQ-style.
+- Export of the quantized model as ONNX, QOP-style.
 
 ## Prerequisites
 
 The examples were tested using:
+- `python>=3.9` (required for QOP ONNX export)
 - `brevitas` installed from dev (`pip install git+https://github.com/Xilinx/brevitas.git@dev`)
-- `torch>=2.1.2`
-- `transformers` installed from main (`pip install git+https://github.com/huggingface/transformers.git@4b236aed7618d90546cd2e8797dab5b4a24c5dce`)
+- `torch>=2.2`
+- `transformers>=4.38.0`
 - `optimum>=1.17.0`
-- `optimum-amd`` installed from main (`pip install git+https://github.com/huggingface/optimum-amd`)
-- Optionally, `accelerate` installed from main (`pip install git+https://github.com/huggingface/accelerate.git`)
+- `optimum-amd` installed from main (`pip install git+https://github.com/huggingface/optimum-amd`)
+- `accelerate>=0.30.0`
+
+Note, you can install all the prerequisites with:
+
+```bash
+pip install 'brevitas @ git+https://github.com/Xilinx/brevitas.git@dev' 'optimum-amd[brevitas] @ git+https://github.com/huggingface/optimum-amd.git@main'
+```
 
 ## Running the Example
 
diff --git a/setup.py b/setup.py
index b87ced6f..f5dbf85c 100644
--- a/setup.py
+++ b/setup.py
@@ -37,7 +37,16 @@
     "quality": QUALITY_REQUIRE,
     "tests": TESTS_REQUIRE,
     "zentorch": ["torch==2.2.1"],
-    "brevitas": ["brevitas", "torch>=2.2", "datasets>=2.17", "onnx", "onnxruntime", "accelerate", "onnx-tool"],
+    "brevitas": [
+        "brevitas",
+        "torch>=2.2",
+        "datasets>=2.17",
+        "onnx",
+        "onnxruntime",
+        "accelerate>=0.30",
+        "onnx-tool",
+        "optimum>=1.17",
+    ],
 }
 
 setup(


Test patch:
diff --git a/.github/workflows/test_brevitas.yaml b/.github/workflows/test_brevitas.yaml
index 5db21168..80846fb2 100644
--- a/.github/workflows/test_brevitas.yaml
+++ b/.github/workflows/test_brevitas.yaml
@@ -35,10 +35,6 @@ jobs:
         python-version: ${{ matrix.python-version }}
     - name: Install dependencies
       run: |
-        # Transformers main may currently be broken due to the KV cache refactor that breaks many things, hence pin to the commit before that.
-        pip install git+https://github.com/huggingface/transformers.git@4b236aed7618d90546cd2e8797dab5b4a24c5dce
-        pip install git+https://github.com/huggingface/accelerate.git
-        pip install optimum
         pip install git+https://github.com/Xilinx/brevitas.git@dev
 
         pip install .[brevitas,tests]