Repository: guidance-ai/guidance (language: python)

Issue:
[Bug] Latest Transformers disagrees with GPT2 on MacOS-ARM
**The bug**

It appears that the latest versions of `transformers` (4.43.*) do not play nicely with GPT2 on MacOS-ARM. This is seen in our PR Gate, with errors:
```
FAILED tests/model_integration/library/test_gen.py::test_stop_list_side_effect - RuntimeError: probability tensor contains either `inf`, `nan` or element < 0
FAILED tests/model_integration/library/test_gen.py::test_unicode - RuntimeError: probability tensor contains either `inf`, `nan` or element < 0
FAILED tests/model_integration/library/test_gen.py::test_pattern_kleene - AssertionError: assert False
 +  where False = <built-in method startswith of str object at 0x11ebb6c70>('!!!!!!!!!!')
 +    where <built-in method startswith of str object at 0x11ebb6c70> = 'stic!!!!!!!!!!'.startswith
FAILED tests/model_integration/library/test_gen.py::test_tool_call - Exception: Attempted to run a transformers model past its maximum context window size of 1024!
```

I have been working around with exclusions in `setup.py` but it would be good if someone with a MacOS-ARM machine could dig into this.

**To Reproduce**

See e.g.:
https://github.com/guidance-ai/guidance/actions/runs/10061999864/job/27815616390

**System info (please complete the following information):**
 - MacOS-ARM
 - Latest source in PR gate



Discussion / hints:
@Harsha-Nori can't reproduce.... @hudson-ai or @nopdive , could you try on your machines, please?
Cannot repro on Sonoma with an M3 Max and `transformers==4.43.1`...

Fix patch:
diff --git a/.github/workflows/workflow-pr-gate.yml b/.github/workflows/workflow-pr-gate.yml
index b73e4b99a..a1c9eded8 100644
--- a/.github/workflows/workflow-pr-gate.yml
+++ b/.github/workflows/workflow-pr-gate.yml
@@ -169,7 +169,7 @@ jobs:
       matrix:
         python-version: ["3.9", "3.10", "3.11", "3.12"]
         model:
-          - "transformers_gpt2_cpu"
+          # - "transformers_gpt2_cpu" See #965
           # - "transformers_phi2_cpu" Seems to get stuck
           # - "transformers_mistral_7b_cpu" See Issue 713
           # - "llamacpp_llama2_7b_cpu" Getting stuck with llama-cpp-python 0.2.77


Test patch:
diff --git a/.github/workflows/action_plain_basic_tests.yml b/.github/workflows/action_plain_basic_tests.yml
index 9ab32f180..789a53ab9 100644
--- a/.github/workflows/action_plain_basic_tests.yml
+++ b/.github/workflows/action_plain_basic_tests.yml
@@ -48,8 +48,8 @@ jobs:
         shell: bash
         run: |
           pytest --cov=guidance --cov-report=xml --cov-report=term-missing \
-          --selected_model ${{ inputs.model }} \
-          ./tests/unit ./tests/model_integration ./tests/model_specific
+            --selected_model ${{ inputs.model }} \
+            ./tests/unit ./tests/model_integration ./tests/model_specific
       - name: Upload coverage reports to Codecov
         uses: codecov/codecov-action@v4
         with: