CircleCI failed test summary
Author: ArthurZucker
Pull request: yes

# What does this PR do?

Create a GitHub Actions workflow to upload CircleCI test summary to a dataset, and send a comment with a Space url where we can visualize the results.

<img width="1542" height="1254" alt="Screenshot 2025-12-08 140746" src="https://github.com/user-attachments/assets/d12ccfc3-59b6-490a-b556-0de2993b945b" />



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42240). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

# Failure summary

## By test

| Test | Failures | Full error(s) |
| --- | --- | --- |
| tests/models/llama/test_tokenization_llama.py::LlamaTokenizationTest::test_add_prefix_space | 1 | 1× /usr/local/lib/python3.10/unittest/case.py:675: AssertionError: Lists differ: [1, 29950, 1032, 920, 526, 366, 2599] != [1, 18637, 920, 526, 366, 2599] |
| tests/models/llama/test_tokenization_llama.py::LlamaIntegrationTest::test_some_edge_cases | 1 | 1× /usr/local/lib/python3.10/unittest/case.py:675: AssertionError: Lists differ: ['▁'] != ['▁▁'] |
| tests/models/llama/test_tokenization_llama.py::CommonSpmIntegrationTests::test_add_dummy_prefix | 1 | 1× /usr/local/lib/python3.10/unittest/case.py:675: AssertionError: Lists differ: [4, 156, 86, 20] != [7, 4, 156, 86, 20] |
| tests/models/llama/test_tokenization_llama.py::CommonSpmIntegrationTests::test_character_after_special_token | 1 | 1× /usr/local/lib/python3.10/unittest/case.py:675: AssertionError: Lists differ: [262, 15, 30, 1, 100] != [156, 30, 1, 100] |
| tests/models/llama/test_tokenization_llama.py::CommonSpmIntegrationTests::test_special_tokens_strip | 1 | 1× /usr/local/lib/python3.10/unittest/case.py:675: AssertionError: Lists differ: [516, 1, 156] != [284, 1, 156] |
| tests/models/glm46v/test_processor_glm46v.py::Glm46VProcessorTest::test_apply_chat_template_video_frame_sampling | 1 | 1× /usr/local/lib/python3.10/site-packages/torch/_ops.py:841: RuntimeError: expected scalar type Long but found Double |

## By model

| Model | Failures | Full error(s) |
| --- | --- | --- |
| llama | 5 | 1× /usr/local/lib/python3.10/unittest/case.py:675: AssertionError: Lists differ: [1, 29950, 1032, 920, 526, 366, 2599] != [1, 18637, 920, 526, 366, 2599]; 1× /usr/local/lib/python3.10/unittest/case.py:675: AssertionError: Lists differ: ['▁'] != ['▁▁']; 1× /usr/local/lib/python3.10/unittest/case.py:675: AssertionError: Lists differ: [4, 156, 86, 20] != [7, 4, 156, 86, 20]; 1× /usr/local/lib/python3.10/unittest/case.py:675: AssertionError: Lists differ: [262, 15, 30, 1, 100] != [156, 30, 1, 100]; 1× /usr/local/lib/python3.10/unittest/case.py:675: AssertionError: Lists differ: [516, 1, 156] != [284, 1, 156] |
| glm46v | 1 | 1× /usr/local/lib/python3.10/site-packages/torch/_ops.py:841: RuntimeError: expected scalar type Long but found Double |

View the CircleCI test collection helper for this PR:\n\nhttps://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=9664edac08729aabb154073c9cd52e944b7cbb3d

thanks. will finalize this on my side

View the CircleCI test collection helper for this PR:\n\nhttps://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=215ea77a013c435cfc01194c5b089173cf96ada0

View the CircleCI test collection helper for this PR:\n\nhttps://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=b2f0d8e0d4d0795f463b975be208866ec04fc587

View the CircleCI test collection helper for this PR:\n\nhttps://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=b5fdb80c080328bb4731b25c6ac2211b02c5296b

View the CircleCI test collection helper for this PR:

https://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=37bc7b5e3f30fa71561e0029b2ce50d57b4c4497

i am back to working on this

View the CircleCI test collection helper for this PR:

https://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=f637e2602674fa7252ccf62985363718c25ab9ce

View the CircleCI test collection helper for this PR:

https://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=45cdd2ca6efc3a0d6a5104fa4338ae6c4d28569a

View the CircleCI test collection helper for this PR:

https://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=3c6d3c09ebd7cdddfbda177472be5a287865777d

<!-- circleci-helper-comment -->

⏳ Waiting for CircleCI to complete for commit `8733209e8498`...

https://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=8733209e84981d3d880dab9b1211f7a544aef25e

View the CircleCI test collection helper for this PR:

https://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=3c6d3c09ebd7cdddfbda177472be5a287865777d

View the CircleCI test collection helper for this PR:

https://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=1cf3e2161de942785e3306671d8d7106754746ab

View the CircleCI test collection helper for this PR:

https://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=53c1c41395055ff45fc3e6ea9008b681692e7f88

View the CircleCI test collection helper for this PR:

https://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=21a39beca8d4ae3bfb2d2906fd0cf1c5c2c9cc08

View the CircleCI test collection helper for this PR:

https://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=e3fa6244b205155c8444df0cda06be47d07701e6

View the CircleCI test collection helper for this PR:

https://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=f403e6f2b1c4c3c73595f00dd84f65cb240335c9

View the CircleCI test collection helper for this PR:

https://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=4154f6a55ea1073d7724bd7d09ff2c94a44486ff

View the CircleCI test collection helper for this PR:

https://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=5d1188136705d058d6bddae24177eefa91a941c5

https://huggingface.co/datasets/transformers-community/circleci-test-results/blob/main/pr-42240/sha-04aa4f066590/failure_summary.json for people who want to check! 

🤗 very nice ty @ydshieh for pushing this 🟢 approved

View the CircleCI test collection helper for this PR:

https://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=8ac9c14372d0c83b153d7e93ed780e7d923fa67b

As discussed internally, it's better to merge this PR only after we fix the space issue and making it public space.

will work on that.

View the CircleCI test collection helper for this PR:

https://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=50947fc72be7b642e15510c04c4c0d2e54122352

View the CircleCI test collection helper for this PR:

https://huggingface.co/spaces/transformers-community/circleci-test-collection-helper?repo=huggingface%2Ftransformers&pr=42240&sha=82cede04d37f98d2b0b743455469b1ce35bded2c

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?repo=huggingface%2Ftransformers&pr=42240&sha=fa61784b2d63fa1c6655e51155c326480e58e5cd

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42240&sha=689acf

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: llama, mixtral

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=42240&sha=0ca767