Repository: huggingface/transformers
Pull request: [tests] fix mamba integration simple inference precision issue

## What does this PR do?
This test fails both on CUDA and XPU. 

Root Cause:
The model's output sequence assertion can pass, but the logits assertion fails. This indicates that there might be some numerical precision issues during inference with torch.float16, since the model is loaded with `torch_dtype=torch.float16`. 

To verify my assumption, I changed the torch.float16 to torch.float32. And the tensor outputs on XPU and CUDA are equal to each other. 

For tasks requiring high precision, such as validation, testing or debugging, `torch.float32` should be used instead of `torch.float16`.

Fix: 
temporarily switch to `torch.float32` ensures that the test is not affected by these precision issues and focuses on validating the model's correctness.

After the fix, it works both on CUDA and XPU:

``
PASSED tests/models/mamba/test_modeling_mamba.py::MambaIntegrationTests::test_simple_generate_1_cpu
PASSED tests/models/mamba/test_modeling_mamba.py::MambaIntegrationTests::test_simple_generate_0_cuda
PASSED tests/models/mamba/test_modeling_mamba.py::MambaIntegrationTests::test_simple_generate_1_cpu
PASSED tests/models/mamba/test_modeling_mamba.py::MambaIntegrationTests::test_simple_generate_0_xpu
```



Changed files:
filename: tests/models/mamba/test_modeling_mamba.py
patch: @@ -451,7 +451,7 @@ def test_simple_generate(self, device):
         tokenizer = AutoTokenizer.from_pretrained("state-spaces/mamba-130m-hf")
         tokenizer.pad_token = tokenizer.eos_token
 
-        model = MambaForCausalLM.from_pretrained("state-spaces/mamba-130m-hf", torch_dtype=torch.float16)
+        model = MambaForCausalLM.from_pretrained("state-spaces/mamba-130m-hf", torch_dtype=torch.float32)
         model.to(device)
         input_ids = tokenizer("Hey how are you doing?", return_tensors="pt")["input_ids"].to(device)
 
@@ -464,14 +464,13 @@ def test_simple_generate(self, device):
 
         EXPECTED_LOGITS_NO_GRAD = torch.tensor(
             [
-                -55.6875, -69.8750, -49.9062, -51.7500, -57.6875, -57.9375, -56.9688,
-                -57.9375, -54.6875, -55.9375, -55.3125, -58.0938, -60.5625, -47.0000,
-                -52.0312, -49.7812, -55.9375, -57.9062, -56.7812, -57.1250, -57.3438,
-                -58.3125, -57.8125, -58.7812, -59.6250, -59.0938, -58.7188, -52.9375,
-                -53.4688, -57.3750, -56.9375, -55.7500, -53.3125, -55.8438, -57.0000,
-                -56.9062, -56.2188, -54.7188, -56.4375, -57.5000
-            ]
-        ,dtype=torch.float32)  # fmt: skip
+                -55.6909, -69.7903, -49.8981, -51.7581, -57.6544, -57.9368, -56.9591,
+                -57.9033, -54.6787, -55.9261, -55.3011, -58.0765, -60.5642, -47.0176,
+                -52.0344, -49.7836, -55.9463, -57.8957, -56.7627, -57.1080, -57.3434,
+                -58.3015, -57.7875, -58.7760, -59.6037, -59.0665, -58.7087, -52.9293,
+                -53.4654, -57.3466, -56.9294, -55.7314, -53.3141, -55.8171, -56.9879,
+                -56.9121, -56.2139, -54.7198, -56.4134, -57.4825
+            ])  # fmt: skip
 
         torch.testing.assert_close(logits[0, 0, :40].cpu(), EXPECTED_LOGITS_NO_GRAD, rtol=1e-3, atol=1e-3)