tests: fix pytorch tensor placement errors
Author: dvrogozh
Pull request: yes

This commit fixes the following errors:
* Fix "expected all tensors to be on the same device" error
* Fix "can't convert device type tensor to numpy"
And few other variants of above where model or inputs are not on the right device.

According to pytorch documentation torch.Tensor.numpy(force=False) performs conversion only if tensor is on CPU (plus few other restrictions) which is not the case. For our case we need force=True since we just need a data and don't care about tensors coherency.

Fixes: #33517
See: https://pytorch.org/docs/2.4/generated/torch.Tensor.numpy.html

CC: @sanchit-gandhi, @amyeroberts


Comments:
This ci failure does not seem to be relevant, I did not touch this part of the code:
```
Traceback (most recent call last):
  File "/root/project/examples/pytorch/multiple-choice/run_swag_no_trainer.py", line 703, in <module>
    main()
  File "/root/project/examples/pytorch/multiple-choice/run_swag_no_trainer.py", line 477, in main
    tokenizer, pad_to_multiple_of=(8 if accelerator.use_fp16 else None)
AttributeError: 'Accelerator' object has no attribute 'use_fp16'
```

Hi @dvrogozh, thanks for opening a PR! 

Could you provide some more context around this issue, specifically linking to a related github issue or providing a minimal reproducible code snippet for the error? 



> Could you provide some more context around this issue, specifically linking to a related github issue

@amyeroberts : see:
* #33517.

For the failing examples tests - I believe this has been fixed upstream. Could you rebase on main -- this should make everything green and ready to merge :) 

Rebased. It seems there are still some issues running tests from main branch.
* `tests_torch` fail with on the following 2 tests, I reproduced locally - it fails with and without my PR:
```
FAILED tests/models/dpt/test_modeling_dpt_auto_backbone.py::DPTModelTest::test_attention_outputs - AttributeError: 'NoneType' object has no attribute 'shape'
FAILED tests/models/dpt/test_modeling_dpt_auto_backbone.py::DPTModelTest::test_retain_grad_hidden_states_attentions - AttributeError: 'NoneType' object has no attribute 'retain_grad'
```
* `examples_torch` failure also seems unrelated:
```
E   ImportError: cannot import name 'preserve_channel_dim' from 'albucore.utils' (/usr/local/lib/python3.10/site-packages/albucore/utils.py)
```

> tests_torch fail with on the following 2 tests, I reproduced locally - it fails with and without my PR

Considering that we did not see these tests on my prev. merge base, that's regression. Indeed:
```
$ git bisect log
git bisect start
# bad: [78b2929c0554b79e0489b451ce4ece14d265ead2] Sdpa dino v2 (#33403)
git bisect bad 78b2929c0554b79e0489b451ce4ece14d265ead2
# good: [b50ff5993a5d8b2a3d8c7558e81684f8803b044a] [`Mamba2`] Move dt calculations to kernel (#33520)
git bisect good b50ff5993a5d8b2a3d8c7558e81684f8803b044a
# good: [653eb40425344b89b5a24e7b07eb3095b04cdc9d] Add sdpa for BioGpt (#33592)
git bisect good 653eb40425344b89b5a24e7b07eb3095b04cdc9d
# good: [077b552f0780c678737700184c109066736ece41] Fix some missing tests in circleci (#33559)
git bisect good 077b552f0780c678737700184c109066736ece41
# good: [7b2b536a811c84831e2c67eb388872b7c83a8263] Fix typos (#33583)
git bisect good 7b2b536a811c84831e2c67eb388872b7c83a8263
# good: [e472e077c24d6f6f080f5535f01c48f09164ec62] Granitemoe (#33207)
git bisect good e472e077c24d6f6f080f5535f01c48f09164ec62
# good: [e71bf70e33d501810951f353f1734cb5be74b32a] Pixtral update example checkpoint (#33633)
git bisect good e71bf70e33d501810951f353f1734cb5be74b32a
# first bad commit: [78b2929c0554b79e0489b451ce4ece14d265ead2] Sdpa dino v2 (#33403)
```

First bad commit 78b2929c0554b79e0489b451ce4ece14d265ead2, after merge of:
* https://github.com/huggingface/transformers/pull/33403

Filed:
* https://github.com/huggingface/transformers/issues/33649

to track this. @avishaiElmakies : fyi.

> `examples_torch` failure also seems unrelated:
> `E   ImportError: cannot import name 'preserve_channel_dim' from 'albucore.utils' (/usr/local/lib/python3.10/site-packages/albucore/utils.py)`

Hm. I reproduced this issue on main without my PR. But I could not find a revision in history which works. I still see same error. I wonder, was one of the packages we depend upon updated behind the scenes so this test got affected? Filed https://github.com/huggingface/transformers/issues/33650 to track.


> error running git clone "git@github.com:huggingface/transformers.git": exit status 128

Ok, waiting once this will get resolved on infrastructure side...

@amyeroberts : ci passed now. No changes in PR, just rebasing to get fixes from main branch.

@dvrogozh Great, thanks for updating!