Repository: pytorch
Issue: Problem with instalation torch2 on a100+cu12.1
Labels: module: binaries,module: cuda,triaged

### 🐛 Describe the bug

I am trying to install torch2 with cu118 on cu121 machine

Using
pip install --upgrade torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

Trying 

import torch
print(torch.__version__)
torch.zeros(2).cuda(0)

And got this:

CUDA call failed lazily at initialization with error: device >= 0 && device < num_gpus INTERNAL ASSERT FAILED at "../aten/src/ATen/cuda/CUDAContext.cpp":50, please report a bug to PyTorch. 

CUDA call was originally invoked at:

['  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main\n    return _run_code(code, main_globals, None,\n', '  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code\n    exec(code, run_globals)\n', '  File "/home/fsuser/.local/lib/python3.8/site-packages/ipykernel_launcher.py", line 17, in <module>\n    app.launch_new_instance()\n', '  File "/home/fsuser/.local/lib/python3.8/site-packages/traitlets/config/application.py", line 1043, in launch_instance\n    app.start()\n', '  File "/home/fsuser/.local/lib/python3.8/site-packages/ipykernel/kernelapp.py", line 725, in start\n    self.io_loop.start()\n', '  File "/home/fsuser/.local/lib/python3.8/site-packages/tornado/platform/asyncio.py", line 215, in start\n    self.asyncio_loop.run_forever()\n', '  File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever\n    self._run_once()\n', '  File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once\n    handle._run()\n', '  File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run\n    self._context.run(self._callback, *self._args)\n', '  File "/home/fsuser/.local/lib/python3.8/site-packages/ipykernel/kernelbase.py", line 513, in dispatch_queue\n    await self.process_one()\n', '  File "/home/fsuser/.local/lib/python3.8/site-packages/ipykernel/kernelbase.py", line 502, in process_one\n    await dispatch(*args)\n', '  File "/home/fsuser/.local/lib/python3.8/site-packages/ipykernel/kernelbase.py", line 409, in dispatch_shell\n    await result\n', '  File "/home/fsuser/.local/lib/python3.8/site-packages/ipykernel/kernelbase.py", line 729, in execute_request\n    reply_content = await reply_content\n', '  File "/home/fsuser/.local/lib/python3.8/site-packages/ipykernel/ipkernel.py", line 422, in do_execute\n    res = shell.run_cell(\n', '  File "/home/fsuser/.local/lib/python3.8/site-packages/ipykernel/zmqshell.py", line 540, in run_cell\n    return super().run_cell(*args, **kwargs)\n', '  File "/home/fsuser/.local/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3006, in run_cell\n    result = self._run_cell(\n', '  File "/home/fsuser/.local/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3061, in _run_cell\n    result = runner(coro)\n', '  File "/home/fsuser/.local/lib/python3.8/site-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner\n    coro.send(None)\n', '  File "/home/fsuser/.local/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3266, in run_cell_async\n    has_raised = await self.run_ast_nodes(code_ast.body, cell_name,\n', '  File "/home/fsuser/.local/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3445, in run_ast_nodes\n    if await self.run_code(code, result, async_=asy):\n', '  File "/home/fsuser/.local/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3505, in run_code\n    exec(code_obj, self.user_global_ns, self.user_ns)\n', '  File "/tmp/ipykernel_15415/129772968.py", line 1, in <module>\n    import torch\n', '  File "<frozen importlib._bootstrap>", line 991, in _find_and_load\n', '  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked\n', '  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked\n', '  File "<frozen importlib._bootstrap_external>", line 848, in exec_module\n', '  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed\n', '  File "/home/fsuser/.local/lib/python3.8/site-packages/torch/__init__.py", line 1146, in <module>\n    _C._initExtension(manager_path())\n', '  File "<frozen importlib._bootstrap>", line 991, in _find_and_load\n', '  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked\n', '  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked\n', '  File "<frozen importlib._bootstrap_external>", line 848, in exec_module\n', '  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed\n', '  File "/home/fsuser/.local/lib/python3.8/site-packages/torch/cuda/__init__.py", line 197, in <module>\n    _lazy_call(_check_capability)\n', '  File "/home/fsuser/.local/lib/python3.8/site-packages/torch/cuda/__init__.py", line 195, in _lazy_call\n    _queued_calls.append((callable, traceback.format_stack()))\n']


### Versions

2.0.0+cu118

cc @seemethere @malfet @ngimel