Repository: pytorch
Issue: RuntimeError: invalid unordered_map<K, T> key
Labels: needs reproduction,module: windows,triaged

### 🐛 Describe the bug

# Bug Report

**Error description:**

```
MViT(
  (patch_embed): PatchEmbed(
    (proj): Conv3d(3, 96, kernel_size=(3, 7, 7), stride=(2, 4, 4), padding=(1, 3, 3))
  )
  (blocks): ModuleList(
    (0): MultiScaleBlock(
      (norm1): LayerNorm((96,), eps=1e-06, elementwise_affine=True)
      (attn): MultiScaleAttention(
        (qkv): Linear(in_features=96, out_features=288, bias=True)
        (proj): Linear(in_features=96, out_features=96, bias=True)
        (pool_q): Conv3d(96, 96, kernel_size=(3, 3, 3), stride=(1, 1, 1), padding=(1, 1, 1), groups=96, bias=False)
        (norm_q): LayerNorm((96,), eps=1e-06, elementwise_affine=True)
        (pool_k): Conv3d(96, 96, kernel_size=(3, 3, 3), stride=(1, 8, 8), padding=(1, 1, 1), groups=96, bias=False)
        (norm_k): LayerNorm((96,), eps=1e-06, elementwise_affine=True)
        (pool_v): Conv3d(96, 96, kernel_size=(3, 3, 3), stride=(1, 8, 8), padding=(1, 1, 1), groups=96, bias=False)
        (norm_v): LayerNorm((96,), eps=1e-06, elementwise_affine=True)............................
        ....... (dropout): Dropout(p=0.5, inplace=False)
    (projection): Linear(in_features=768, out_features=400, bias=True)
    (act): Softmax(dim=1)
  )
)
**Got Model architecture as intermediate part of output, truncated some parts**

Traceback (most recent call last):
  File "C:\Users\P.Vijay Srinivasan\AppData\Local\Programs\Python\Python310\lib\pdb.py", line 1726, in main
    pdb._runscript(mainpyfile)
  File "C:\Users\P.Vijay Srinivasan\AppData\Local\Programs\Python\Python310\lib\pdb.py", line 1586, in _runscript
    self.run(statement)
  File "C:\Users\P.Vijay Srinivasan\AppData\Local\Programs\Python\Python310\lib\bdb.py", line 597, in run
    exec(cmd, globals, locals)
  File "<string>", line 1, in <module>
  File "c:\users\p.vijay srinivasan\downloads\nyx wolves\shoplifting_2\slowfast\tools\run_net.py", line 55, in <module>
    main()
  File "c:\users\p.vijay srinivasan\downloads\nyx wolves\shoplifting_2\slowfast\tools\run_net.py", line 28, in main
    launch_job(cfg=cfg, init_method=args.init_method, func=train)
  File "C:\Users\P.Vijay Srinivasan\Downloads\NYX Wolves\shoplifting_2\SlowFast\slowfast\utils\misc.py", line 430, in launch_job
    func(cfg=cfg)
  File "C:\Users\P.Vijay Srinivasan\Downloads\NYX Wolves\shoplifting_2\SlowFast\tools\train_net.py", line 543, in train
    flops, params = misc.log_model_info(model, cfg, use_train_input=True)
  File "C:\Users\P.Vijay Srinivasan\Downloads\NYX Wolves\shoplifting_2\SlowFast\slowfast\utils\misc.py", line 189, in log_model_info
    flops = get_model_stats(model, cfg, "flop", use_train_input)
  File "C:\Users\P.Vijay Srinivasan\Downloads\NYX Wolves\shoplifting_2\SlowFast\slowfast\utils\misc.py", line 168, in get_model_stats
    count_dict, *_ = model_stats_fun(model, inputs)
  File "C:\Users\P.Vijay Srinivasan\AppData\Local\Programs\Python\Python310\lib\site-packages\fvcore\nn\flop_count.py", line 147, in flop_count
    for op, flop in flop_counter.by_operator().items():
  File "C:\Users\P.Vijay Srinivasan\AppData\Local\Programs\Python\Python310\lib\site-packages\fvcore\nn\jit_analysis.py", line 265, in by_operator
    stats = self._analyze()
  File "C:\Users\P.Vijay Srinivasan\AppData\Local\Programs\Python\Python310\lib\site-packages\fvcore\nn\jit_analysis.py", line 551, in _analyze
    graph = _get_scoped_trace_graph(self._model, self._inputs, self._aliases)
  File "C:\Users\P.Vijay Srinivasan\AppData\Local\Programs\Python\Python310\lib\site-packages\fvcore\nn\jit_analysis.py", line 176, in _get_scoped_trace_graph
    graph, _ = _get_trace_graph(module, inputs)
  File "C:\Users\P.Vijay Srinivasan\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\jit\_trace.py", line 1285, in _get_trace_graph
    outs = ONNXTracedModule(
  File "C:\Users\P.Vijay Srinivasan\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\P.Vijay Srinivasan\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\P.Vijay Srinivasan\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\jit\_trace.py", line 133, in forward
    graph, out = torch._C._create_graph_by_tracing(
  File "C:\Users\P.Vijay Srinivasan\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\jit\_trace.py", line 124, in wrapper
    outs.append(self.inner(*trace_inputs))
  File "C:\Users\P.Vijay Srinivasan\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\P.Vijay Srinivasan\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1568, in _call_impl
    result = forward_call(*args, **kwargs)
  File "C:\Users\P.Vijay Srinivasan\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1508, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "C:\Users\P.Vijay Srinivasan\Downloads\NYX Wolves\shoplifting_2\SlowFast\slowfast\models\video_model_builder.py", line 1215, in forward
    x, bcthw = self.patch_embed(x)
  File "C:\Users\P.Vijay Srinivasan\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\P.Vijay Srinivasan\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1568, in _call_impl
    result = forward_call(*args, **kwargs)
  File "C:\Users\P.Vijay Srinivasan\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\module.py", line 1508, in _slow_forward
    result = self.forward(*input, **kwargs)
  File "C:\Users\P.Vijay Srinivasan\AppData\Local\Programs\Python\Python310\lib\site-packages\fairscale\nn\checkpoint\checkpoint_activations.py", line 191, in _checkpointed_forward
    output = CheckpointFunction.apply(
  File "C:\Users\P.Vijay Srinivasan\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\autograd\function.py", line 539, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
RuntimeError: invalid unordered_map<K, T> key
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> c:\users\p.vijay srinivasan\appdata\local\programs\python\python310\lib\site-packages\torch\autograd\function.py(539)apply()
-> return super().apply(*args, **kwargs)  # type: ignore[misc]

**RuntimeError: invalid unordered_map<K, T> key**
```


**- Code**
[run_net.py-](https://github.com/facebookresearch/SlowFast/blob/main/tools/run_net.py)

### Versions

Python platform: Windows-10-10.0.19045-SP0
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture=9
CurrentClockSpeed=2100
DeviceID=CPU0
Family=107
L2CacheSize=2048
L2CacheSpeed=
Manufacturer=AuthenticAMD
MaxClockSpeed=2100
Name=AMD Ryzen 5 3500U with Radeon Vega Mobile Gfx
ProcessorType=3
Revision=6145

Versions of relevant libraries:
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.23.5
[pip3] onnx==1.15.0
[pip3] onnxruntime==1.16.3
[pip3] onnxscript==0.1.0.dev20240119
[pip3] pytorchvideo==0.1.5
[pip3] torch==2.1.2
[pip3] torchaudio==2.1.2
[pip3] torchvision==0.16.2
[pip3] tritonclient==2.41.0

cc @peterjc123 @mszhanyi @skyline75489 @nbcsm @vladimir-aubrecht @iremyux @Blackhex @cristianPanaite