gopikrsmscs/torch-issues huggingface dataset:
Barriers to using torch.compile directly in PyTorch library code
Labels: triaged, oncall: pt2, module: inductor, module: dynamo

### 🐛 Describe the bug

There are many places in the PyTorch eager API where we could conceivably write a kernel for something, but we it's annoying / time consuming to do so / increases our binary size. 

For example, in https://github.com/pytorch/pytorch/pull/101115 we don't have a deterministic implementation of upsample_bilinear backwards and implementing one would be pretty annoying (the most plausible approach is to redo the CUDA kernel with the iteration space on grad_input rather than grad_output (which necessitates atomic adds). However, we could instead use torch.compile to compile us an implementation of the backwards! Assuming that torch.compile has a deterministic implementation of scatter (it doesn't, but we could conceivably do it by going to eager's deterministic implementation.)

So ideally, you'd like to modify code in PyTorch's Python frontend to look like this:

```
    if input.dim() == 4 and mode == "bilinear":
        assert align_corners is not None
        if antialias:
            return torch._C._nn._upsample_bilinear2d_aa(input, output_size, align_corners, scale_factors)
        if torch.are_deterministic_algorithms_enabled() and input.is_cuda:
            return torch.compile(torch._C._nn.upsample_bilinear2d, dynamic=True)(input, output_size, align_corners, scale_factors)
        else:
            return torch._C._nn.upsample_bilinear2d(input, output_size, align_corners, scale_factors)
```

However, there are a few problems to doing this:

1. This breaks torch.compile! If you modify a function which is allow_in_graph'ed, during Dynamo execution we will send fake tensors through the arguments, but if you recursively hit torch.compile it will choke as it doesn't know how to deal with fake tensors.
2. The warmup time is bad! It takes a long time to compile, and if you hit the function at multiple sizes we will compile a few times before quiescing dynamic=True. https://github.com/pytorch/pytorch/pull/101027 could potentially help here.
3. It goops up observability! Most users of our logging and debug tools assume that they will only get logs for their actual model, and no PT2 logs when they are running stock eager code. This breaks this assumption, since random eager code can trigger compile. (I also ran into this with https://github.com/pytorch/pytorch/pull/99809)
4. Performance might be bad! These are small graphs and guard overhead is a lot more notable in these situations.

What I am kind of imagining is that we need some alternative frontend to torch.compile which is specialized for "single function optimize me" use case. This frontend can be export oriented; e.g., we won't recompile unless you explicitly bump a version number or something and should aggressively cache its compile products to disk.

cc @msaroufim @wconstab @ngimel @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @soumith @Chillee 

### Versions

master
Tensorboard graph tracing with torch fx API
Labels: triaged, module: tensorboard, module: fx

### 🚀 The feature, motivation and pitch

The current implementation of adding a model graph to tensorboard is not perfect:
- A brittle heuristic is used to assemble the scopes, in- and outputs of Modules and operations in the graph, based on strings extracted from the trace
- This leads to [silently](https://github.com/pytorch/pytorch/issues/33691) [wrong](https://github.com/pytorch/pytorch/issues/65652), [confusing](https://github.com/pytorch/pytorch/issues/39292) or [useless](https://github.com/pytorch/pytorch/issues/37387) graphs sometimes
- sometimes operations have duplicate inputs / outputs in the graph
- operations are labeled with a meaningless number instead of the name of the operation 
- in general, pytorch model graphs are just subpar compared to tensorflow models
- [The graphing breaks completely with zero-dim tensors](https://github.com/pytorch/pytorch/issues/88707)

🚀 My idea / feature request is now to build the tensorboard graph from a symbolic trace using the newer `torch.fx` API.
As I understand it, it contains all the necessary information and good scopes that help correct nesting of modules in the graph.
Graphing a general `fx` IR to a tensorboard may also be an excellent way of visually debugging `fx` transformations.

This maybe can also help with this issue, where some users want a model graph without having to provide concrete inputs values: #37334.

### Alternatives

- Continue using the status quo with the buggy parsing of torch jit traces
- Currently, the trace parsing discards `ClassType` nodes. Investigate if including information from these nodes may help create more accurate graphs

### Additional context

See some of the linked issues for images where the tensorboard graph is not satisfactory. 
The code for parsing the jit trace is here: https://github.com/pytorch/pytorch/blob/e9ebda29d87ce0916ab08c06ab26fd3766a870e5/torch/utils/tensorboard/_pytorch_graph.py#L236

cc @ezyang @SherlockNoMad @soumith @EikanWang @jgong5 @wenzhe-nrv
addmv doesn't do type promotion correctly,
Labels: triaged, module: type promotion, module: linear algebra

Google colab test:

Input:
```
input = torch.rand([3], dtype=torch.complex128, requires_grad=True)
vec1 = torch.rand([3, 2], dtype=torch.float64, requires_grad=True)
vec2 = torch.rand([2], dtype=torch.float64, requires_grad=True)

res = torch.Tensor.addmv(input, vec1, vec2)
desired_dtype = torch.promote_types(torch.complex128, torch.float64)
print(res.dtype)
print(desired_dtype)
```

Output:
```
torch.float64
torch.complex128
<ipython-input-3-e284f66968e0>:5: UserWarning: Casting complex values to real discards the imaginary part (Triggered internally at ../aten/src/ATen/native/Copy.cpp:276.)
  res = torch.Tensor.addmv(input, vec1, vec2)
```

_Originally posted by @gerhean in https://github.com/pytorch/pytorch/issues/99984#issuecomment-1542566779_
            

cc @nairbv @mruberry @jianyuh @nikitaved @pearu @walterddr @IvanYashchuk @xwang233 @Lezcano
[BE] Refactor logic for MultiTensorApply
Labels: triaged, better-engineering, actionable, module: mta

### 🚀 The feature, motivation and pitch

Context: https://github.com/pytorch/pytorch/pull/100811#discussion_r1187576042

It would be a valuable next step to refactor the multitensorapply code to allow for better engineering in terms of clarity. The code currently is convoluted and we should break up the for-loop into separate preprocessing and kernel call steps. Desirable outcomes:
- allow people to more easily understand what is going on
- lessen the chance that bugs will be introduced accidentally

### Alternatives

do nothing

### Additional context

_No response_

cc @crcrpar @mcarilli @ngimel
Sequence annotation in type hints is wrong
Labels: module: typing, triaged

### 🐛 Describe the bug

We have generated .pyi files that specify annotation of SymInt[], float[], and possibly other types as Sequence[T]:
```
// torch.ones in torch/_VF.pyi
def ones(size: Sequence[Union[_int, SymInt]], *, out: Optional[Tensor] = None, dtype: Op
tional[_dtype] = None, layout: Optional[_layout] = None, device: Optional[Union[_device,
 str, None]] = None, pin_memory: Optional[_bool] = False, requires_grad: Optional[_bool]
 = False) -> Tensor: ...
```
This is incorrect, because we only support Tuple and List via python_arg_parser (and Sequence represents anything that subscribes to the Sequence API, like ranges).

Some options are:
- we correct these annotations (probably the easiest option)
- we support PySequence in python_arg_parser (https://docs.python.org/3.8/c-api/sequence.html)

### Versions

main

cc @ezyang @malfet @rgommers @xuzhao9 @gramster
Lazily format C++ stack trace if it is not used
Labels: module: performance, triaged

### 🐛 Describe the bug

Meta-only context: https://fb.workplace.com/groups/1405155842844877/?multi_permalinks=6965759763451096&hoisted_section_header_type=recently_seen

We can speed up stack trace collection by only formatting it when we actually need to print it. To do this, we can use @zdevito's new stack trace code described at https://dev-discuss.pytorch.org/t/fast-combined-c-python-torchscript-inductor-tracebacks/1158/3

This should speed up how quickly our test suite runs.

### Versions

master

cc @ngimel
torch.autograd.detect_anomaly should report the original forward trace as part of the error, rather than as out of band warning
Labels: module: autograd, triaged, actionable

### 🐛 Describe the bug

Today, when an exception is raised from a backward function, and anomaly mode was enabled, we will emit a warning saying what forward caused the exception:

```
/data/users/ezyang/a/pytorch/torch/autograd/__init__.py:319: UserWarning: Error detected in SumBackward0. Traceback of forward call that caused the error:                                                                                        
  File "/data/users/ezyang/a/pytorch/test/dynamo/test_misc.py", line 4872, in f                                          
    return h(a)                                                                                                          
 (Triggered internally at /data/users/ezyang/a/pytorch/torch/csrc/autograd/python_anomaly_mode.cpp:119.)                 
  result = Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
```

There should be a way to force this information to instead get attached to the exception being thrown. This is important to do as high level error reporting frameworks will typically surface the exception message to the user, but not necessarily warnings (which have to be separately extracted from logs.)

### Versions

master

cc @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7
Tensor __getitem__ not documented, sparse grad?
Labels: module: sparse, triaged, module: advanced indexing

### 📚 The doc issue

I was search for some documentation on `Tensor.__getitem__` but I don't find any, at least not [here](https://pytorch.org/docs/stable/tensors.html#torch.Tensor). Maybe it is elsewhere?

Esp, I wonder if it uses `torch.gather` internally when you give it some scalar index.

And I wonder if it uses `sparse_grad` for `torch.gather` or not.

I also wonder, why does `torch.gather` has such a `sparse_grad` option at all? Why is it not always sparse? Why is the default to have it dense? What are the downsides in always enabling this? The [`torch.gather` documentation](https://pytorch.org/docs/stable/generated/torch.gather.html) doesn't really say anything about this.

If `Tensor.__getitem__` does not use `torch.gather`, is the gradient sparse or not?

E.g. when I have a tensor `x` of shape [T, B, F], and I loop over T, would it be a good idea to have this code (when training)?
```python
for t in range(T):
   y = f(x[t])
   ...
```
If `__getitem__` is not using sparse gradients, wouldn't it be better to have sparse gradients in this case? Wouldn't it always be better to have sparse gradients?


### Suggest a potential alternative/fix

Add some documentation on `Tensor.__getitem__`.

Extend the documentation of `torch.gather` `sparse_grad` option.

Briefly address my questions here.


cc @alexsamardzic @nikitaved @pearu @cpuhrsch @amjames @bhosmer
[Placeholder] PyTorch 2.0 Dynamo/Inductor Hack{day/week}
Labels: triaged, oncall: pt2, module: dynamo

We will have a list of graph breaks with the minimized repro for the hack{day/week}.
(Assigned task to @yanboliang temporarily before we start the hackday)

## Dynamo Graph Breaks ##
* [x] [Easy] https://github.com/pytorch/pytorch/issues/101154
* [x] [Easy] https://github.com/pytorch/pytorch/issues/101155
* [x] [Easy] https://github.com/pytorch/pytorch/issues/91662
* [ ] [Medium] https://github.com/pytorch/pytorch/issues/97115
* [x] [Easy] https://github.com/pytorch/pytorch/issues/102338
* [x] [Easy] https://github.com/pytorch/pytorch/issues/102340
* [x] [Easy] https://github.com/pytorch/pytorch/issues/101980
* [x] [Easy] https://github.com/pytorch/pytorch/issues/102877
* [x] [Easy] https://github.com/pytorch/pytorch/issues/102878
* [x] [Easy] https://github.com/pytorch/pytorch/issues/102879
* [x] [Easy] https://github.com/pytorch/pytorch/issues/103125

## Other Dynamo Issues ##
* [x] [Easy] https://github.com/pytorch/pytorch/issues/102053
* [x] [Easy] https://github.com/pytorch/pytorch/issues/100854
* [x] [Medium] https://github.com/pytorch/pytorch/issues/99639
* [x] [Medium] https://github.com/pytorch/pytorch/issues/98973
* [x] [Medium] https://github.com/pytorch/pytorch/issues/99569
  + [ ] [Medium] ^^ followup to the above issue, which was worked-around with a graph break but underlying AOT Autograd + nn.Parameter issue still exists
* [ ] [Easy] https://github.com/pytorch/pytorch/issues/99752
* [x] [Easy] https://github.com/pytorch/pytorch/issues/93340
* [x] [Easy] https://github.com/pytorch/pytorch/issues/103139

## Newly added ones during hackaday
* [ ] From Avik - https://www.internalfb.com/phabricator/paste/view/P763199946
* [ ] From Zhengxu - https://github.com/pytorch/pytorch/issues/103210


## Other PT2 stack issues
- [ ] Debuggability + Inductor: https://github.com/pytorch/pytorch/issues/97778
- [ ] [Medium] https://github.com/pytorch/pytorch/issues/101832
- [ ] Model benchmarks - https://github.com/pytorch/pytorch/issues/96759
- [x] Make explain pretty - https://github.com/pytorch/pytorch/pull/102869


## Hard ones
- [ ] AOT Autograd - https://github.com/pytorch/pytorch/issues/96456
- [ ] [Hard] https://github.com/pytorch/pytorch/issues/100386
- [ ] [Hard] [Vague] https://github.com/pytorch/pytorch/issues/102063
ONNX TorchDynamo Exporter  - Ability to export and load ONNX files without parameters
Labels: module: onnx, triaged, oncall: pt2

## 🚀 Feature Request

### Motivation 

I'm working on a large ML benchmarking effort that heavily utilizes ONNX files. This generates hundreds to thousands of ONNX models that we can't store economically. Most of the size of the models comes from the parameters, which are not essential for applications such as benchmarking (on certain accelerators).

### Feature Description

A feature that I would love to see as part of the new [TorchDynamo Exporter](https://pytorch.org/docs/main/onnx.html#preview-torch-onnx-torchdynamo-exporter) is the ability to:
* Export/Save models without storing parameters, significantly reducing file size, and
* Load models exported/saved without parameters by generating random parameters.



cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @anijain2305
Extending compatibility of LibTorch
Labels: module: cpp, feature, triaged, needs design

### 🚀 The feature, motivation and pitch

I am noticing that the current C-API is limited to modern C++ standards. I am working on a code that assumes a pure C90 standard, and am not able to compile it with LibTorch functionality since the C-API is too modern. 

Would it be possible to LibTorch functional with historical pure C standards? 

### Alternatives

_No response_

### Additional context

_No response_

cc @jbschlosser
PyTorch installs the file mkldnn.cmake that looks for the package MKLDNN that doesn't exist
Labels: module: build, triaged, module: mkldnn

### 🐛 Describe the bug

/usr/local/lib/python3.9/site-packages/torch/share/cmake/Caffe2/public/mkldnn.cmake looks for ```MKLDNN```.
There is a GitHub project https://github.com/Intel-tensorflow/mkl-dnn that was last updated in 2019. Its website though redirects to oneDNN.

As a result, the DGL project (https://github.com/dmlc/dgl) fails to build:
```
CMake Error at /usr/local/lib/python3.9/site-packages/torch/share/cmake/Caffe2/public/mkldnn.cmake:7 (find_package):
  By not providing "FindMKLDNN.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "MKLDNN", but
  CMake did not find one.

  Could not find a package configuration file provided by "MKLDNN" with any
  of the following names:

    MKLDNNConfig.cmake
    mkldnn-config.cmake

  Add the installation prefix of "MKLDNN" to CMAKE_PREFIX_PATH or set
  "MKLDNN_DIR" to a directory containing one of the above files.  If "MKLDNN"
  provides a separate development package or SDK, be sure it has been
  installed.
Call Stack (most recent call first):
  /usr/local/lib/python3.9/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:106 (include)
  /usr/local/lib/python3.9/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
  CMakeLists.txt:29 (find_package)
```


### Versions

PyTorch-2.0.0
FreeBSD 13.2

cc @malfet @seemethere @gujinghui @PenghuiCheng @XiaobingSuper @jianyuh @jgong5 @mingfeima @sanchitintel @ashokei @jingxu10 @min-jean-cho @yanbing-j @Guobing-Chen @Xia-Weiwen
DISABLED test_vmapjvpvjp_linalg_lu_factor_ex_cuda_float32 (__main__.TestOperatorsCUDA)
Labels: triaged, module: flaky-tests, skipped, module: functorch

Platforms: linux

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_vmapjvpvjp_linalg_lu_factor_ex_cuda_float32&suite=TestOperatorsCUDA) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/undefined).

Over the past 3 hours, it has been determined flaky in 2 workflow(s) with 2 failures and 2 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_vmapjvpvjp_linalg_lu_factor_ex_cuda_float32`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `functorch/test_ops.py` or `functorch/test_ops.py`

cc @zou3519 @Chillee @samdow @soumith @kshitij12345 @janeyx99
[MPS] Track failures of test_module.py for MPS backend
Labels: triaged, module: backend, module: mps

### 🐛 Describe the bug

This issue tracks the failures in test_modules.py for the MPS backend.

- BatchNorm1d/BatchNorm2d: fails due to unsupported float64 passed to MPS's batch_norm implementation with error:
`TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.`

- These tests fail due to functional issues (mismatching elements):
1. SELU
2. Softplus 
3. PReLU 
4. ReLU6 
5. Threshold

I'll update this issue whenever each of the above failures get fixed.
These issues were discovered in the PR #95334

### Versions

PyTorch version: 2.1.0a0+gite975f83
Is debug build: True

CMake version: version 3.26.3

Python version: 3.9.16 (main, Mar  8 2023, 04:29:24)  [Clang 14.0.6 ] (64-bit runtime)

CPU:
Apple processor

cc @bdhirsh @kulinseth @albanD @malfet @DenisVieriu97 @abhudev
Revise glossary
Labels: module: docs, triaged

### 📚 The doc issue

The glossary provided by #40640 could use some attention

- [ ] revise existing terms
- [ ] create new glossary entries (and links) for various sections of the docs
     - [ ] compiler terms to augment the existing JIT terms
     - [ ] generator
     - [ ] torchrun
     - [ ] torchscript
     - [ ] backends
     - [ ] autograd 
... 
pls post a list of suggestions 

### Suggest a potential alternative/fix

_No response_

cc @svekars @carljparker
MPS backend is not supported on MacOS 12.6.3
Labels: needs reproduction, triaged, module: mps

### 🐛 Describe the bug

ERROR: test_output_match_zero__cpu_uint8 (__main__.TestConsistencyCPU)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/ec2-user/pytorch/torch/testing/_internal/common_device_type.py", line 414, in instantiated_test
    raise rte
  File "/Users/ec2-user/pytorch/torch/testing/_internal/common_device_type.py", line 401, in instantiated_test
    result = test(self, **param_kwargs)
  File "/Users/ec2-user/pytorch/torch/testing/_internal/common_device_type.py", line 846, in test_wrapper
    return test(*args, **kwargs)
  File "/Users/ec2-user/pytorch/test/test_mps.py", line 10350, in test_output_match
    mps_sample = cpu_sample.transform(
  File "/Users/ec2-user/pytorch/torch/testing/_internal/opinfo/core.py", line 264, in transform
    tt(self.input),
  File "/Users/ec2-user/pytorch/torch/testing/_internal/opinfo/core.py", line 251, in tt
    return _tt(t)
  File "/Users/ec2-user/pytorch/torch/testing/_internal/opinfo/core.py", line 248, in _tt
    return f(t)
  File "/Users/ec2-user/pytorch/test/test_mps.py", line 10351, in <lambda>
    lambda x: x.detach().to("mps").requires_grad_(x.requires_grad) if isinstance(x, torch.Tensor) else x)
RuntimeError: The MPS backend is supported on MacOS 12.3+.Current OS version can be queried using `sw_vers`

----------------------------------------------------------------------
Ran 4506 tests in 57.884s

FAILED (errors=2687, skipped=159, expected failures=1561)
(pt) ec2-user@ip-172-31-44-228 test %
(pt) ec2-user@ip-172-31-44-228 test % sw_vers
ProductName:    macOS
ProductVersion: 12.6.3
BuildVersion:   21G419


### Versions

Collecting environment information...
PyTorch version: 2.1.0a0+gita3989b2
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.6.3 (x86_64)
GCC version: Could not collect
Clang version: 14.0.0 (clang-1400.0.29.202)
CMake version: version 3.22.1
Libc version: N/A

Python version: 3.9.16 (main, Mar  8 2023, 04:29:44)  [Clang 14.0.6 ] (64-bit runtime)
Python platform: macOS-10.16-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Intel(R) Core(TM) i7-8700B CPU @ 3.20GHz

Versions of relevant libraries:
[pip3] numpy==1.24.3
[pip3] torch==2.1.0a0+gita3989b2
[conda] mkl                       2023.1.0         h59209a4_43558
[conda] mkl-include               2023.1.0         hecd8cb5_43558
[conda] numpy                     1.24.3                   pypi_0    pypi
[conda] torch                     2.1.0a0+gita3989b2           dev_0    <develop>


cc @frank-wei @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev
onnx.export fails if do_constant_folding=False
Labels: module: onnx, triaged

### 🐛 Describe the bug

I am currently using the code below to convert a diffusion unet model to ONNX format. Noticed the unet model is larger than 2GB, which exceeds protobuf limitation, so the model weights will be exported as separate files.

```python
import onnx
import torch
from diffusers import UNet2DConditionModel

unet = UNet2DConditionModel.from_pretrained("CompVis/stable-diffusion-v1-4",
                                            torch_dtype=torch.float16,
                                            revision="fp16",
                                            subfolder="unet")
unet.cuda()
# unet.float()

with torch.inference_mode(), torch.autocast("cuda"):
    inputs = torch.randn(2,4,64,64, dtype=torch.float16, device='cuda'), torch.randn(1, dtype=torch.float16, device='cuda'), torch.randn(2, 77, 768, dtype=torch.float16, device='cuda')

    # Export the model
    torch.onnx.export(unet,
                    inputs,
                    "unet.onnx",
                    opset_version=14,
                    do_constant_folding=False,  # whether to execute constant folding for optimization
                    # do_constant_folding=True, 
                    input_names = ['input_0', 'input_1', 'input_2'],
                    output_names = ['output_0'])
```

The code works as expected when using either float32 precision or the "do_constant_folding=True" flag. However, when attempting to export the model using float16 version and "do_constant_folding=False", the code produces a libprotobuf error.

```
libprotobuf ERROR ../third_party/protobuf/src/google/protobuf/message_lite.cc:457] onnx_torch.ModelProto exceeded maximum protobuf size of 2GB: 2891617469
```

### Versions

1.12.1+cu113
logger instead of print in lr_scheduler.py
Labels: module: optimizer, module: logging, triaged, actionable, module: LrScheduler

### 🚀 The feature, motivation and pitch

In lr_scheduler.py, there are a couple calls to print().  For example, `ReduceLROnPlateau._reduce_lr` calls `print`.  If it called a logger instead, the logs could be turned off, or at least would show up in python logging streams as part of stderr.

### Alternatives

_No response_

### Additional context

_No response_

cc @vincentqb @jbschlosser @albanD @janeyx99 @crcrpar
Accuracy issues with Jitterated complex kernels for acos, acosh, asin, asinh, tan and tanh
Labels: module: cuda, good first issue, triaged, module: jiterator

## Issue description

The following complex kernels cause CI failures when implemented with `jitted_gpu_kernel`. 

- [ ] [acos](https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/native/cuda/UnaryGeometricAcosKernel.cu)
- [ ] [acosh](https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/native/cuda/UnaryGeometricAcoshKernel.cu)
- [ ] [asin](https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/native/cuda/UnaryGeometricAsinKernel.cu)
- [ ] [asinh](https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/native/cuda/UnaryGeometricAsinhKernel.cu)
- [ ] [tan](https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/native/cuda/UnaryGeometricTanKernel.cu)
- [ ] [tanh](https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/native/cuda/UnaryGeometricTanhKernel.cu)

Some may simply require a small tolerance increase to the tests but for larger differences we need to figure out which implementation is correct, thrust or jitterator. 

Note that `gpu_kernel` implemenations use `thrust` while `jitted_gpu_kernel` calls the  complex math implementations we vendor from `libc++`. These can be found in [`llvm_complex.cpp`](https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/cuda/llvm_complex.cpp).



cc @ngimel @mruberry @ezyang @soumith @msaroufim @wconstab @bdhirsh @anijain2305
add new private operator copy-on-write torch._lazy_clone
Labels: module: internals, triaged, open source, ciflow/trunk, topic: not user facing, ciflow/mps, ciflow/inductor, no-stale

add new private operator copy-on-write torch._lazy_clone

Summary:
This is an early access operator with some limitations. It will more
eagerly copy than desired because we haven't completed our audit
partitioning data pointer access between const_data_ptr and
mutable_data_ptr.

The idea is to add some logging when a copy is triggered to help
prioritize which operations will be audited and fixed.

Test Plan: Added a new unit test.

---
Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/pytorch/pytorch/pull/100821).
* __->__ #100821
* #100820

cc @ezyang @bhosmer @smessmer @ljk53 @bdhirsh
implement a function to materialize a copy-on-write storage
Labels: module: internals, triaged, open source, ciflow/trunk, topic: not user facing, ciflow/mps

implement a function to materialize a copy-on-write storage

Summary:
This is triggered if the storage is copy-on-write whenever mutable
access to a DataPtr is given.

Test Plan: 100% code coverage

---
Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/pytorch/pytorch/pull/100820).
* #100821
* __->__ #100820

cc @ezyang @bhosmer @smessmer @ljk53 @bdhirsh
Wrong type for `get_lr` inside lr_scheduler.pyi
Labels: module: optimizer, module: typing, triaged, actionable, module: LrScheduler

### 🐛 Describe the bug

`lr_scheduler.pyi` describes `def get_lr(self) -> float: ...` but the implementation returns `List[float]`

https://github.com/pytorch/pytorch/blob/31f311a816c026bbfca622d6121d6a7fab44260d/torch/optim/lr_scheduler.pyi#L19
https://github.com/pytorch/pytorch/blob/31f311a816c026bbfca622d6121d6a7fab44260d/torch/optim/lr_scheduler.py#L394

### Versions

Current master branch

cc @vincentqb @jbschlosser @albanD @janeyx99 @ezyang @malfet @rgommers @xuzhao9 @gramster
ONNX Opset 16 GridSample Does Not Support 5D Volumetric Input Tensor
Labels: module: onnx, triaged

### 🚀 The feature, motivation and pitch

[#92209](https://github.com/pytorch/pytorch/issues/92209)

I need to use the gridsample operator in my code. I can use onnx in Pytorch1.12.0 to do this successfully, but there are errors in infer, because 5D input is not supported, so I checked the update log for Pytorch 2.0, found that as of now, the latest pytorch onnx does not support 5D input, so please ask:

When Will ONNX's GridSample support for 5D volumetric input be implemented?



### Alternatives

_No response_

### Additional context

_No response_
compile torch2.0 in debug mode
Labels: triaged, topic: build

### 🐛 Describe the bug

Hello, I want to compile torch2.0 in debug mode. However, I met some problems.

[1/1817] Performing build step for 'nccl_external'
FAILED: nccl_external-prefix/src/nccl_external-stamp/nccl_external-build nccl/lib/libnccl_static.a /pytorch/build/nccl_external-prefix/src/nccl_external-stamp/nccl_external-build 
/pytorch/build/nccl/lib/libnccl_static.a 
cd /pytorch/third_party/nccl/nccl && make -j64 -l64 CXX=/mnt/lustre/share/gcc-10.1.0/bin/c++ CUDA_HOME=/share/cuda-11.2-cudnn8.1 
NVCC=/share/cuda-11.2-cudnn8.1/bin/nvcc NVCC_GENCODE=-gencode=arch=compute_80,code=sm_80 BUILDDIR=/pytorch/build/nccl VERBOSE=0 && /anaconda3/envs/torch_compile/bin/cmake -E touch /pytorch/build/nccl_external-prefix/src/nccl_external-stamp/nccl_external-build
make -C src build BUILDDIR=/pytorch/build/nccl
NVCC_GENCODE is -gencode=arch=compute_80,code=sm_80
make[1]: Entering directory `
/pytorch/third_party/nccl/nccl/src'
NVCC_GENCODE is -gencode=arch=compute_80,code=sm_80
make[2]: Entering directory `
/pytorch/third_party/nccl/nccl/src/collectives/device'
In file included from /share/cuda-11.2-cudnn8.1/bin/crt/link.stub:129:
/tmp/tmpxft_00012f69_00000000-4_devlink.fatbin.c:33985860: internal compiler error: Segmentation fault
33985860 | ".text\n");
         | 
0xc6221f crash_signal
        ../../gcc/toplev.c:328
0xe9dc53 build_string(int, char const*)
        ../../gcc/tree.c:2229
0x71134b cp_parser_string_literal
        ../../gcc/cp/parser.c:4274
0x722151 cp_parser_asm_definition
        ../../gcc/cp/parser.c:20275
0x722151 cp_parser_block_declaration
        ../../gcc/cp/parser.c:13513
0x746c82 cp_parser_declaration
        ../../gcc/cp/parser.c:13433
0x7473a4 cp_parser_translation_unit
        ../../gcc/cp/parser.c:4734
0x7473a4 c_parse_file()
        ../../gcc/cp/parser.c:43975
0x80f82b c_common_parse_file()
        ../../gcc/c-family/c-opts.c:1190
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <https://gcc.gnu.org/bugs/> for instructions.
make[2]: *** [/pytorch/build/nccl/obj/collectives/device/devlink.o] Error 1
make[2]: Leaving directory `/pytorch/third_party/nccl/nccl/src/collectives/device'
make[1]: *** [/pytorch/build/nccl/obj/collectives/device/colldevice.a] Error 2
make[1]: Leaving directory `
/pytorch/third_party/nccl/nccl/src'
make: *** [src.build] Error 2
ninja: build stopped: subcommand failed.
Building wheel torch-2.1.0a0+gitd4bf76c
-- Building version 2.1.0a0+gitd4bf76c
cmake --build . --target install --config Debug -- -j 1

I have tried set the env variable MAX_JOBS=1, and this problem still occurs. Do you have any suggestions?

### Versions

OS: CentOS Linux 7 (Core) (x86_64)
GCC version: (GCC) 10.1.0
Clang version: Could not collect
CMake version: version 3.22.1
Libc version: glibc-2.17

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @anijain2305
[MPS] Unary ops yield wrong results if striding is different
Labels: high priority, triaged, module: regression, module: correctness (silent), module: mps

### 🐛 Describe the bug

Consider the following code
```python
x=torch.arange(4.0, device='mps').reshape(2, 2)
y=torch.empty(2, 2, device='mps').t()
print(torch.neg(x, out=y))
```
it will print
```
tensor([[-0., -2.],
        [-1., -3.]], device='mps:0')
```
but should have printed
```
tensor([[-0., -1.],
        [-2., -3.]], device='mps:0')
```
I guess error stems from this check:
https://github.com/pytorch/pytorch/blob/c9593bc0e15b661dc90286eeaf503f31a3c9df78/aten/src/ATen/native/mps/operations/UnaryOps.mm#L84-L88

### Versions

2.0/nightly

cc @ezyang @gchanan @zou3519 @kulinseth @albanD @DenisVieriu97 @razarmehr @abhudev
Nightly torch.compile fails with dynamically patched `nn.module.forward`
Labels: high priority, module: nn, triaged, oncall: pt2, module: dynamo

### 🐛 Describe the bug

When an `nn.Module` forward is modified during runtime, `torch.compile` fails.

```
import torch
import torch.nn as nn
import functools

model = nn.Linear(5, 5)
old_forward = model.forward


@functools.wraps(old_forward)
def new_forward(*args, **kwargs):
    return old_forward(*args, **kwargs)


model.forward = new_forward
model = torch.compile(model, backend='eager')
model(torch.randn(1, 5))
```

Output
```
Traceback (most recent call last):
  File "/home/jgu/Projects/dynamite/env/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1228, in get_fake_value
    return wrap_fake_exception(
  File "/home/jgu/Projects/dynamite/env/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 848, in wrap_fake_exception
    return fn()
  File "/home/jgu/Projects/dynamite/env/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1229, in <lambda>
    lambda: run_node(tx.output, node, args, kwargs, nnmodule)
  File "/home/jgu/Projects/dynamite/env/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1292, in run_node
    raise RuntimeError(
RuntimeError: Failed running call_method forward(*(Linear(in_features=5, out_features=5, bias=True), FakeTensor(..., size=(1, 5))), **{}):
Please convert all Tensors to FakeTensors first or instantiate FakeTensorMode with 'allow_non_fake_inputs'. Found in aten.t.default(*(Parameter containing:
tensor([[ 0.1118,  0.2579,  0.2571, -0.2395,  0.2391],
        [-0.3907, -0.2695,  0.2111, -0.4282, -0.2784],
        [ 0.1674,  0.3213, -0.1111,  0.2801,  0.3222],
        [-0.0022,  0.2045, -0.2440,  0.2014, -0.3790],
        [ 0.4095, -0.2653, -0.2879,  0.2875, -0.1032]], requires_grad=True),), **{})
(scroll up for backtrace)
```

Expected Output: exit 0

### Impact

I accidentally hit this when running a model that implicitly uses hugging face's accelerate module with torch.compile. HF's accelerate dynamically patches model.forward to do device offload. https://github.com/huggingface/accelerate/blob/main/src/accelerate/hooks.py#L118

### Versions

torch==2.1.0.dev20230428+cu117

The above example works with torch2.0 but fails on nightly



cc @ezyang @gchanan @zou3519 @albanD @mruberry @jbschlosser @walterddr @mikaylagawarecki @msaroufim @wconstab @ngimel @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @soumith @desertfire
`torch::jit::EliminateExceptions` lowering pass never completes on specific model
Labels: oncall: jit

### 🐛 Describe the bug

## Bug Description and Overview
When running the `torch::jit::EliminateExceptions` lowering pass on a specific model, the program never completes.
https://github.com/pytorch/pytorch/blob/40df6e164777e834b5f5b50e066632ed40bd25ef/torch/csrc/jit/passes/remove_exceptions.h#L20

A simple model which elicits this error (when run through the `torch::jit::EliminateExceptions` lowering pass), as adapted from https://github.com/pytorch/TensorRT/issues/1823 is:
```python
import torch

class UpSample(torch.nn.Module):
    def __init__(self):
        super(UpSample, self).__init__()
        self.upsample = torch.nn.Upsample(
            scale_factor=2, mode="bilinear", align_corners=False
        )

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        return self.upsample(x) + self.upsample(x)

model = torch.jit.script(UpSample().eval().to("cuda"))
input = torch.randn((5, 5, 5, 5)).to("cuda")
```

The corresponding IR for the above code is reproduced here: https://github.com/pytorch/TensorRT/issues/1823#issuecomment-1509005086.

## Context and Findings
The bug was traced to the use of `replaceAllUsesWith` (see https://github.com/pytorch/TensorRT/pull/1859#issuecomment-1534153675), which can cause invalid IR to be generated, especially in cases where there are nested `prim::If` calls. https://github.com/pytorch/TensorRT/pull/1859 gives a sample solution to this issue, substituting the `replaceAllUsesWith` operator with `replaceFirstUseWith`.

Would this substitution retain the original functionality/intent of the `torch::jit::EliminateExceptions` lowering pass?

### Versions

Versions of relevant libraries:
```
[pip3] torch==2.1.0.dev20230419+cu117
```

cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel
torch.cuda.amp.GradScaler initialization 
Labels: triaged, module: amp (automated mixed precision)

### 🐛 Describe the bug

If I just create a scaler using the following code 

`scaler = torch.cuda.amp.GradScaler()`
 
 and do not use this `scaler` in my code, then somehow training faces issue with convergence. 
 
 I have faced it with at least 2 models. 

### Versions

I have faced this in torch-1.6, torch-1.12. 

cc @mcarilli @ptrblck @leslie-fang-intel @jgong5
[Discussion] Investigate possibilities for Windows Arm64 BLAS and LAPACK
Labels: module: windows, triaged, module: linear algebra

We are trying to investigate what implementations of BLAS/LAPACK are compatible with PyTorch and Windows Arm64.

Considering the options listed [here](https://github.com/pytorch/pytorch/blob/master/cmake/Dependencies.cmake#L246), we have the following conclusions:
- [ATLAS](https://math-atlas.sourceforge.net/): outdated (last update was in 2015)
- [BLIS](https://github.com/flame/blis): prebuild Windows [binaries](https://ci.appveyor.com/project/shpc/blis), [instructions](https://github.com/flame/blis/blob/e14424f55b15d67e8d18384aea45a11b9b772e02/docs/FAQ.md#can-i-build-blis-on-windows) on how to build it on Windows
    - When running the instructions for building on Windows on an Arm64 machine, there is a SEH unwinding issue in the LLVM 
- [OpenBLAS](https://github.com/xianyi/OpenBLAS): Compiled with PyTorch and it seems to be working:
    -  Some core tests are failing on access violation or similar issues
    - WIP testing pipeline is [here](https://github.com/Windows-on-ARM-Experiments/pytorch-ci/actions/runs/4587922322)
- Generic BLAS: TBD
- [EigenBLAS](https://eigen.tuxfamily.org/dox/index.html): [here](https://gist.github.com/danielTobon43/8ef3d15f84a43fb15f1f4a49de5fcc75) are some instructions about building Eigen on Windows. There is also some Windows Arm64 option on https://vcpkg.io/en/packages.html
- [MKL](https://www.intel.com/content/www/us/en/developer/tools/oneapi/onemkl.html): for Intel processors
- [VecLib](https://developer.apple.com/documentation/accelerate/veclib):  for Apple devices
- [Nvidia MAGMA](https://developer.nvidia.com/magma): optimization for GPU (out of our scope for now) 
- [FlexiBLAS](https://github.com/mpimd-csc/flexiblas):  no Windows support https://github.com/mpimd-csc/flexiblas/issues/9
- [libflame](https://github.com/flame/libflame): It is a sister project to BLIS. Last update in August 2019
- [Netlib Reference Implementation](https://github.com/flame/libflame): [how to build on windows](https://icl.utk.edu/lapack-for-windows/lapack/)
- [APL](https://developer.arm.com/Tools%20and%20Software/Arm%20Performance%20Libraries#:~:text=Arm%20Performance%20Libraries%20provide%20optimized,performance%20in%20multi%2Dprocessor%20environments.): No support for Windows

## Open questions
- What did we miss with the implementations described above?
- What other implementations are worth being taken into consideration?

cc @peterjc123 @mszhanyi @skyline75489 @nbcsm @vladimir-aubrecht @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @Lezcano
DISABLED test_inplace_gradgrad_remainder_cuda_float64 (__main__.TestBwdGradientsCUDA)
Labels: module: autograd, triaged, module: flaky-tests, skipped

Platforms: linux

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_inplace_gradgrad_remainder_cuda_float64&suite=TestBwdGradientsCUDA) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/undefined).

Over the past 3 hours, it has been determined flaky in 7 workflow(s) with 7 failures and 7 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_inplace_gradgrad_remainder_cuda_float64`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `test_ops_gradients.py`

ConnectionTimeoutError: Connect timeout for 5000ms, GET https://raw.githubusercontent.com/pytorch/pytorch/master/test/test_ops_gradients.py -2 (connected: false, keepalive socket: false, socketHandledRequests: 1, socketHandledResponses: 0)
headers: {}

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7
Add support for MaxPool3D on the MPS backend
Labels: triaged, module: mps

### 🚀 The feature, motivation and pitch

Currently, Pooling operations are only supported on MPS for 1D and 2D inputs. While MPS doesn't have native support for 3d pooling operations, it does support 4d pooling operations (e.g. [`maxPooling4DWithSourceTensor()`](https://developer.apple.com/documentation/metalperformanceshadersgraph/mpsgraph/3750695-maxpooling4dwithsourcetensor?language=objc)). 3d tensors can be expanded to become 4d tensors, passed to 4d pooling operations, and then squeezed back to 3d tensors. This wouldn't affect the accuracy of the results when compared to direct 3d pooling operations. So, we can start working to add support for 3d pooling operations on the MPS backend.

cc @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev @mattiaspaul

### Alternatives

_No response_

### Additional context

This feature, along with the introduction of 3d convolutions in #99246 would open up lots of possible model architectures for 3D inputs. 
Full disclosure: I already have a working implementation of MaxPool3D support for MPS, and @mattiaspaul already has something in the works to address this as well.
Optimal Batch Size Selection in Torchdynamo Benchmarks for Different GPU Memory Sizes
Labels: triaged, oncall: pt2

### 🚀 The feature, motivation and pitch

The torchdynamo benchmarking scripts come with default batch size for each model based on A100 40GB GPU tests. When adapting these scripts for training on smaller sized GPUs, we see OOM errors. The performance and accuracy tests will fail for most models and the benchmarking results becomes less informative. Thus it makes more sense to assign an optimal batch size according to the actual GPU memory size for each model, where the optimal batch size map will be kept as a list in the directory `benchmarks/dynamo`. By adding this batch size selection feature in the benchmarking scripts for all model suites, the benchmarking can be adapted by wider communities and suits their needs for testing and comparison among different GPU types.



### Alternatives

_No response_

### Additional context

_No response_

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @anijain2305
Will Deep Implicit Models ever become first class citizens in PyTorch?
Labels: module: autograd, triaged, oncall: pt2, module: functorch

### 🚀 The feature, motivation and pitch

PyTorch makes it unnecessarily difficult to work with [Deep Implicit Models](https://implicit-layers-tutorial.org/). Deep Implicit Layers/Models are characterized by the need to implement custom backward passes to be efficient, making use of the [implicit function theorem](https://en.wikipedia.org/wiki/Implicit_function_theorem).

In particular, the following issues persist:

- JIT is incompatible with custom backwards. The only way around this is writing custom C++ extensions.  https://github.com/pytorch/pytorch/issues/35749
- Implementing custom backward hooks for a whole `nn.Module` is extremely counter-intuitive, consider this example from Chapter 4:

```python
def forward(self, x):
    # compute forward pass and re-engage autograd tape
    with torch.no_grad():
        z, self.forward_res = self.solver(
            lambda z : self.f(z, x), torch.zeros_like(x), **self.kwargs)
    z = self.f(z,x)
    
    # set up Jacobian vector product (without additional forward calls)
    z0 = z.clone().detach().requires_grad_()
    f0 = self.f(z0,x)
    def backward_hook(grad):
        g, self.backward_res = self.solver(
            lambda y : autograd.grad(f0, z0, y, retain_graph=True)[0] + grad,
            grad, **self.kwargs)
        return g
            
    z.register_hook(backward_hook)
    return z
```

One needs to `no_grad` the forward, and then manually surgically insert the custom backward.

- Custom backward of nn.Modules are also incompatible with JIT, and it's unclear how to work around it.

### Alternatives

JAX supports per-function custom backward overrides via decorators: https://jax.readthedocs.io/en/latest/notebooks/Custom_derivative_rules_for_Python_code.html

```python
import jax.numpy as jnp
from jax import custom_jvp

@custom_jvp
def f(x, y):
  return jnp.sin(x) * y

@f.defjvp
def f_jvp(primals, tangents):
  x, y = primals
  x_dot, y_dot = tangents
  primal_out = f(x, y)
  tangent_out = jnp.cos(x) * x_dot * y + jnp.sin(x) * y_dot
  return primal_out, tangent_out
```

If supported, this would solve all 3 issues.

### Additional context

_No response_

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7 @soumith @msaroufim @wconstab @ngimel @bdhirsh @Chillee @samdow @kshitij12345 @janeyx99
Fix static lib
Labels: triaged, open source, ciflow/binaries, release notes: releng

Fix linking issues when building with BUILD_SHARED_LIBS=OFF.
Added 2 CI jobs: one that builds with BUILD_SHARED_LIBS=OFF with cuda12.1, and another job that creates a project from a cmake file using the previously built libtorch and runs it.
This PR is using https://github.com/pytorch/builder/pull/1465 to test the changes work.
The third commit changes the builder branch to my branch in order to do the test.
Fixes #87499
GPU VRAM usage significantly higher for Lenet5 models when compared to other frameworks
Labels: triaged, better-on-discuss-forum

### 🐛 Describe the bug

Hello everybody.

I’ve been experimenting with different models and different frameworks, and I’ve noticed that, when using GPU, training Lenet-5 on the MNIST dataset on PyTorch leads to higher GPU VRAM usage when compared to the Keras and TensorFlow v1.X implementations. 

I'm on a Ubuntu 18.04.4 system equipped with an NVIDIA Quadro RTX 4000 GPU with 8GB of VRAM and an Intel(R) Core(TM) i9-9900K CPU running at 3.60GHz.

Here are boxplots that showcase the performance of numerous Lenet5:


![Screenshot 2023-05-04 015324 - Copy (2)](https://user-images.githubusercontent.com/132307143/236125280-c7e019fb-2dec-4a1a-b1e3-9ae3c395b2e4.png)


Any ideas on what may be causing this?


### Versions

The version of relevant libraries are:

numpy==1.19.5
torch==1.10.0
torchaudio==0.10.0
torchvision==0.11.1
mkl==2022.2.1
mkl-fft==1.3.0
mkl-random==1.2.1
mkl-service==2.4.0