jiebi/CodeConvo huggingface dataset:
Change:
import torch from torch.distributions import constraints from torch.distributions.categorical import Categorical <del> from torch.distributions.utils import clamp_probs, broadcast_all, log_sum_exp </del> <ins> from torch.distributions.utils import clamp_probs, broadcast_all, _log_sum_exp </ins> from torch.distributions.distribution import Distribution from torch.distributions.transformed_distribution import TransformedDistribution from torch.distributions.transforms import ExpTransform

Discussion:
Leaving this issue here for higher visibility. Originally reported by here: Also mentioned by Piggybacking off of this issue, has strange behavior (reported here: )
I have fixed this locally. Could I send a PR?
Yes, go ahead!
Change:
begin{array}{ll} i_t = sigma(W_{ii} x_t + b_{ii} + W_{hi} h_{(t-1)} + b_{hi})  f_t = sigma(W_{if} x_t + b_{if} + W_{hf} h_{(t-1)} + b_{hf})  <del> g_t = tanh(W_{ig} x_t + b_{ig} + W_{hc} h_{(t-1)} + b_{hg})  </del> <ins> g_t = tanh(W_{ig} x_t + b_{ig} + W_{hg} h_{(t-1)} + b_{hg})  </ins> o_t = sigma(W_{io} x_t + b_{io} + W_{ho} h_{(t-1)} + b_{ho})  c_t = f_t c_{(t-1)} + i_t g_t  h_t = o_t tanh(c_t)

Discussion:
Hi, Looking at the formulas of LSTM in , in the third equation (gt), the suffix of the second W should be changed from W{hc} to W{hg}. The correct formula: gt=tanh(W{ig}xt+b{ig}+W{hg}h{(t−1)}+b_{hg}) Cheers, Navid
Change:
tmp3 = torch.cuda.FloatTensor(t.size()) self.assertEqual(tmp3.data_ptr(), ptr[0], 'allocation not re-used') <ins> def test_noncontiguous_pinned_memory(self): # See issue #3266 x = torch.arange(0, 10).view((2, 5)) self.assertEqual(x.t(), x.t().pin_memory()) </ins> def test_caching_pinned_memory(self): cycles_per_ms = get_cycles_per_ms()

Discussion:
Hello, I was experimenting with a custom which receives a batch of different sized integer sequences and pads them to form 2D tensors. I noticed a weird issue for which I created the following MWE: It's probably because of the different memory ordering on Python's side and CUDA's allocator. Tested on K40 and GTX1080 with driver 375.39, CUDA8.
Pin memory is missing a call inside. Should be an easy fix
fixed via
Change:
- [Releases and Contributing](#releases-and-contributing) - [The Team](#the-team) <del> | Python |  **`Linux CPU`**   |  **`Linux GPU`** | |--------|--------------------|------------------| | 2.7.8  | [![Build Status](https://travis-ci.com/apaszke/pytorch.svg?token=shqHbUq29zKDxuqzGcjC&branch=master)](https://travis-ci.com/apaszke/pytorch) | | | 2.7    | [![Build Status](https://travis-ci.com/apaszke/pytorch.svg?token=shqHbUq29zKDxuqzGcjC&branch=master)](https://travis-ci.com/apaszke/pytorch) | [![Build Status](http://build.pytorch.org:8080/buildStatus/icon?job=pytorch-master-py2)](https://build.pytorch.org/job/pytorch-master-py2)  | | 3.5    | [![Build Status](https://travis-ci.com/apaszke/pytorch.svg?token=shqHbUq29zKDxuqzGcjC&branch=master)](https://travis-ci.com/apaszke/pytorch) | [![Build Status](http://build.pytorch.org:8080/buildStatus/icon?job=pytorch-master-py3)](https://build.pytorch.org/job/pytorch-master-py3)  | | Nightly| [![Build Status](https://travis-ci.com/apaszke/pytorch.svg?token=shqHbUq29zKDxuqzGcjC&branch=master)](https://travis-ci.com/apaszke/pytorch) | | </del> <ins> | System | Python | Status | | --- | --- | --- | | Linux CPU | 2.7.8, 2.7, 3.5, nightly | [![Build Status](https://travis-ci.org/pytorch/pytorch.svg?branch=master)](https://travis-ci.org/pytorch/pytorch) | | Linux GPU | 2.7 | [![Build Status](http://build.pytorch.org:8080/buildStatus/icon?job=pytorch-master-py2)](https://build.pytorch.org/job/pytorch-master-py2) | | Linux GPU | 3.5 | [![Build Status](http://build.pytorch.org:8080/buildStatus/icon?job=pytorch-master-py3)](https://build.pytorch.org/job/pytorch-master-py3) | </ins> ## More about PyTorch

Discussion:
We should explore using TorchDynamo to capture/optimize optimizer code. I don't anticipate any major issues here, but we should make sure it plays nice with AOT Autograd. cc
linking issue related to performance of this:
Change:
]] [[ <del> name: free return: self arguments: - THTensor* self ]] [[ name: retain return: self arguments: - THTensor* self ]] [[ </del> python_name: index name: THPTensor_(getValue)<true> only_register: True

Discussion:
Dataloader call getitem as many times as indices in the current batch. In case datasets support a list of indices in one call, or a native python slice object, add a getbatch (optional) to pytorch dataset class. If dataloader sees the dataset has such a method implemented, make it fetch the batch corresping to the list of indices with one getbatch call rather than many getitem calls. This saves the trouble of calling getitem many times in cases where for example a disk file has to be opened then closed for every call and the underlying data structure supports a slicing or lists of indices for example hdf5 files.
will support something similar
is the latest version of this.
Change:
class InstanceNorm2d(_InstanceNorm): r"""Applies Instance Normalization over a 4d input that is seen as a mini-batch of 3d inputs .. math:: y = frac{x - mean[x]}{ sqrt{Var[x]} + epsilon} * gamma + beta The mean and standard-deviation are calculated per-dimension separately for each object in a mini-batch. Gamma and beta are learnable parameter vectors of size C (where C is the input size).

Discussion:
why 1.21.6? because that's the oldest supported numpy on windows/python3.10: all cc
Here is the test for pinning numpy to 1.21.6 The reason its was not pinned to this version is that conda can't properly resolve all the dependencies at this step here:
It looks like you are correct, I will put up a PR to pin lower then already pinned:
Here is the PR:
Change:
The upper triangular part of the matrix is defined as the elements on and above the diagonal. <del> The argument :attr:`k` controls which diagonal to consider. </del> <ins> The argument :attr:`diagonal` controls which diagonal to consider. </ins> <del> - :attr:`k` = 0, is the main diagonal. - :attr:`k` > 0, is above the main diagonal. - :attr:`k` < 0, is below the main diagonal. </del> <ins> - :attr:`diagonal` = 0, is the main diagonal. - :attr:`diagonal` > 0, is above the main diagonal. - :attr:`diagonal` < 0, is below the main diagonal. </ins> Args: input (Tensor): the input `Tensor` <del> k (int, optional): the diagonal to consider </del> <ins> diagonal (int, optional): the diagonal to consider </ins> out (Tensor, optional): The result `Tensor` Example::

Discussion:
The states that the argument controls which diagonal to consider. However, this argument is actually named in pytorch 0.2.0.
Change:
If :attr:`upper` is ``False``, then lower triangular portion is used. <ins> .. note:: The eigenvalues are returned in ascending order. If :attr:`input` is a batch of matrices, then the eigenvalues of each matrix in the batch is returned in ascending order. </ins> .. note:: Irrespective of the original strides, the returned matrix `V` will be transposed, i.e. with strides `V.contiguous().transpose(-1, -2).stride()`.

Discussion:
Documentation for does not indicate whether or not singular values are returned in descending order. My suspicion is that they are since both LAPACK and MAGMA's ?gesdd return them in descending order. It seems like this should be clarified in the docs to avoid unnecessary sorting, or mistaken assumptions. <!-- A clear and concise description of what content in is an issue. If this has to do with the general website, please file an issue at instead. If this has to do with , please file an issue at --
Change:
embedding_dim (int): the size of each embedding vector max_norm (float, optional): If given, will renormalize the embeddings to always have a norm lesser than this norm_type (float, optional): The p of the p-norm to compute for the max_norm option <del> scale_grad_by_freq (boolean, optional): if given, this will scale gradients by the frequency of </del> <ins> scale_grad_by_freq (bool, optional): if given, this will scale gradients by the frequency of </ins> the words in the dictionary. mode (string, optional): 'sum' | 'mean'. Specifies the way to reduce the bag. Default: 'mean' <del> sparse (boolean, optional): if ``True``, gradient w.r.t. weight matrix will be a sparse tensor. See Notes for </del> <ins> sparse (bool, optional): if ``True``, gradient w.r.t. weight matrix will be a sparse tensor. See Notes for </ins> more details regarding sparse gradients. Attributes:

Discussion:
I have a straightforward use case. Given a token (or word), lookup the embedding of the word, pass that through a linear layer and perform L2 regression. If the token corresponds to the of the embedding layer, throws an error saying OS: macOS High Sierra PyTorch version: 0.3.1 How you installed PyTorch (conda, pip, source): Tried both conda and pip Python version: 3.6.4 CUDA/cuDNN version: CPU GPU models and configuration: NA GCC version (if compiling from source): NA A script to reproduce the bug:
I got the same trouble, and it throw this error after some epochs of training. Does anyone have some solutions?
Change:
g = tanh(W_{ig} x + b_{ig} + W_{hc} h + b_{hg})  o = mathrm{sigmoid}(W_{io} x + b_{io} + W_{ho} h + b_{ho})  c' = f * c + i * g  <del> h' = o * tanh(c_t)  </del> <ins> h' = o * tanh(c')  </ins> end{array} Args:

Discussion:
When converting a tensor to dlpack, the strides of single-value tensors () should be normalized to . This is the fallout conclusion from . Here is a demo of what is wrong: Collecting environment information... PyTorch version: 1.10.0a0+git2aa19f3 Is debug build: False CUDA used to build PyTorch: Could not collect ROCM used to build PyTorch: N/A OS: Ubuntu 20.04.4 LTS (x8664) GCC version: (GCC) 9.4.0 Clang version: 12.0.1 ( ) CMake version: version 3.21.1 Libc version: glibc-2.31 Python version: 3.9.6 (default, Jul 11 2021, 03:39:48) [GCC 9.3.0] (64-bit runtime) Python platform: Linux-5.4.0-122-generic-x8664-with-glibc2.31 Is CUDA available: False CUDA runtime version: 11.2.152 GPU models and configuration: GPU 0: NVIDIA GeForce GTX 1660 SUPER Nvidia driver version: 470.141.03 cuDNN version: /usr/lib/x8664-linux- HIP runtime version: N/A MIOpen runtime version: N/A Is XNNPACK available: False Versions of relevant libraries: [pip3] mypy==0.812 [pip3] mypy-extensions==0.4.3 [pip3] numpy==1.21.2 [pip3] torch==1.10.0a0+git2aa19f3 [pip3] torchaudio==0.10.0a0+ [conda] magma-cuda112 2.5.2 1 pytorch [conda] mkl 2021.3.0 h726a3e6557 conda-forge [conda] mkl-include 2021.3.0 h726a3e6557 conda-forge [conda] numpy 1.21.2 py39hdbf815f0 conda-forge [conda] torch 1.12.0a0+git1f29b31 dev0 <develop[conda] torchaudio 0.10.0a0+ pypi0 pypi
does this capture the problem?
Change:
def calculate_gain(nonlinearity, param=None): """Return the recommended gain value for the given nonlinearity function. The values are as follows: ============ ========================================== nonlinearity gain ============ ==========================================

Discussion:
why 1.21.6? because that's the oldest supported numpy on windows/python3.10: all cc
Here is the test for pinning numpy to 1.21.6 The reason its was not pinned to this version is that conda can't properly resolve all the dependencies at this step here:
It looks like you are correct, I will put up a PR to pin lower then already pinned:
Here is the PR:
Change:
def index_select(self, dim, index): return IndexSelect(dim)(self, index) <ins> def gather(self, dim, index): return Gather(dim)(self, index) def scatter(self, dim, index, source): return Scatter(dim)(self, index, source) def scatter_(self, dim, index, source): return Scatter(dim, True)(self, index, source) </ins> def masked_copy(self, mask, variable): return MaskedCopy()(self, mask, variable)

Discussion:
<!-- A clear and concise description of what content in is an issue. If this has to do with the general website, please file an issue at instead. If this has to do with , please file an issue at --In the getting started section section the following quote appears: I guess this is from previous versions where CUDA support was not built-in and this can be removed, correct? cc
This seems to be easy fixable docs issue
If gets merged this is fixed.
Thanks for the PR Just merged it.
Change:
# Example of using Container class Net(nn.Container): def __init__(self): <del> super(Net, self).__init__( conv1 = nn.Conv2d(1, 20, 5), relu  = nn.ReLU() ) </del> <ins> super(Net, self).__init__() self.conv1 = nn.Conv2d(1, 20, 5) self.relu  = nn.ReLU() </ins> def forward(self, input): output = self.relu(self.conv1(x)) return output

Discussion:
When I enable the MPS backend on Mac in libtorch, it now crashes with this error when exiting: If I remove: the crash disappears. It only affects exiting/destructing the app. Most likely it is the MPS memory allocator closing down. nightly 1st August 2022. Backtrace (basically it is torch::jit::Module destructor, leading to C10::StorageImpl destructor): cc
, can you please provide a small test-case to repro the issue ?
I'me using next C++ code, but can't reproduce this issue void toTest() { torch::Tensor inputTensor = torch::randn({1, 3, 256, 256}); (torch::kMPS); } Tested with the latest mps_graph branch build. Could you please provide a minimal reproducible test?
The crash is in torch::jit::Module so you'd likely need to be running a forward. I'll try to get a reproduce. It's not happening in Python which makes it a bit harder.
I've tried to reproduce with a Module, for that i've created test module using code for resnet18 from this sample And next created next C++ code std::vector&lt;torch::jit::IValueinputs; inputs.pushback(torch::ones({1, 3, 224, 224}).to(torch::kMPS)); at::Tensor output = module.forward(inputs).toTensor().to(torch::kCPU); std::cout &lt;&lt; (1,0, 5) &lt;&lt; 'n'; With this code I see no issues, it runs forward and prints expected output: -1.2350 -0.0649 0.2181 0.0655 -0.2242 [ CPUFloatType{1,5} ]
, removing the "High priority" label, as we are not able to reproduce the issue. Please provide us a failing case and we will take a look.
We weren't able to reproduce this issue, so I'll close it. Please reopen the issue if you see again. Thanks.
Change:
# (3) initialize mean square values and square gradient storage if not 'm' in state: <del> state['m'] = x.new().resize_as_(dfdx).fill_(1) </del> <ins> state['m'] = x.new().resize_as_(dfdx).zero_() </ins> state['tmp'] = x.new().resize_as_(dfdx)

Discussion:
We should unify this with our normal list creating code for the reasonable cases cc
Similar issue to this, I believe:

The hub tests are broken on macOS. I'm seeing SSL certificate related errors when trying to hit GitHub. Run the macOS tests on CI with TestHub enabled. I tried installing the latest version of from conda-forge (which pulled in ) but this didn't fix the issue. cc
Looks like the same error I'm getting trying to use Ubuntu 18.04 with fresh install of conda and Pytorch 1.2
I think this will all be solved with .

Porting TH operators is essential for code simplicity and performance reasons. Porting guides and Q&A are available in umbrella issue: Feel free to add as a reviewer to get a prioritized review.

This is of particular use in list comprehensions: This should work:
I'll need to adapt list comprehensions to use Iterables. I'll take a look!

I haven't looked into this in depth but this looks fishy: More here: cc
is this still an issue in 1.3.1?
I can repro this on master:
The problem is: removes the parameters from the module's parameter list and makes them normal class attributes: applies only to parameters, not to the weight attributes.  (which is called from acts on which includes the value of the weight attributes and the bias parameters. So, it contains a mix of CUDA and CPU tensors.
This is my workaround for right now: This applies weight norm to an rnn module and overrides the function to deal with the error.
This appears to be fixed. do you want to verify?
I tested this on nightly and it worked for me, so closing.

As mentioned in title, I want to make sure if pytorch can deal with the situation that each input in a batch has different shapes and thus different computation graphs.
Please don't open duplicated issue for discussion. Context will be missed. Try to discuss about it here:

Be Careful What You Backpropagate: A Case For Linear Output Activations & Gradient Boosting I can work on this if this can be to pytorch. Please let me know. Thanks!
This should be fairly straightforward to add in user code using . Eventually it could be to core, but its utility should probably be validated in external repos first.
Change:
Example:: >>> tensor = torch.tensor((), dtype=torch.float64) <del> >>> tensor.new_ones((2, 3)) tensor([[ 1.,  1.,  1.], [ 1.,  1.,  1.]], dtype=torch.float64) </del> <ins> >>> tensor.new_zeros((2, 3)) tensor([[ 0.,  0.,  0.], [ 0.,  0.,  0.]], dtype=torch.float64) </ins> """.format(**new_common_args))

Discussion:
The example given in the docs for newzeros() is actually an example for newones(). Should be a trivial fix. (Sorry, not quite sure what the etiquette/process is for fixing small errors like these). Thanks. The example should read:
Thanks for the catch, If you'd like to take a stab at fixing it, you can fork the pytorch repo and contribute a pull request. Otherwise, I can make the replacement.
fixed via
Change:
return obj; } <ins> PyObject * THSPTensor_(NewEmpty)() { THSTensorPtr tensor(THSTensor_(new)(LIBRARY_STATE_NOARGS)); THSTensor_(initStorage)(tensor.get()); return THSPTensor_(New)(tensor.release()); } </ins> static void THSPTensor_(dealloc)(THSPTensor* self) { if (self->cdata)

Discussion:
This causes to not work properly on newly created empty sparse tensors. In the example below, both should print 1 instead.
Change:
return type(self), (self.tolist(),) def __repr__(self): <del> return repr(str(self)) </del> <ins> return str(self) </ins> def __str__(self): # All strings are unicode in Python 3, while we have to encode unicode

Discussion:
In torch, printing a tensor will print the whole tensor, even if that one is huge. This is sometimes annoying, especially when using a notebook, as it crashes it. Numpy has a very nice way to display big tensors by truncating them. Would it be possible so have something similar for pytorch?
Yeah, we haven't implemented it yet, but that's definitely on our roadmap
Implemented in .
I should say that I have some artefacts that appear with while printing a tensor. Here is an example output: gives Curiously, if I simply do It works fine and print nicely, with the truncation for large tensors. I'm using Python 2.7.6 (probably the one that came with my system and not from anaconda). Maybe it's just a bad setup in my machine?
hmmm, it works fine on python 2.7.12. wonder what changed between .6 and .12
ok i got a repro. I opened my ipython and did this:
my example was flawed, sorry. Using works for me, but when only entering it prints the s, as in your example.
Is it possible to tell the truncation point?
yes, use the function
Change:
] if WITH_CUDNN: main_libraries += ['cudnn'] <del> library_dirs.append(CUDNN_LIB_DIR) </del> <ins> library_dirs.insert(0, CUDNN_LIB_DIR) </ins> # NOTE: these are at the front, in case there's another cuDNN in CUDA path include_dirs.insert(0, CUDNN_INCLUDE_DIR) if not IS_WINDOWS:

Discussion:
If there was two libcudnn files ( and ) in the folder wouldn't it result in taking the older version ()? Maybe adding a warning or something to insure its the right lib?
I think and are quite possibly related. EDIT: also

Compilation goes without a problem but importing torch gives ImportError: /lib/python3.6/site-packages/torch/C.cpython-36m-x8664-linux- undefined symbol: cudnnSetConvolutionGroupCount Compiling from source v0.2.0 works fine. Set up is Python 3.6.2 :: Anaconda custom (64-bit) Cuda compilation tools, release 8.0, V8.0.61 gcc (GCC) 4.8.5 centos-release-scl-rh-2-
It's likely a configuration issue of your system. You built with cuDNN7, but cuDNN6 is loaded at runtime. Can you please run and paste the output in a gist?
OK, here's the .
That's weird. It doesn't even seem to be linked. Can you post the output of ? Did you build from source? Can you clean the repo ( ASSUMING YOU DID NO CHANGES YOURSELF - THIS MIGHT DESTROY THEM), build form scratch, and post the build log in a gist as well?
Hey Adam, I made a mistake. This is the correct . And the ldd just in case.
Ok. cuDNN isn't linked. How did you install PyTorch?
Compiled from source, following the instructions.
Please rebuild and post the build log
Done, here's the .
The build log looks correct. I think the problem is that you have multiple Python environments with different PyTorch packages. In your run time log, torch is loaded from , but in your build log it's installed in . Cleaning up the global install should fix it for you.
Ok, so I made sure pytorch wasn't installed in the global python. Checked & removed from the Anaconda . Now the only shows Still, get Again, runtime and the of the from the error
Eh, I'm running out of tricks. Maybe there's sth wrong with the compiler? you can see that there's a flag when building . Try prepending before the build command
The sysadmin installed both cuDNN 6.0 and 7.0 and that's causing the problem. I followed but still found it necessary to insert CUDNNLIBDIR at the beginning of the list with .

Now, after compiling the import doesn't throw any errors, so I'm guessing it's ok.
hmm yeah, it should also go in the front of . Can you please submit a PR with the fix?
Will do
i believe this patch in the second last comment has to be adapted for your refactor.
Change:
return output @staticmethod <del> @once_differentiable </del> def backward(ctx, grad_output, grad_indices=None): <del> grad_input = grad_output.new(ctx.input_size).zero_() </del> <ins> grad_input = Variable(grad_output.data.new(ctx.input_size).zero_(), requires_grad=True) </ins> if ctx.return_indices: indices, = ctx.saved_tensors else: indices = ctx.indices dim = ctx.dim if ctx.dim is not None else grad_output.dim() - 1 <del> return (grad_input.scatter_(dim, indices, grad_output),) + (None,) * ctx.num_flags </del> <ins> return (grad_input.scatter(dim, Variable(indices), grad_output),) + (None,) * ctx.num_flags </ins> class Sort(_MultiSelectionFunction):

Discussion:
Just a very small grammatical issue. ! Suggested fix: "mutates" to "mutate" <!-- A clear and concise description of what content in is an issue. If this has to do with the general website, please file an issue at instead. If this has to do with , please file an issue at --
Thanks for pointing this out. Please feel free to open a pull request for this.
Thanks for adding labels. Forgot to do that. I will make a pull request
Change:
#  binary_linux_libtorch_3.6m_cpu_test: #    environment: #      BUILD_ENVIRONMENT: "libtorch 3.6m cpu" <del> #    resource_class: gpu.medium </del> <ins> #    resource_class: gpu.nvidia.small </ins> #    <<: *binary_linux_test # #  binary_linux_libtorch_3.6m_cu90_test: #    environment: #      BUILD_ENVIRONMENT: "libtorch 3.6m cu90" <del> #    resource_class: gpu.medium </del> <ins> #    resource_class: gpu.nvidia.small </ins> #    <<: *binary_linux_test #

Discussion:
As it still using deprecated resource class, see See example of the failure in
Followup: we need to plan better --we knew it was gonna happen and none of us fixed it early enough. Action item: go through tasks about deprecation in sprint planning