jiebi/CodeConvo huggingface dataset:
Change:
<ins> From PyTorch: Copyright (c) 2016-     Facebook, Inc            (Adam Paszke) Copyright (c) 2014-     Facebook, Inc            (Soumith Chintala) Copyright (c) 2011-2014 Idiap Research Institute (Ronan Collobert) Copyright (c) 2012-2014 Deepmind Technologies    (Koray Kavukcuoglu) Copyright (c) 2011-2012 NEC Laboratories America (Koray Kavukcuoglu) Copyright (c) 2011-2013 NYU                      (Clement Farabet) Copyright (c) 2006-2010 NEC Laboratories America (Ronan Collobert, Leon Bottou, Iain Melvin, Jason Weston) Copyright (c) 2006      Idiap Research Institute (Samy Bengio) Copyright (c) 2001-2004 Idiap Research Institute (Ronan Collobert, Samy Bengio, Johnny Mariethoz) From Caffe2: Copyright (c) 2016-present, Facebook Inc. All rights reserved. All contributions by Facebook: Copyright (c) 2016 Facebook Inc. All contributions by Google: Copyright (c) 2015 Google Inc. All rights reserved. All contributions by Yangqing Jia: Copyright (c) 2015 Yangqing Jia All rights reserved. All contributions from Caffe: Copyright(c) 2013, 2014, 2015, the respective contributors All rights reserved. All other contributions: Copyright(c) 2015, 2016 the respective contributors All rights reserved. Caffe2 uses a copyright model similar to Caffe: each contributor holds copyright over their contributions to Caffe2. The project versioning records all such contribution and copyright details. If a contributor wants to further mark their specific copyright on a particular contribution, they should indicate their copyright solely in the commit message of the change when it is committed. </ins> All rights reserved. Redistribution and use in source and binary forms, with or without

Discussion:
The three-clause BSD license in file LICENSE says in clause 1 "Redistributions of source code must retain the above copyright notice..." However, there is no longer a copyright notice in the file; it appears to have been removed in commit (see ).
cc
Change:
# Target only our CI GPU machine's CUDA arch to speed up the build export TORCH_CUDA_ARCH_LIST=5.2 <ins> if [[ "$BUILD_ENVIRONMENT" == *trusty-py3.6-gcc5.4* ]]; then export DEBUG=1 fi </ins> WERROR=1 python setup.py install # Add the test binaries so that they won't be git clean'ed away

Discussion:
Turning on DEBUG=1 turns off inlining. However, if you declare a function as , it won't get object code generated for it, which means there will be undefined symbol errors. You might not notice this with a release build, since everything gets inlined away. We can catch errors like this by doing CI with DEBUG=1. Probably just pick one platform and run that only.
Change:
} } <del> variable_list flatten(const TensorList& tensors) { </del> <ins> static variable_list flatten(const TensorList& tensors) { </ins> return cast_tensor_list(tensors); } <del> variable_list flatten(const Tensor& x, const TensorList& y) { </del> <ins> static variable_list flatten(const Tensor& x, const TensorList& y) { </ins> std::vector<Variable> r; r.reserve(1 + y.size()); r.emplace_back(x);

Discussion:
See Docker run fails with , which is due to the fact that is specified in docker container: Which results in: CI cc
Check that adding to manywheels image results in a failure:
Tentatively marking hi-pri since nightly testing are failing
This is resolved, here is the successful run:
Change:
if (info != 0) THError("MAGMA orgqr : Argument %d : illegal value.", -info); <ins> THCTensor_(free)(state, a); </ins> THCTensor_(free)(state, work); magma_free_pinned(tau_data);

Discussion:
It seems like the function leaks memory. I'm using 0.2.0_4 on anaconda python 3.6 CUDA 8.0 using the standard conda installation. So the memory footprint increases up to 1.9G though each takes about 100M. compared to where it takes up more memory but doesn't leak memory and the memory foot print fluctuates between 414M and 432M and back to 414M. Aside from that, is there a reason for using Magma instead of cuBLAS?
Thanks for catching this! Looks like we're leaking : I think at the time we the Magma bindings cuBLAS didn't provide a QR solver. I'm not sure there's any reason to prefer it now.
Change:
return value.reshape(required_shape).sum(-1) <del> def log_sum_exp(tensor, keepdim=True): </del> <ins> def _log_sum_exp(tensor, keepdim=True): </ins> r""" Numerically stable implementation for the `LogSumExp` operation. The summing is done along the last dimension.

Discussion:
Leaving this issue here for higher visibility. Originally reported by here: Also mentioned by Piggybacking off of this issue, has strange behavior (reported here: )
I have fixed this locally. Could I send a PR?
Yes, go ahead!
Change:
l = torch.randn(15) source = TensorDataset(t, l) for i in range(15): <del> self.assertEqual(t[i:i + 1], source[i][0]) self.assertEqual(l[i:i + 1], source[i][1]) </del> <ins> self.assertEqual(t[i], source[i][0]) self.assertEqual(l[i], source[i][1]) </ins> class ErrorDataset(Dataset):

Discussion:
TensorDataset adds casts 1D target tensor to 2D which is not acceptable by NLL loss layer.
Fixed in .
Change:
#include <cstdint> <del> namespace F = torch::nn::functional; </del> namespace torch { namespace nn {

Discussion:
As part of pull request , F is redefined globally in nn/modules/batchnorm.h F is then used once in batchnorm.h and again in instancenorm.h not a bug, but a problem for anyone using F in some other way. it's solvable, but I don't see this pattern anywhere else in libtorch, was wondering if this use is an anomaly, and if we could use thanks cc
Thanks a lot for reporting! Yes this is an anomaly, and will be fixed by
Change:
* Inception (warning: this model is highly sensitive to changes in operator implementation) * ResNet <del> * SqueezeNet </del> * SuperResolution * VGG * `word_language_model <https://github.com/pytorch/examples/tree/master/word_language_model>`_

Discussion:
Squeezenet is not exportable via onnx since it uses in which is not supported by ONNX. Please update documentation accordingly.
Change:
return res def emit_record_trace(env, declaration): <ins> # Note [clang-802.0.42 tuple overload bug] # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ # Originally, my plan for emit_$ecord_trace was to keep it as # simple as possible, if at the expense of some somewhat ugly # overloads.  So this meant we had a 'recordTrace' function # with overloads like this: # #   recordTrace(..., const Variable& out) #   recordTrace(..., const std::tuple<Variable, Variable>& out) # # Unfortunately, this triggers a bug in clang-802.0.42 # (widely used in macOS Sierra 10.12.6) wherein a Variable is # implicitly convertible into a std::tuple<Variable, Variable>; # a minimal repro can be seen below here: # #   #include <tuple> #   struct T {}; #   void f(const std::tuple<T, T>&) {} #   void g(T& x) { f(x); } # # To work around this bug, the code generator is a bit more # complicated, and is taught how to handle this situation. </ins> local = {} <ins> arguments = declaration['arguments'] tensor_args = [arg for arg in arguments if arg['simple_type'] in {'Tensor', 'TensorList'}] if len(tensor_args) == 1 and tensor_args[0]['simple_type'] == 'TensorList': # Special case for TensorList.  This only works when there # is a single argument local['trace_inputs'] = "cast_tensor_list({})".format(declaration['arguments'][0]['name']) else: local['trace_inputs'] = CodeTemplate("{ ${tensor_args} }").substitute(env) </ins> local['record_attributes'] = [] for arg in declaration['arguments']: if arg['simple_type'] in {'Tensor', 'TensorList'}:

Discussion:
when executing:
I'm seeing this as well as of last night. Its preceded by many warnings about : But the first error is the one about . I went back, and I get a good install as late as commit
Change:
v = self.ValueTensor([5]).cuda(0) self.assertRaises(RuntimeError, lambda: self.SparseTensor(i, v, torch.Size([3]))) <ins> def _test_new_device(self, size, device): with torch.cuda.device(device): x = torch.cuda.sparse.DoubleTensor(*size) self.assertEqual(x.get_device(), device) x1 = x.new() x2 = x.new(2, 3) self.assertEqual(x1.get_device(), device) self.assertEqual(x2.get_device(), device) </ins> @cuda_only <del> @unittest.skipIf(torch.cuda.device_count() < 2, "only one GPU detected") def test_new_device(self): with torch.cuda.device(1): x = torch.cuda.sparse.DoubleTensor(30, 2) self.assertEqual(x.get_device(), 1) </del> <ins> def test_new_device_single_gpu(self): self._test_new_device((), 0) self._test_new_device((30, 20), 0) self._test_new_device((30, 20, 10), 0) </ins> <del> y = x.new(2, 3) self.assertEqual(y.get_device(), 1) </del> <ins> @cuda_only @unittest.skipIf(torch.cuda.device_count() < 2, "only one GPU detected") def test_new_device_multi_gpu(self): self._test_new_device((), 1) self._test_new_device((30, 20), 1) self._test_new_device((30, 20, 10), 1) </ins> class TestUncoalescedSparse(TestSparse):

Discussion:
This causes to not work properly on newly created empty sparse tensors. In the example below, both should print 1 instead.
Change:
if bias is not None: self.bias.data.uniform_(-0.1, 0.1) <del> def forward(self, input): # See the autograd section for explanation of what happens here. return Linear()(input, self.weight, self.bias) </del> <ins> def forward(self, input): # See the autograd section for explanation of what happens here. return Linear()(input, self.weight, self.bias) </ins> Writing custom C extensions

Discussion:
Broken download links for libtorch v1.8.0 with CUDA 10.2 for Linux (pre-cxx11 and cxx11 ABI) and Windows (Release and Debug versions) on the PyTorch website. Steps to reproduce the behavior: on any of the following links leads to an "Access Denied" error: The chosen libtorch version is downloaded. PyTorch Version: .1.8.0 OS: Linux and Windows CUDA version: 10.2
The correct links are listed below. cc
Thanks!
Same issue also happens for CUDA 11.1 when trying to download windows precompiled binaries Changing to in the links suggested by still leads to Access Denied page
Okay I wrote a simple script to test if all of our libtorch packages are actually downloadable, sharing results here: <details<summarypkgs </summaryCommand I used to generate these (from the root of ): </detailsOutput being: I did find that this issue is correct in that the installation instructions were incorrect for CUDA 10.2 and I created a PR to resolve this:
Change:
} private: <del> friend class VariableVersion; </del> <ins> friend struct VariableVersion; </ins> int expected_version; std::shared_ptr<VersionBlock> version_block;  // may be null };

Discussion:
when executing:
I'm seeing this as well as of last night. Its preceded by many warnings about : But the first error is the one about . I went back, and I get a good install as late as commit
Change:
def batch_norm(input, running_mean, running_var, weight=None, bias=None, training=False, momentum=0.1, eps=1e-5): <ins> size = list(input.size()) if reduce(mul, size[2:], size[0]) == 1: raise ValueError('Expected more than 1 value per channel, got input size {}'.format(size)) </ins> f = torch._C._functions.BatchNorm(running_mean, running_var, training, momentum, eps, torch.backends.cudnn.enabled) return f(input, weight, bias)

Discussion:
Right now it makes everything .
To be clear, for a tensor, the problem is only when all of N, H, and W are 1. (So BatchNorm2d on batch size of 1 is OK as long as you don't have a 1x1 image). What's the desired behavior? The only reasonable behavior I can think of is: Raise an exception when the dimensions of which you are normalizing are one or Output zero (+ optional affine transform) I'm not sure the outputing zero is a good idea. I can't think of a case where that's what you want.
Change:
sum  ); } <del> THArgCheckWithCleanup((sum > 0), THCleanup(THDoubleTensor_free(cum_dist);), 2, </del> <ins> THArgCheckWithCleanup((sum > 0), THCleanup(THDoubleTensor_free(cum_dist); if (start_dim == 1) THTensor_(resize1d)(prob_dist, n_categories);), 2, </ins> "invalid multinomial distribution (sum of probabilities <= 0)"); /* normalize cumulative probability distribution so that last val is 1 i.e. doesn't assume original prob_dist row sums to one */

Discussion:
I was reading the documentation of method in the . I tried this in IPython environment. Of course, error occurs because 5 is larger than the size of weights and I didn't set . The error message is like this: But after this, the dimensions of weights changed. If I run as following without runtime exception, then the dimensions of weights do not change. This is caused by not cleaning up after exception raised in this . Maybe the solution is adding cleaning up code in the .
The problem emerged in two other situations as well, here are the examples: I'm sending a pull request with the fix. Thanks for reporting the problem. :)
Change:
assert data_tensor.size(0) == target_tensor.size(0) self.data_tensor = data_tensor self.target_tensor = target_tensor <del> if self.data_tensor.dim() == 1: self.data_tensor = self.data_tensor.view(-1, 1) if self.target_tensor.dim() == 1: self.target_tensor = self.target_tensor.view(-1, 1) </del> def __getitem__(self, index): return self.data_tensor[index], self.target_tensor[index]

Discussion:
TensorDataset adds casts 1D target tensor to 2D which is not acceptable by NLL loss layer.
Fixed in .
Change:
def grid_sampler(input, grid, padding_mode): <del> if cudnn.is_acceptable(input.data) and padding_mode == 'zeros' and input.dim() == 4: </del> <ins> if (cudnn.is_acceptable(input.data) and padding_mode == 'zeros' and input.dim() == 4 and input.size(1) <= 1024):  # as of cudnn 7102, will not work for larger than 1024 </ins> return torch.cudnn_grid_sampler(input, grid) else: return GridSampler.apply(input, grid, padding_mode)

Discussion:
PyTorch version: 0.4.0 Is debug build: No CUDA used to build PyTorch: 9.0.176 OS: Ubuntu 16.04.3 LTS GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.5) 5.4.0 CMake version: version 3.9.4 Python version: 3.6 Is CUDA available: Yes CUDA runtime version: 9.0.176 GPU models and configuration: GPU 0: TITAN X (Pascal) GPU 1: TITAN X (Pascal) Nvidia driver version: 384.98 cuDNN version: Probably one of the following: /usr/lib/x8664-linux- /usr/lib/x8664-linux- /usr/lib/x8664-linux-gnu/libcudnnstaticv7.a /usr/local/lib/python2.7/dist- /usr/local/lib/python3.5/dist- Versions of relevant libraries: [pip3] numpy (1.14.1) [pip3] numpydoc (0.6.0) [pip3] torch (0.4.0) [pip3] torchvision (0.2.0) [conda] cuda90 1.0 h6433d270 pytorch [conda] magma-cuda90 2.3.0 1 soumith [conda] pytorch 0.4.0 py36cuda9.0.176cudnn7.1.21 [cuda90] pytorch [conda] torchvision 0.2.1 py361 pytorch
() returns 7102
Possibly related:
I'm not sure why, but I can't reproduce this on master. I can reproduce on 0.4 though.
Never mind, this error happens when CUDNN is updated regardless of pytorch version.
Change:
#include <tuple> #include <TH/THMath.h> <del> #include "THP.h" #include "copy_utils.h" #include "DynamicTypes.h" </del> <ins> #include "torch/csrc/THP.h" #include "torch/csrc/copy_utils.h" #include "torch/csrc/DynamicTypes.h" </ins> <del> #include "generic/Tensor.cpp" #include <TH/THGenerateAllTypes.h> #include "generic/Tensor.cpp" #include <TH/THGenerateHalfType.h> </del> <ins> //generic_include TH torch/csrc/generic/Tensor.cpp </ins>

Discussion:
Trying to run on github actions via AWS instance gives me: I will of course set up now, but it shouldn't crash like that. May be it should use as a fallback? pt-2.0 cc
Is this a multitenant setup? Also curious which distro are you running, I was under the impression that most linux distros set home by default
this is AWS image. one default user. I think it's user, but it wants you to run as root, which is weird. I have manually now set to overcome this.
We should update triton to use pathlib or something
Change:
if [[ $WITH_CUDA -eq 1 ]]; then GLOO_FLAGS="-DUSE_CUDA=1 -DNCCL_ROOT_DIR=$INSTALL_DIR" fi <ins> CWRAP_FILES=" $BASE_DIR/torch/csrc/Declarations.cwrap; $BASE_DIR/torch/lib/ATen/Local.cwrap; $BASE_DIR/torch/lib/THNN/generic/THNN.h; $BASE_DIR/torch/lib/THCUNN/generic/THCUNN.h" </ins> # Used to build an individual library, e.g. build TH function build() {

Discussion:
There are previous issues addressing this, but I want to write this up with a plan in a new issue. We should make differentiable instead of having a wrapper class. The Variable class adds conceptual overhead without sufficient benefit. It makes it harder for new users to adopt PyTorch by adding a second API which is similar but not exactly the same as Tensor. There are few prerequisites: on s need to be at least as fast as operations on () need scalar support at the Python level () The constructor will be deprecated and become a no-op. The accessor will be deprecated and return a view that has requiresgrad=False. The constructors will take an optional keyword argument There will be a number of internal changes, but we shouldn't break most existing code: performance reasons, we will no longer build up the graph of operations by default when unless the operation uses stochastic variables. We'll use a context manager or global variable to enable graph creation. new differentiable Tensor will need support all the operations currently on Tensor, including in-place and operations. -place operations on leaf Variables which requiregrad are not currently supported. In the new API, the tensor's value will be updated and it will remain a leaf node. (This is important for optimizers, which update leaf Variables in-place). There are probably a few other subtle differences between Tensor and Variable APIs. I'll update this issue as we address them. See also: 23/738 # Greg will be fixing the tests, Tensor vs. Variable requiresgrad 2/1837 # Just testbroadcast from TestTorch OK OK # error at top level OK OK OK OK OK OK OK OK 10/120 # missing sspaddmm, dsmm/mm, addmm 1/257 TestTorch.testbroadcast 2OK
Properly handling in-place might not be as easy as it seems. Sometimes you actually need to move the Variable in the graph. Consider this case:
One of the cases of differences between variable and tensor is computation of var/std:
IMO Variable gives user a clear difference between graph node and data. .data attribute and Variable constructor provide convenient and explicit ways to take control of the graph (or graphs).
This is done. (Still some clean-up to do)
Is the conda PyTorch out of date? I'm getting the following error: .
Yes, we haven't released 0.4, which has the Variable Tensor merge, yet
Change:
# Don't store the actual traceback to avoid memory cycle _queued_calls.append((callable, traceback.format_stack())) <ins> _lazy_call(_check_capability) </ins> class DeferredCudaCallError(Exception): pass

Discussion:
There's quite a few user-reports of installing the CUDA 7.5 binary of pytorch and then running on GTX 1080 / 1070, and there will be reports of installing 7.5 / 8.0 binaries and running on V100. Raising a user-warning in such situations and letting them know will fix the problem.
The version of CUDA used to compile the binary should be available thanks to this PR:
I think is a more robust way of getting the CUDA used to compile PyTorch, as currently if there are multiple CUDA installed in the machine, the heuristics used in might not match those used by CMake FindCUDA. I think that we should modify the to return the cuda version used during compilation. I have the impression that it will not require changing much because doesn't require cuda to execute. What do you think?
Change:
__global__ void renormRowsL1(T* dist, long rows, long cols) { extern __shared__  unsigned char my_smem[]; T *smem = reinterpret_cast<T *>(my_smem); <ins> T zero = ScalarConvert<int, T>::to(0); T val; </ins> for (int64_t row = blockIdx.x; row < rows; row += gridDim.x) { T sum = ScalarConvert<int, T>::to(0); for (int64_t col = threadIdx.x; col < cols; col += blockDim.x) { <del> sum = THCNumerics<T>::add(sum, dist[row * cols + col]); </del> <ins> val = dist[row * cols + col]; assert(THCNumerics<T>::ge(val, zero)); sum = THCNumerics<T>::add(sum, val); </ins> } <del> sum = reduceBlock(smem, blockDim.x, sum, ReduceAdd<T, T>(), ScalarConvert<int, T>::to(0)); </del> <ins> sum = reduceBlock(smem, blockDim.x, sum, ReduceAdd<T, T>(), zero); </ins> if (threadIdx.x == 0) { <ins> assert(THCNumerics<T>::gt(sum, zero)); </ins> smem[0] = sum; } __syncthreads();

Discussion:
Looking at the values returned from tn.multinomial(1) and counting them, it seems like it returns the values of the multinomial distribution taken from t, i.e. that it has ignored the signs of the values. However, it seem like always returns a tensor filled with zeros.
looking into this
Change:
if declaration['inplace']: env['return_value'] = 'self' <del> env['return_name'] = 'self' </del> env['result'] = 'static_cast<Variable&>(self)' <ins> env['trace_outputs'] = '{ self }' </ins> else: env['return_value'] = '{}(std::move(ret))'.format(declaration['return_type']) <del> env['return_name'] = 'ret' </del> env['result'] = 'std::get<0>(ret)' if len(declaration['returns']) > 1 else 'ret' <ins> if len(declaration['returns']) > 1: # NB: This won't work if we get heterogenous outputs outs = ['std::get<{}>(ret)'.format(i) for i, v in enumerate(declaration['returns']) if v['type'] == 'Tensor'] else: outs = ['ret'] env['trace_outputs'] = CodeTemplate("{ ${outs} }").substitute(outs=outs) </ins> if any(arg['simple_type'] in {'Generator', 'Storage'} for arg in arguments): env['record_trace'] = []

Discussion:
when executing:
I'm seeing this as well as of last night. Its preceded by many warnings about : But the first error is the one about . I went back, and I get a good install as late as commit
Change:
} cudaError_t CudaStorageWeakRefAllocator::free(void* ptr) { <del> PyGILState_STATE gstate = PyGILState_Ensure(); PyObject_SetAttrString(object.get(), "cdata", Py_None); object = nullptr; PyGILState_Release(gstate); </del> <ins> { AutoGIL gil; PyObject_SetAttrString(object.get(), "cdata", Py_None); object = nullptr; } </ins> cudaError_t err = allocator->free(allocatorContext, ptr); delete this; return err;

Discussion:
When both PyTorch and Pandas are imported, calling results in a segfault. The issue was originally reported by It looks like calling returns a null pointer here: Examining in GDB: I suspect it's both PyTorch and Pandas doing threading magic and somehow getting in each other's way. I haven't yet been able to provide a minimal example that triggers this behaviour and does not use my package; it's likely that I'm doing something odd that causes this. More documentation here: Possibly related issues: and Python version: 3.6 PyTorch version: v0.2.0 Pandas version: 0.20.1
Change:
<ins> #include "torch/csrc/utils/object_ptr.h" #include <Python.h> template<> void THPPointer<PyObject>::free() { if (ptr) Py_DECREF(ptr); } template class THPPointer<PyObject>; </ins>

Discussion:
Go to Look at the html. It contains the following: ! This problem manifests in google search. Searching for 1.8.1 docs brings up the "1.8.0 docs": ! We can probably fix this by doing a quick sed on the website but we should develop a solution for future releases. I think this has happened before in the past and am not sure what causes it. cc
Change:
def forward(input, weight, hidden): if batch_first: <del> input.transpose(0, 1) </del> <ins> input = input.transpose(0, 1) </ins> nexth, output = func(input, hidden, weight) if batch_first: <del> output.transpose(0, 1) </del> <ins> output = output.transpose(0, 1) </ins> return output, nexth

Discussion:
The last line here fails on CPU or when CUDNN is otherwise unavailable: This is because accidentally assumes 's in-place semantics rather than the functional semantics of ( gets it right): I can push a PR that fixes this, or one of the devs can put it in the next bugfix PR:
If you have a ready solution for your issues feel free to send the PR straight away. Contributions are welcome!
Thanks