jiebi/CodeConvo huggingface dataset:
Change:
The following code: produces this error Also, removing the ReLU from the network: causes the error to change to: However, when the second convolution is removed as well: then the code runs error free I'm on a ppc64le architecture with a P100 GPU, CUDA 8.0 and PyTorch version '0.4.0a0+'
I'm also having this problem using branch v0.3.0
i could reproduce this as well. we'll look into this.
I'm having the same error :/
same error with gradient penalty for WGAN
When you execute , are you cause memory leak
no there shouldn't be a memory leak.
I am having an out of memory error as well, only when applying backward to the output of this method. I am using 3 titans GPUs so memory is not a problem I would assume. def computeGP(netD, realdata, realembed, fakedata, LAMBDA): BATCHSIZE = (0) alpha = (BATCHSIZE, 1) alpha = (BATCHSIZE, int(realdata.nelement() / BATCHSIZE)).contiguous().view(BATCHSIZE, 3, 64, 64) alpha = () interpolates = alpha realdata + ((1 - alpha) fakedata) interpolates = () interpolates = autograd.Variable(interpolates, requiresgrad=True) discinterpolates, = netD(interpolates, realembed) gradients = (outputs=discinterpolates, inputs=interpolates, (()).cuda(), creategraph=True, retaingraph=True, onlyinputs=True)[0] gradientpenalty = (((2, dim=1) - 1) * 2).mean() LAMBDA return gradient_penalty `
I am also using it with gradient penalty for WGAN, But When I use gradient_penalty.backward(), the memory leak is happen, My pytorch is 0.4.0a0+.
I can reproduce the memory leak with this: However the leak disappears when only one convolution is used: Also, if nn.Linear is used, there is no leak. Torch version: 0.4.0a0+
I've opened a separate issue for that memory leak:

Discussion:
// For Convolution strategies that don't implicitly handle grad_bias, we add a helper // function here to perform it using simple Tensor operators static at::Tensor compute_grad_bias(const at::Tensor& grad_output) { <del> // grad_output is in N, C, H, W, we re-shape and reduce over spatial dims and batches </del> <ins> // grad_output is in N, C, H, W, we re-shape and reduce over spatial dims and batches </ins> return grad_output.contiguous().view({grad_output.size(0), grad_output.size(1), -1}).sum(0).sum(1); }

gI = apply_fn<Transpose>(0, 1)(gIt); } } <ins> if (should_compute_output(0) && !ggO.defined()) ggO = at::zeros_like(gO); if (should_compute_output(1) && !gI.defined()) gI = at::zeros_like(input); if (should_compute_output(2) && !gW.defined()) gW = at::zeros_like(weight); </ins> return {ggO, gI, gW}; }
Change:
While the fast path for cuDNN RNNs introduced in includes a check intended to fall back to copying weights if weights are moved or aliased, this only works if the aliasing happens after the weights are first compacted. But the most natural place to tie weights is inside a module's method, which runs before , the typical time when is first invoked. Here's an MWE. The three runs (CPU, weight tying before , and weight tying after ) should all give the same results, and they do before , but they have not since:

Discussion:
weight_data[:] = 4 self.assertEqual(weight_data, all_vars[4].data) <ins> @unittest.skipIf(not TEST_CUDNN, 'CUDNN not available') def test_cudnn_weight_tying(self): rnns = [ nn.LSTM(10, 20, batch_first=True, bidirectional=True), nn.GRU(10, 20, batch_first=True, bidirectional=True), nn.RNN(10, 20, batch_first=True, bidirectional=True) ] for rnn in rnns: rnn.bias_ih_l0_reverse = rnn.bias_ih_l0 rnn.cuda() input = Variable(torch.randn(5, 4, 10).cuda(), requires_grad=True) hx = Variable(torch.randn(2, 5, 20).cuda(), requires_grad=True) all_vars = [input, hx] + list(rnn.parameters()) opt = torch.optim.SGD(rnn.parameters(), lr=0.1) opt.zero_grad() if isinstance(rnn, nn.LSTM): cx = Variable(torch.randn(2, 5, 20).cuda(), requires_grad=True) all_vars[2:2] = [cx] hx = (hx, cx) with warnings.catch_warnings(record=True) as w: output = rnn(input, hx) self.assertEqual(len(w), 1) self.assertIn('weights are not part of single contiguous chunk of memory', w[0].message.args[0]) output[0].sum().backward() opt.step() with warnings.catch_warnings(record=True) as w: output_cuda = rnn(input, hx) rnn.cpu() hx = (hx[0].cpu(), hx[1].cpu()) if isinstance(rnn, nn.LSTM) else hx.cpu() output_cpu = rnn(input.cpu(), hx) self.assertEqual(output_cuda, output_cpu) </ins> @unittest.skipIf(not TEST_CUDA, 'CUDA not available') def test_cuda_rnn_fused(self): def copy_rnn(rnn1, rnn2):

self._data_ptrs = [] return <ins> # If any parameters alias, we fall back to the slower, copying code path. This is # a sufficient check, because overlapping parameter buffers that don't completely # alias would break the assumptions of the uniqueness check in # Module.named_parameters(). unique_data_ptrs = set(p.data_ptr() for l in self.all_weights for p in l) if len(unique_data_ptrs) != sum(len(l) for l in self.all_weights): self._data_ptrs = [] return </ins> with torch.cuda.device_of(any_param): # This is quite ugly, but it allows us to reuse the cuDNN code without larger # modifications. It's really a low-level API that doesn't belong in here, but

fn.w_desc = rnn.init_weight_descriptor(fn, fn.weight_buf) # Slice off views into weight_buf <del> params = rnn.get_parameters(fn, handle, fn.weight_buf) </del> all_weights = [[p.data for p in l] for l in self.all_weights] <ins> params = rnn.get_parameters(fn, handle, fn.weight_buf) </ins> # Copy weights and update their storage rnn._copyParams(all_weights, params)
Change:
It seems that can change memory outside x, when x is a cuda tensor. If x is non-cuda tensor, we get: In contrast, when x is cuda tensor, does not make any error It's hard to share the whole code, but I have noticed that such operation outside a tensor did affect the performance of existing network, so I'm afraid that this op can change arbitrary memory on GPU which can be dangerous. Could you check this out?
This snippet is fine - it's enough for us to reproduce the problem. It appears we're missing some out-of-bounds checks (we have them for other indexing functions). Thanks for reporting.
working on this

Discussion:
// Lua indices begin at 1 IndexType dstIndex_ = indices.data[IndexToOffset<int64_t, IndexType, IdxDim>::get(dstIndex, indices)] - TH_INDEX_BASE; <del> assert(dstIndex < dstFillDimSize); </del> <ins> assert(dstIndex_ < dstFillDimSize); </ins> // We stride over the output ignoring the indexed dimension // (innerSize), whose offset calculation is handled differently
Change:
Hi, I was trying to update from source but I get an error "no suitable constructor exists to convert from "int" to "__half"" when trying to build I am working with python 2.7 cuda 7.5 and GCC 4.8.3 in openSUSE. Any suggest? Here is the output error when building from source:
ATen Excluded: set(['bernoulli', 'bernoulli']) [ 0%] Built target atenfilesaregenerated [ 0%] Building NVCC (Device) object [ 1%] Building NVCC (Device) object [ 1%] Building NVCC (Device) object [ 1%] Building NVCC (Device) object [ 1%] Building NVCC (Device) object [ 2%] Building NVCC (Device) object [ 2%] Building NVCC (Device) object [ 2%] Building NVCC (Device) object (77): error: no suitable constructor exists to convert from "int" to "half" (77): error: no suitable constructor exists to convert from "float" to "half" 2 errors detected in the compilation of "/tmp/tmpxft0000587a00000000-". CMake Error at (message): Error generating file recipe for target '' failed make[2]: [] Error 1 make[2]: Waiting for unfinished jobs.... CMakeFiles/Makefile2:194: recipe for target '' failed make[1]: [] Error 2 Makefile:129: recipe for target 'all' failed make: [all] Error 2
NVIDIA cuDNN v6.x or above
same problem here. gcc (Ubuntu 4.8.4), python 2.7, cuda 7.5, cuDNN 6.0.21
Same problem here. CUDA: v7.5 CUDNN: v6.0

Discussion:
return THC_float2half(0); #else THError("Cublas_Hdot requires CUDA 8.0+"); <del> return THC_half2float(0); </del> <ins> return THC_float2half(0); </ins> #endif } #endif
Change:
This issue tracks the components / tests that are not working on Windows: [ ] : currently disabled on Windows because Windows doesn't have and we need to look for substitutes [ ] : Fuser is disabled on Windows because the current implementation uses symbols from Linux-specific headers and . We will need to find alternatives for Windows. [ ] : some parts of and are disabled because Windows doesn't support opening an already opened file (see discussion at and ) [ ] : currently doesn't work with Windows ( is the porting diff) [ ] : in causes intermittent CUDA out-of-memory error on Windows [ ] , : DataLoader with multiple workers causes intermittent CUDA out-of-memory error on Windows. [x] [x] (done in ) [x] [x] - - [x] [x] [x] - For more discussions, also see: cc
The first one is solved by but I think DataLoader can be further improved when is fininshed.
Cool I will mark it as resolved :)
The has been in .
We can try to revert and , because the memory leak in the CPU side could also cause CUDA errors.
Are they fixed by I think we can revert them after we merge the PR.
I think this may be related. Since once the memory of the CPU side is low, the will fail with too.
I think it could be. For what it's worth, when I tried to inspect the CUDA OOM error, showed no process that was taking memory, but running CUDA tests on the machine would still fail.
Now that is merged into master, could you please try to revert the changes on and ?
Awesome! Just to understand it better: does fix both numworker=1 and numworker1 cases?
Yes, they are both solved.
I guess we should mark the cpp_extension test as completed, for it's now enabled in CI.
Closing this issue due to age and because its references are long out of date. For example, distributed tests now have their own jobs.

Discussion:
raise ValueError('num_workers cannot be negative; ' 'use num_workers=0 to disable multiprocessing.') <ins> if sys.platform == "win32" and self.num_workers > 0: raise ValueError('num_workers > 0 is not supported on Windows') </ins> if batch_sampler is None: if sampler is None: if shuffle:
Change:
Turning on DEBUG=1 turns off inlining. However, if you declare a function as , it won't get object code generated for it, which means there will be undefined symbol errors. You might not notice this with a release build, since everything gets inlined away. We can catch errors like this by doing CI with DEBUG=1. Probably just pick one platform and run that only.

Discussion:
# Target only our CI GPU machine's CUDA arch to speed up the build export TORCH_CUDA_ARCH_LIST=5.2 <ins> if [[ "$BUILD_ENVIRONMENT" == *trusty-py3.6-gcc5.4* ]]; then export DEBUG=1 fi </ins> WERROR=1 python setup.py install # Add the test binaries so that they won't be git clean'ed away
Change:
The documentation of the parameter to is one of the strangest I've seen... what does (the default) do???
Ha it is somewhat weird indeed. When , the returned U and V matrices are square orthonormal matrices. I will send out a PR to clarify this.

Discussion:
`U, S, V = torch.svd(A)` returns the singular value decomposition of a real matrix `A` of size `(n x m)` such that :math:`A = USV^T`. <del> `U` is of shape :math:`(n times min(n, m))`. </del> <ins> `U` is of shape :math:`(n times n)`. </ins> <del> `S` is a diagonal matrix of shape :math:`(min(n, m) times min(n, m))`, represented as a vector of size :math:`min(n, m)` containing the diagonal entries. </del> <ins> `S` is a diagonal matrix of shape :math:`(n times m)`, represented as a vector of size :math:`min(n, m)` containing the diagonal entries. </ins> <del> `V` is of shape :math:`(m times min(n, m))`. </del> <ins> `V` is of shape :math:`(m times m)`. </ins> <del> :attr:`some` represents the number of singular values to be computed. If `some=True`, it computes some and `some=False` computes all. </del> <ins> If :attr:`some` is ``True`` (default), the returned `U` and `V` matrices will contain only :math:`min(n, m)` orthonormal columns. </ins> .. note:: Irrespective of the original strides, the returned matrix `U` will be transposed, i.e. with strides `(1, n)` instead of `(n, 1)`.

Args: input (Tensor): the input 2-D tensor <del> some (bool, optional): controls the number of singular values to be computed </del> <ins> some (bool, optional): controls the shape of returned `U` and `V` </ins> out (tuple, optional): the output tuple of tensors Example::
Change:
Ref , the input only supports format. This is not consistent with which supports also .
it doesn't make much sense to support instance norm for because it is usually normalizing over dimension.
Okay, so the documentation should be updated?

Discussion:
class InstanceNorm1d(_InstanceNorm): <del> r"""Applies Instance Normalization over a 2d or 3d input that is seen as a mini-batch. </del> <ins> r"""Applies Instance Normalization over a 3d input that is seen as a mini-batch. </ins> .. math::
Change:
A minimal example: Although has and this type is listed as supported, the array can't be converted. Consider that is initialized with an upper bound of type . I'm not completely sure, whether this issue is related to torch or numpy, but numpy initializes the array with values as expected, whereas torch throws an inconsistent error message.
Edit: nevermind, I didn't run the same exact thing ! Works for me on master and 0.3. Seems like it could be related to

Discussion:
np.int64, np.int32, np.int16, <del> np.uint8 </del> <ins> np.uint8, np.longlong, </ins> ] for dtype in dtypes: array = np.array([1, 2, 3, 4], dtype=dtype)

case NPY_DOUBLE: return kDouble; case NPY_FLOAT: return kFloat; case NPY_HALF: return kHalf; <ins> case NPY_LONGLONG: </ins> case NPY_INT64: return kLong; case NPY_INT32: return kInt; case NPY_INT16: return kShort;
Change:
Minimal example: Torch initializes the tensor with values which are different from the numpy array. I've seen issues for similar kinds of problems (see ). Maybe the easiest solution to all these initialization problems might be, to check for the same values of the original and initialized array/tensor instead of initializing tensors silently with wrong numbers. What do you think? Furthermore, I've opened an issue which is also related this one in terms of initializing a numpy array with a number of type and then initializing a torch tensor from that array, see Feel free to merge them if you think that both issues can be solved together.
Could you please provide some information about what version of pytorch and numpy you're using? ! It works as expected for me on master
I'm using torch 0.3.0.post4 (latest one on website of PyTorch) and numpy 1.13.3.
I'm running the same thing, this is strange. In the first line , I assumed that came from . Is this the case?
Nope, it's indeed the python built-in type. Actually this issue came up when I tried to get the shape of (i.e. , , returns ).
Thanks for the clarification, I'm now seeing your results. Looks like the tensor constructor doesn't work well with python2.7 long types in this case
Sure, no problem. Thanks for looking into it!
The problem seems to be that np.longlong and np.int64 are the same thing (at least on a 64bit system, I think), but the NPYLONGLONG and NPYINT64 macros are different... I'll put up a pr for this soon.

Discussion:
np.int64, np.int32, np.int16, <del> np.uint8 </del> <ins> np.uint8, np.longlong, </ins> ] for dtype in dtypes: array = np.array([1, 2, 3, 4], dtype=dtype)

case NPY_DOUBLE: return kDouble; case NPY_FLOAT: return kFloat; case NPY_HALF: return kHalf; <ins> case NPY_LONGLONG: </ins> case NPY_INT64: return kLong; case NPY_INT32: return kInt; case NPY_INT16: return kShort;
Change:
odd difference between cuda Tensor and Variable: encountered in torch.distributions.Categorical, built yesterday from master
related to hmm, here's the issue is for variables, so probably not (immediately) related
different code path but similar issue
This is a regression from the latest 0.3 binaries, which don't have this error.

Discussion:
THArgCheck(THCTensor_(nDimension)(state, src) <= MAX_CUTORCH_DIMS, 2, CUTORCH_DIM_WARNING); THArgCheck(THCudaLongTensor_nDimension(state, index) <= MAX_CUTORCH_DIMS, 2, CUTORCH_DIM_WARNING); <del> int srcDims = THCTensor_(nDimension)(state, src); THArgCheck(srcDims > 0, 2, "Source tensor is empty"); </del> <ins> if (numIndices == 0) { return; } </ins> if (accumulate) { // wrap indices so to replace negative indices

('ones', small_3d, lambda t: [1, 2, 3, 4, 5],), ('permute', new_t(1, 2, 3, 4), lambda t: [2, 1, 3, 0],), ('put_', new_t(2, 5, 3), lambda t: [long_type(t)([[0], [-2]]), t([[3], [4]])],), <ins> ('put_', new_t(2, 3), lambda t: [long_type(t)([]), t([])], 'empty'), </ins> ('put_', new_t(2, 2), lambda t: [long_type(t)([[1], [-3]]), t([[1], [2]]), True], 'accumulate'), ('prod', small_2d_oneish, lambda t: [],), ('prod', small_3d, lambda t: [1], 'dim'),

idx = Variable(torch.LongTensor()) self.assertEqual(x[idx].numel(), 0) <ins> # empty assignment should have no effect but not throw an exception y = x.clone() y[idx] = -1 self.assertEqual(x, y) mask = torch.zeros(4, 3).byte() y[mask] = -1 self.assertEqual(x, y) </ins> def test_basic_advanced_combined(self): # From the NumPy indexing example x = Variable(torch.arange(0, 12).view(4, 3))
Change:
Hello, I'm trying to build pytorch (I tried both v3_buildfix and master branch) on a raspberry pi 3 but i'm getting that error: Note: I'm building without CUDA and DISTRIBUTED support. OS: Ubuntu 16.04 (Mate) gcc: arm-linux-gnueabihf-gcc (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.5) 5.4.0 cmake: v3.5.1
Can you try building with and let me know if that fixes it?
Thanks, it compiles now ;)

Discussion:
#include <initializer_list> #include <iostream> #include <functional> <ins> #include <cstddef> </ins> #ifdef _MSC_VER #ifdef Type

static void setattr(jit::Node* n, jit::Symbol name, const at::IntList& v)  { n->is_(name, v); } static void setattr(jit::Node* n, jit::Symbol name, bool v)                { n->i_(name, v); } static void setattr(jit::Node* n, jit::Symbol name, double v)              { n->f_(name, v); } <del> template<unsigned long N> </del> <ins> template<std::size_t N> </ins> static void setattr(jit::Node* n, jit::Symbol name, std::array<bool, N> v) { n->is_(name, std::vector<int64_t>(v.begin(), v.end())); } VariableType::VariableType(Context* context, Type* baseType)
Change:
Hi, In the is written: "lr values are [0.05, 0.005, ...]." If I understand it correctly, it should be [0.5, 0.05, 0.005, ...].
That is correct. Please feel free to send a PR to fix it! (otherwise, I can)
If you don't mind to change it yourself, I'm not contributor yet and don't want to start just for one comment. I guess is the same case.

Discussion:
last_epoch (int): The index of last epoch. Default: -1. Example: <del> >>> # Assuming optimizer uses lr = 0.5 for all groups </del> <ins> >>> # Assuming optimizer uses lr = 0.05 for all groups </ins> >>> # lr = 0.05     if epoch < 30 >>> # lr = 0.005    if 30 <= epoch < 60 >>> # lr = 0.0005   if 60 <= epoch < 90

last_epoch (int): The index of last epoch. Default: -1. Example: <del> >>> # Assuming optimizer uses lr = 0.5 for all groups </del> <ins> >>> # Assuming optimizer uses lr = 0.05 for all groups </ins> >>> # lr = 0.05     if epoch < 30 >>> # lr = 0.005    if 30 <= epoch < 80 >>> # lr = 0.0005   if epoch >= 80
Change:
My compiler (Apple LLVM version 8.1.0 (clang-802.0.42) complained about this, but I think it's reasonable. In the specialization THCNumerics<uint8tthe abs-function is defined in a selfrecursive way ie uint8t abs(uint8t a) { return abs(a); } Because an uint8t has no sign it should imo defined as uint8t abs(uint8t a) { return a; }
Yup that's an error. Can you please send a PR?
I am sorry. I am inexperienced with that and have not currently time. But I will soon invest some time in learning it.
I'm volunteer to push a patch for this issue.

Discussion:
static inline __host__ __device__  uint8_t mul(uint8_t a, uint8_t b) { return a * b; } static inline __host__ __device__  uint8_t sub(uint8_t a, uint8_t b) { return a - b; } static inline __host__ __device__  uint8_t div(uint8_t a, uint8_t b) { return a / b; } <del> static inline __host__ __device__  uint8_t abs(uint8_t a) { return abs(a); } </del> <ins> static inline __host__ __device__  uint8_t abs(uint8_t a) { return a; } </ins> }; template <>
Change:
Trivial report, unfortunately I can't make a PR at the moment, but I can submit a report. line 329 in should replace with

Discussion:
if mode not in {'min', 'max'}: raise ValueError('mode ' + mode + ' is unknown!') if threshold_mode not in {'rel', 'abs'}: <del> raise ValueError('threshold mode ' + mode + ' is unknown!') </del> <ins> raise ValueError('threshold mode ' + threshold_mode + ' is unknown!') </ins> if mode == 'min' and threshold_mode == 'rel': rel_epsilon = 1. - threshold self.is_better = lambda a, best: a < best * rel_epsilon
Change:
It seems like running has a dependency on first running . This command also seems to also error out when installing via a wheel. This creates a problem for tools like  which rely on the output of to examine dependencies. Is there anyway to allow this command to not error out when installing via a wheel? Full stack trace: I understand the preferred package manager for this project is conda but being officially endorsed from , pipenv has a fairly large audience:
thanks do you know when's the next planned release?
Hopefully within a month

Discussion:
pass def run(self): <ins> # Check if you remembered to check out submodules def check_file(f): if not os.path.exists(f): print("Could not find {}".format(f)) print("Did you run 'git submodule update --init'?") sys.exit(1) check_file(os.path.join(lib_path, "gloo", "CMakeLists.txt")) check_file(os.path.join(lib_path, "nanopb", "CMakeLists.txt")) check_file(os.path.join(lib_path, "pybind11", "CMakeLists.txt")) </ins> libs = [] if WITH_NCCL and not WITH_SYSTEM_NCCL: libs += ['nccl']

lib_path = os.path.join(cwd, "torch", "lib") <del> # Check if you remembered to check out submodules def check_file(f): if not os.path.exists(f): print("Could not find {}".format(f)) print("Did you run 'git submodule update --init'?") sys.exit(1) check_file(os.path.join(lib_path, "gloo", "CMakeLists.txt")) check_file(os.path.join(lib_path, "nanopb", "CMakeLists.txt")) check_file(os.path.join(lib_path, "pybind11", "CMakeLists.txt")) </del> tmp_install_path = lib_path + "/tmp_install" include_dirs += [ cwd,
Change:
Hi, I've updated pytorch to the lastest master () from github, but got an error when I use : What to do for workaround? Thanks! Jin
Here's a workaround. I think the problem is that pytorch hasn't initialized CUDA yet. At any rate, the following works for me:
Thanks but your workaround doesn't work for me. The error message still shows. Jin
Sorry, I meant the following: Could you please try if that works for you?
Yep. Just what I thought and was going to add comment here :smile: It works. Thank you! Jin

Discussion:
global Storage Tensor = _import_dotted_name(t) Storage = _import_dotted_name(t.replace('Tensor', 'Storage')) <ins> if 'cuda' in t: import torch.cuda torch.cuda.init() </ins> _C._set_default_tensor_type(Tensor)
Change:
Pybind11 has a bugfix here: which is not included in pytorch master. In brief, the bug causes two python modules, when both compiled with buggy version of pybind11, to conflict and crash at import. I've last week when debugging its conflict with pytorch. Hope pytorch can also upgrade to avoid potential conflict with other libraries.

Discussion:
<del> Subproject commit 9f6a636e547fc70a02fa48436449aad67080698f </del> <ins> Subproject commit add56ccdcac23a6c522a2c1174a866e293c61dab </ins>