jiebi/CodeConvo huggingface dataset:
Code change:
In torch, printing a tensor will print the whole tensor, even if that one is huge. This is sometimes annoying, especially when using a notebook, as it crashes it. Numpy has a very nice way to display big tensors by truncating them. Would it be possible so have something similar for pytorch?
Yeah, we haven't implemented it yet, but that's definitely on our roadmap
Implemented in .
I should say that I have some artefacts that appear with while printing a tensor. Here is an example output: gives Curiously, if I simply do It works fine and print nicely, with the truncation for large tensors. I'm using Python 2.7.6 (probably the one that came with my system and not from anaconda). Maybe it's just a bad setup in my machine?
hmmm, it works fine on python 2.7.12. wonder what changed between .6 and .12
ok i got a repro. I opened my ipython and did this:
my example was flawed, sorry. Using works for me, but when only entering it prints the s, as in your example.
Is it possible to tell the truncation point?
yes, use the function

Related discussion:
text: return type(self), (self.tolist(),) def __repr__(self): <del> return repr(str(self)) </del> <ins> return str(self) </ins> def __str__(self): # All strings are unicode in Python 3, while we have to encode unicode
Code change:
Right now it makes everything .
To be clear, for a tensor, the problem is only when all of N, H, and W are 1. (So BatchNorm2d on batch size of 1 is OK as long as you don't have a 1x1 image). What's the desired behavior? The only reasonable behavior I can think of is: Raise an exception when the dimensions of which you are normalizing are one or Output zero (+ optional affine transform) I'm not sure the outputing zero is a good idea. I can't think of a case where that's what you want.

Related discussion:
text: from numbers import Integral import warnings import math <ins> from operator import mul from functools import reduce </ins> import torch from torch._C import _infer_size
Code change:
Right now it makes everything .
To be clear, for a tensor, the problem is only when all of N, H, and W are 1. (So BatchNorm2d on batch size of 1 is OK as long as you don't have a 1x1 image). What's the desired behavior? The only reasonable behavior I can think of is: Raise an exception when the dimensions of which you are normalizing are one or Output zero (+ optional affine transform) I'm not sure the outputing zero is a good idea. I can't think of a case where that's what you want.

Related discussion:
text: def batch_norm(input, running_mean, running_var, weight=None, bias=None, training=False, momentum=0.1, eps=1e-5): <ins> size = list(input.size()) if reduce(mul, size[2:], size[0]) == 1: raise ValueError('Expected more than 1 value per channel, got input size {}'.format(size)) </ins> f = torch._C._functions.BatchNorm(running_mean, running_var, training, momentum, eps, torch.backends.cudnn.enabled) return f(input, weight, bias)
Code change:
Hi, I have installed two versions of CUDA. But I want to use one of it, can I specify it during setup process?
Try running
I have tried it, () still returns False. Thx for your reply!
Do you have the cuda driver installed?
Of course I do. It raise assertion error which shows "The NVIDIA driver on your system is too old(found version 8000)".
that's the answer to your problem. upgrade your driver.
Thx!

Related discussion:
text: | :attr:`stride` controls the stride for the cross-correlation. | If :attr:`padding` is non-zero, then the input is implicitly zero-padded on both sides <del> for :attr:`padding` number of points | :attr:`dilation` controls the spacing between the kernel points. It is harder to describe, but this `link`_ has a nice visualization of what :attr:`dilation` does. | :attr:`groups` controls the connections between inputs and outputs. </del> <ins> for :attr:`padding` number of points. | :attr:`dilation` controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but this `link`_ has a nice visualization of what :attr:`dilation` does. | :attr:`groups` controls the connections between inputs and outputs. `in_channels` and `out_channels` must both be divisible by `groups`. </ins> |       At groups=1, all inputs are convolved to all outputs. |       At groups=2, the operation becomes equivalent to having two conv layers side by side, each seeing half the input channels, and producing half the output channels, and both subsequently concatenated. <ins> At groups=`in_channels`, each input channel is convolved with its own set of filters (of size `out_channels // in_channels`). </ins> .. note::
Code change:
Hi, I have installed two versions of CUDA. But I want to use one of it, can I specify it during setup process?
Try running
I have tried it, () still returns False. Thx for your reply!
Do you have the cuda driver installed?
Of course I do. It raise assertion error which shows "The NVIDIA driver on your system is too old(found version 8000)".
that's the answer to your problem. upgrade your driver.
Thx!

Related discussion:
text: | :attr:`stride` controls the stride for the cross-correlation. | If :attr:`padding` is non-zero, then the input is implicitly zero-padded on both sides <del> for :attr:`padding` number of points | :attr:`dilation` controls the spacing between the kernel points. It is harder to describe, but this `link`_ has a nice visualization of what :attr:`dilation` does. | :attr:`groups` controls the connections between inputs and outputs. </del> <ins> for :attr:`padding` number of points. | :attr:`dilation` controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but this `link`_ has a nice visualization of what :attr:`dilation` does. | :attr:`groups` controls the connections between inputs and outputs. `in_channels` and `out_channels` must both be divisible by `groups`. </ins> |       At groups=1, all inputs are convolved to all outputs. |       At groups=2, the operation becomes equivalent to having two conv layers side by side, each seeing half the input channels, and producing half the output channels, and both subsequently concatenated. <ins> At groups=`in_channels`, each input channel is convolved with its own set of filters (of size `out_channels // in_channels`). </ins> The parameters :attr:`kernel_size`, :attr:`stride`, :attr:`padding`, :attr:`dilation` can either be:
Code change:
Hi, I have installed two versions of CUDA. But I want to use one of it, can I specify it during setup process?
Try running
I have tried it, () still returns False. Thx for your reply!
Do you have the cuda driver installed?
Of course I do. It raise assertion error which shows "The NVIDIA driver on your system is too old(found version 8000)".
that's the answer to your problem. upgrade your driver.
Thx!

Related discussion:
text: composed of several input planes. This module can be seen as the gradient of Conv1d with respect to its input. <del> It is sometimes (but incorrectly) refered to as a deconvolutional operation. </del> <ins> It is also known as a fractionally-strided convolution or a deconvolution (although it is not an actual deconvolution operation). | :attr:`stride` controls the stride for the cross-correlation. | If :attr:`padding` is non-zero, then the input is implicitly zero-padded on both sides for :attr:`padding` number of points. | If :attr:`output_padding` is non-zero, then the output is implicitly zero-padded on one side for :attr:`output_padding` number of points. | :attr:`dilation` controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but this `link`_ has a nice visualization of what :attr:`dilation` does. | :attr:`groups` controls the connections between inputs and outputs. `in_channels` and `out_channels` must both be divisible by `groups`. |       At groups=1, all inputs are convolved to all outputs. |       At groups=2, the operation becomes equivalent to having two conv layers side by side, each seeing half the input channels, and producing half the output channels, and both subsequently concatenated. At groups=`in_channels`, each input channel is convolved with its own set of filters (of size `out_channels // in_channels`). </ins> .. note::
Code change:
Hi, I have installed two versions of CUDA. But I want to use one of it, can I specify it during setup process?
Try running
I have tried it, () still returns False. Thx for your reply!
Do you have the cuda driver installed?
Of course I do. It raise assertion error which shows "The NVIDIA driver on your system is too old(found version 8000)".
that's the answer to your problem. upgrade your driver.
Thx!

Related discussion:
text: output_padding (int or tuple, optional): Zero-padding added to one side of the output groups (int, optional): Number of blocked connections from input channels to output channels bias (bool, optional): If True, adds a learnable bias to the output <ins> dilation (int or tuple, optional): Spacing between kernel elements </ins> Shape: - Input: :math:`(N, C_{in}, L_{in})`
Code change:
Hi, I have installed two versions of CUDA. But I want to use one of it, can I specify it during setup process?
Try running
I have tried it, () still returns False. Thx for your reply!
Do you have the cuda driver installed?
Of course I do. It raise assertion error which shows "The NVIDIA driver on your system is too old(found version 8000)".
that's the answer to your problem. upgrade your driver.
Thx!

Related discussion:
text: composed of several input planes. This module can be seen as the gradient of Conv2d with respect to its input. <del> It is sometimes (but incorrectly) refered to as a deconvolutional operation. </del> <ins> It is also known as a fractionally-strided convolution or a deconvolution (although it is not an actual deconvolution operation). </ins> | :attr:`stride` controls the stride for the cross-correlation. | If :attr:`padding` is non-zero, then the input is implicitly zero-padded on both sides <del> for :attr:`padding` number of points </del> <ins> for :attr:`padding` number of points. </ins> | If :attr:`output_padding` is non-zero, then the output is implicitly zero-padded on one side <del> for :attr:`output_padding` number of points | :attr:`dilation` controls the spacing between the kernel points. It is harder to describe, but this `link`_ has a nice visualization of what :attr:`dilation` does. | :attr:`groups` controls the connections between inputs and outputs. </del> <ins> for :attr:`output_padding` number of points. | :attr:`dilation` controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but this `link`_ has a nice visualization of what :attr:`dilation` does. | :attr:`groups` controls the connections between inputs and outputs. `in_channels` and `out_channels` must both be divisible by `groups`. </ins> |       At groups=1, all inputs are convolved to all outputs. |       At groups=2, the operation becomes equivalent to having two conv layers side by side, each seeing half the input channels, and producing half the output channels, and both subsequently concatenated. <ins> At groups=`in_channels`, each input channel is convolved with its own set of filters (of size `out_channels // in_channels`). </ins> The parameters :attr:`kernel_size`, :attr:`stride`, :attr:`padding`, :attr:`output_padding` can either be: <del> - a single ``int`` -- in which case the same value is used for the height and width dimension </del> <ins> - a single ``int`` -- in which case the same value is used for the height and width dimensions </ins> - a ``tuple`` of two ints -- in which case, the first `int` is used for the height dimension, and the second `int` for the width dimension
Code change:
Hi, I have installed two versions of CUDA. But I want to use one of it, can I specify it during setup process?
Try running
I have tried it, () still returns False. Thx for your reply!
Do you have the cuda driver installed?
Of course I do. It raise assertion error which shows "The NVIDIA driver on your system is too old(found version 8000)".
that's the answer to your problem. upgrade your driver.
Thx!

Related discussion:
text: The transposed convolution operator multiplies each input value element-wise by a learnable kernel, and sums over the outputs from all input feature planes. <del> **This module can be seen as the exact reverse of Conv3d**. It is sometimes (but incorrectly) refered to as a deconvolutional operation. </del> <ins> This module can be seen as the gradient of Conv3d with respect to its input. It is also known as a fractionally-strided convolution or a deconvolution (although it is not an actual deconvolution operation). </ins> | :attr:`stride` controls the stride for the cross-correlation. | If :attr:`padding` is non-zero, then the input is implicitly zero-padded on both sides <del> for :attr:`padding` number of points </del> <ins> for :attr:`padding` number of points. </ins> | If :attr:`output_padding` is non-zero, then the output is implicitly zero-padded on one side <del> for :attr:`output_padding` number of points | :attr:`groups` controls the connections between inputs and outputs. </del> <ins> for :attr:`output_padding` number of points. | :attr:`dilation` controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but this `link`_ has a nice visualization of what :attr:`dilation` does. | :attr:`groups` controls the connections between inputs and outputs. `in_channels` and `out_channels` must both be divisible by `groups`. </ins> |       At groups=1, all inputs are convolved to all outputs. |       At groups=2, the operation becomes equivalent to having two conv layers side by side, each seeing half the input channels, and producing half the output channels, and both subsequently concatenated. <ins> At groups=`in_channels`, each input channel is convolved with its own set of filters (of size `out_channels // in_channels`). </ins> The parameters :attr:`kernel_size`, :attr:`stride`, :attr:`padding`, :attr:`output_padding` can either be: <del> - a single ``int`` -- in which case the same value is used for the height and width dimension </del> <ins> - a single ``int`` -- in which case the same value is used for the depth, height and width dimensions </ins> - a ``tuple`` of three ints -- in which case, the first `int` is used for the depth dimension, the second `int` for the width dimension and the third `int` for the width dimension
Code change:
I think I encountered a memory leak situation potentially due to . I create a to reproduce my observations. Basically, for three different (meaningless) models, once the function is used, the memory usage will keep increasing. To run the code, where the flag controls whether is used. Please run it long enough. I'm on Ubuntu 14.04 with python3.6. I tried two versions of pytorch, 0.1.122 and 0.1.12+. Side Note: Even without using , the memory usauge can also increase for a while, but it will finally stablize at a level. What's more, when (i.e. ) is used, it can take very long before the memory usage stablizes. I'm not sure whether this is a problem or not.
I think I'm sure there is a memory leak due to , and I found a weird way to do a temporary fix under pytorch version "0.1.122". The simplest example is as follows: Basically, if we replace with , and pass in an unpacked list of numbers, (i.e., instead of ), the memory leak is gone. Using will also have the leak issue. Unfortunately, I don't know why this is working. Finally, since there are so many in pytorch code, to do a quick hacky fix, one can change the function in the file as follows:
After some more research, I think we have a much bigger problem here. My current conjecture is that: whenever we use ( object) as an argument to a function, there will be a memory leak. An example is the function in , where of the code is which uses . Again, a minimal example for reproduction is as follows: As in the case, once I change the code to , memory leak is gone. So, in short, the problem is on the usage of .
The same issue. +1
Thanks for tracking this down and for the repro script!
Thanks for the quick fix! BTW, will the fix be included in the release version (0.1.12_?)?
it'll be released in
I see. Thanks again.

Related discussion:
text: template<typename FnType, FnType fn, typename ...Args> static PyObject* wrap_tuple_fn(Args ... args) { <del> PyObject *result = (*fn)(std::forward<Args>(args)...); </del> <ins> THPObjectPtr result((*fn)(std::forward<Args>(args)...)); </ins> if (!result) return NULL; <del> if (PyTuple_Check(result)) { return PyObject_CallFunctionObjArgs((PyObject*)&THPSizeType, result, NULL); </del> <ins> if (PyTuple_Check(result.get())) { return PyObject_CallFunctionObjArgs((PyObject*)&THPSizeType, result.get(), NULL); </ins> } <del> Py_INCREF(result); return result; </del> <ins> return result.release(); </ins> } static auto sq_concat = PyTuple_Type.tp_as_sequence->sq_concat;
Code change:
Torchvision is considered a basic requirement of the tutorials. Perhaps it makes sense to include it in the docker build.
It's easy to install it by command ;)
It's easy to install torch itself using a similar command? :). I was thinking a docker image should have torchvision included.
Runtime docker file already has torchvision.

Related discussion:
text: <del> FROM nvidia/cuda:8.0-devel-ubuntu16.04 </del> <ins> FROM nvidia/cuda:8.0-cudnn6-devel-ubuntu16.04 </ins> RUN echo "deb http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64 /" > /etc/apt/sources.list.d/nvidia-ml.list <del> ENV CUDNN_VERSION 6.0.20 </del> RUN apt-get update && apt-get install -y --no-install-recommends  build-essential  cmake  git  curl  <ins> vim  </ins> ca-certificates  libjpeg-dev  <del> libpng-dev  libcudnn6=$CUDNN_VERSION-1+cuda8.0  libcudnn6-dev=$CUDNN_VERSION-1+cuda8.0 &&  </del> <ins> libpng-dev && </ins> rm -rf /var/lib/apt/lists/* RUN curl -o ~/miniconda.sh -O  https://repo.continuum.io/miniconda/Miniconda3-4.2.12-Linux-x86_64.sh  &&
Code change:
Torchvision is considered a basic requirement of the tutorials. Perhaps it makes sense to include it in the docker build.
It's easy to install it by command ;)
It's easy to install torch itself using a similar command? :). I was thinking a docker image should have torchvision included.
Runtime docker file already has torchvision.

Related discussion:
text: CMAKE_PREFIX_PATH="$(dirname $(which conda))/../"  pip install -v . <ins> RUN git clone https://github.com/pytorch/vision.git && cd vision && pip install -v . </ins> WORKDIR /workspace RUN chmod -R a+w /workspace
Code change:
Be Careful What You Backpropagate: A Case For Linear Output Activations & Gradient Boosting I can work on this if this can be to pytorch. Please let me know. Thanks!
This should be fairly straightforward to add in user code using . Eventually it could be to core, but its utility should probably be validated in external repos first.

Related discussion:
text: # (3) initialize mean square values and square gradient storage if not 'm' in state: <del> state['m'] = x.new().resize_as_(dfdx).fill_(1) </del> <ins> state['m'] = x.new().resize_as_(dfdx).zero_() </ins> state['tmp'] = x.new().resize_as_(dfdx)
Code change:
Be Careful What You Backpropagate: A Case For Linear Output Activations & Gradient Boosting I can work on this if this can be to pytorch. Please let me know. Thanks!
This should be fairly straightforward to add in user code using . Eventually it could be to core, but its utility should probably be validated in external repos first.

Related discussion:
text: # State initialization if len(state) == 0: state['step'] = 0 <del> state['square_avg'] = grad.new().resize_as_(grad).fill_(1) </del> <ins> state['square_avg'] = grad.new().resize_as_(grad).zero_() </ins> square_avg = state['square_avg'] alpha = group['alpha']
Code change:
Intermittent test failures occurring on AdaptiveMaxPool3d. and I have seen this in the past where Max pooling has exactly the same values in the same window, then it's differently resolved on CPU/GPU. For example [0 2.5 2.5 3] can give a max index of or on CUDA depending on the runtime. So generating input values that are within epsilon of each other is important (especially at half precision). fix that one.
Will do. Thanks for identifying the issue.

Related discussion:
text: self.assertEqual(torch.mm(flattened_tensor, flattened_tensor.t()), torch.eye(rows) * gain ** 2, prec=1e-6) <ins> # Generates rand tensor with non-equal values. This ensures that duplicate # values won't be causing test failure for modules like MaxPooling. # size should be small, otherwise randperm fails / long overflows. def _rand_tensor_non_equal(*size): total = reduce(mul, size, 1) return torch.randperm(total).view(*size).double() </ins> def add_test(test): test_name = test.get_name()
Code change:
Intermittent test failures occurring on AdaptiveMaxPool3d. and I have seen this in the past where Max pooling has exactly the same values in the same window, then it's differently resolved on CPU/GPU. For example [0 2.5 2.5 3] can give a max index of or on CUDA depending on the runtime. So generating input values that are within epsilon of each other is important (especially at half precision). fix that one.
Will do. Thanks for identifying the issue.

Related discussion:
text: dict( module_name='AdaptiveMaxPool1d', constructor_args=(3,), <del> input=torch.rand(1, 3, 5), </del> <ins> input=_rand_tensor_non_equal(1, 3, 5), </ins> ), dict( module_name='AdaptiveMaxPool2d', constructor_args=(3,), <del> input=torch.rand(1, 3, 5, 6), </del> <ins> input=_rand_tensor_non_equal(1, 3, 5, 6), </ins> desc='single', ), dict( module_name='AdaptiveMaxPool2d', constructor_args=((3, 4),), <del> input=torch.rand(1, 3, 5, 6), </del> <ins> input=_rand_tensor_non_equal(1, 3, 5, 6), </ins> desc='tuple', ), dict( module_name='AdaptiveMaxPool3d', constructor_args=(3,), <del> input=torch.rand(2, 3, 5, 6, 7), </del> <ins> input=_rand_tensor_non_equal(2, 3, 5, 6, 7), </ins> desc='single', ), dict( module_name='AdaptiveMaxPool3d', constructor_args=((3, 4, 5),), <del> input=torch.rand(2, 3, 5, 6, 7), </del> <ins> input=_rand_tensor_non_equal(2, 3, 5, 6, 7), </ins> desc='tuple', ), dict( module_name='AdaptiveMaxPool3d', constructor_args=(3,), <del> input=torch.rand(2, 3, 12, 9, 3), </del> <ins> input=_rand_tensor_non_equal(2, 3, 12, 9, 3), </ins> desc='single_nonatomic', ), dict( module_name='AdaptiveMaxPool3d', constructor_args=((3, 4, 5),), <del> input=torch.rand(2, 3, 6, 4, 10), </del> <ins> input=_rand_tensor_non_equal(2, 3, 6, 4, 10), </ins> desc='tuple_nonatomic', ), dict(
Code change:
When using classic SGD optimizer with momentum with sparse embeddings the memory keeps garbage collecting / allocating leading to slow down and out of memory error eventually. ! ! The issue dissapears when momentum is not used ! or when embeddings are not sparse ! I'm using the last pytorch version on conda:
I tried out your script with momentum 0.1 on master, it takes roughly 10800mb gpu memory max. This is caused by using sparse buffer. I'm sending out a PR for this.

Related discussion:
text: if momentum != 0: param_state = self.state[p] if 'momentum_buffer' not in param_state: <del> buf = param_state['momentum_buffer'] = d_p.clone() </del> <ins> buf = param_state['momentum_buffer'] = p.data.new().resize_as_(p.data).zero_() buf.mul_(momentum).add_(d_p) </ins> else: buf = param_state['momentum_buffer'] buf.mul_(momentum).add_(1 - dampening, d_p)
Code change:
Looking at the values returned from tn.multinomial(1) and counting them, it seems like it returns the values of the multinomial distribution taken from t, i.e. that it has ignored the signs of the values. However, it seem like always returns a tensor filled with zeros.
looking into this

Related discussion:
text: THArgCheckWithCleanup(n_sample > 0, THCleanup(if (start_dim == 1) THTensor_(resize1d)(prob_dist, n_categories);), 2, <del> "cannot sample n_sample < 0 samples"); </del> <ins> "cannot sample n_sample <= 0 samples"); </ins> if (!with_replacement) {