jiebi/CodeConvo huggingface dataset:
Change:
last_epoch (int): The index of last epoch. Default: -1. Example: <del> >>> # Assuming optimizer uses lr = 0.5 for all groups </del> <ins> >>> # Assuming optimizer uses lr = 0.05 for all groups </ins> >>> # lr = 0.05     if epoch < 30 >>> # lr = 0.005    if 30 <= epoch < 60 >>> # lr = 0.0005   if 60 <= epoch < 90

Discussion:
Hi, In the is written: "lr values are [0.05, 0.005, ...]." If I understand it correctly, it should be [0.5, 0.05, 0.005, ...].
That is correct. Please feel free to send a PR to fix it! (otherwise, I can)
If you don't mind to change it yourself, I'm not contributor yet and don't want to start just for one comment. I guess is the same case.
Change:
else: if exp_max > prec + 1 or exp_max < 0: sz = max(min_sz, 7) <del> scale = math.pow(10, exp_max - 1) </del> <ins> scale = math.pow(10, max(exp_max - 1, __MIN_LOG_SCALE)) </ins> else: if exp_max == 0: sz = 7

Discussion:
on print(torch.DoubleTensor(1)) I got ZeroDivisionError: float division by zero but torch.DoubleTensor(0)) or torch.DoubleTensor(2)) work just fine. I work in Jupyter notebook pytorch 0.1.12
Note that returns undefined memory. Could you find the specific content of the tensor that gave this error, by printing ? I'm not sure if this bug has been fixed in the latest release already.
the error is pretty interesting. I restarted Jupyter notebook. # many freaking signatures: print(torch.FloatTensor(1)) print(torch.IntTensor(1)) print(torch.DoubleTensor(1)) # no error right after initial run. #print(torch.DoubleTensor(0)) print(torch.DoubleTensor(2)) print(torch.ByteTensor(1)) print(torch.CharTensor(1)) print(torch.ShortTensor(1)) print(torch.LongTensor(1)) steps to reproduce the error: 1). run, no errors 2) repeat, no error 3) uncomment ..DoubleTensor(0), run, got an error (see below): 4) comment ..DouleTensor(0), run still have this error 2.+24 [torch.FloatTensor of size 1] 7.+08 [torch.IntTensor of size 1] ZeroDivisionError Traceback (most recent call last) &lt;ipython-input-5-3629c250c505in &lt;module() 2 print(torch.FloatTensor(1)) 3 print(torch.IntTensor(1)) ----4 print(torch.DoubleTensor(1)) # ZeroDivisionError: float division by zero 5 #print(torch.DoubleTensor(0)) 6 print(torch.DoubleTensor(2)) C:UsersleogAppDataLocalcondacondaenvspy35libsite-packagestorch in str(self) 126 # characters to replace unicode characters with.

127 if sys.versioninfo (3,): --128 return (self) 129 else: 130 if hasattr(, 'encoding'): C:UsersleogAppDataLocalcondacondaenvspy35libsite-packagestorch in str(self) 284 return '[{} with no dimension]n'.format(torch.typename(self)) 285 elif self.ndimension() == 1: --286 strt = vectorstr(self) 287 elif self.ndimension() == 2: 288 strt = matrixstr(self) C:UsersleogAppDataLocalcondacondaenvspy35libsite-packagestorch in vectorstr(self) 270 if () &lt; PRINTOPTS.threshold: 271 return (strt + --272 'n'.join(ident + (val / scale) for val in self) + 273 'n') 274 else: C:UsersleogAppDataLocalcondacondaenvspy35libsite-packagestorch in &lt;genexpr(.0) 270 if () &lt; PRINT_OPTS.threshold: 271 return (strt + --272 'n'.join(ident + (val / scale) for val in self) + 273 'n') 274 else: ZeroDivisionError: float division by zero
Ok, I got a repro. Here is the failing case: Fails with the error you mentioned. This seems to be an edge case though, as both and works. The issue is in , where becomes zero because is a too large negative number. Maybe we should clip to be positive?
Change:
Tensor forward(const Tensor& input) { this->_check_input_dim(input); <del> return F::detail::instance_norm( </del> <ins> return torch::nn::functional::detail::instance_norm( </ins> input, this->running_mean, this->running_var, this->weight, this->bias, this->is_training() || !this->options.track_running_stats(), this->options.momentum(), this->options.eps()); }

Discussion:
As part of pull request , F is redefined globally in nn/modules/batchnorm.h F is then used once in batchnorm.h and again in instancenorm.h not a bug, but a problem for anyone using F in some other way. it's solvable, but I don't see this pattern anywhere else in libtorch, was wondering if this use is an anomaly, and if we could use thanks cc
Thanks a lot for reporting! Yes this is an anomaly, and will be fixed by
Change:
, dependencies() {} }; <del> struct FunctionTask { BackwardTask* base; std::shared_ptr<Function> fn; GradBuffer grad; FunctionTask(BackwardTask* base, std::shared_ptr<Function> fn, GradBuffer grad) : base(base) , fn(fn) , grad(std::move(grad)) {} }; </del> auto ReadyQueue::push_front(FunctionTask item) -> void { { std::lock_guard<std::mutex> lock(mutex);

Discussion:
Version include <stringin some cpp files: define order in The pre-defined struct is not passed by clang in macOS, so we should reorder them.
Simple fix
Change:
self.assertFalse(found_data[data_point_idx]) found_data[data_point_idx] += 1 break <del> self.assertEqual(target, self.labels.narrow(0, data_point_idx, 1)) </del> <ins> self.assertEqual(target, self.labels[data_point_idx]) </ins> found_labels[data_point_idx] += 1 self.assertEqual(sum(found_data.values()), (i + 1) * batch_size) self.assertEqual(sum(found_labels.values()), (i + 1) * batch_size)

Discussion:
TensorDataset adds casts 1D target tensor to 2D which is not acceptable by NLL loss layer.
Fixed in .
Change:
self.assertRaises(AssertionError, nn.Embedding, num_embeddings=10, embedding_dim=20, padding_idx=25) self.assertRaises(AssertionError, nn.Embedding, num_embeddings=10, embedding_dim=20, padding_idx=-25) <ins> # test backward when input contains padding_idx padding_idx = 0 embedding = nn.Embedding(5, 2, padding_idx=padding_idx) for n in (1, 2): for other_indices in ([], [1, 3], [2]): indices = torch.LongTensor(other_indices + [padding_idx] * n) pre = embedding.weight[padding_idx].clone() embedding(indices).sum().backward() after = (embedding.weight + embedding.weight.grad)[padding_idx] embedding.zero_grad() self.assertEqual(after, pre) </ins> def test_embedding_max_norm(self): embedding = nn.Embedding(22, 5, max_norm=1.0) input = Variable(torch.LongTensor([2, 8, 8, 6]))

Discussion:
I have a straightforward use case. Given a token (or word), lookup the embedding of the word, pass that through a linear layer and perform L2 regression. If the token corresponds to the of the embedding layer, throws an error saying OS: macOS High Sierra PyTorch version: 0.3.1 How you installed PyTorch (conda, pip, source): Tried both conda and pip Python version: 3.6.4 CUDA/cuDNN version: CPU GPU models and configuration: NA GCC version (if compiling from source): NA A script to reproduce the bug:
I got the same trouble, and it throw this error after some epochs of training. Does anyone have some solutions?
Change:
super(TestCase, self).assertEqual(len(x), len(y), message) for x_, y_ in zip(x, y): self.assertEqual(x_, y_, prec, message) <ins> elif isinstance(x, bool) and isinstance(y, bool): super(TestCase, self).assertEqual(x, y, message) </ins> elif isinstance(x, Number) and isinstance(y, Number): if abs(x) == float('inf') or abs(y) == float('inf'): if allow_inf:

Discussion:
PyTorch version: 0.3.0 Python version: 3.5 GPU models and configuration: 8x NVIDIA Titan I found that requiresgrad setting in module is ignored when module is wrapped with nn.DataParallel. For example, This code should be executed because very small part of network activation (only fc layer) is stored for backward computation, but it results to run-time error: Internally, I found a crack in function which is in . In replicate function, it copies all parameter in module (# of replica times) with . In broadcasting code, it just defines new with default constructor parameter, which is always set to True. I think there can be some choices to fix this issue. is intended behavior for DataParallel, so we should use volatile (or ) to implement transfer learning in DataParallel model should be fixed to handle requires_grad when copying parameters. should be synchronizing in I think any of solution doesn't need lots of effort to fix it.
Change:
Attributes: weight_ih_l[k] : the learnable input-hidden weights of the k-th layer (W_ir|W_ii|W_in), of shape <del> (input_size x 3*hidden_size) </del> <ins> `(input_size x 3*hidden_size)` </ins> weight_hh_l[k] : the learnable hidden-hidden weights of the k-th layer (W_hr|W_hi|W_hn), of shape <del> (hidden_size x 3*hidden_size) bias_ih_l[k] : the learnable input-hidden bias of the k-th layer (b_ir|b_ii|b_in), of shape (3*hidden_size) bias_hh_l[k] : the learnable hidden-hidden bias of the k-th layer (W_hr|W_hi|W_hn), of shape (3*hidden_size) </del> <ins> `(hidden_size x 3*hidden_size)` bias_ih_l[k] : the learnable input-hidden bias of the k-th layer (b_ir|b_ii|b_in), of shape `(3*hidden_size)` bias_hh_l[k] : the learnable hidden-hidden bias of the k-th layer (W_hr|W_hi|W_hn), of shape `(3*hidden_size)` </ins> Examples:: >>> rnn = nn.GRU(10, 20, 2)

Discussion:
From : Happened because of some change in the environment as it is also affecting historical builds. cc
This issue seems to come from within libuv, a dependency of TensorPipe. First guess: there's two conflicting versions of libuv (e.g., a preinstalled one and the one shipped as a submodule of TensorPipe), and we build the one shipped by TensorPipe but somehow it ends up including the headers of the other one? In particular, the constant it's complaining about was in , which became part of libuv 1.41.0, which is the version that TensorPipe bundles. But it seems those jobs are installing version 1.40.0 via conda: So the question is why are the conda headers picked up and prioritized? I feel this will require some CMake digging, and it's possible that the fix will have to land in libuv's upstream CMake files. Ugh.
BTW, if you want to validate this theory you can try setting this flag to : Which will cause TensorPipe to use the preinstalled libuv instead of building its own one. But please don't land this "workaround", as it will only lead to users coming to complain that they couldn't build PyTorch because their system has a libuv version that is too old for what TensorPipe needs.
this is definitely what is happening: libuv-1.40 is installed from conda, which does not have in the header: But I'm still confused why it started to happen just now [Edit] Can it be update that happened 4 hours ago?
Finally found a PR that caused a regression: (and testing were clearly regressed in CI: )
Change:
flat_weight = first_data.new().set_(first_data.storage(), 0, torch.Size([self._param_buf_size])) else: flat_weight = None <ins> self.check_forward_args(input, hx, batch_sizes) </ins> func = self._backend.RNN( self.mode, self.input_size,

Discussion:
I run RNN sucessfully with the code. and I run embedding example. Then, got RuntimeError. I set to use nothing seems wrong. And the error info seems not the dim problem? The error is so strange.
What exactly do you mean by "embedding example"? Could you provide a code snippet that causes the RuntimeError?
To better reproduce the error, I use the official code example of The origin runs well: Due to use , I have to make batch first. And then , it raises error.
Thank you for the example! The say that setting means that the first dim of input is the batch, ie, input has size . however is unchanged, so the following should work: I do agree the error message could be better, though
Change:
return platform <ins> def get_python_platform(): import platform return platform.platform() def get_libc_version(): import platform if get_platform() != 'linux': return 'N/A' return '-'.join(platform.libc_ver()) </ins> def get_pip_packages(run_lambda): """Returns `pip list` output. Note: will also find conda-installed pytorch and numpy packages."""

Discussion:
TensorPipe uses some advanced features of the Linux kernel. It does so on a "best effort" basis (i.e., use them if available, fall back on something else if not). This logic is unfortunately hard to test because it would require specific kernel versions on our CI machines. So it occasionally happens that we find out about issues when users report them. To help investigate these reports we've found it useful to know the kernel version and the glibc version used by the user (both when building and when running, if they're different). It would be great if these could be to the report produced by collect_env.
CC
Change:
{ /* Get normalized cumulative distribution from prob distribution */ double sum = 0; <ins> double val; </ins> for (j=0; j<n_categories; j++) { <del> sum += THStorage_(get)(  </del> <ins> val = THStorage_(get)(  </ins> prob_dist->storage,  prob_dist->storageOffset+i*prob_dist->stride[0]+j*prob_dist->stride[1]  ); <ins> THArgCheckWithCleanup((val >= 0), THCleanup(THDoubleTensor_free(cum_dist); if (start_dim == 1) THTensor_(resize1d)(prob_dist, n_categories);), 2, "invalid multinomial distribution (encountering probability entry < 0)"); sum += val; </ins> THDoubleStorage_set( cum_dist->storage,  cum_dist->storageOffset+j*cum_dist->stride[0], 

Discussion:
Looking at the values returned from tn.multinomial(1) and counting them, it seems like it returns the values of the multinomial distribution taken from t, i.e. that it has ignored the signs of the values. However, it seem like always returns a tensor filled with zeros.
looking into this
Change:
dict( module_name='AdaptiveMaxPool1d', constructor_args=(3,), <del> input=torch.rand(1, 3, 5), </del> <ins> input=_rand_tensor_non_equal(1, 3, 5), </ins> ), dict( module_name='AdaptiveMaxPool2d', constructor_args=(3,), <del> input=torch.rand(1, 3, 5, 6), </del> <ins> input=_rand_tensor_non_equal(1, 3, 5, 6), </ins> desc='single', ), dict( module_name='AdaptiveMaxPool2d', constructor_args=((3, 4),), <del> input=torch.rand(1, 3, 5, 6), </del> <ins> input=_rand_tensor_non_equal(1, 3, 5, 6), </ins> desc='tuple', ), dict( module_name='AdaptiveMaxPool3d', constructor_args=(3,), <del> input=torch.rand(2, 3, 5, 6, 7), </del> <ins> input=_rand_tensor_non_equal(2, 3, 5, 6, 7), </ins> desc='single', ), dict( module_name='AdaptiveMaxPool3d', constructor_args=((3, 4, 5),), <del> input=torch.rand(2, 3, 5, 6, 7), </del> <ins> input=_rand_tensor_non_equal(2, 3, 5, 6, 7), </ins> desc='tuple', ), dict( module_name='AdaptiveMaxPool3d', constructor_args=(3,), <del> input=torch.rand(2, 3, 12, 9, 3), </del> <ins> input=_rand_tensor_non_equal(2, 3, 12, 9, 3), </ins> desc='single_nonatomic', ), dict( module_name='AdaptiveMaxPool3d', constructor_args=((3, 4, 5),), <del> input=torch.rand(2, 3, 6, 4, 10), </del> <ins> input=_rand_tensor_non_equal(2, 3, 6, 4, 10), </ins> desc='tuple_nonatomic', ), dict(

Discussion:
Intermittent test failures occurring on AdaptiveMaxPool3d. and I have seen this in the past where Max pooling has exactly the same values in the same window, then it's differently resolved on CPU/GPU. For example [0 2.5 2.5 3] can give a max index of or on CUDA depending on the runtime. So generating input values that are within epsilon of each other is important (especially at half precision). fix that one.
Will do. Thanks for identifying the issue.
Change:
modules = self.__dict__['_modules'] if name in modules: return modules[name] <del> return object.__getattribute__(self, name) </del> <ins> return object.__getattr__(self, name) </ins> def __setattr__(self, name, value): <ins> def remove_from(*dicts): for d in dicts: if name in d: del d[name] </ins> params = self.__dict__.get('_parameters') if isinstance(value, Parameter): if params is None: raise AttributeError( "cannot assign parameters before Module.__init__() call") <ins> remove_from(self.__dict__, self._buffers, self._modules) </ins> self.register_parameter(name, value) elif params is not None and name in params: if value is not None:

Discussion:
They do one pair of these calls for each slicing operation.
Does this actually have a performance impact? These calls tend to be very quick
Yes, it's a major reason why small unrolled LSTMs (which involve in addition to two GEMMs: a chunk into four gates, for 4 pairs of calls, five unary pointwise ops, for 5 more, and five binary pointwise ops, for another 5) are much slower than they are in Theano and TensorFlow.
Also, there are excessive cudaGetDevice calls for each pointwise operation (9 cudaGetDevice) even without chunking, as in the following repro code: As for small unrolled LSTMs it does show up.
This should no longer be an issue, now that is implemented as a native ATen function. I think we're still using the Python version for tensors, but this will be resolved soon when we stop using tensors altogether.
Change:
grad.zero_() def updateParameters(self, learningRate): <del> params, gradParams = self.parameters() if params: for p, gp in zip(params, gradParams): p.add_(-learningRate, gp) </del> <ins> if self.parameters() is not None: params, gradParams = self.parameters() if params: for p, gp in zip(params, gradParams): p.add_(-learningRate, gp) </ins> def training(self): self.train = True

Discussion:
I implemented a model using because I have to do particular things and I'm starting from a PyTorch implementation. I think this method may be broken. If I run this code: I get this error: This happens whenever the Sequential model has layers without parameters. Removing ReLUs and LogSoftMax solves the problem. I think this happens because returns for s which don't have parameters and raises the error. I tried this both on Python2.7 and Python 3.5 and it's the same.
Solved by
Change:
CMAKE_PREFIX_PATH="$(dirname $(which conda))/../"  pip install -v . <ins> RUN git clone https://github.com/pytorch/vision.git && cd vision && pip install -v . </ins> WORKDIR /workspace RUN chmod -R a+w /workspace

Discussion:
Torchvision is considered a basic requirement of the tutorials. Perhaps it makes sense to include it in the docker build.
It's easy to install it by command ;)
It's easy to install torch itself using a similar command? :). I was thinking a docker image should have torchvision included.
Runtime docker file already has torchvision.
Change:
#define THSTensorPtr  TH_CONCAT_3(THS,Real,TensorPtr) #define THSPTensorPtr  TH_CONCAT_3(THSP,Real,TensorPtr) <del> template<class T> class THPPointer { public: THPPointer(): ptr(nullptr) {}; THPPointer(T *ptr): ptr(ptr) {}; THPPointer(THPPointer &&p) { free(); ptr = p.ptr; p.ptr = nullptr; }; ~THPPointer() { free(); }; T * get() { return ptr; } T * release() { T *tmp = ptr; ptr = NULL; return tmp; } operator T*() { return ptr; } THPPointer& operator =(T *new_ptr) { free(); ptr = new_ptr; return *this; } THPPointer& operator =(THPPointer &&p) { free(); ptr = p.ptr; p.ptr = nullptr; return *this; } T * operator ->() { return ptr; } operator bool() { return ptr != nullptr; } private: void free(); T *ptr = nullptr; }; typedef THPPointer<PyObject> THPObjectPtr; </del> typedef THPPointer<THPGenerator> THPGeneratorPtr; template <typename T>

Discussion:
Go to Look at the html. It contains the following: ! This problem manifests in google search. Searching for 1.8.1 docs brings up the "1.8.0 docs": ! We can probably fix this by doing a quick sed on the website but we should develop a solution for future releases. I think this has happened before in the past and am not sure what causes it. cc
Change:
output_var_.requires_grad = self->cdata.requires_grad; auto dummy_prev_fn = std::make_shared<Variable>( std::unique_ptr<Tensor>(output_var_.data->clone_shallow()), false, false); <del> self->cdata.previous_functions[i] = std::make_pair<>(dummy_prev_fn, 0); </del> <ins> // Replace all references to the variable auto& previous_functions = self->cdata.previous_functions; for (int inp = 0; inp < self->num_inputs; inp++) { if (previous_functions[inp].first.get() == &output_var_) { previous_functions[inp] = std::make_pair<>(dummy_prev_fn, 0); } } </ins> } else { // output_var_.requires_grad throw std::runtime_error("a leaf Variable that requires grad has been used in an in-place operation."); }

Discussion:
import torch as th from torch.autograd import Variable x = Variable(th.Tensor([1 , 2 , 3]) , requiresgrad = False) y = Variable(th.Tensor([100]) , requiresgrad = True) x = x.cuda(0) y = y.cuda(0) x[0] = y print(x.requiresgrad) If ((0) (0)) is executed then the output is False, True otherwise. I think the result that x.requiresgrad = True is more natural.
I further checked some in-place operator in GPU mode(say, x = Variable(...).cuda(0) ), and find out that after the operation is done, the requiresgrad of the Variable doesn't change. For example: import torch as th from torch.autograd import Variable x = Variable(torch.Tensor([1]) , requiresgrad = False) y = Variable(torch.Tensor([2]) , requiresgrad = True) x = x.cuda(0) y = y.cuda(0) x.add(y) print(x.requires_grad) the output is False
It doesn't have to do with Variable holding a CUDA tensor, but rather with the fact that it's no longer a leaf. I've sent a fix in .
Thanks for the report!
Change:
#include <vector> #include <stack> #include <tuple> <del> #include "THCP.h" </del> <ins> #include "torch/csrc/cuda/THCP.h" </ins> <del> #include "override_macros.h" </del> <ins> #include "torch/csrc/cuda/override_macros.h" </ins> #include "torch/csrc/copy_utils.h" #include "DynamicTypes.h" <del> #define THC_GENERIC_FILE "torch/csrc/generic/Tensor.cpp" #include <THC/THCGenerateAllTypes.h> </del> <ins> //generic_include THC torch/csrc/generic/Tensor.cpp </ins> <del> #include "undef_macros.h" #include "restore_macros.h" </del> <ins> #include "torch/csrc/cuda/undef_macros.h" #include "torch/csrc/cuda/restore_macros.h" </ins>

Discussion:
Trying to run on github actions via AWS instance gives me: I will of course set up now, but it shouldn't crash like that. May be it should use as a fallback? pt-2.0 cc
Is this a multitenant setup? Also curious which distro are you running, I was under the impression that most linux distros set home by default
this is AWS image. one default user. I think it's user, but it wants you to run as root, which is weird. I have manually now set to overcome this.
We should update triton to use pathlib or something
Change:
THArgCheck(THCTensor_(nDimension)(state, src) <= MAX_CUTORCH_DIMS, 2, CUTORCH_DIM_WARNING); THArgCheck(THCudaLongTensor_nDimension(state, index) <= MAX_CUTORCH_DIMS, 2, CUTORCH_DIM_WARNING); <del> int srcDims = THCTensor_(nDimension)(state, src); THArgCheck(srcDims > 0, 2, "Source tensor is empty"); </del> <ins> if (numIndices == 0) { return; } </ins> if (accumulate) { // wrap indices so to replace negative indices

Discussion:
odd difference between cuda Tensor and Variable: encountered in torch.distributions.Categorical, built yesterday from master
related to hmm, here's the issue is for variables, so probably not (immediately) related
different code path but similar issue
This is a regression from the latest 0.3 binaries, which don't have this error.
Change:
`U, S, V = torch.svd(A)` returns the singular value decomposition of a real matrix `A` of size `(n x m)` such that :math:`A = USV^T`. <del> `U` is of shape :math:`(n times min(n, m))`. </del> <ins> `U` is of shape :math:`(n times n)`. </ins> <del> `S` is a diagonal matrix of shape :math:`(min(n, m) times min(n, m))`, represented as a vector of size :math:`min(n, m)` containing the diagonal entries. </del> <ins> `S` is a diagonal matrix of shape :math:`(n times m)`, represented as a vector of size :math:`min(n, m)` containing the diagonal entries. </ins> <del> `V` is of shape :math:`(m times min(n, m))`. </del> <ins> `V` is of shape :math:`(m times m)`. </ins> <del> :attr:`some` represents the number of singular values to be computed. If `some=True`, it computes some and `some=False` computes all. </del> <ins> If :attr:`some` is ``True`` (default), the returned `U` and `V` matrices will contain only :math:`min(n, m)` orthonormal columns. </ins> .. note:: Irrespective of the original strides, the returned matrix `U` will be transposed, i.e. with strides `(1, n)` instead of `(n, 1)`.

Discussion:
The documentation of the parameter to is one of the strangest I've seen... what does (the default) do???
Ha it is somewhat weird indeed. When , the returned U and V matrices are square orthonormal matrices. I will send out a PR to clarify this.