jiebi/CodeConvo huggingface dataset:
Change:
import torch as th from torch.autograd import Variable x = Variable(th.Tensor([1 , 2 , 3]) , requiresgrad = False) y = Variable(th.Tensor([100]) , requiresgrad = True) x = x.cuda(0) y = y.cuda(0) x[0] = y print(x.requiresgrad) If ((0) (0)) is executed then the output is False, True otherwise. I think the result that x.requiresgrad = True is more natural.
I further checked some in-place operator in GPU mode(say, x = Variable(...).cuda(0) ), and find out that after the operation is done, the requiresgrad of the Variable doesn't change. For example: import torch as th from torch.autograd import Variable x = Variable(torch.Tensor([1]) , requiresgrad = False) y = Variable(torch.Tensor([2]) , requiresgrad = True) x = x.cuda(0) y = y.cuda(0) x.add(y) print(x.requires_grad) the output is False
It doesn't have to do with Variable holding a CUDA tensor, but rather with the fact that it's no longer a leaf. I've sent a fix in .
Thanks for the report!

Discussion:
y._backward_hooks['test'] = error b.backward(torch.ones(5, 5)) <ins> def test_requires_grad_inplace(self): a = Variable(torch.randn(5, 5)) b = Variable(torch.randn(5, 5), requires_grad=True) a += b self.assertTrue(a.requires_grad) # non-leaf Variable a = Variable(torch.randn(5, 5)) + 0 b = Variable(torch.randn(5, 5), requires_grad=True) a += b self.assertTrue(a.requires_grad) </ins> def test_previous_functions(self): x = Variable(torch.randn(5, 5), requires_grad=True) y = Variable(torch.randn(5, 5), requires_grad=True)

Py_INCREF(input_var); output_var = input_var; input_var_.creator = THPFunction_asFunction(self); <ins> input_var_.requires_grad = self->cdata.requires_grad; </ins> } else { // If the leaf Variable has been returned, we have to move it after the // current function to ensure the gradient is computed correctly.

output_var_.requires_grad = self->cdata.requires_grad; auto dummy_prev_fn = std::make_shared<Variable>( std::unique_ptr<Tensor>(output_var_.data->clone_shallow()), false, false); <del> self->cdata.previous_functions[i] = std::make_pair<>(dummy_prev_fn, 0); </del> <ins> // Replace all references to the variable auto& previous_functions = self->cdata.previous_functions; for (int inp = 0; inp < self->num_inputs; inp++) { if (previous_functions[inp].first.get() == &output_var_) { previous_functions[inp] = std::make_pair<>(dummy_prev_fn, 0); } } </ins> } else { // output_var_.requires_grad throw std::runtime_error("a leaf Variable that requires grad has been used in an in-place operation."); }
Change:
Should we add a ShortTensor Type, or just convert int16 to IntTensor ?
We already have a type available in pytorch. We have to enable conversion in the relevant function here:
Auto travis-ci failed at python2.7 :(
Do you have any plan to support int8 numpy conversion while you have too?
Char tensor uses char which is not guaranteed to be signed by the C standard. We'd need to change our C code to use a
I try , it complains , any idea?
The error message is quite self explanatory. PyTorch doesn't support tensors at the moment.

Discussion:
np.float, np.int64, np.int32, <ins> np.int16, </ins> np.uint8 ] for dtype in dtypes:

return PyObject_CallFunctionObjArgs(THPLongTensorClass, array, NULL); } else if (type == NPY_INT32) { return PyObject_CallFunctionObjArgs(THPIntTensorClass, array, NULL); <ins> } else if (type == NPY_INT16) { return PyObject_CallFunctionObjArgs(THPShortTensorClass, array, NULL); </ins> } else if (type == NPY_UINT8) { return PyObject_CallFunctionObjArgs(THPByteTensorClass, array, NULL); }

#ifdef TH_REAL_IS_INT #define NUMPY_TYPE_ENUM NPY_INT32 #endif <ins> #ifdef TH_REAL_IS_SHORT #define NUMPY_TYPE_ENUM NPY_INT16 #endif </ins> #ifdef TH_REAL_IS_BYTE #define NUMPY_TYPE_ENUM NPY_UINT8 #endif
Change:
A few times, VolumetricMaxPooling (in ) kept failing on the continuous builds, and needed to investigate that. So it turns out, there were these one set of indices that were in the same Pooling window that differed by less than epsilon. So, the numeric gradient was hitting boundary conditions (max-pooling is discontinuous of course). ! ! The obvious fix is to modify the test to not have the input tensor have these boundary conditions. However, as an aside, I am not sure why the boundary condition is only hit sometimes and not always, because the same random numbers are generated always, and division is unstable but it is deterministic (right? or have i always got this wrong)
Repeat running will produce error occasional , more detail:

Discussion:
input_size=(1, 2, 4, 4, 4)), OldModuleTest(nn.VolumetricMaxPooling, (2, 2, 2), <del> input_size=(2, 3, 5, 5, 5)), </del> <ins> input=(torch.randn(2, 3, 5, 5, 5) * 1000)), </ins> OldModuleTest(nn.VolumetricMaxPooling, (2, 2, 2, 2, 2, 2), input_size=(2, 3, 5, 5, 5),
Change:
Version include <stringin some cpp files: define order in The pre-defined struct is not passed by clang in macOS, so we should reorder them.
Simple fix

Discussion:
namespace torch { namespace autograd { <ins> struct FunctionTask { BackwardTask* base; std::shared_ptr<Function> fn; GradBuffer grad; FunctionTask(BackwardTask* base, std::shared_ptr<Function> fn, GradBuffer grad) : base(base) , fn(fn) , grad(std::move(grad)) {} }; </ins> struct ReadyQueue { std::deque<FunctionTask> queue; std::condition_variable not_empty;

, dependencies() {} }; <del> struct FunctionTask { BackwardTask* base; std::shared_ptr<Function> fn; GradBuffer grad; FunctionTask(BackwardTask* base, std::shared_ptr<Function> fn, GradBuffer grad) : base(base) , fn(fn) , grad(std::move(grad)) {} }; </del> auto ReadyQueue::push_front(FunctionTask item) -> void { { std::lock_guard<std::mutex> lock(mutex);

#include "function.h" <ins> #include <string> </ins> #include <THPP/THPP.h> #include "variable.h"

#include "tuple_parser.h" <ins> #include <string> </ins> #include "torch/csrc/DynamicTypes.h" #include "torch/csrc/autograd/python_variable.h" #include "python_numbers.h"
Change:
I am new to pythonwhen i solve the promblem with the help below I find some confusion in the code I set ‘dimension=1self.dimension = dimension’it seem ok for mebut i don’t kown how the value of ’dimension‘ was initialled. Thank you !
I already Konw it comes from 'module = JoinTable(dimension, nInputDims)' But when I convert the model to pytorch , error appears: Traceback (most recent call last): File "", line 173, in <moduleGnetf =generator.forward(input) File "/usr/local/lib/python2.7/dist-", line 33, in forward return self.updateOutput(input) File "/usr/local/lib/python2.7/dist-", line 36, in updateOutput currentOutput = module.updateOutput(currentOutput) File "/usr/local/lib/python2.7/dist-", line 37, in updateOutput (dim, offset, (dim)).copy_(currentOutput) RuntimeError: inconsistent tensor size at /home/lxl/pytorch-master/torch/lib/TH/generic/THTensorCopy.c:51
I Use "generator.modules[0] = nn.JoinTable(1)",it was fine ,but error again: Traceback (most recent call last): File "", line 171, in <moduleGnetf =generator.forward(input) File "/usr/local/lib/python2.7/dist-", line 33, in forward return self.updateOutput(input) File "/usr/local/lib/python2.7/dist-", line 36, in updateOutput currentOutput = module.updateOutput(currentOutput) File "/usr/local/lib/python2.7/dist-", line 96, in updateOutput if is None: AttributeError: 'SpatialFullConvolution' object has no attribute 'finput'
How old is the Lua model file you're trying to import? Can you please try to load it in Lua, save again, and load it in PyTorch? Also, please update PyTorch to the newest version.
The model is convert from the cudnn model trained by myseft the code below is the convert code BTWmy torch was installed on 17th Dec,2016 the pytorch version i use Metadata-Version: 1.0 Name: torch Version: 0.1.10+ I Build it from source today

Discussion:
tW = targetTensor.size(tDims - 1) adjW = self._calculateAdj(tW, self.kW, self.padW, self.dW) adjH = self._calculateAdj(tH, self.kH, self.padH, self.dH) <del> if self.finput is None: </del> <ins> if not hasattr(self, 'finput') or self.finput is None: </ins> self.finput = input[0].new() <del> if self.fgradInput is None: </del> <ins> if not hasattr(self, 'fgradInput') or self.fgradInput is None: </ins> self.fgradInput = input[0].new() else: <del> if self.finput is None: </del> <ins> if not hasattr(self, 'finput') or self.finput is None: </ins> self.finput = input.new() <del> if self.fgradInput is None: </del> <ins> if not hasattr(self, 'fgradInput') or self.fgradInput is None: </ins> self.fgradInput = input.new() inputTensor = self._makeContiguous(inputTensor)
Change:
Running this on OSX: More details of failure in Ubuntu 14.04.1 in
Maybe it's caused by a ? I just test this code and it doesn't work (on Mac OS X 10.12.3 with Python 3.6):
That thread links to this one , which says it’s an OS bug and proposes a workaround patch that hasn’t been merged. There’s also a numpy issue thread, which links to this Torch7 commit that implements a workaround
We're doing a loop in case it writes less, but we always pass the full amount of remaining bytes. Putting a cap (std::min with INT_MAX) on the size of a single write might help.
I can't reproduce it on CentOS. I think it might have been a report for a PyTorch version prior to fixing .

Discussion:
char *bytes = (char *) data; int64_t remaining = sizeof(real) * size; while (remaining > 0) { <del> ssize_t result = write(fd, bytes, remaining); </del> <ins> // we write and read in 1GB blocks to avoid bugs on some OSes ssize_t result = write(fd, bytes, THMin(remaining, 1073741824)); </ins> if (result < 0) throw std::system_error(result, std::system_category()); bytes += result;

char *bytes = (char *) data; int64_t remaining = sizeof(real) * storage->size; while (remaining > 0) { <del> ssize_t result = read(fd, bytes, remaining); </del> <ins> // we write and read in 1GB blocks to avoid bugs on some OSes ssize_t result = read(fd, bytes, THMin(remaining, 1073741824)); </ins> if (result <= 0) // 0 means EOF, which is also an error throw std::system_error(result, std::system_category()); bytes += result;
Change:
Do you plan to publish an official entry on docker hub?
any update? I think it's an important enhancement.
I believe this would just be pushing the image from ? If so, I can add it to Docker Hub if you like?
Configure automated builds on Docker Hub
The dockerfile you referenced builds from current master, I don't know if that's desirable. There is a runtime dockerfile in tools/docker that installs current binary release, I'd think that's what should be hosted on the dockerhub.
yup, that makes more sense. Thanks
Oops, looks like runtime dockerfile would be broken now (see discussion in #beginner, conda packages conflict with conda numpy).
Just tested, the runtime dockerfile seems to work just fine. Pulls in numpy 1.13.1 which seems to not have any issues.
pre-built images are now available, you can run them with:

Discussion:
``` docker build -t pytorch . ``` <del> Alternatively, if you want a runtime image, build with ``` docker build -t pytorch . -f tools/docker/Dockerfile_runtime ``` and run with nvidia-docker: </del> <ins> Alternatively, if you want to use a runtime image, you can use the pre-built one from Docker Hub and run with nvidia-docker: </ins> ``` <del> nvidia-docker run --rm -ti --ipc=host pytorch </del> <ins> nvidia-docker run --rm -ti --ipc=host pytorch/pytorch:latest </ins> ``` Please note that PyTorch uses shared memory to share data between processes, so if torch multiprocessing is used (e.g. for multithreaded data loaders) the default shared memory segment size that container runs with is not enough, and you
Change:
Usually calling backwards twice gives this message: But conv2d just segfaults. $ pt A B Segmentation fault (core dumped)

Discussion:
c = nn.Conv2d(3, 3, 3) self.assertRaises(RuntimeError, lambda: c(None)) <ins> def test_Conv2d_backward_twice(self): input = Variable(torch.randn(2, 3, 5, 5)) c = nn.Conv2d(3, 3, 3) o1 = c(input) o1.sum().backward() self.assertRaisesRegex(RuntimeError, 'Specify retain_variables=True', lambda: o1.sum().backward()) </ins> @unittest.skipIf(not TEST_CUDA, 'CUDA not available') def test_Conv2d_large_workspace(self): # These sizes require huge cuDNN workspaces. Make sure we choose a

if (is_padding_neg()) throw std::runtime_error("negative padding is not supported"); if (is_output_padding_neg()) throw std::runtime_error("negative output_padding is not supported"); <del> AutoGPU guard(input_.data->getDevice()); auto input = input_.unpack_data()->contiguous(); </del> <ins> auto input = input_.unpack_data(); AutoGPU guard(input->getDevice()); input = input->contiguous(); </ins> std::unique_ptr<Tensor> weight(weight_.unpack_data()->clone_shallow()); auto bias = bias_.unpack_data(); auto grad_output = grad_outputs[0]->data->contiguous();

PyObject *THPFunction_saved_tensors(THPFunction *self, void *_unused) { <del> THPUtils_assert(!self->has_freed_buffers, "Trying to backward through the " "graph second time, but the buffers have already been freed. Please " "specify retain_variables=True when calling backward for the first time."); </del> <ins> THPUtils_assert(!self->has_freed_buffers, ERR_BACKWARD_TWICE); </ins> if (!self->saved_variables) return PyTuple_New(0);

PyObject *THPFunction_saved_variables(THPFunction *self, void *_unused) { <del> THPUtils_assert(!self->has_freed_buffers, "Trying to backward through the " "graph second time, but the buffers have already been freed. Please " "specify retain_variables=True when calling backward for the first time."); </del> <ins> THPUtils_assert(!self->has_freed_buffers, ERR_BACKWARD_TWICE); </ins> if (!self->saved_variables) return PyTuple_New(0);

} auto SavedVariable::unpack() -> std::shared_ptr<Variable> { <del> if (!data) return nullptr; </del> <ins> if (!data) { if (version) { throw std::runtime_error(ERR_BACKWARD_TWICE); } return nullptr; } </ins> int current_version = **version; if (expected_version != current_version) {

return new_var; } <ins> const char* ERR_BACKWARD_TWICE = "Trying to backward through the graph a second time, but the buffers have " "already been freed. Specify retain_variables=True when calling backward " "the first time."; </ins> }} // namespace torch::autograd

namespace torch { namespace autograd { <ins> extern const char* ERR_BACKWARD_TWICE; </ins> struct Variable : std::enable_shared_from_this<Variable> { struct SavedVariable {
Change:
"Randomly zeroes some of the elements of the input tensor. The elements to zero are randomized on every forward call." This is incorrect; the function also scales up by 1/(1-p), which the implementation correctly does.

Discussion:
class Dropout(Module): <del> r"""Randomly zeroes some of the elements of the input tensor. </del> <ins> r"""During training, randomly zeroes some of the elements of the input tensor with probability *p* using samples from a bernoulli distribution. </ins> The elements to zero are randomized on every forward call. <ins> This has proven to be an effective technique for regularization and preventing the co-adaptation of neurons as described in the paper `Improving neural networks by preventing co-adaptation of feature detectors`_ . Furthermore, the outputs are scaled by a factor of *1/(1-p)* during training. This means that during evaluation the module simply computes an identity function. </ins> Args: p: probability of an element to be zeroed. Default: 0.5 inplace: If set to True, will do this operation in-place. Default: false

>>> m = nn.Dropout(p=0.2) >>> input = autograd.Variable(torch.randn(20, 16)) >>> output = m(input) <ins> .. _Improving neural networks by preventing co-adaptation of feature detectors: https://arxiv.org/abs/1207.0580 </ins> """ def __init__(self, p=0.5, inplace=False):
Change:
the error only happens in very specific conditions. (change the view of input tensor and use weight converted from numpy array.) The output of functional conv2d is far off in scale (e12 or so.)
this happens because generates double precision tensor by default, whereas torch generates single precision. We need to add guards for tensor types in THNN and THCUNN.
It's a problem with function, that only . Should be a simple fix.
Whoops! Yeah it just ignores the "thpp::Type type" parameter
what type is expected here?
a is a FloatTensor but weights is a DoubleTensor. Change: to: (or change a to (...).double())
cool thanks!

Discussion:
self.assertIn('buf', l.state_dict()) self.assertIs(l.state_dict()['buf'], buf) <ins> def test_Conv2d_inconsistent_types(self): inputs = Variable(torch.randn(4, 1, 7, 7).float()) weights = Variable(torch.randn(1, 1, 3, 3).double()) # inconsistent types should raise an exception self.assertRaises(RuntimeError, lambda: nn.functional.conv2d(inputs, weights)) # but it should work with the same type nn.functional.conv2d(inputs.float(), weights.float()) </ins> @unittest.skipIf(not TEST_CUDA, 'CUDA not available') def test_Conv2d_large_workspace(self): # These sizes require huge cuDNN workspaces. Make sure we choose a

if (tensor->isCuda() != isCuda) { throw invalid_tensor(isCuda ? "CUDA" : "CPU", tensor->isCuda() ? "CUDA" : "CPU"); } <ins> if (tensor->type() != type) { throw invalid_tensor(thpp::toString(type), thpp::toString(tensor->type())); } </ins> } }
Change:
When building from source, I got this error. My system is OSX 10.12.4, conda 2.7, CUDA 8.0. Any thoughts? Thanks!
Yes, same exact issue over here, with (Miniconda) Python 3.6.0.
Thank you,

Discussion:
TARGET_LINK_LIBRARIES(THPP LINK_PRIVATE ${TH_LIBRARIES} ${THS_LIBRARIES}) IF(CUDA_FOUND) TARGET_LINK_LIBRARIES(THPP LINK_PRIVATE ${THC_LIBRARIES} ${THCS_LIBRARIES}) <ins> TARGET_LINK_LIBRARIES(THPP ${CUDA_LIBRARIES}) </ins> ENDIF() INSTALL(TARGETS THPP

LDFLAGS="-L$INSTALL_DIR/lib " LD_POSTFIX=".so.1" LD_POSTFIX_UNVERSIONED=".so" <del> if [[ $(uname) == 'Darwin' ]]; then LDFLAGS="$LDFLAGS -Wl,-rpath,@loader_path" </del> <ins> if [[ $(uname) == 'Darwin' ]]; then LDFLAGS="$LDFLAGS -Qunused-arguments -Wl,-rpath,@loader_path" </ins> LD_POSTFIX=".1.dylib" LD_POSTFIX_UNVERSIONED=".dylib" else
Change:
The state attribute in the opt. class is a defaultdict which helps to add new parameter states in the state dict as they are being used while training. Once the optimizer's statedict is saved and loaded, the state attribute is a dict and not a defaultdict which assumes that before saving, all the parameters in the network are present in the state dict. When loading the opt. using a saved statedict the state attribute should be a defaultdict. I currently do the following to have this behavior Corresponding
Hi all, it is my first time trying to contribute to an open-source project. I took a look on the bug. It seems that it is related to the function "statedict()" of class "". When the state is transformed to packedstate, it uses dict instead of defaultdict, which is inconsistent with the type of state. The solution I am proposing is to let the packedstate be defaultdict to maintain consistency with Here is my proposed change: packedstate = defaultdict(dict, {(id(k) if isinstance(k, Variable) else k): v for k, v in ()}) Is there anything I am missing? Is "" in the pytorch/test enough to validate my change? Where could I add a test point to ensure the type of state remain the same after saving and loading? Thanks a lot for your patience.
closed via thanks to !! Thanks a lot !

Discussion:
id_map = {old_id: p for old_id, p in zip(chain(*(g['params'] for g in saved_groups)), chain(*(g['params'] for g in groups)))} <del> state = {id_map.get(k, k): v for k, v in state_dict['state'].items()} </del> <ins> state = defaultdict( dict, {id_map.get(k, k): v for k, v in state_dict['state'].items()}) </ins> # Update parameter groups, setting their 'params' value def update_group(group, new_group):
Change:
I was reading the documentation of method in the . I tried this in IPython environment. Of course, error occurs because 5 is larger than the size of weights and I didn't set . The error message is like this: But after this, the dimensions of weights changed. If I run as following without runtime exception, then the dimensions of weights do not change. This is caused by not cleaning up after exception raised in this . Maybe the solution is adding cleaning up code in the .
The problem emerged in two other situations as well, here are the examples: I'm sending a pull request with the fix. Thanks for reporting the problem. :)

Discussion:
n_dist = THTensor_(size)(prob_dist, 0); n_categories = THTensor_(size)(prob_dist, 1); <del> THArgCheck(n_sample > 0, 2, "cannot sample n_sample < 0 samples"); </del> <ins> THArgCheckWithCleanup(n_sample > 0, THCleanup(if (start_dim == 1) THTensor_(resize1d)(prob_dist, n_categories);), 2, "cannot sample n_sample < 0 samples"); </ins> if (!with_replacement) { <del> THArgCheck((!with_replacement) && (n_sample <= n_categories), 2,  "cannot sample n_sample > prob_dist:size(1) samples without replacement"); </del> <ins> THArgCheckWithCleanup((!with_replacement) && (n_sample <= n_categories), THCleanup(if (start_dim == 1) THTensor_(resize1d)(prob_dist, n_categories);), 2, "cannot sample n_sample > prob_dist:size(1) samples without replacement"); </ins> } /* cumulative probability distribution vector */

sum  ); } <del> THArgCheckWithCleanup((sum > 0), THCleanup(THDoubleTensor_free(cum_dist);), 2, </del> <ins> THArgCheckWithCleanup((sum > 0), THCleanup(THDoubleTensor_free(cum_dist); if (start_dim == 1) THTensor_(resize1d)(prob_dist, n_categories);), 2, </ins> "invalid multinomial distribution (sum of probabilities <= 0)"); /* normalize cumulative probability distribution so that last val is 1 i.e. doesn't assume original prob_dist row sums to one */
Change:
I have a network defined as: --. In this particular case, I want to keep the two conv layer weights fixed, and only adapt the middle layer weights. In the code below I have three ways of defining the conv layers parameters, one with buffers, one with parameters that have , and one with regular parameters (which doesn't keep the weights fixed, but I'm using it as a reference). If I use the first two definitions (commented out in the code below), when I call it results in a kernel crash. If I use the third definition, it works fine albeit without maintaining fixed weights. In order to apply the linear layer I reshape the output and inputs, and I believe the crash stems from somewhere there (on a side note: Is this the right way to apply a linear layer on a time sequence?)
Looks like the problem is with . If I reimplement this network using 2D convolutions with unit widths it works fine.

Discussion:
} if (k == 3) { <del> if (should_compute_output(0)) { grad_input = view3d(*grad_input); </del> <ins> if (grad_input) { grad_input = view3d(*grad_input); } if (grad_weight) { grad_weight = view3d(*grad_weight); </ins> } <del> grad_weight = view3d(*grad_weight); </del> } auto outputs =  as_tensor_list(std::move(grad_input),