jiebi/CodeConvo huggingface dataset:
Change:
self._data_ptrs = [] return <ins> # If any parameters alias, we fall back to the slower, copying code path. This is # a sufficient check, because overlapping parameter buffers that don't completely # alias would break the assumptions of the uniqueness check in # Module.named_parameters(). unique_data_ptrs = set(p.data_ptr() for l in self.all_weights for p in l) if len(unique_data_ptrs) != sum(len(l) for l in self.all_weights): self._data_ptrs = [] return </ins> with torch.cuda.device_of(any_param): # This is quite ugly, but it allows us to reuse the cuDNN code without larger # modifications. It's really a low-level API that doesn't belong in here, but

Discussion:
While the fast path for cuDNN RNNs introduced in includes a check intended to fall back to copying weights if weights are moved or aliased, this only works if the aliasing happens after the weights are first compacted. But the most natural place to tie weights is inside a module's method, which runs before , the typical time when is first invoked. Here's an MWE. The three runs (CPU, weight tying before , and weight tying after ) should all give the same results, and they do before , but they have not since:
Change:
(4.53, 3.83, -6.64, 2.06, -2.47, 4.70))).t() b = torch.Tensor(((8.58, 8.26, 8.48, -5.28, 5.72, 8.93), (9.35, -4.43, -0.70, -0.26, -7.36, -2.52))).t() <del> _test(a, b, expectedNorm) </del> <ins> _test_overdetermined(a, b, expectedNorm) </ins> # test underdetermined expectedNorm = 0

Discussion:
b = (600,1) A = (600,2) X, = (b,A) X.size() #torch.Size([2, 1]) Y, = (b.cuda(),A.cuda()) Y.size() #torch.Size([600, 1])
that is interesting. says that the first rows returned ( here) include the solution of the least-squares problem. I've verified that the first 2 rows of X and Y are the same so you can probably use that for now while we figure out what the rest of the output is.
Change:
ensure_attr('ClassNLLCriterion', 'weights') ensure_attr('ParallelCriterion', 'repeatTarget') ensure_attr('MultiMarginCriterion', 'weights') <ins> ensure_attr('SpatialConvolution', 'bias', 'finput', 'fgradInput', 'gradWeight', 'gradBias') </ins> attr_map('ReLU', {'val': 'value'}) attr_map('Threshold', {'val': 'value'}) attr_map('Unsqueeze', {'pos': 'dim'})

Discussion:
Subclassing torch.Tensor with fails due to a on the latest nightly builds, whereas it previously worked without issue. Run this snippet of code: And you'll recieve an error message like this: Changing the to results in the exact same error message. There should be no error message as it should be working like it was previously in Torch 1.8.1 and below. I think that the issue started on Torch version: , and I can confirm that it still exists as of , . The issue started on the nightly builds at least 2 days ago, if that helps resolve things. It's been causing some of Captum's CircleCI tests to fail, and the tests show the first occurrence of the error on . cc
This is due to either or
I think the problem is I'm not supposed to delete because that means the super() call doesn't work. Instead, I should just verify that the passed in class is a subclass, then this should work.
Change:
) @unittest.skipIf(BACKEND != 'gloo', "Only Gloo backend supports CUDA allReduce") <ins> @skip_if_no_cuda_distributed </ins> def test_all_reduce_sum_cuda(self): group, group_id, rank = self._init_global_test() self._test_all_reduce_helper(

Discussion:
is causing a crash when we try to build pytorch. Crash looks like this: (all times in US/West) Our 7AM, 8AM builds and earlier were all fine but 9AM, 10AM, 11AM builds all crashed with that same traceback. The new import was in which I think went in between 8AM and 9AM today See command below: See here if needed for : <!-- A clear and concise description of what you expected to happen. --Please copy and paste the output from our (or fill out the checklist below manually). You can get the script and run it with: PyTorch Version (e.g., 1.0): building from master OS (e.g., Linux): linux How you installed PyTorch (, , source): source Build command you used (if compiling from source): see above under 'reproduce' Python version: 3.6
maybe easier to read for build command used:
please try in the CircleCi environment?
in our build script seems to have fixed this. Thanks !
Change:
# test is no more than 4 higher than the 10th available at the # start. This attempts to catch file descriptor leaks, but allows # one-off initialization that may use up a file descriptor <del> available_fds = self._get_next_fds(10) self.test_case.assertLessEqual( available_fds[-1] - self.next_fds[-1], 5) </del> <ins> # TODO: Disabled because this check is too flaky # available_fds = self._get_next_fds(10) # self.test_case.assertLessEqual( #     available_fds[-1] - self.next_fds[-1], 5) </ins> self.test_case.assertFalse(self.has_shm_files()) return False

Discussion:
causes a host-device synchronization on CUDA. This is because it must first calculate the count of nonzero elements, and then copy this count to host. There are cases where I can easily calculate the count myself (from existing host data), or already know the count, which would make this unnecessary. For some given of shape : Instead of: Do this trickery: This avoids the host-device synchronization, but on the other hand, is more complicated, and a bit slower. I'm not sure if there are any other ways? Btw, I'm using to measure the speed. But actually I'm not sure if this really properly covers the problem of host-device synchronization with . Does it? E.g. when comparing this trickery code to , I just see that is slower. No response_
dupe of i think
Ah yes, I think you are right.
Change:
export CMAKE_PREFIX_PATH=${PYTORCH_ENV_DIR}/miniconda3/ # Test PyTorch <ins> if [[ "${JOB_BASE_NAME}" == *cuda9.2* ]]; then # Eigen gives "explicit specialization of class must precede its first use" error # when compiling with Xcode 9.1 toolchain, so we have to use Xcode 8.2 toolchain instead. export DEVELOPER_DIR=/Library/Developer/CommandLineTools else export DEVELOPER_DIR=/Applications/Xcode9.app/Contents/Developer fi </ins> export MACOSX_DEPLOYMENT_TARGET=10.9 export CXX=clang++ export CC=clang

Discussion:
Mac OS X builds failing this way. This appears to be correlated with worker-macos-high-sierra-2 and worker-macos-high-sierra-16.
Change:
output (Tensor): output Tensor """ <ins> if mode != 'bilinear': raise NotImplementedError("nn.functional.grid_sample got unsupported mode: '{}'".format(mode)) </ins> return vision.grid_sampler(input, grid, padding_mode)

Discussion:
The only valid mode is 'bilinear', but one can pass anything to it and it will not complain:
Apparently, the is not passed to the backend function for . I think it should be removed.
Yes. I think the idea is that we'd like to support other modes at some point.
Change:
python setup.py install cd test/ echo "Ninja version: $(ninja --version)" <del> sh run_test.sh </del> <ins> sh run_test.sh -- -v </ins> echo "BUILD PASSED"

Discussion:
If a test segfaults, it's hard to determine which test it is because the output looks like: ! I can work on this if we think this is a good idea.
Last time I asked about this, he didn't want to because it adds a lot of scrolling. I also did some work to move the CI tests to so that we can output XML status files, but (1) this is unlikely to help in case of segfault, and (2) you can't do this straightforwardly, because we run some test files multiple times with different environment variables, and the XML exporter isn't really set up to handle this case. For the record, I am personally OK with just slapping on the tests and calling it a day.
in the conbuild sounds good to me too
closed via
Change:
are zeros. A sparse tensor is represented as a pair of dense tensors: a tensor <del> of values and a tensor of indices.  A sparse tensor can be constructed </del> <ins> of values and a 2D tensor of indices.  A sparse tensor can be constructed </ins> by providing these two tensors, as well as the size of the sparse tensor <del> (which cannot be inferred from these tensors!) </del> <ins> (which cannot be inferred from these tensors!)  Suppose we want to define a sparse tensor with the entry 3 at location (0, 2), entry 4 at location (1, 0), and entry 5 at location (1, 2).  We would then write: </ins> <del> >>> i = torch.LongTensor([[0, 1], [2, 0]]) >>> v = torch.FloatTensor([3, 4]) </del> <ins> >>> i = torch.LongTensor([[0, 1, 1], [2, 0, 2]]) >>> v = torch.FloatTensor([3, 4, 5]) </ins> >>> torch.sparse.FloatTensor(i, v, torch.Size([2,3])).to_dense() 0  0  3 <del> 4  0  0 </del> <ins> 4  0  5 [torch.FloatTensor of size 2x3] Note that the input to LongTensor is NOT a list of index tuples.  If you want to write your indices this way, you should transpose before passing them to the sparse constructor: >>> i = torch.LongTensor([[0, 2], [1, 0], [1, 2]]) >>> v = torch.FloatTensor([3,      4,      5    ]) >>> torch.sparse.FloatTensor(i.t(), v, torch.Size([2,3])).to_dense() 0  0  3 4  0  5 </ins> [torch.FloatTensor of size 2x3] You can also construct hybrid sparse tensors, where only the first n

Discussion:
The indices don't align with the values. In the example given on the website, by this logic, seems like it's indexing the columns first and that the indices are in reverse order to their values. Seems like corresponds to 4 and corresponds to 3. i = torch.LongTensor([[0, 1], [2, 0]]) v = torch.FloatTensor([3, 4]) (i, v, torch.Size([2,3])).todense() 0 0 3 4 0 0 [torch.FloatTensor of size 2x3] If I change the indices to and , I get i = torch.LongTensor([[0, 1], [0,1]]) v = torch.FloatTensor([3, 4]) (i, v, torch.Size([2,3])).todense() 3 0 0 0 4 0 [torch.FloatTensor of size 2x3] seems like unexpected behavior. Also if I try - should be intuitive - putting a value in position i = torch.LongTensor([[0, 1], [2, 0], [1,1]]) v = torch.FloatTensor([3, 4, 5]) (i, v, torch.Size([2,3])).to_dense() Traceback (most recent call last): File "<stdin", line 1, in <moduleRuntimeError: invalid argument 2: number of dimensions must be nDimI + nDimV at /pytorch/torch/lib/THS/generic/THSTensor.c:169 Help please?
This is a doc problem, thank you. The docs are misleading here, because it is not that is a coordinate, but 0 from the first tensor, and 2 from the second tensor. I'll try to rewrite these docs so they are clearer.
Basically, our constructor expects to be (transpose of what you gave it).
We should consider changing that
Change:
1.2469  0.0064 -1.6250 [torch.FloatTensor of size 3x3] <del> >>> torch.tril(a, k=1) </del> <ins> >>> torch.tril(a, diagonal=1) </ins> 1.3225  1.7304  0.0000 -0.3052 -0.3111 -0.1809 1.2469  0.0064 -1.6250 [torch.FloatTensor of size 3x3] <del> >>> torch.tril(a, k=-1) </del> <ins> >>> torch.tril(a, diagonal=-1) </ins> 0.0000  0.0000  0.0000 -0.3052  0.0000  0.0000

Discussion:
The states that the argument controls which diagonal to consider. However, this argument is actually named in pytorch 0.2.0.
Change:
// should have saved the grad accumulator. Even if the Variable no longer // alive, the accumulator should be kept alive by the references in the graph). if (requires_grad && !grad_fn && weak_grad_fn.expired() && grad_accumulator.expired()) <del> throw std::logic_error("No grad accumulator for a saved leaf!"); </del> <ins> throw std::logic_error("No grad accumulator for a saved leaf!"); </ins> new_var->grad_accumulator = grad_accumulator; return new_var;

Discussion:
there is a bug. the code is from The code for installing cuda10.2 should be corrected to The code for installing cuda10.2 should be corrected to cc
Thank you for reporting the typo

This is how I implement the decoder of a sequence to sequence model I'm not sure about the new autograd mechanics but this worked in the previous version. If I didn't make the unrolling codes a function it will work. It will also work with CPU. I compiled from source with Python 2.7, CUDA 8.0 and Cudnn 6
Thanks for the minimal reproduction! It broke OpenNMT and a big internal model but we weren't looking forward to isolating what parts of those had the issue.
That's great, I'll take a look today. Thanks!
I've pushed a fix to .
How does the inference work in this case? I mean, once you train it how do you test it in the test phase?
Change:
self += modules def __getitem__(self, idx): <del> if not (-len(self) <= idx < len(self)): raise IndexError('index {} is out of range'.format(idx)) if idx < 0: idx += len(self) return self._modules[str(idx)] </del> <ins> if isinstance(idx, slice): return ModuleList(list(self._modules.values())[idx]) else: if not (-len(self) <= idx < len(self)): raise IndexError('index {} is out of range'.format(idx)) if idx < 0: idx += len(self) return self._modules[str(idx)] </ins> def __setitem__(self, idx, module): return setattr(self, str(idx), module)

Discussion:
it'd be great to pick out elements of of a sequential with array slicing:
I agree this idea!
However,
this has come up again (last in ). Shall we reconsider?
I attempted getting a to work. Is this fine? If this is fine, I can issue a PR too. Let me know what you think. I believe this modification needs to be done in the class.
cc: This issue has been open for quite sometime.
Change:
#include <initializer_list> #include <iostream> #include <functional> <ins> #include <cstddef> </ins> #ifdef _MSC_VER #ifdef Type

Discussion:
Hello, I'm trying to build pytorch (I tried both v3_buildfix and master branch) on a raspberry pi 3 but i'm getting that error: Note: I'm building without CUDA and DISTRIBUTED support. OS: Ubuntu 16.04 (Mate) gcc: arm-linux-gnueabihf-gcc (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.5) 5.4.0 cmake: v3.5.1
Can you try building with and let me know if that fixes it?
Thanks, it compiles now ;)
Change:
call %CONDA_PARENT_DIR%Miniconda3Scriptsactivate.bat %CONDA_PARENT_DIR%Miniconda3 if "%INSTALL_FRESH_CONDA%"=="1" ( <del> call conda install -y -q python=%PYTHON_VERSION% numpy cffi pyyaml boto3 libuv </del> <ins> call conda install -y -q python=%PYTHON_VERSION% numpy"<1.23" cffi pyyaml boto3 libuv </ins> if errorlevel 1 exit /b if not errorlevel 0 exit /b call conda install -y -q -c conda-forge cmake=3.22.3

Discussion:
Numpy shouldn't require a pin with PyTorch. This issue is to ensure that should get reverted and all issues addressed (issues look like ) CI
Never mind, we apparently should have been pinning it.
Change:
l = nn.Linear(10, 5).float().cuda() i = Variable(torch.randn(20, 10).float().cuda(1)) l.cuda(1) <del> expected_out = l(i).data l.cuda(0) out = dp.data_parallel(l, i, (0, 1)) self.assertEqual(out.get_device(), 0) self.assertEqual(out.data, expected_out) </del> <ins> expected_out = l(i) loss = expected_out.sum() loss.backward() expected_grads = [] for param in l.parameters(): expected_grads.append(param.grad.clone()) dev_ids_list = [(0, 1), (1, 0)] for dev_id in dev_ids_list: with torch.cuda.device(dev_id[0]): l.cuda() l.zero_grad() out = dp.data_parallel(l, i, dev_id) loss = out.sum() loss.backward() self.assertEqual(out.get_device(), dev_id[0]) self.assertEqual(out.data, expected_out.data) for expected, param in zip(expected_grads, l.parameters()): self.assertEqual(param.grad.data, expected.data) </ins> # Check for None device_ids <ins> l = l.cuda() </ins> out = dp.data_parallel(l, i) @unittest.skipIf(not TEST_MULTIGPU, "multi-GPU not supported")

Discussion:
The loss will never decrease when I train a model using multi-gpus but without gpu0. This is an example I modified from the : If the result after one epoch is If the result after one epoch is
should always contain . If you want to specify which GPUs to use, you can set the environment variable , either by running your script like this: or set it programmatically in your script like this: So that deviceids 0, 1 will map to GPUs 1, 2, respectively. (Note that for the second method, you should put that line before any cuda relevant function call.)
Thank you.
<!-- drci-comment-start --:pagefacingup: Preview :pagefacingup: Preview :question: Need help or want to give feedback on the CI? Visit our Note: Links to docs will display an error until the docs builds have been completed. This comment was automatically generated by Dr. CI and updates every 15 minutes. <!-- drci-comment-end --
Change:
- name: adaptive_max_pool3d(Tensor self, IntList[3] output_size) cname: VolumetricAdaptiveMaxPooling <del> - name: avg_pool2d(Tensor self, IntList[2] kernel_size, IntList[2] stride={}, IntList[2] padding=0, bool ceil_mode=false, bool count_include_pad=false) </del> <ins> - name: avg_pool2d(Tensor self, IntList[2] kernel_size, IntList[2] stride={}, IntList[2] padding=0, bool ceil_mode=false, bool count_include_pad=true) </ins> cname: SpatialAveragePooling default_init: stride: kernel_size <del> - name: avg_pool3d(Tensor self, IntList[3] kernel_size, IntList[3] stride={}, IntList[3] padding=0, bool ceil_mode=false, bool count_include_pad=false) </del> <ins> - name: avg_pool3d(Tensor self, IntList[3] kernel_size, IntList[3] stride={}, IntList[3] padding=0, bool ceil_mode=false, bool count_include_pad=true) </ins> cname: VolumetricAveragePooling default_init: stride: kernel_size

Discussion:
, thank you! The documentation for avgpool2d/3d says Indeed, the default was just that up to PyTorch 0.2. In 0.3-0.4 the default is False for 2d/3d and True 1d and the doc says it was True for all. The change in default happened when replaced the previous mechanism of calling THNN. Now I was about to submit a PR reverting the default to True, but the subtle twist is that TF uses False by default. avgpool2d is used in the wild, e.g. by ResNet. Should I change the docstring or should I change the default in Do you have an opinion ( maybe)? Best regards Thomas
So after submitting this, I saw that the module interface uses countincludepad=True all along, so the right option seems to be to have the same in the functional interface. I'll submit a PR.
we should indeed default functional interface to True as well. Thanks for the PR
Change:
self.assertEqual(reference[None, 2:5, None, None], reference.unsqueeze(0)[:, 2:5].unsqueeze(2).unsqueeze(2)) # indexing with step <del> reference = self._consecutive((10, 10, 10)) </del> <ins> reference = consec((10, 10, 10)) </ins> self.assertEqual(reference[1:5:2], torch.stack([reference[1], reference[3]], 0)) self.assertEqual(reference[1:6:2], torch.stack([reference[1], reference[3], reference[5]], 0)) self.assertEqual(reference[1:9:4], torch.stack([reference[1], reference[5]], 0))

Discussion:
Arguably, indexing a Tensor by a LongTensor with no dimension should return a LongTensor with no dimension. Consider the following case: This breaks indexing by indices when the list of indices is empty, which can be hard to detect. In my use-case, I use such a masking to select rows of a tensor corresponding to "valid" entries in a vector; I had not considered that it would not work when there are no valid entries.
Thanks! I'll take a look.
Change:
} #define RUN_T(INDEX_T)                                                   <del> TensorInfo<real, INDEX_T> inputInfo =                                 getTensorInfo<THCTensor, INDEX_T>(state, input);                  TensorInfo<real, INDEX_T> topKInfo =                                  getTensorInfo<THCTensor, INDEX_T>(state, topK);                   </del> <ins> TensorInfo<real, INDEX_T> inputInfo =                                  getTensorInfo<THCTensor, INDEX_T>(state, input);                     TensorInfo<real, INDEX_T> topKInfo =                                   getTensorInfo<THCTensor, INDEX_T>(state, topK);                      </ins> TensorInfo<int64_t, INDEX_T> indicesInfo =                             getTensorInfo<THCudaLongTensor, INDEX_T>(state, indices);            

Discussion:
Repro: Expect two small tensors of small integers to be printed. Instead, only the first case works. In the second case, you get a series of failed assertions starting with this: I wonder if it is related to
is this still an outstanding issue? I cannot reproduce this with an install from the master branch.
The size which causes failure doesn't seem deterministic. Using a pretty recent build (a few days old), I now have to increase the "9" to a larger number to make it fail. Sometimes it will fail at 10, sometimes at 11. Here's an updated case that seems to fail reliably at some size, for me:
Change:
{"_write_file", (PyCFunction)THPStorage_(writeFile), METH_O, NULL}, {"_new_with_file", (PyCFunction)THPStorage_(newWithFile), METH_O | METH_STATIC, NULL}, #ifndef THC_GENERIC_FILE <ins> {"from_buffer", (PyCFunction)THPStorage_(fromBuffer), METH_VARARGS | METH_KEYWORDS | METH_STATIC, NULL}, </ins> {"_share", (PyCFunction)THPStorage_(_share), METH_NOARGS, NULL}, {"_new_shared", (PyCFunction)THPStorage_(_newShared), METH_VARARGS | METH_STATIC, NULL}, #endif

Discussion:
<!-- A clear and concise description of what the bug is. --The target name is used by a test executable in mkldnn. Steps to reproduce the behavior: BUILDTEST=1 build <!-- If you have a code sample, error messages, stack traces, please provide it here as well --<!-- A clear and concise description of what you expected to happen. --CMake configure passes. Please copy and paste the output from our (or fill out the checklist below manually). You can get the script and run it with: PyTorch Version (e.g., 1.0): v1.5.0-rc2 OS (e.g., Linux): Windows How you installed PyTorch (, , source): source Build command you used (if compiling from source): python build Python version: 3.x CUDA/cuDNN version: 9.2 / 10.1 / 10.2 / NA GPU models and configuration: Tesla M60 Any other relevant information: <!-- Add any other context about the problem here. --We should not build tests for mkldnn.
I have encountered the same error, how do I resolve it?
It is actually mkldnn that is out of sync. So running the following script should help.
Thanks a lot for this I posted this on the PyTorch Discussions but to no avail, and now I've finally understood the reason of this error. Thanks a lot. Surprisingly, for me the build works out if I build from the master repo, but changing to a different version always fails.
Change:
.format(torch.typename(value), name)) modules[name] = value else: <del> object.__setattr__(self, name, value) </del> <ins> buffers = self.__dict__.get('_buffers') if buffers is not None and name in buffers: if value is not None and not torch.is_tensor(value): raise TypeError("cannot assign '{}' as buffer '{}' " "(torch.Tensor or None expected)" .format(torch.typename(value), name)) buffers[name] = value else: object.__setattr__(self, name, value) </ins> def __delattr__(self, name): if name in self._parameters: del self._parameters[name] <ins> elif name in self._buffers: del self._buffers[name] </ins> elif name in self._modules: del self._modules[name] else:

Discussion:
They do one pair of these calls for each slicing operation.
Does this actually have a performance impact? These calls tend to be very quick
Yes, it's a major reason why small unrolled LSTMs (which involve in addition to two GEMMs: a chunk into four gates, for 4 pairs of calls, five unary pointwise ops, for 5 more, and five binary pointwise ops, for another 5) are much slower than they are in Theano and TensorFlow.
Also, there are excessive cudaGetDevice calls for each pointwise operation (9 cudaGetDevice) even without chunking, as in the following repro code: As for small unrolled LSTMs it does show up.
This should no longer be an issue, now that is implemented as a native ATen function. I think we're still using the Python version for tensors, but this will be resolved soon when we stop using tensors altogether.
Change:
INCLUDE_DIRECTORIES(${CMAKE_CURRENT_SOURCE_DIR}) ADD_LIBRARY(THPP SHARED ${all_cpp}) <del> SET_PROPERTY(TARGET THPP PROPERTY CXX_STANDARD 11) </del> SET_TARGET_PROPERTIES(THPP PROPERTIES VERSION 1 SOVERSION 1) <ins> if(NOT ${CMAKE_VERSION} VERSION_LESS "3.1") SET_PROPERTY(TARGET THPP PROPERTY CXX_STANDARD 11) endif(NOT ${CMAKE_VERSION} VERSION_LESS "3.1") </ins> TARGET_LINK_LIBRARIES(THPP PRIVATE ${TH_LIBRARIES}) INSTALL(TARGETS THPP

Discussion:
Currently, CMake 3.0 or higher is required for THPP. This is annoying for building from source on Ubuntu 14.04 systems, as default CMake is 2.8, and updating it requires either compiling from source or adding some random repos:
it should work fine if you change it to 2.8. I'm testing this out in a sec, and will push it into master
Nope, it does not for me.