jiebi/CodeConvo huggingface dataset:
Change:
"torch/csrc/cuda/Module.cpp", "torch/csrc/cuda/Storage.cpp", "torch/csrc/cuda/Stream.cpp", <del> "torch/csrc/cuda/Tensor.cpp", </del> "torch/csrc/cuda/AutoGPU.cpp", "torch/csrc/cuda/utils.cpp", "torch/csrc/cuda/serialization.cpp", <ins> *split_types("torch/csrc/cuda/Tensor.cpp"), </ins> ] if WITH_NCCL:

Discussion:
Trying to run on github actions via AWS instance gives me: I will of course set up now, but it shouldn't crash like that. May be it should use as a fallback? pt-2.0 cc
Is this a multitenant setup? Also curious which distro are you running, I was under the impression that most linux distros set home by default
this is AWS image. one default user. I think it's user, but it wants you to run as root, which is weird. I have manually now set to overcome this.
We should update triton to use pathlib or something
Change:
pip_packages=pip_list_output, conda_packages=get_conda_packages(run_lambda), os=get_os(run_lambda), <ins> libc_version=get_libc_version(), </ins> gcc_version=get_gcc_version(run_lambda), clang_version=get_clang_version(run_lambda), cmake_version=get_cmake_version(run_lambda),

Discussion:
TensorPipe uses some advanced features of the Linux kernel. It does so on a "best effort" basis (i.e., use them if available, fall back on something else if not). This logic is unfortunately hard to test because it would require specific kernel versions on our CI machines. So it occasionally happens that we find out about issues when users report them. To help investigate these reports we've found it useful to know the kernel version and the glibc version used by the user (both when building and when running, if they're different). It would be great if these could be to the report produced by collect_env.
CC
Change:
def cross(self, other, dim=-1): return Cross(dim)(self, other) <ins> def inverse(self): return Inverse.apply(self) </ins> def multinomial(self, num_samples=1, with_replacement=False): return Multinomial(num_samples, with_replacement)(self)

Discussion:
Command to install the latest nightly binary: installs : However, PyTorch itself reports : I guess this version tag might be defined during the build process from e.g. and then forwarded from , but also source builds should use based on the tag. Unsure, if this is just a reporting issue or if we are building the nightly binaries with and executing them with , so creating this issue to dig a bit further into it. CC cc
This is really weird, I've put up measure in place to validate against it... Will investigate today (and moving to pytorch repo)
can you please check that 2.2.0 reports (and is build) against the correct version of nccl library?
Confirmed for nightlies : Same issue for RC:
Looking at the build log Looks like we are compiling with: USESYSTEMNCCL
On Windows I see following exception: We compiling with should we return None rather then exception ?
In this case we are indeed using 2.18.1: Do you know why we are using the system-wide NCCL installation instead of the one? Should we delete the system NCCL and use by default?
Looks like its set here: Looks like it was by: Let me propose the PR to unset this flag : USESYSTEM_NCCL
I guess we are using the system NCCL installation to be able to during the runtime. If we would use the module I assume we would link NCCL into the PyTorch binaries. If so, we should make sure to update the NCCL version whenever is updated. This could break our dependency on the NCCL pip package.
Yes. Also when we update this nccl here:
We used following path  What would be good path for nccl 2.19.3 and cuda 12.1 ? As per this issue: Will be using 12.0 ncck 2.19.3
Installing system NCCL into the nightly build docker container feels great, but we need to figure out, how to keep it in sync with PyTorch. I wonder if one can, for example, just check nccl version from this requirements file and build from source?
yes probably we should build and upload nccl for our supported cuda versions, same way as we build docker containers, by following details here:
Verified with final rc
Change:
if memo is None: memo = set() if self not in memo: <del> super(Container, self).modules(memo) </del> <ins> for m in super(Container, self).modules(memo): yield m </ins> for module in self.children(): for m in module.modules(memo): yield m

Discussion:
While using torch on a Macbook pro with M1 ship, using python 3.8.10 and torch 1.12.1, I came across a weird bug that happened because I forgot to put a tensor on MPS. The reproducible code is the following: As you can see, if is on MPS and on CPU, the line wrongly replaces the index in place of the index . I would expect to have an error saying both tensors should be on the same device instead. I get an error: cc
The error seems to happen somewhere within Also I'm getting a warning "The operator 'aten::nonzero' is not currently supported on the MPS backend and will fall back to run on the CPU. This may have performance implications" when printing tensors on MPS.
Hi - thanks for reporting this issue. This is actually an allowed usecase (copying CPU tensors into MPS tensors / MPS -CPU / MPS -MPS): The issue is on MPS side when copying a CPU tensor into a strided MPS tensor. I'll take a look.
Change:
#include "THCTensorTypeUtils.cuh" #include "THCHalfAutoNumerics.cuh" #include "SharedMem.cuh" <ins> #include <algorithm> </ins> //////////////////////////////////////////////////////////////////////////////// // Spatial kernel (fast with large inner_size and small dim_size) ////////////////////////////////////////////////////////////////////////////////

Discussion:
mac 10.12.6 xcode 8.3.3 clang Apple LLVM version 8.0.0 (clang-800.0.42.1)
same here. Adding fixes it.
thanks! ! building now...
success!yeah! thanks again.
Change:
return grad_input <del> # TODO: gather </del> <ins> class Gather(Function): def __init__(self, dim): super(Gather, self).__init__() self.dim = dim def forward(self, input, index): assert not self.needs_input_grad[1], "Gather can't differentiate "  "the index" self.input_size = input.size() self.save_for_backward(index) return input.gather(self.dim, index) def backward(self, grad_output): index, = self.saved_tensors grad_input = grad_output.new(self.input_size).zero_() return grad_input.scatter_(self.dim, index, grad_output), None class Scatter(InplaceFunction): def __init__(self, dim, inplace=False): super(Scatter, self).__init__(inplace) self.dim = dim def forward(self, input, index, source): assert not self.needs_input_grad[1], "Scatter can't differentiate "  "the index" if self.inplace: self.mark_dirty(input) else: input = input.clone() self.save_for_backward(index) return input.scatter_(self.dim, index, source) def backward(self, grad_output): index, = self.saved_tensors grad_input = grad_source = None if self.needs_input_grad[0]: grad_input = grad_output.clone() grad_input.scatter_(self.dim, index, 0) if self.needs_input_grad[2]: grad_source = grad_output.gather(self.dim, index) return grad_input, None, grad_source </ins> # TODO: kthvalue # TODO: repeat # TODO: sort

Discussion:
<!-- A clear and concise description of what content in is an issue. If this has to do with the general website, please file an issue at instead. If this has to do with , please file an issue at --In the getting started section section the following quote appears: I guess this is from previous versions where CUDA support was not built-in and this can be removed, correct? cc
This seems to be easy fixable docs issue
If gets merged this is fixed.
Thanks for the PR Just merged it.
Change:
self.assertEqual(a[()], a) self.assertEqual(a[()].data_ptr(), a.data_ptr()) <del> # @unittest.skip('failing') </del> def test_empty_fancy_index(self): # Empty list index creates an empty array a = tensor([1, 2, 3])

Discussion:
In PyTorch master 0.4 , variables cannot be indexed. This used to work in PyTorch 0.2 :
This is breaking some of Pyro's inference algorithms on PyTorch master:
Change:
def _test_sparse_mask_hybrid_fixed(self): i = self.IndexTensor([ <del> [1, 3, 3, 0, 4], [2, 1, 1, 2, 3], </del> <ins> [1, 3, 0, 4], [2, 1, 2, 3], </ins> ]) <del> v = self.ValueTensor([[1, 2], [2, 3], [3, 4], [4, 5], [5, 6]]) x = self.SparseTensor(i, v, torch.Size([5, 4, 2])) </del> <ins> v = self.ValueTensor([[1, 2], [2, 3], [3, 4], [4, 5]]) # TODO: This is also testing that, if coalesce is a no-op, # the indices don't get permuted. I don't know if we actually # want to give this invariant. x = self.SparseTensor(i, v, torch.Size([5, 4, 2])).coalesce() </ins> dense = self.ValueTensor([ [[1, 3], [2, 2], [3, 3], [4, 2]], [[5, 7], [6, 7], [7, 9], [8, 9]],

Discussion:
and I talked about the intended semantics of sparsemask when given a sparse tensor as an argument, and we came to the following conclusions: with an uncoalesced sparse tensor does not make sense, full stop. Unlike values, where your update might be linear and work correctly on an uncoalesced sparse tensor, there isn't any good reason why you want to pluck out values from the dense tensor multiple times, it is basically always a bug. the (eventual) documentation for sparsemask, we will recommend calling coalesce before doing any operations involving sparsemask (or nonlinear updates to indices/values). should add a test to sparsemask which checks if the flag of the input tensor is true, and raise an error if it is not. This is a cheap safety check which will prevent people from forgetting to call coalesce (as in (2)). There is a performance consequence to calling coalesce every iteration of training. If the gradient is producing coalesced vectors, we can alleviate this problem by adding a preprocessing step to coalesce() which tests if the input is already coalesced, and bugs out in that case. However, I think this is out of scope for this particular ticket for now.
Change:
# State initialization if len(state) == 0: state['step'] = 0 <del> state['square_avg'] = grad.new().resize_as_(grad).fill_(1) </del> <ins> state['square_avg'] = grad.new().resize_as_(grad).zero_() </ins> square_avg = state['square_avg'] alpha = group['alpha']

Discussion:
We should unify this with our normal list creating code for the reasonable cases cc
Similar issue to this, I believe:

The hub tests are broken on macOS. I'm seeing SSL certificate related errors when trying to hit GitHub. Run the macOS tests on CI with TestHub enabled. I tried installing the latest version of from conda-forge (which pulled in ) but this didn't fix the issue. cc
Looks like the same error I'm getting trying to use Ubuntu 18.04 with fresh install of conda and Pytorch 1.2
I think this will all be solved with .

Porting TH operators is essential for code simplicity and performance reasons. Porting guides and Q&A are available in umbrella issue: Feel free to add as a reviewer to get a prioritized review.

This is of particular use in list comprehensions: This should work:
I'll need to adapt list comprehensions to use Iterables. I'll take a look!

I haven't looked into this in depth but this looks fishy: More here: cc
is this still an issue in 1.3.1?
I can repro this on master:
The problem is: removes the parameters from the module's parameter list and makes them normal class attributes: applies only to parameters, not to the weight attributes.  (which is called from acts on which includes the value of the weight attributes and the bias parameters. So, it contains a mix of CUDA and CPU tensors.
This is my workaround for right now: This applies weight norm to an rnn module and overrides the function to deal with the error.
This appears to be fixed. do you want to verify?
I tested this on nightly and it worked for me, so closing.

As mentioned in title, I want to make sure if pytorch can deal with the situation that each input in a batch has different shapes and thus different computation graphs.
Please don't open duplicated issue for discussion. Context will be missed. Try to discuss about it here:

Be Careful What You Backpropagate: A Case For Linear Output Activations & Gradient Boosting I can work on this if this can be to pytorch. Please let me know. Thanks!
This should be fairly straightforward to add in user code using . Eventually it could be to core, but its utility should probably be validated in external repos first.
Change:
self.assertLessEqual(params.data.dist(solution), initial_dist) <ins> def _test_rosenbrock_sparse(self, constructor): params_t = torch.Tensor([1.5, 1.5]) params = Variable(torch.Tensor([1.5, 1.5]), requires_grad=True) params_c = Variable(torch.Tensor([1.5, 1.5]), requires_grad=True) optimizer = constructor([params]) optimizer_c = constructor([params_c]) solution = torch.Tensor([1, 1]) initial_dist = params.data.dist(solution) def eval(params, sparse_grad, w): optimizer.zero_grad() loss = rosenbrock(params) loss.backward() # params.grad.data now has the computed gradient. Turn # it into a sparse tensor params.grad.data.copy_(drosenbrock(params.data)) # This is really goofy if w: i = torch.LongTensor([[0]]) v = torch.DoubleTensor([params.grad.data[0]]) else: i = torch.LongTensor([[1]]) v = torch.DoubleTensor([params.grad.data[1]]) x = sparse.DoubleTensor(i, v, torch.Size([2])) if sparse_grad: params.grad.data = x else: params.grad.data = x.to_dense() return loss for i in range(2000): w = torch.rand(1)[0] > 0.5 optimizer.step(functools.partial(eval, params, True, w)) optimizer_c.step(functools.partial(eval, params_c, False, w)) self.assertEqual(params.data, params_c.data) self.assertLessEqual(params.data.dist(solution), initial_dist) </ins> def _test_basic_cases_template(self, weight, bias, input, constructor): weight = Variable(weight, requires_grad=True) bias = Variable(bias, requires_grad=True)

Discussion:
and I talked about the intended semantics of sparsemask when given a sparse tensor as an argument, and we came to the following conclusions: with an uncoalesced sparse tensor does not make sense, full stop. Unlike values, where your update might be linear and work correctly on an uncoalesced sparse tensor, there isn't any good reason why you want to pluck out values from the dense tensor multiple times, it is basically always a bug. the (eventual) documentation for sparsemask, we will recommend calling coalesce before doing any operations involving sparsemask (or nonlinear updates to indices/values). should add a test to sparsemask which checks if the flag of the input tensor is true, and raise an error if it is not. This is a cheap safety check which will prevent people from forgetting to call coalesce (as in (2)). There is a performance consequence to calling coalesce every iteration of training. If the gradient is producing coalesced vectors, we can alleviate this problem by adding a preprocessing step to coalesce() which tests if the input is already coalesced, and bugs out in that case. However, I think this is out of scope for this particular ticket for now.
Change:
example_value = tensor[invalid_value_mask.eq(0)][0] tensor[invalid_value_mask] = example_value if invalid_value_mask.any(): <del> min_sz = 3 </del> <ins> min_sz = max(min_sz, 3) </ins> int_mode = True # TODO: use fmod?

Discussion:
It seems that the following link that can be reached from here (check the screenshot) is broken: Page: Broken link: Screenshot: ! cc
this should be transferred to , but I don't have permissions to do that. Fix is available at pytorch/hub.
Change:
toDeviceTensor<THCIndex_t, 1>(state, target), toDeviceTensor<real, 1>(state, output), weights ? THCTensor_(data)(state, weights) : NULL, <ins> n_classes, </ins> ignore_index); <ins> THCudaCheck(cudaGetLastError()); </ins> if (weights) { THCTensor_(free)(state, weights); }

Discussion:
Claim: When I use an invalid label with F.crossentropy with reduce=False, I don't receive an error message. I do receive an error message when reduce=True. Code: Result: No error message received. Expected error message: Context: I was using , accidentally used an invalid label, and got extremely large losses and extremely small losses, and inconsistently so. Environment: OS: Ubuntu 16.04 on Docker and host PyTorch version: 0.3.0.post4 How you installed PyTorch: conda Python version: 3.6.3
Change:
``` docker build -t pytorch . ``` <del> Alternatively, if you want a runtime image, build with ``` docker build -t pytorch . -f tools/docker/Dockerfile_runtime ``` and run with nvidia-docker: </del> <ins> Alternatively, if you want to use a runtime image, you can use the pre-built one from Docker Hub and run with nvidia-docker: </ins> ``` <del> nvidia-docker run --rm -ti --ipc=host pytorch </del> <ins> nvidia-docker run --rm -ti --ipc=host pytorch/pytorch:latest </ins> ``` Please note that PyTorch uses shared memory to share data between processes, so if torch multiprocessing is used (e.g. for multithreaded data loaders) the default shared memory segment size that container runs with is not enough, and you

Discussion:
Do you plan to publish an official entry on docker hub?
any update? I think it's an important enhancement.
I believe this would just be pushing the image from ? If so, I can add it to Docker Hub if you like?
Configure automated builds on Docker Hub
The dockerfile you referenced builds from current master, I don't know if that's desirable. There is a runtime dockerfile in tools/docker that installs current binary release, I'd think that's what should be hosted on the dockerhub.
yup, that makes more sense. Thanks
Oops, looks like runtime dockerfile would be broken now (see discussion in #beginner, conda packages conflict with conda numpy).
Just tested, the runtime dockerfile seems to work just fine. Pulls in numpy 1.13.1 which seems to not have any issues.
pre-built images are now available, you can run them with:
Change:
return SymbolicCallArg(expr, tree.numel) def generate_workspace_allocation(self, nbytes, device, zero_fill): <ins> if isinstance(nbytes, sympy.Expr): nbytes = V.graph.sizevars.size_hint(nbytes) </ins> line = self.make_allocation( "workspace", device, torch.uint8, shape=(nbytes,), stride=(1,) )

Discussion:
Repro: Error: cc
Un-assign myself to let work on it.
, I can't reproduce the issue. The is no longer in the codebase. I gave a try of , which worked well. Could you double check if you can reproduce the issue?
Yes, I can still reproduce with your cmdline. Can you retry after deleting your local cache?
Change:
T bern_uniform = bernoulli[idx]; int _mask = (int) THCNumerics<T>::lt(bern_uniform, q[rand_ind]); output[idx] = J[rand_ind]*(1 -_mask) + (rand_ind+1L) * _mask; <del> } </del> <ins> } </ins> } template <typename T>

Discussion:
Looking at the values returned from tn.multinomial(1) and counting them, it seems like it returns the values of the multinomial distribution taken from t, i.e. that it has ignored the signs of the values. However, it seem like always returns a tensor filled with zeros.
looking into this
Change:
(e.g. BatchNorm stats) are broadcast form the module in process of rank 0, to all other replicas in the system in every iteration. <ins> .. warning:: Forward and backwrad hooks defined on :attr:`module` and its submodules won't be invoked anymore, unless the hooks are initialized in the :meth:`forward` method. </ins> Args: module: module to be parallelized device_ids: CUDA devices (default: all devices)

Discussion:
PyTorch version: 0.3.0 Python version: 3.5 GPU models and configuration: 8x NVIDIA Titan I found that requiresgrad setting in module is ignored when module is wrapped with nn.DataParallel. For example, This code should be executed because very small part of network activation (only fc layer) is stored for backward computation, but it results to run-time error: Internally, I found a crack in function which is in . In replicate function, it copies all parameter in module (# of replica times) with . In broadcasting code, it just defines new with default constructor parameter, which is always set to True. I think there can be some choices to fix this issue. is intended behavior for DataParallel, so we should use volatile (or ) to implement transfer learning in DataParallel model should be fixed to handle requires_grad when copying parameters. should be synchronizing in I think any of solution doesn't need lots of effort to fix it.
Change:
include_dirs = [] extra_link_args = [] <del> extra_compile_args = ['-std=c++11'] </del> <ins> extra_compile_args = ['-std=c++11', '-Wno-write-strings'] </ins> cwd = os.path.dirname(os.path.abspath(__file__)) lib_path = os.path.join(cwd, "torch", "lib")

Discussion:
<!-- A clear and concise description of what the bug is. --The target name is used by a test executable in mkldnn. Steps to reproduce the behavior: BUILDTEST=1 build <!-- If you have a code sample, error messages, stack traces, please provide it here as well --<!-- A clear and concise description of what you expected to happen. --CMake configure passes. Please copy and paste the output from our (or fill out the checklist below manually). You can get the script and run it with: PyTorch Version (e.g., 1.0): v1.5.0-rc2 OS (e.g., Linux): Windows How you installed PyTorch (, , source): source Build command you used (if compiling from source): python build Python version: 3.x CUDA/cuDNN version: 9.2 / 10.1 / 10.2 / NA GPU models and configuration: Tesla M60 Any other relevant information: <!-- Add any other context about the problem here. --We should not build tests for mkldnn.
I have encountered the same error, how do I resolve it?
It is actually mkldnn that is out of sync. So running the following script should help.
Thanks a lot for this I posted this on the PyTorch Discussions but to no avail, and now I've finally understood the reason of this error. Thanks a lot. Surprisingly, for me the build works out if I build from the master repo, but changing to a different version always fails.
Change:
THLongStorage_free(topKSize); #define RUN_K(INDEX_T, DIM, DIR)                                         <del> gatherTopK<real, INDEX_T, DIM, DIR>                                          </del> <ins> gatherTopK<real, INDEX_T, DIM, DIR>                                    </ins> <<<grid, block, 0, THCState_getCurrentStream(state)>>>(              inputInfo,                                                         sliceSize,                                                        

Discussion:
Repro: Expect two small tensors of small integers to be printed. Instead, only the first case works. In the second case, you get a series of failed assertions starting with this: I wonder if it is related to
is this still an outstanding issue? I cannot reproduce this with an install from the master branch.
The size which causes failure doesn't seem deterministic. Using a pretty recent build (a few days old), I now have to increase the "9" to a larger number to make it fail. Sometimes it will fail at 10, sometimes at 11. Here's an updated case that seems to fail reliably at some size, for me:
Change:
for(int i = 0; i < input->nDimension; i++){ no_elements *= input->size[i]; } <del> no_elements *= scale_factor * scale_factor; </del> <ins> no_elements *= scale_factor; </ins> int d1; int d2;

Discussion:
Original post at: Original post content: code: Tested on Linux with pytorch 0.2.0+ & GTX 750 and Mac OS X Sierra with pytorch 0.2.0+ & GT 750M It seems to be work well with mode=‘linear’ and memory of gpu is larger than 1GB. I think it’s something wrong. Comment: successfully reproduced using the above snippet on master.