jiebi/CodeConvo huggingface dataset:
Change:
for weight in self.parameters(): weight.data.uniform_(-stdv, stdv) <ins> def check_forward_args(self, input, hidden, batch_sizes): is_input_packed = batch_sizes is not None expected_input_dim = 2 if is_input_packed else 3 if input.dim() != expected_input_dim: raise RuntimeError( 'input must have {} dimensions, got {}'.format( expected_input_dim, input.dim())) if self.input_size != input.size(-1): raise RuntimeError( 'input.size(-1) must be equal to input_size. Expected {}, got {}'.format( fn.input_size, input.size(-1))) if is_input_packed: mini_batch = batch_sizes[0] else: mini_batch = input.size(0) if self.batch_first else input.size(1) num_directions = 2 if self.bidirectional else 1 expected_hidden_size = (self.num_layers * num_directions, mini_batch, self.hidden_size) def check_hidden_size(hx, expected_hidden_size, msg='Expected hidden size {}, got {}'): if tuple(hx.size()) != expected_hidden_size: raise RuntimeError(msg.format(expected_hidden_size, tuple(hx.size()))) if self.mode == 'LSTM': check_hidden_size(hidden[0], expected_hidden_size, 'Expected hidden[0] size {}, got {}') check_hidden_size(hidden[1], expected_hidden_size, 'Expected hidden[1] size {}, got {}') else: check_hidden_size(hidden, expected_hidden_size) </ins> def forward(self, input, hx=None): is_packed = isinstance(input, PackedSequence) if is_packed:

Discussion:
I run RNN sucessfully with the code. and I run embedding example. Then, got RuntimeError. I set to use nothing seems wrong. And the error info seems not the dim problem? The error is so strange.
What exactly do you mean by "embedding example"? Could you provide a code snippet that causes the RuntimeError?
To better reproduce the error, I use the official code example of The origin runs well: Due to use , I have to make batch first. And then , it raises error.
Thank you for the example! The say that setting means that the first dim of input is the batch, ie, input has size . however is unchanged, so the following should work: I do agree the error message could be better, though
Change:
} __device__ __forceinline__ void operator()(T* out, T* in) { <del> *out = mask(in) ? *in : ScalarConvert<int, T>::to(0); </del> <ins> *out = mask(out) ? *in : ScalarConvert<int, T>::to(0); </ins> } __device__ __forceinline__ void operator()(T* v) {

Discussion:
On recent PyTorch master (built from source; 0.4.0a0+) on a P100, CUDA 9:
I think the results on CPU meets the defination of triu (upper triangular part of the matrix). It is really hard to handle with a tensor from expand operation with multi-thread not only on GPU but also on CPU. The unexpected result will also happen on CPU if we apply openmp to it. I think it is necessary to eleminate the difference between the implementation and defination.
It doesn’t matter if it’s cpu or not. triu doesn’t have an underscore which means it doesn’t operate in-place and should always give the same result (in new memory)
Change:
std::make_shared<autograd::Eval>()->replaceSubgraph(inputs, outputs); } <del> Node* recordTraceHelper(std::string op, // TODO: make this a Symbol at::ArrayRef<Variable> inputs, at::ArrayRef<Variable> outputs) { </del> <ins> Node* recordTrace(std::string op, // TODO: make this a Symbol at::ArrayRef<Variable> inputs, at::ArrayRef<Variable> outputs) { </ins> auto state = getTracingState(inputs); auto& graph = state->graph; // TODO: Technically, we could reduce the scope of the lock, but since we

Discussion:
when executing:
I'm seeing this as well as of last night. Its preceded by many warnings about : But the first error is the one about . I went back, and I get a good install as late as commit
Change:
add_docstr(torch._C.triu, """ <del> triu(input, k=0, out=None) -> Tensor </del> <ins> triu(input, diagonal=0, out=None) -> Tensor </ins> Returns the upper triangular part of the matrix (2D Tensor) :attr:`input`, the other elements of the result Tensor :attr:`out` are set to 0.

Discussion:
The states that the argument controls which diagonal to consider. However, this argument is actually named in pytorch 0.2.0.
Change:
tensor_view = tensor.view(5, 3) tensor_view.fill_(random.uniform(0, 1)) self.assertEqual((tensor_view-tensor).abs().max(), 0) <ins> self.assertEqual(empty.view_as(empty), empty) self.assertEqual(empty.view(0), empty) </ins> def test_expand(self): result = torch.Tensor()

Discussion:
Please check and fix all the PyTorch web links format from to , like in the official site.
closed via
Change:
// most fields THPFunction* self = (THPFunction*)obj; new (&self->cdata) torch::autograd::PyFunction(obj); <ins> new (&self->cdata_ptr) std::weak_ptr<torch::autograd::PyFunction>(); </ins> self->cdata.num_inputs = -1; self->cdata.is_stochastic = PyObject_IsInstance(obj, THPStochasticFunctionClass); return obj;

Discussion:
there is a bug. the code is from The code for installing cuda10.2 should be corrected to The code for installing cuda10.2 should be corrected to cc
Thank you for reporting the typo

This is how I implement the decoder of a sequence to sequence model I'm not sure about the new autograd mechanics but this worked in the previous version. If I didn't make the unrolling codes a function it will work. It will also work with CPU. I compiled from source with Python 2.7, CUDA 8.0 and Cudnn 6
Thanks for the minimal reproduction! It broke OpenNMT and a big internal model but we weren't looking forward to isolating what parts of those had the issue.
That's great, I'll take a look today. Thanks!
I've pushed a fix to .
How does the inference work in this case? I mean, once you train it how do you test it in the test phase?
Change:
self.assertEqual(input_var.requires_grad, broadcasted_var.requires_grad) @unittest.skipIf(not TEST_MULTIGPU, "multi-GPU not supported") <ins> def test_broadcast_no_grad(self): x = torch.randn(1, 2, dtype=torch.cuda.float32, requires_grad=True) with torch.no_grad(): broadcasted = Broadcast.apply((0, 1), x) self.assertTrue(x.requires_grad) for output in broadcasted: self.assertFalse(output.requires_grad) @unittest.skipIf(not TEST_MULTIGPU, "multi-GPU not supported") </ins> def test_replicate(self): module = nn.Linear(10, 5).float().cuda() input = Variable(torch.randn(2, 10).float().cuda())

Discussion:
OS: ubuntu 16.04 PyTorch version: 0.4.0a0+ How you installed PyTorch (conda, pip, source): source Python version: 3.6.3 CUDA/cuDNN version: v9.1, cudnn-9.1-linux-x64-v7 GPU models and configuration: 4x Tesla M60 GCC version (if compiling from source): 5.4.0 When using context () on a model with dataparallel operation (same with ) on 1 GPU the flag requires_grad does not come back to True. If only one GPU is used then everything works fine. Script: On 2 GPUs, this outputs which is wrong in the last line. On 1 GPU (set ) the output is correct.
is looking into it
Change:
int batch_size = THTensor_(size)(input, 0); THTensor_(resize1d)(output, batch_size); <ins> int invalid_target = -1;  // We cannot throw an exception inside omp parallel </ins> int i; #pragma omp parallel for private(i) for (i = 0; i < batch_size; i++) { int cur_target = THTensor_fastGet1d(target, i) - TH_INDEX_BASE; <del> if (cur_target == ignore_index) { THTensor_fastSet1d(output, i, 0.0f); continue; </del> <ins> if (cur_target >= 0 && cur_target < n_classes) { if (cur_target == ignore_index) { THTensor_fastSet1d(output, i, 0.0f); continue; } real cur_weight = weights ? THTensor_fastGet1d(weights, cur_target) : 1.0f; THTensor_fastSet1d(output, i, -THTensor_fastGet2d(input, i, cur_target) * cur_weight); } else { THAtomicCompareAndSwap(&invalid_target, -1, cur_target); </ins> } <del> real cur_weight = weights ? THTensor_fastGet1d(weights, cur_target) : 1.0f; THTensor_fastSet1d(output, i, -THTensor_fastGet2d(input, i, cur_target) * cur_weight); </del> <ins> } if (invalid_target >= 0) { THError("Target %d out of bounds", invalid_target); </ins> } return;

Discussion:
Claim: When I use an invalid label with F.crossentropy with reduce=False, I don't receive an error message. I do receive an error message when reduce=True. Code: Result: No error message received. Expected error message: Context: I was using , accidentally used an invalid label, and got extremely large losses and extremely small losses, and inconsistently so. Environment: OS: Ubuntu 16.04 on Docker and host PyTorch version: 0.3.0.post4 How you installed PyTorch: conda Python version: 3.6.3
Change:
limits = new_limits # Find the first and the last bin defining the support of the histogram: <del> cum_counts = np.cumsum(np.greater(counts, 0, dtype=np.int32)) </del> <ins> cum_counts = np.cumsum(np.greater(counts, 0)) </ins> start, end = np.searchsorted(cum_counts, [0, cum_counts[-1] - 1], side="right") start = int(start) end = int(end) + 1

Discussion:
The use of in is deprecated a while ago, and the deprecation expired in Numpy 1.24+: Source: Minimal example: Error message with Numpy 1.24+: This error does not occur in Numpy 1.23.5. Numpy should have print a deprecation warning about this: but I think PyTorch has suppressed the warning.
(related: there are a few more numpy1.24 compatibility issues at )
so how can use add_histogram function in numpy 1.24.1?
Since this issue has been resolved, I believe you can use  as before (i.e., as in NumPy < 1.24).
I was using Python 3.9, torch 1.9.1 (with CUDA 11.1) and numpy 1.24.2 to run something today and still ran into this exact problem, don't really know about pytorch 2.0+ though. Thank you for the post though, otherwise might have cost me several more hours to figure this out.
I have this issue during yolo v7 training - cum_counts = (np.greater(counts, 0, dtype=np.int32)) TypeError: No loop matching the specified signature and casting was found for ufunc greater.
Hi, I'm still getting this error with numpy 1.24 and tensorboard 2.15.
same problem with numpy==1.24.3 torch==1.13.1+cu117 tensorboard==2.13.0
Same problem with numpy == 1.24.0 tensorboard == 2.14.0 torch == 1.13.1
Same problem with numpy == 1.26.4 tensorboard == 2.16.2 torch == 1.13.1+cu116
Using can solve this issue
same problem when I used with numpy==1.26.4 torch==1.13.0 tensorboard==2.16.2
go to the path of the in your or your main env like this -C:Userslegionanaconda3envsganLibsite-packagestorchutilstensorboard open and change following line to
Change:
if (threadIdx.x == 0) { // Make sure the sum of our distribution didn't overflow assert(!isinf(sum)); <ins> assert(THCNumerics<AccT>::gt(sum, accZero)); </ins> asmem[0] = sum; smem[0] = sampled[curDist];

Discussion:
Looking at the values returned from tn.multinomial(1) and counting them, it seems like it returns the values of the multinomial distribution taken from t, i.e. that it has ignored the signs of the values. However, it seem like always returns a tensor filled with zeros.
looking into this
Change:
<del> FROM nvidia/cuda:9.1-cudnn7-devel-ubuntu16.04 RUN echo "deb http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64 /" > /etc/apt/sources.list.d/nvidia-ml.list </del> <ins> FROM nvidia/cuda:9.0-cudnn7-devel-ubuntu16.04 </ins> RUN apt-get update && apt-get install -y --no-install-recommends  build-essential 

Discussion:
OS: MacOS Sierra PyTorch version: How you installed PyTorch (conda, pip, source): Just pulled the dockerfile I pulled the dockerfile just now and tried to build it using a tool that we plan on open-sourcing at some point. The command translates to a simple .
CC
It seems to work now.
Change:
# for-loop bodies don't define scopes, so we have to save the variables # we want to close over in some way def do_test(self, name=name, self_size=self_size, args=new_args, test_name=test_name, <del> test_output_indices=test_output_indices): </del> <ins> output_process_fn=output_process_fn): </ins> def check(name): is_magic_method = name[:2] == '__' and name[-2:] == '__' is_inplace = name[-1] == "_" and not is_magic_method

Discussion:
I am having problem in running the backward propagation with svd. In particular, it seems working when the first dimension of the matrix x (below) is greater than the second. But I get the following error if the first is smaller than the second. PyTorch version: 2018.04.20 Is debug build: No CUDA used to build PyTorch: 8.0.61 OS: Ubuntu 16.04.4 LTS GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 CMake version: Could not collect Python version: 3.5 Is CUDA available: No CUDA runtime version: No CUDA GPU models and configuration: No CUDA Nvidia driver version: No CUDA cuDNN version: No CUDA Versions of relevant libraries: [pip3] numpy (1.13.3) [pip3] torch (2018.4.20) [pip3] torchvision (0.2.0) [conda] pytorch 0.3.1 py35cuda8.0.61cudnn7.0.52 pytorch [conda] pytorch-nightly 2018.04.20 py35cuda8.0.61cudnn7.1.21 pytorch [conda] torchvision 0.2.0 py35heaa392f_1 pytorch
Repro'ed on 0.4 branch. works btw.
What does "Repro'ed" means? Should I install from that branch? I need to compute the kernel of a matrix, therefore I need the last columns of the matrix V (which is why I have to pass ).
It's a shorthand for "reproduced", which means that the bug still hasn't been fixed. Sorry, we'll try to get to it soon.
Change:
but they must be :ref:`broadcastable <broadcasting-semantics>`. .. math:: <del> text{out}_i = min(text{tensor}_i, text{other}_i) """ + r""" </del> <ins> text{{out}}_i = min(text{{tensor}}_i, text{{other}}_i) </ins> .. note:: When the shapes do not match, the shape of the returned output tensor follows the :ref:`broadcasting rules <broadcasting-semantics>`.

Discussion:
with , both max and min are broken <img width="880" alt="Screen Shot 2020-03-23 at 1 10 07 PM" src="https://user-"on , min is broken <img width="875" alt="Screen Shot 2020-03-23 at 1 13 23 PM" src="https://user-"
Fixed by and
Change:
endforeach(flag_var) endif() <del> set (CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -fno-omit-frame-pointer") set (CMAKE_LINKER_FLAGS_DEBUG "${CMAKE_STATIC_LINKER_FLAGS_DEBUG} -fno-omit-frame-pointer") </del> <ins> set (CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -fno-omit-frame-pointer -O0") set (CMAKE_LINKER_FLAGS_DEBUG "${CMAKE_STATIC_LINKER_FLAGS_DEBUG} -fno-omit-frame-pointer -O0") </ins> if (USE_ASAN) set (CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -fsanitize=address") set (CMAKE_LINKER_FLAGS_DEBUG "${CMAKE_STATIC_LINKER_FLAGS_DEBUG} -fsanitize=address")

Discussion:
I think this may have been intentional, so I want to float this first before I apply the change. Historically, PyTorch did DEBUG builds with , because it prevents inlining, which means that you can inspect variables in all stack frames. However, in master, DEBUG mode appears to be compiled with optimizations, which means that any inlined functions are non-inspectable. The following patch will turn off optimizations when compiled in DEBUG mode. Is there any reason not to apply this?
Change:
// See Note [Numpy memory management] &THNumpyArrayAllocator, new NumpyArrayAllocator(numpy_array))); <ins> THStorage_(clearFlag)(storage.get(), TH_STORAGE_RESIZABLE); </ins> result = THTensor_(newWithStorage)(LIBRARY_STATE storage, 0, sizes, strides); } else

Discussion:
When both PyTorch and Pandas are imported, calling results in a segfault. The issue was originally reported by It looks like calling returns a null pointer here: Examining in GDB: I suspect it's both PyTorch and Pandas doing threading magic and somehow getting in each other's way. I haven't yet been able to provide a minimal example that triggers this behaviour and does not use my package; it's likely that I'm doing something odd that causes this. More documentation here: Possibly related issues: and Python version: 3.6 PyTorch version: v0.2.0 Pandas version: 0.20.1
Change:
def parameters(self, memo=None): if memo is None: memo = set() <del> super(Container, self).parameters(memo) </del> <ins> for p in super(Container, self).parameters(memo): yield p </ins> for module in self.children(): for p in module.parameters(memo): yield p

Discussion:
While using torch on a Macbook pro with M1 ship, using python 3.8.10 and torch 1.12.1, I came across a weird bug that happened because I forgot to put a tensor on MPS. The reproducible code is the following: As you can see, if is on MPS and on CPU, the line wrongly replaces the index in place of the index . I would expect to have an error saying both tensors should be on the same device instead. I get an error: cc
The error seems to happen somewhere within Also I'm getting a warning "The operator 'aten::nonzero' is not currently supported on the MPS backend and will fall back to run on the CPU. This may have performance implications" when printing tensors on MPS.
Hi - thanks for reporting this issue. This is actually an allowed usecase (copying CPU tensors into MPS tensors / MPS -CPU / MPS -MPS): The issue is on MPS side when copying a CPU tensor into a strided MPS tensor. I'll take a look.
Change:
HingeEmbeddingLoss, MarginRankingLoss backend.register_function('Linear', Linear) <del> backend.register_function('BatchNorm', BatchNorm) </del> backend.register_function('ConvNd', ConvNd) backend.register_function('RNN', RNN) backend.register_function('RNNTanhCell', RNNTanhCell)

Discussion:
As Arthur Szlam reports, fb-internal cudnn is still lagging behind, and giving batch-size 1024 with batchnorm is raising an error from cudnn. Need to check for compile-time version and disable this codepath
The 1024 limitation was removed in 5.1.10
Change:
import torch from torch.distributions import constraints from torch.distributions.distribution import Distribution <del> from torch.distributions.utils import probs_to_logits, logits_to_probs, log_sum_exp, lazy_property, broadcast_all </del> <ins> from torch.distributions.utils import probs_to_logits, logits_to_probs, _log_sum_exp, lazy_property, broadcast_all </ins> class Categorical(Distribution):

Discussion:
Leaving this issue here for higher visibility. Originally reported by here: Also mentioned by Piggybacking off of this issue, has strange behavior (reported here: )
I have fixed this locally. Could I send a PR?
Yes, go ahead!
Change:
def fn(*inputs): output = getattr(inputs[0], name)(*inputs[1:]) <del> if test_output_indices is None: return output else: return tuple(output[i] for i in test_output_indices) </del> <ins> return output_process_fn(output) </ins> if not is_inplace and name not in EXCLUDE_GRADCHECK: run_grad_and_gradgrad_checks(self, name, test_name, fn,

Discussion:
I am having problem in running the backward propagation with svd. In particular, it seems working when the first dimension of the matrix x (below) is greater than the second. But I get the following error if the first is smaller than the second. PyTorch version: 2018.04.20 Is debug build: No CUDA used to build PyTorch: 8.0.61 OS: Ubuntu 16.04.4 LTS GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 CMake version: Could not collect Python version: 3.5 Is CUDA available: No CUDA runtime version: No CUDA GPU models and configuration: No CUDA Nvidia driver version: No CUDA cuDNN version: No CUDA Versions of relevant libraries: [pip3] numpy (1.13.3) [pip3] torch (2018.4.20) [pip3] torchvision (0.2.0) [conda] pytorch 0.3.1 py35cuda8.0.61cudnn7.0.52 pytorch [conda] pytorch-nightly 2018.04.20 py35cuda8.0.61cudnn7.1.21 pytorch [conda] torchvision 0.2.0 py35heaa392f_1 pytorch
Repro'ed on 0.4 branch. works btw.
What does "Repro'ed" means? Should I install from that branch? I need to compute the kernel of a matrix, therefore I need the last columns of the matrix V (which is why I have to pass ).
It's a shorthand for "reproduced", which means that the bug still hasn't been fixed. Sorry, we'll try to get to it soon.
Change:
-DCMAKE_SHARED_LINKER_FLAGS="$LDFLAGS"  -DCMAKE_INSTALL_LIBDIR="$INSTALL_DIR/lib"  -DCUDA_NVCC_FLAGS="$C_FLAGS"  <del> -DCWRAP_FILES_BASE="$BASE_DIR/torch"  </del> <ins> -Dcwrap_files="$CWRAP_FILES"  </ins> -DTH_INCLUDE_PATH="$INSTALL_DIR/include"  -DTH_LIB_PATH="$INSTALL_DIR/lib"  -DTH_LIBRARIES="$INSTALL_DIR/lib/libTH$LD_POSTFIX" 

Discussion:
There are previous issues addressing this, but I want to write this up with a plan in a new issue. We should make differentiable instead of having a wrapper class. The Variable class adds conceptual overhead without sufficient benefit. It makes it harder for new users to adopt PyTorch by adding a second API which is similar but not exactly the same as Tensor. There are few prerequisites: on s need to be at least as fast as operations on () need scalar support at the Python level () The constructor will be deprecated and become a no-op. The accessor will be deprecated and return a view that has requiresgrad=False. The constructors will take an optional keyword argument There will be a number of internal changes, but we shouldn't break most existing code: performance reasons, we will no longer build up the graph of operations by default when unless the operation uses stochastic variables. We'll use a context manager or global variable to enable graph creation. new differentiable Tensor will need support all the operations currently on Tensor, including in-place and operations. -place operations on leaf Variables which requiregrad are not currently supported. In the new API, the tensor's value will be updated and it will remain a leaf node. (This is important for optimizers, which update leaf Variables in-place). There are probably a few other subtle differences between Tensor and Variable APIs. I'll update this issue as we address them. See also: 23/738 # Greg will be fixing the tests, Tensor vs. Variable requiresgrad 2/1837 # Just testbroadcast from TestTorch OK OK # error at top level OK OK OK OK OK OK OK OK 10/120 # missing sspaddmm, dsmm/mm, addmm 1/257 TestTorch.testbroadcast 2OK
Properly handling in-place might not be as easy as it seems. Sometimes you actually need to move the Variable in the graph. Consider this case:
One of the cases of differences between variable and tensor is computation of var/std:
IMO Variable gives user a clear difference between graph node and data. .data attribute and Variable constructor provide convenient and explicit ways to take control of the graph (or graphs).
This is done. (Still some clean-up to do)
Is the conda PyTorch out of date? I'm getting the following error: .
Yes, we haven't released 0.4, which has the Variable Tensor merge, yet
Change:
z3-solver==4.12.2.0 tensorboard==2.13.0 optree==0.9.1 <ins> # NB: test_hparams_* from test_tensorboard is failing with protobuf 5.26.0 in # which the stringify metadata is wrong when escaping double quote protobuf==3.20.2 </ins>

Discussion:
Platforms: macos, mac This test was disabled because it is failing on main branch (). cc
<!-- validation-comment-start --<bodyHello there! From the DISABLED prefix in this issue title, it looks like you are attempting to disable a test in PyTorch CI. The information I have parsed is below: Test name: Platforms for which to skip the test: mac, macos Disabled by Within ~15 minutes, will be disabled in PyTorch CI for these platforms: mac, macos. Please verify that your test name looks correct, e.g., . To modify the platforms list, please include a line in the issue body, like below. The default action will disable the test for all platforms if no platforms list is specified. We currently support the following platforms: asan, dynamo, inductor, linux, mac, macos, rocm, slow, win, windows.</body<!-- validation-comment-end --
Change:
return PyLong_FromLong((int64_t) driverVersion); } <ins> PyObject * THCPModule_getCompiledVersion(PyObject *self) { return PyLong_FromLong((long) CUDA_VERSION); } </ins> PyObject * THCPModule_getRNGState(PyObject *_unused) { HANDLE_TH_ERRORS

Discussion:
There's quite a few user-reports of installing the CUDA 7.5 binary of pytorch and then running on GTX 1080 / 1070, and there will be reports of installing 7.5 / 8.0 binaries and running on V100. Raising a user-warning in such situations and letting them know will fix the problem.
The version of CUDA used to compile the binary should be available thanks to this PR:
I think is a more robust way of getting the CUDA used to compile PyTorch, as currently if there are multiple CUDA installed in the machine, the heuristics used in might not match those used by CMake FindCUDA. I think that we should modify the to return the cuda version used during compilation. I have the impression that it will not require changing much because doesn't require cuda to execute. What do you think?