jiebi/CodeConvo huggingface dataset:
Code change:
The default value of `model_dir` is ``$TORCH_HOME/models`` where ``$TORCH_HOME`` defaults to ``~/.torch``. The default directory can be <del> overriden with the ``$TORCH_MODEL_ZOO`` environement variable. </del> <ins> overriden with the ``$TORCH_MODEL_ZOO`` environment variable. </ins> Args: url (string): URL of the object to download

Related discussion:
text: When converting a tensor to dlpack, the strides of single-value tensors () should be normalized to . This is the fallout conclusion from . Here is a demo of what is wrong: Collecting environment information... PyTorch version: 1.10.0a0+git2aa19f3 Is debug build: False CUDA used to build PyTorch: Could not collect ROCM used to build PyTorch: N/A OS: Ubuntu 20.04.4 LTS (x8664) GCC version: (GCC) 9.4.0 Clang version: 12.0.1 ( ) CMake version: version 3.21.1 Libc version: glibc-2.31 Python version: 3.9.6 (default, Jul 11 2021, 03:39:48) [GCC 9.3.0] (64-bit runtime) Python platform: Linux-5.4.0-122-generic-x8664-with-glibc2.31 Is CUDA available: False CUDA runtime version: 11.2.152 GPU models and configuration: GPU 0: NVIDIA GeForce GTX 1660 SUPER Nvidia driver version: 470.141.03 cuDNN version: /usr/lib/x8664-linux- HIP runtime version: N/A MIOpen runtime version: N/A Is XNNPACK available: False Versions of relevant libraries: [pip3] mypy==0.812 [pip3] mypy-extensions==0.4.3 [pip3] numpy==1.21.2 [pip3] torch==1.10.0a0+git2aa19f3 [pip3] torchaudio==0.10.0a0+ [conda] magma-cuda112 2.5.2 1 pytorch [conda] mkl 2021.3.0 h726a3e6557 conda-forge [conda] mkl-include 2021.3.0 h726a3e6557 conda-forge [conda] numpy 1.21.2 py39hdbf815f0 conda-forge [conda] torch 1.12.0a0+git1f29b31 dev0 <develop[conda] torchaudio 0.10.0a0+ pypi0 pypi
does this capture the problem?
Code change:
# automatically filled by the dynamic loader. import os as _dl_flags <ins> # if we have numpy, it *must* be imported before the call to setdlopenflags() # or there is risk that later c modules will segfault when importing numpy try: import numpy as np except: pass </ins> # first check if the os package has the required flags if not hasattr(_dl_flags, 'RTLD_GLOBAL') or not hasattr(_dl_flags, 'RTLD_NOW'): try:

Related discussion:
text: As shown in below. Is this something expected? This seems most closely related to , but it is closed without any specific solutions.
Importing after also causes segv:
Happens with sklearn as well. . Reversing the order of imports seems to fix things for me though.
Can you reinstall sklearn or scipy? I think I've seen a case where that helped.
I've tried reinstalling things. Doesn't fix it. Could it be because scikit seems to build using openblas and not MKL?
I don't think so, it's a problem with numpy initialization. AFAIK the same was happening with TF, not sure if it's solved now.
same. run import scipy/ import sklearn after torch will cause error. but it's all right if load these two first.
Code change:
} __global__ void createBatchGemmBuffer3(const real** buffer1, const real ** buffer2, const real ** buffer3, real* data1, <del> real * data2, real * data3, long stride1, long stride2, long stride3, long num_batches) { const long idx = blockIdx.x * blockDim.x + threadIdx.x; </del> <ins> real * data2, real * data3, int64_t stride1, int64_t stride2, int64_t stride3, int64_t num_batches) { const int64_t idx = blockIdx.x * blockDim.x + threadIdx.x; </ins> if (idx < num_batches) { buffer1[idx] = data1 + idx * stride1; buffer2[idx] = data2 + idx * stride2;

Related discussion:
text: I am trying to build PyTorch from source on macOS High Sierra Version 10.13.3 with an NVIDIA GeForce GT 750M. I'm following the instructions at prints prints prints prints When I enter , as per the instructions, I receive the following error message:
Carlos - this may be minor but I eventually found problems with ananconda for this an other CUDA-enabled builds. Couple of things that may help: a - ensure that your activated conda environment is env actually the env you want. I was checking the path with and it was showing the incorrect path to python and pip half of the time. I ended up creating new conda environments (switching to miniconda by the way) and passing the python=3.6 flag to ensure a full environment was deployed. Maybe somehow reverts back to an incorrect path b - in your activated environment, c - make sure a recent OS update didn't bump your CommandLineTools to v 9.0 (I found 8.0 was the only that worked, not 8.1) I built this on 10.12, CUDA 9, cudNN 7.0.
Could you see if can build for you? I don't have CUDA 9 on mac, so it'd be great if you can let me know if it works or not. :)
I'm no longer getting an error at installation, but still returns for some reason.
Code change:
), dict( module_name='Conv3d', <ins> constructor_args=(3, 4, (2, 3, 4), 1, 0, 1, 1, False), input_size=(2, 3, 3, 4, 5), cudnn=True, desc='no_bias' ), dict( module_name='Conv3d', </ins> constructor_args=(3, 4, 2, 2), input_size=(2, 3, 5, 5, 5), cudnn=True,

Related discussion:
text: Hi, I'm trying to create a module that, given an n-dimensional returns an n+2 dimensional arranged correspondingly from slices of a 3-dimensional . In theory this can be accomplished with plus a reshape, except that I need the matrix corresponding to the to be the identity matrix instead of a zero matrix. This could be accomplished by allowing the user to specify what default vector is returned corresponding to . Would that be possible? Thanks, Shawn
this is certainly possible to implement, we have to change our C backends slightly to accommodate this. will put it on a list of things to do
Closing this because it is possible to specify a custom value to use for the vector at . An example of doing so is in the .
Code change:
y_cpu_nhwc.data<float>(), y_cpu_nhwc.size()); int max_diff_idx = -1; (y_cpu_vec - y_gpu_vec).cwiseAbs().maxCoeff(&max_diff_idx); <del> EXPECT_FLOAT_EQ(y_cpu_vec[max_diff_idx], y_gpu_vec[max_diff_idx]); </del> <ins> EXPECT_NEAR(y_cpu_vec[max_diff_idx], y_gpu_vec[max_diff_idx], 1e-1); </ins> max_diff_idx = -1; (y_cpu_vec - y_cpu_nhwc_vec).cwiseAbs().maxCoeff(&max_diff_idx);

Related discussion:
text: CC If no one has time to investigate we should disable this test.
Sounds like a float-point precision issue? Maybe replacing EXPECTFLOATEQ with EXPECT_NEAR can help.
Code change:
import sys import os <del> # TODO: make this more robust WITH_CUDA = os.path.exists('/Developer/NVIDIA/CUDA-7.5/include') or os.path.exists('/usr/local/cuda/include') </del> <ins> CUDA_HOME = os.getenv('CUDA_HOME', '/usr/local/cuda') WITH_CUDA = os.path.exists(CUDA_HOME) </ins> WITH_CUDNN = WITH_CUDA DEBUG = False

Related discussion:
text: right now if is not in LDLIBRARYPATH, though it was found at compile-time, these lines will fail: Avoid this, by idk doing something... I think i have a few good ideas.
We need to add /usr/local/cuda/lib64 (or that's mac dir) to the rpath of _C
the problem i dont think is to add it to rpath, it's because we
Right. We could try with the default cuda installation path if it's not found from LDLIBRARYPATH.
Actually, do we really want to use the compile path? We should try to detect it at runtime, right? Or are we going to ship CUDA libs in our binaries?
we first load using runtime paths, and if it fails, fallback to path known at compile time.
I fear this is going to get brittle, especially if you are shipping CUDA libs. e.g. a user swaps HW and picks up a new driver and/or toolkit. As says, I'm pretty sure you want to use the runtime paths. Similarly, CUDA 7.5 won't run on Pascal and we have a cudnn built for 7.5 that will not do the right thing on CUDA 8, so those need to go together. If a user installs the CUDA bits via deb/rpm the right things are supposed to happen. If you push bits, we need to figure out how to make sure things line up. But this is perhaps a much larger packaging conversation.
Yeah, I've implemented it to first try loading it without any path and then try and . I'll push the commit tomorrow.
Code change:
dims = THCudaLongTensor_nDimension(state, indices); THArgCheck(dims <= MAX_CUTORCH_DIMS, 4, CUTORCH_DIM_WARNING); <del> ptrdiff_t numIndices = THCudaLongTensor_nElement(state, indices); int srcDims = THCTensor_(nDimension)(state, src); cudaStream_t stream = THCState_getCurrentStream(state); THArgCheck(THCudaLongTensor_nDimension(state, indices) == 1, 3, "expecting vector of indices"); THArgCheck(dim < srcDims, 4, "Indexing dim is out of bounds"); THArgCheck(srcDims > 0, 2, "Source tensor is empty"); THArgCheck(numIndices == src->size[dim], 4, "length of src.size[dim] is not equal to length of indices"); int indContig = THCudaLongTensor_isContiguous(state, indices); </del> // The `src` is partitioned into two parts: // -the size of each slice we are indexing, which is the // total size of the tensor ignoring dimension `dim`; // -the number of indices we are choosing, which is the total size // of the tensor `indices`. <ins> ptrdiff_t sliceSize = THCTensor_(getSliceSize)(state, dst, dim, indices, src); </ins> ptrdiff_t srcTotalSize = THCTensor_(nElement)(state, src); int64_t dstAddDimSize = THCTensor_(size)(state, dst, dim); <del> ptrdiff_t sliceSize = srcTotalSize / numIndices; </del> <ins> ptrdiff_t numIndices = THCudaLongTensor_nElement(state, indices); cudaStream_t stream = THCState_getCurrentStream(state); int indContig = THCudaLongTensor_isContiguous(state, indices); </ins> int mpc = THCState_getCurrentDeviceProperties(state)->multiProcessorCount;

Related discussion:
text: A = (5, 4) B = (0, 9).view(3, 3) C = (0, 15).view(3, 5) idxs = torch.LongTensor([0, 2, 4]) A.indexadd(0, idxs, B) # RuntimeError: inconsistent tensor size, expected r [4], t [4] and src [3] to have the same number of elements, but got 4, 4 and 3 elements respectively at (...)/aten/src/TH/generic/THTensorMath.c:1008 A.indexadd(0, idxs, C) # RuntimeError: inconsistent tensor size, expected r [4], t [4] and src [5] to have the same number of elements, but got 4, 4 and 5 elements respectively at (...)/aten/src/TH/generic/THTensorMath.c:1008 So far so good. But if we use CUDA... A = (5, 4).cuda() B = (0, 9).view(3, 3).cuda() C = (0, 15).view(3, 5).cuda() idxs = torch.LongTensor([0, 2, 4]).cuda() A.indexadd(0, idxs, B) print(A) # 0 1 2 0 # 0 0 0 0 # 3 4 5 0 # 0 0 0 0 # 6 7 8 0 # [ of size 5x4 (GPU 0)] OK, this looks wrong... A.zero() A.indexadd(0, idxs, C) print(A) # 0 1 2 3 # 4 0 0 0 # 5 6 7 8 # 9 0 0 0 # 10 11 12 13 # [ of size 5x4 (GPU 0)] Now this looks definitely wrong. Increase C's dimension to something like (3, 500), and it overwrites other tensors or triggers asserts. Same thing happens with indexcopy_.
I'll take it if no one's looking at it yet.
Code change:
if (len(self.needs_input_grad) > 1 and self.needs_input_grad[2]) or self.use_cudnn: grad_bias = bias.new(bias.size()).zero_() <del> if self.use_cudnn: </del> <ins> if self.use_cudnn and self.training: # cudnn does not support backward in evaluate mode </ins> torch._C._cudnn_batch_norm_backward( input, grad_output, grad_input, grad_weight, grad_bias, weight,

Related discussion:
text: Gradient changes, though inputs, outputs, and internal state stay same. Even when I run forward in evaluation mode and backward in training I still get the gradient changed. It works fine with cudnn disabled.
cudnn does not support backprop in evaluate mode, we should add a fallback to nn for this case
Hi, thanks for the report. It turns out that the cudnn library (at least up to v5) doesn't support backward in evaluation mode. We should assert that in the code.
Hi, thanks for the responses. What about another case when you run forward in evaluation mode and backward in training? It produces same issue, but seems to work fine in lua torch. Here is the code with changes: If there is another way to implement virtual batch normalization - please share.
this should be fixed in master. It will be incorporated into the next release that will have binaries late today.
Out of curiosity, why do backward in evaluate mode? (Or rather, what is virtual batch normalization?)
a lot of research involves this. For example neural style transfer, or etc.
Code change:
array = np.array([1, 2, 3, 4], dtype=dtype) self.assertEqual(torch.from_numpy(array), torch.Tensor([1, 2, 3, 4])) <ins> # check storage offset x = np.linspace(1, 125, 125) x.shape = (5, 5, 5) x = x[1] expected = torch.range(1, 125).view(5, 5, 5)[1] self.assertEqual(torch.from_numpy(x), expected) # check noncontiguous x = np.linspace(1, 25, 25) x.shape = (5, 5) expected = torch.range(1, 25).view(5, 5).t() self.assertEqual(torch.from_numpy(x.T), expected) # check noncontiguous with holes x = np.linspace(1, 125, 125) x.shape = (5, 5, 5) x = x[:, 1] expected = torch.range(1, 125).view(5, 5, 5)[:, 1] self.assertEqual(torch.from_numpy(x), expected) </ins> @unittest.skipIf(not TEST_NUMPY, "Numpy not found") def test_numpy_index(self): i = np.int32([0, 1, 2])

Related discussion:
text: I followed the guide The command was working just fine until a couple days ago, but now it returns I manually checked and there is no cu100/[...] What should I do? I have CUDA 10.0 I would like to use pytorch...
You could try as a temporary workground. cc
This is most likely related to this: I've re-uploaded the stable html to re-include the cu100 binaries, and submitted a PR to make sure this doesn't happen again
Code change:
array = np.array([1, 2, 3, 4], dtype=dtype) self.assertEqual(torch.from_numpy(array), torch.Tensor([1, 2, 3, 4])) <ins> # check storage offset x = np.linspace(1, 125, 125) x.shape = (5, 5, 5) x = x[1] expected = torch.range(1, 125).view(5, 5, 5)[1] self.assertEqual(torch.from_numpy(x), expected) # check noncontiguous x = np.linspace(1, 25, 25) x.shape = (5, 5) expected = torch.range(1, 25).view(5, 5).t() self.assertEqual(torch.from_numpy(x.T), expected) # check noncontiguous with holes x = np.linspace(1, 125, 125) x.shape = (5, 5, 5) x = x[:, 1] expected = torch.range(1, 125).view(5, 5, 5)[:, 1] self.assertEqual(torch.from_numpy(x), expected) </ins> @unittest.skipIf(not TEST_NUMPY, "Numpy not found") def test_numpy_index(self): i = np.int32([0, 1, 2])

Related discussion:
text: Repro: Strides seem to be copied correctly, and the first row is also ok. All other rows are garbage.
Code change:
def load_url(url, model_dir=None, map_location=None): r"""Loads the Torch serialized object at the given URL. <del> If the object is already present in `model_dir`, it's deserialied and </del> <ins> If the object is already present in `model_dir`, it's deserialized and </ins> returned. The filename part of the URL should follow the naming convention ``filename-<sha256>.ext`` where ``<sha256>`` is the first eight or more digits of the SHA256 hash of the contents of the file. The hash is used to

Related discussion:
text: When converting a tensor to dlpack, the strides of single-value tensors () should be normalized to . This is the fallout conclusion from . Here is a demo of what is wrong: Collecting environment information... PyTorch version: 1.10.0a0+git2aa19f3 Is debug build: False CUDA used to build PyTorch: Could not collect ROCM used to build PyTorch: N/A OS: Ubuntu 20.04.4 LTS (x8664) GCC version: (GCC) 9.4.0 Clang version: 12.0.1 ( ) CMake version: version 3.21.1 Libc version: glibc-2.31 Python version: 3.9.6 (default, Jul 11 2021, 03:39:48) [GCC 9.3.0] (64-bit runtime) Python platform: Linux-5.4.0-122-generic-x8664-with-glibc2.31 Is CUDA available: False CUDA runtime version: 11.2.152 GPU models and configuration: GPU 0: NVIDIA GeForce GTX 1660 SUPER Nvidia driver version: 470.141.03 cuDNN version: /usr/lib/x8664-linux- HIP runtime version: N/A MIOpen runtime version: N/A Is XNNPACK available: False Versions of relevant libraries: [pip3] mypy==0.812 [pip3] mypy-extensions==0.4.3 [pip3] numpy==1.21.2 [pip3] torch==1.10.0a0+git2aa19f3 [pip3] torchaudio==0.10.0a0+ [conda] magma-cuda112 2.5.2 1 pytorch [conda] mkl 2021.3.0 h726a3e6557 conda-forge [conda] mkl-include 2021.3.0 h726a3e6557 conda-forge [conda] numpy 1.21.2 py39hdbf815f0 conda-forge [conda] torch 1.12.0a0+git1f29b31 dev0 <develop[conda] torchaudio 0.10.0a0+ pypi0 pypi
does this capture the problem?
Code change:
include("${CMAKE_UTILS_PATH}") torch_cuda_get_nvcc_gencode_flag(NVCC_GENCODE) <del> string (REPLACE ";" " " NVCC_GENCODE "${NVCC_GENCODE}") string (REPLACE "-gencode " "-gencode=" NVCC_GENCODE "${NVCC_GENCODE}") </del> <ins> string(REPLACE "-gencode;" "-gencode=" NVCC_GENCODE "${NVCC_GENCODE}") </ins> message(STATUS "Set NVCC_GENCODE for building NCCL: ${NVCC_GENCODE}") ADD_CUSTOM_COMMAND(

Related discussion:
text: Steps to reproduce: a CI CUDA docker image, e.g.,  PyTorch, and run Expected result: It works Actual result: Workaround: set forces only one to be passed.
CC is this the nccl+sccache problem you were seeing?
cc has the exact error. CI log doesn't actually post the error.
sccache probably doesn't allow passing multiple right now, I can look into it if needed.
I am having this same error. How can you figure out which gencode to pass? I'm trying to build from source from a Dockerfile as well, as seen in this gist:
this is super helpful, thanks.
just wanted to make sure you saw that this is pulling from a ppc64le (IBM Power system) docker image, so you'll run into different issues if trying to build on an x64 machine.
I modified it very slightly for x64: It seems to build pytorch successfully for me (on latest master I suppose).
looking into this
Code change:
const auto& dim = tensor_names[idx]; TORCH_CHECK(dim.isBasic(), "align_to: All input dims must be named. Found unnamed dim at index ", <del> dim, " of Tensor", tensor_names); </del> <ins> idx, " of Tensor", tensor_names); </ins> auto it = std::find(names.begin(), names.end(), dim); TORCH_CHECK(it != names.end(), "align_to: Cannot find dim ", dim, " from Tensor", names,

Related discussion:
text: only supports fully named inputs; all input dimensions must have a name. When passing it an unnamed input, it errors out with the following message: It should really say "Found unnamed dim at index 0 of Tensor[None, None]".
fixed in
Code change:
const auto& dim = tensor_names[idx]; TORCH_CHECK(dim.isBasic(), "align_to: All input dims must be named. Found unnamed dim at index ", <del> dim, " of Tensor", tensor_names); </del> <ins> idx, " of Tensor", tensor_names); </ins> auto it = std::find(names.begin(), names.end(), dim); TORCH_CHECK(it != names.end(), "align_to: Cannot find dim ", dim, " from Tensor", names,

Related discussion:
text: This is an older implementation that I think doesn't make any sense anymore. We should throw a NYI exception for it. The expected behavior for this should be:
fixed in
Code change:
#ifdef NUMPY_TYPE_ENUM THTensor* THPTensor_(fromNumpy)(PyObject *numpy_array) { PyArrayObject *array = (PyArrayObject*)numpy_array; <del> THStoragePtr storage = THStorage_(newWithDataAndAllocator)( (real*)PyArray_DATA(array), PyArray_NBYTES(array) / sizeof(real), &THNumpyArrayAllocator, new NumpyArrayAllocator(numpy_array)); </del> // Numpy and Torch disagree on empty tensors. In Torch, an empty // tensor is a tensor with zero dimensions. In Numpy, an empty tensor

Related discussion:
text: I followed the guide The command was working just fine until a couple days ago, but now it returns I manually checked and there is no cu100/[...] What should I do? I have CUDA 10.0 I would like to use pytorch...
You could try as a temporary workground. cc
This is most likely related to this: I've re-uploaded the stable html to re-include the cu100 binaries, and submitted a PR to make sure this doesn't happen again
Code change:
#ifdef NUMPY_TYPE_ENUM THTensor* THPTensor_(fromNumpy)(PyObject *numpy_array) { PyArrayObject *array = (PyArrayObject*)numpy_array; <del> THStoragePtr storage = THStorage_(newWithDataAndAllocator)( (real*)PyArray_DATA(array), PyArray_NBYTES(array) / sizeof(real), &THNumpyArrayAllocator, new NumpyArrayAllocator(numpy_array)); </del> // Numpy and Torch disagree on empty tensors. In Torch, an empty // tensor is a tensor with zero dimensions. In Numpy, an empty tensor

Related discussion:
text: Repro: Strides seem to be copied correctly, and the first row is also ok. All other rows are garbage.
Code change:
return PyObject_CallFunctionObjArgs(THPLongTensorClass, array, NULL); } else if (type == NPY_INT32) { return PyObject_CallFunctionObjArgs(THPIntTensorClass, array, NULL); <ins> } else if (type == NPY_INT16) { return PyObject_CallFunctionObjArgs(THPShortTensorClass, array, NULL); </ins> } else if (type == NPY_UINT8) { return PyObject_CallFunctionObjArgs(THPByteTensorClass, array, NULL); }

Related discussion:
text: Should we add a ShortTensor Type, or just convert int16 to IntTensor ?
We already have a type available in pytorch. We have to enable conversion in the relevant function here:
Auto travis-ci failed at python2.7 :(
Do you have any plan to support int8 numpy conversion while you have too?
Char tensor uses char which is not guaranteed to be signed by the C standard. We'd need to change our C code to use a
I try , it complains , any idea?
The error message is quite self explanatory. PyTorch doesn't support tensors at the moment.
Code change:
THCTensor_(data)(state, result_), ldc, result_->stride[0], num_batches); } else { <del> for (long i = 0; i < num_batches; ++i) { </del> <ins> for (int64_t i = 0; i < num_batches; ++i) { </ins> THCudaBlas_Hgemm( state, transpose_batch1,

Related discussion:
text: I am trying to build PyTorch from source on macOS High Sierra Version 10.13.3 with an NVIDIA GeForce GT 750M. I'm following the instructions at prints prints prints prints When I enter , as per the instructions, I receive the following error message:
Carlos - this may be minor but I eventually found problems with ananconda for this an other CUDA-enabled builds. Couple of things that may help: a - ensure that your activated conda environment is env actually the env you want. I was checking the path with and it was showing the incorrect path to python and pip half of the time. I ended up creating new conda environments (switching to miniconda by the way) and passing the python=3.6 flag to ensure a full environment was deployed. Maybe somehow reverts back to an incorrect path b - in your activated environment, c - make sure a recent OS update didn't bump your CommandLineTools to v 9.0 (I found 8.0 was the only that worked, not 8.1) I built this on 10.12, CUDA 9, cudNN 7.0.
Could you see if can build for you? I don't have CUDA 9 on mac, so it'd be great if you can let me know if it works or not. :)
I'm no longer getting an error at installation, but still returns for some reason.
Code change:
See :func:`torch.ormqr` """) <ins> add_docstr_all('permute', r""" permute(*dims) -> Tensor Permute the dimensions of this tensor. Args: *dims (int...): The desired ordering of dimensions Example: >>> x = torch.randn(2, 3, 5) >>> x.size() torch.Size([2, 3, 5]) >>> x.permute(2, 0, 1).size() torch.Size([5, 2, 3]) """) </ins> add_docstr_all('potrf', r""" potrf(upper=True) -> Tensor

Related discussion:
text: I also found this issue when I tried to swap axes of a tensor. I googled an older doc in the source code. Hope it helps.