jiebi/CodeConvo huggingface dataset:
Code change:
tmpdir = tempfile.mkdtemp() ext_suf = '.pyd' if os.sys.platform == 'win32' else '.so' libname = cffi_wrapper_name + ext_suf <del> ffi.compile(tmpdir=tmpdir, verbose=verbose, target=libname) shutil.copy(os.path.join(tmpdir, libname), os.path.join(target_dir, libname)) </del> <ins> outfile = ffi.compile(tmpdir=tmpdir, verbose=verbose, target=libname) shutil.copy(outfile, os.path.join(target_dir, libname)) </ins> finally: shutil.rmtree(tmpdir)

Related discussion:
text: (old title) When build C extension, the error: FileNotFoundError was got. OS: Windows 10 pro PyTorch version: 0.4.0a0+ How you installed PyTorch (conda, pip, source): source Python version: 3.6.4 CUDA/cuDNN version: CUDA 9.0 GCC version (if compiling from source): msvc 14/15 (then compiling with CUDA I use VS2015, but when build extension, the program automatically use vs2017) When building c extension on Windows, I got the error: (The Chinese above means compiler success compile the library, and generated .lib and .exp) And the same error was got on a Linux Work Station. (gcc is 5.4.0)
In , copy the linked file from , for example , but in my Windows, and a Linux work station, the linked one is in: ,for example . So, there must be a bug, or error in pytorch's ffi or python's ffi. (pyhton 3.6.4, cffi 1.11.4), If it broke down because of change in cffi, I think I can create a PR. I do think this is because of the change of cffi api,
CC who enabled extension build for Windows on
Is this change documented in somewhere like Python SDK?
Code change:
for(iw = 0; iw < kW; iw++) { real val = *(ip + it*istrideT + ih*istrideH + iw*istrideW); <del> if (val > maxval) </del> <ins> if ((val > maxval) || isnan(val)) </ins> { maxval = val; maxindex = (it+istartT)*isizeH*isizeW + (ih+istartH)*isizeW + (iw+istartW);

Related discussion:
text: max pooling functions are not consistent with max functions. Below an example, every max pooling (be it 1d, 2d or 3d, adaptive or not) acts the same, on cpu or on cuda. Essentially, there are two fondamental differences : max pooling of all values is while for it's max pooling of nan and valid values is valid values, which means s get ignored, while for , as soon as there is a value, the result is . More generally, choosing explicetely how to deal with as in numpy () could be a solution, but maybe this is related to CuDNN's max pooling ? Built from latest sources (as of 05/17) PyTorch version: 0.5.0a0+ Is debug build: No CUDA used to build PyTorch: 9.1.85 OS: Ubuntu 16.04.4 LTS GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 CMake version: version 3.9.4 Python version: 3.6 Is CUDA available: Yes CUDA runtime version: 9.1.85 GPU models and configuration: GPU 0: Quadro M1000M Nvidia driver version: 390.30 cuDNN version: Probably one of the following: /usr/lib/x8664-linux- /usr/lib/x8664-linux-gnu/libcudnnstaticv7.a Versions of relevant libraries: [conda] magma-cuda91 2.3.0 1 pytorch [conda] torch 0.5.0a0+ <pip[conda] torch 0.3.1b0+ <pip[conda] torch 0.5.0a0+ <pip
Hi, . Thank you for providing the example. I'll take a look.
Hmhm. I seem to get a double free in the ...
Another question: Do we really want to provide an option to ignore NaN? My view is that "if you have nan in your net, you're screwed", so I would just return NaN there, personally.
My practical usecase is for KITTI groundtruth depthmaps and FlowMaps which are 2D sparse arrays.
Code change:
for(iw = 0; iw < kW; iw++) { real val = *(ip + it*istrideT + ih*istrideH + iw*istrideW); <del> if (val > maxval) </del> <ins> if ((val > maxval) || isnan(val)) </ins> { maxval = val; maxindex = (it+istartT)*isizeH*isizeW + (ih+istartH)*isizeW + (iw+istartW);

Related discussion:
text: For an algorithm that use FlowNet-like architecture that outputs predictions at multiple scale levels, we can either compare predictions to downscaled GT or upscaled predictions to GT. The first being obviously less computationally expensive, the ignore NaN would help downscaling such sparse 2D maps. For the moment we do something I find ugly, you can see it Essentially it zeroes the s, takes the map and construct two maps of positive and negative values which are then maxpooled and back together. I am actually open for a strict "no-nan" policy on pooling functions, but in that case better enforce it before someone writes a code that tries to benefit from maxpooling ignore s feature/bug and if you have a clever way of pooling sparse 2D tensors, I'm open to it, but I guess it's a topic for pytorch forums ;)
Personally, I think it is more sane have NaN -NaN in the pooling and offer a parametrizable (where you get to pick the values) that does or so.
So the proposed fix does NaN -NaN similar to max. I didn't try to fix gradients for the nan case. This would involve keeping the values and I don't think that is worth it (in particular because I would not expect the pooling layer to be last, and otherwise we'd probably get NaN as grad_out). If you are reasonably happy with it, I'd move it to a PR.
+1 for NaN -NaN, "abyssus abyssum invocat" We could also add an optional mask which would be a ByteTensor of the same size, specifying whether or not the considered pixel is used for the pooling, that could be used for any kind of pulling. An functionality would then be to provide the mask
I could work with that. to continue to be picky, the potential drawback is that it won't work for other pooling methods such as average pooling or median pooling. The problem here is that the max operation inherently ignores nonmax values which can be leveraged for "ignore some pixels" operation, but it has a "non universal" feel to it since it woun't be as easy for other kinds of 2D operations.
To be clear, the fix is good for me, but I figured a related discussion on selective pooling (whether to ditch NaN values or anything you want to ignore) could happen (maybe not on this issue ?)
Code change:
raise ValueError('num_workers cannot be negative; ' 'use num_workers=0 to disable multiprocessing.') <ins> if sys.platform == "win32" and self.num_workers > 0: raise ValueError('num_workers > 0 is not supported on Windows') </ins> if batch_sampler is None: if sampler is None: if shuffle:

Related discussion:
text: This issue tracks the components / tests that are not working on Windows: [ ] : currently disabled on Windows because Windows doesn't have and we need to look for substitutes [ ] : Fuser is disabled on Windows because the current implementation uses symbols from Linux-specific headers and . We will need to find alternatives for Windows. [ ] : some parts of and are disabled because Windows doesn't support opening an already opened file (see discussion at and ) [ ] : currently doesn't work with Windows ( is the porting diff) [ ] : in causes intermittent CUDA out-of-memory error on Windows [ ] , : DataLoader with multiple workers causes intermittent CUDA out-of-memory error on Windows. [x] [x] (done in ) [x] [x] - - [x] [x] [x] - For more discussions, also see: cc
The first one is solved by but I think DataLoader can be further improved when is fininshed.
Cool I will mark it as resolved :)
The has been in .
We can try to revert and , because the memory leak in the CPU side could also cause CUDA errors.
Are they fixed by I think we can revert them after we merge the PR.
I think this may be related. Since once the memory of the CPU side is low, the will fail with too.
I think it could be. For what it's worth, when I tried to inspect the CUDA OOM error, showed no process that was taking memory, but running CUDA tests on the machine would still fail.
Now that is merged into master, could you please try to revert the changes on and ?
Awesome! Just to understand it better: does fix both numworker=1 and numworker1 cases?
Yes, they are both solved.
I guess we should mark the cpp_extension test as completed, for it's now enabled in CI.
Closing this issue due to age and because its references are long out of date. For example, distributed tests now have their own jobs.
Code change:
// with the mode. struct ModeUnsignedBoolPair match = {0, false}; <del> match = reduceBlockN<struct ModeUnsignedBoolPair, MatchReduceOp<struct ModeUnsignedBoolPair>, 2> </del> <ins> match = reduceBlockWithNThreadLocalReductions<struct ModeUnsignedBoolPair, MatchReduceOp<struct ModeUnsignedBoolPair>, 2> </ins> (ubpmem, ubpp, sliceSize, MatchReduceOp<struct ModeUnsignedBoolPair>(), match); // Finally, we have the mode, and an index where it occurs. We use a single thread

Related discussion:
text: Not sure what is the reason for these errors, any suggestions? I suspect the clang version is not supported? Here is the
clang have more strict checking , pull request will fixed it.
fixed, thanks to
Code change:
} else { // transposed if (input.size(1) != weight.size(0)) { std::stringstream ss; <del> ss << "Given transposed=" << transposed << ", weight" << weight.sizes() << ", so expected input" << input.sizes() << " to have " </del> <ins> ss << "Given transposed=" << transposed << ", weight of size " << weight.sizes() << ", expected input" << input.sizes() << " to have " </ins> << weight.size(0) << " channels, but got " << input.size(1) << " channels instead"; throw std::runtime_error(ss.str());

Related discussion:
text: got the following err msg with conv2D and ver 0.4: RuntimeError: expected stride to be a single integer value or a list of 1 values to match the convolution dimensions, but got stride=[1, 1]. I was not able to see the problem so I uninstalled 0.4 and installed 0.3.1 Still got an err but this time it said that the expected input should be a 4d tensor and it got a 3D tensor. This helped understand the issue and I fixed it (just add a dimension). reinstalled 0.4 and its working (no surprise...). I think 0.4 should have the same err feedback otherwise its really hard to undrstand the problem How you installed PyTorch (conda, pip, source): conda Build command you used (if compiling from source): OS: macos PyTorch version: 0.4 Python version: 3.5 CUDA/cuDNN version: no GPU models and configuration: GCC version (if compiling from source): CMake version: Versions of any other relevant libraries:
Can you please post a small self-contained code snippet that would let us reproduce the problem?
The strange error messge was also mentioned a few times in the forum. Here is a code snippet for PyTorch :
This is because we direct all conv ops to and infer dim + throw error message there.
Hmm that's not great. We might want to pass the expected dimensionality of the convolution to the generic implementation, so that we can improve the error messages.
Thx for the responses. Just a reminder, for the same err the provided err msg in version 0.3.1 was very informative
Yes, this should definitely be fixed. Sorry about this!
The error message of seems to be a bit misleading, too. Code: Should I create a new issue or is it related to the current one?
that appears fixed on master! :D
Code change:
if (weight_dim != k) { std::stringstream ss; <del> ss << "Expected " << k << "-dimensional weight for " << k << "-dimensional input " << input.sizes() << ", but got weight of size " << weight.sizes() << " instead"; </del> <ins> ss << "Expected " << weight_dim << "-dimensional input for " << weight_dim << "-dimensional weight " << weight.sizes() << ", but got input of size " << input.sizes() << " instead"; </ins> throw std::runtime_error(ss.str()); } if (weight.size(0) < groups) {

Related discussion:
text: got the following err msg with conv2D and ver 0.4: RuntimeError: expected stride to be a single integer value or a list of 1 values to match the convolution dimensions, but got stride=[1, 1]. I was not able to see the problem so I uninstalled 0.4 and installed 0.3.1 Still got an err but this time it said that the expected input should be a 4d tensor and it got a 3D tensor. This helped understand the issue and I fixed it (just add a dimension). reinstalled 0.4 and its working (no surprise...). I think 0.4 should have the same err feedback otherwise its really hard to undrstand the problem How you installed PyTorch (conda, pip, source): conda Build command you used (if compiling from source): OS: macos PyTorch version: 0.4 Python version: 3.5 CUDA/cuDNN version: no GPU models and configuration: GCC version (if compiling from source): CMake version: Versions of any other relevant libraries:
Can you please post a small self-contained code snippet that would let us reproduce the problem?
The strange error messge was also mentioned a few times in the forum. Here is a code snippet for PyTorch :
This is because we direct all conv ops to and infer dim + throw error message there.
Hmm that's not great. We might want to pass the expected dimensionality of the convolution to the generic implementation, so that we can improve the error messages.
Thx for the responses. Just a reminder, for the same err the provided err msg in version 0.3.1 was very informative
Yes, this should definitely be fixed. Sorry about this!
The error message of seems to be a bit misleading, too. Code: Should I create a new issue or is it related to the current one?
that appears fixed on master! :D
Code change:
return Subscript(base, [build_SliceExpr(ctx, base, expr.slice)]) elif sub_type is ast.ExtSlice: return Subscript(base, build_ExtSlice(ctx, base, expr.slice)) <ins> elif sys.version_info >= (3, 9):  # In Python3.9 array indicies are not wrapped in ast.Index if sub_type is ast.Tuple: # N-dimensional indexing using Tuple: x[(i, j, k)] is equivalent to x[i, j, k] indices = [] for index_expr in expr.slice.elts: if isinstance(index_expr, ast.Slice): indices.append(build_SliceExpr(ctx, base, index_expr)) else: indices.append(build_expr(ctx, index_expr)) return Subscript(base, indices) return Subscript(base, [build_expr(ctx, expr.slice)]) </ins> else:  # Ellipsis (can only happen in Python 2) raise NotSupportedError(base.range(), "ellipsis is not supported")

Related discussion:
text: Following example: fails in Python-3.9 with cc
And the reason for that is very simple:
Another interesting offender:
Code change:
const float lr = *lr_ptr; if (!nesterov) { CUDA_1D_KERNEL_LOOP(i, N) { <del> moment_out[i] = mu * moment[i] * lr * grad[i]; </del> <ins> moment_out[i] = mu * moment[i] + lr * grad[i]; </ins> param_out[i] = param[i] - moment_out[i]; } } else {

Related discussion:
text: +69 I read: To me, it should be: The CPU code is not affected.
The two lines of code are identical?
No, one is mu times moment time lr times grad instead of mu times moment plus lr times grad.
I see. Send a PR? :)
Will do :)
Code change:
if isinstance(expr.slice.value, ast.Tuple): # N-dimensional indexing using Tuple: x[(i, j, k)] is equivalent to x[i, j, k] # XXX: Indexing using a list is **different**! It triggers advanced indexing. <del> indices = [] for index_expr in expr.slice.value.elts: indices.append(build_expr(ctx, index_expr)) </del> <ins> indices = [build_expr(ctx, index_expr) for index_expr in expr.slice.value.elts] </ins> return Subscript(base, indices) else: return Subscript(base, [build_expr(ctx, expr.slice.value)])

Related discussion:
text: Following example: fails in Python-3.9 with cc
And the reason for that is very simple:
Another interesting offender:
Code change:
'zero-dimensional.*cannot be concatenated'): torch.cat([x, y]) <del> def test_cat_empty(self): </del> <ins> @staticmethod def _test_cat_empty(self, use_cuda=False): </ins> # FIXME: this is legacy behavior and should be removed # when we support empty tensors with arbitrary sizes <del> x = torch.randn(4, 3, 32, 32) empty = torch.randn(0) </del> <ins> if use_cuda: dtype = torch.cuda.float32 else: dtype = torch.float32 x = torch.randn((4, 3, 32, 32), dtype=dtype) empty = torch.randn((0,), dtype=dtype) </ins> res1 = torch.cat([x, empty], dim=1) res2 = torch.cat([empty, x], dim=1) self.assertEqual(res1, res2) <del> conv = torch.nn.Conv2d(3, 3, kernel_size=1) </del> <ins> conv = torch.nn.Conv2d(3, 3, kernel_size=1).float() if use_cuda: conv = conv.cuda() </ins> res1 = torch.cat([conv(x), empty], dim=1) res2 = torch.cat([empty, conv(x)], dim=1) self.assertEqual(res1, res2)

Related discussion:
text: gdb points that the error might be in trying to get a size from an empty tensor: I'm using PyTorch version 0.4.0a0+
There's a check that should exclude zero-dim tensors from ( should be 0 in this case), so I'm wondering why that's not happening right now... edit: Nevermind, I was running an old build. I pulled the latest master and with being an empty tensor (with shape (0,)), cat crashes.
Related: We should probably rewrite to better handle these cases
Okay, I found the bug. The CUDA version of check doesn't check the case where the input contains all empty tensors, while the CPU version does. I'll put up a fix soon.
Code change:
{ tcntr = y*iwidth + x; real val = *(ip + tcntr); <del> if (val > maxval) </del> <ins> if ((val > maxval) || isnan(val)) </ins> { maxval = val; maxindex = tcntr;

Related discussion:
text: max pooling functions are not consistent with max functions. Below an example, every max pooling (be it 1d, 2d or 3d, adaptive or not) acts the same, on cpu or on cuda. Essentially, there are two fondamental differences : max pooling of all values is while for it's max pooling of nan and valid values is valid values, which means s get ignored, while for , as soon as there is a value, the result is . More generally, choosing explicetely how to deal with as in numpy () could be a solution, but maybe this is related to CuDNN's max pooling ? Built from latest sources (as of 05/17) PyTorch version: 0.5.0a0+ Is debug build: No CUDA used to build PyTorch: 9.1.85 OS: Ubuntu 16.04.4 LTS GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 CMake version: version 3.9.4 Python version: 3.6 Is CUDA available: Yes CUDA runtime version: 9.1.85 GPU models and configuration: GPU 0: Quadro M1000M Nvidia driver version: 390.30 cuDNN version: Probably one of the following: /usr/lib/x8664-linux- /usr/lib/x8664-linux-gnu/libcudnnstaticv7.a Versions of relevant libraries: [conda] magma-cuda91 2.3.0 1 pytorch [conda] torch 0.5.0a0+ <pip[conda] torch 0.3.1b0+ <pip[conda] torch 0.5.0a0+ <pip
Hi, . Thank you for providing the example. I'll take a look.
Hmhm. I seem to get a double free in the ...
Another question: Do we really want to provide an option to ignore NaN? My view is that "if you have nan in your net, you're screwed", so I would just return NaN there, personally.
My practical usecase is for KITTI groundtruth depthmaps and FlowMaps which are 2D sparse arrays.
Code change:
{ tcntr = y*iwidth + x; real val = *(ip + tcntr); <del> if (val > maxval) </del> <ins> if ((val > maxval) || isnan(val)) </ins> { maxval = val; maxindex = tcntr;

Related discussion:
text: For an algorithm that use FlowNet-like architecture that outputs predictions at multiple scale levels, we can either compare predictions to downscaled GT or upscaled predictions to GT. The first being obviously less computationally expensive, the ignore NaN would help downscaling such sparse 2D maps. For the moment we do something I find ugly, you can see it Essentially it zeroes the s, takes the map and construct two maps of positive and negative values which are then maxpooled and back together. I am actually open for a strict "no-nan" policy on pooling functions, but in that case better enforce it before someone writes a code that tries to benefit from maxpooling ignore s feature/bug and if you have a clever way of pooling sparse 2D tensors, I'm open to it, but I guess it's a topic for pytorch forums ;)
Personally, I think it is more sane have NaN -NaN in the pooling and offer a parametrizable (where you get to pick the values) that does or so.
So the proposed fix does NaN -NaN similar to max. I didn't try to fix gradients for the nan case. This would involve keeping the values and I don't think that is worth it (in particular because I would not expect the pooling layer to be last, and otherwise we'd probably get NaN as grad_out). If you are reasonably happy with it, I'd move it to a PR.
+1 for NaN -NaN, "abyssus abyssum invocat" We could also add an optional mask which would be a ByteTensor of the same size, specifying whether or not the considered pixel is used for the pooling, that could be used for any kind of pulling. An functionality would then be to provide the mask
I could work with that. to continue to be picky, the potential drawback is that it won't work for other pooling methods such as average pooling or median pooling. The problem here is that the max operation inherently ignores nonmax values which can be leveraged for "ignore some pixels" operation, but it has a "non universal" feel to it since it woun't be as easy for other kinds of 2D operations.
To be clear, the fix is good for me, but I figured a related discussion on selective pooling (whether to ditch NaN values or anything you want to ignore) could happen (maybe not on this issue ?)
Code change:
t2 = torch.from_numpy(t.numpy().transpose()) self.assertEqual(t1, t2) <ins> def test_inplace_division(self): t = torch.rand(5, 5) id_before = id(t) t /= 2 id_after = id(t) self.assertEqual(id_before, id_after) </ins> # Functions to test negative dimension wrapping METHOD = 1 INPLACE_METHOD = 2

Related discussion:
text: This is an ipython session. Note that the doesn't remain the same for /= even though it works for div_
Just to be clear, the reason this is an issue is that it means that functions can't do inplace operations on function arguments.
Also, this works fine for +=
Code change:
Args: input (Tensor): the tensor to compare other (Tensor or float): the tensor or value to compare <del> out (Tensor, optional): the output tensor. Must be a `ByteTensor` or the same type as `input`. </del> <ins> out (Tensor, optional): the output tensor. Must be a `ByteTensor` </ins> Returns: Tensor: A ``torch.ByteTensor`` containing a 1 at each location where comparison is true

Related discussion:
text: [pytorch] The docs say that for tensor comparison operators (gt,lt etc) it should be possible to pass out argument typed as input (), yet when I try to do it, I hit an error Should the docs be fixed, or is it a bug?
If it's really useful we can add it back; but we'll fix the docs for now
Code change:
#include "THCTensor.hpp" #include "THCHalf.h" #include "THCHalfAutoNumerics.cuh" <ins> #include "THCNumerics.cuh" </ins> #include "common.h" // kernels borrowed from Caffe

Related discussion:
text: max pooling functions are not consistent with max functions. Below an example, every max pooling (be it 1d, 2d or 3d, adaptive or not) acts the same, on cpu or on cuda. Essentially, there are two fondamental differences : max pooling of all values is while for it's max pooling of nan and valid values is valid values, which means s get ignored, while for , as soon as there is a value, the result is . More generally, choosing explicetely how to deal with as in numpy () could be a solution, but maybe this is related to CuDNN's max pooling ? Built from latest sources (as of 05/17) PyTorch version: 0.5.0a0+ Is debug build: No CUDA used to build PyTorch: 9.1.85 OS: Ubuntu 16.04.4 LTS GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 CMake version: version 3.9.4 Python version: 3.6 Is CUDA available: Yes CUDA runtime version: 9.1.85 GPU models and configuration: GPU 0: Quadro M1000M Nvidia driver version: 390.30 cuDNN version: Probably one of the following: /usr/lib/x8664-linux- /usr/lib/x8664-linux-gnu/libcudnnstaticv7.a Versions of relevant libraries: [conda] magma-cuda91 2.3.0 1 pytorch [conda] torch 0.5.0a0+ <pip[conda] torch 0.3.1b0+ <pip[conda] torch 0.5.0a0+ <pip
Hi, . Thank you for providing the example. I'll take a look.
Hmhm. I seem to get a double free in the ...
Another question: Do we really want to provide an option to ignore NaN? My view is that "if you have nan in your net, you're screwed", so I would just return NaN there, personally.
My practical usecase is for KITTI groundtruth depthmaps and FlowMaps which are 2D sparse arrays.