jiebi/CodeConvo huggingface dataset:
Change:
std::vector<at::Tensor> unpack(at::TensorList tl, const char *name, int pos) const; std::vector<at::Tensor> unpack_idxs(at::TensorList tl, const char *name, int pos) const; <del> Variable as_variable(Tensor tensor) const; std::tuple<Variable, Variable> as_variable(std::tuple<Tensor, Tensor> tensor) const; std::tuple<Variable, Variable, Variable> as_variable(std::tuple<Tensor, Tensor, Tensor> tensor) const; std::tuple<Variable, Variable, Variable, Variable> as_variable(std::tuple<Tensor, Tensor, Tensor, Tensor> tensor) const; std::vector<Variable> as_variable(TensorList tensor) const; Variable maybe_wrap(Tensor data, const Variable & self, bool inplace) const; </del> private: at::Type* baseType; std::string str;

Discussion:
See Docker run fails with , which is due to the fact that is specified in docker container: Which results in: CI cc
Check that adding to manywheels image results in a failure:
Tentatively marking hi-pri since nightly testing are failing
This is resolved, here is the successful run:
Change:
python ..ci_scriptsdelete_image.py 7z x %IMAGE_COMMIT_TAG%.7z <del> sh run_test.sh </del> <ins> sh run_test.sh -- -v </ins> EOL

Discussion:
If a test segfaults, it's hard to determine which test it is because the output looks like: ! I can work on this if we think this is a good idea.
Last time I asked about this, he didn't want to because it adds a lot of scrolling. I also did some work to move the CI tests to so that we can output XML status files, but (1) this is unlikely to help in case of segfault, and (2) you can't do this straightforwardly, because we run some test files multiple times with different environment variables, and the XML exporter isn't really set up to handle this case. For the record, I am personally OK with just slapping on the tests and calling it a day.
in the conbuild sounds good to me too
closed via
Change:
int collapseIndicesDim = indicesInfo.collapseDims(dim);                 int64_t inputSlices = 1;                                               <del> int64_t topKSlices = 1;                                                for (int i = 0; i < numDims; ++i) {                                    </del> <ins> for (int i = 0; i < inputInfo.dims; ++i) {                             </ins> inputSlices *= inputInfo.sizes[i];                                   <ins> }                                                                      int64_t topKSlices = 1;                                                for (int i = 0; i < topKInfo.dims; ++i) {                              </ins> topKSlices *= topKInfo.sizes[i];                                     }                                                                      

Discussion:
Repro: Expect two small tensors of small integers to be printed. Instead, only the first case works. In the second case, you get a series of failed assertions starting with this: I wonder if it is related to
is this still an outstanding issue? I cannot reproduce this with an install from the master branch.
The size which causes failure doesn't seem deterministic. Using a pretty recent build (a few days old), I now have to increase the "9" to a larger number to make it fail. Sometimes it will fail at 10, sometimes at 11. Here's an updated case that seems to fail reliably at some size, for me:
Change:
>>> hx = Variable(torch.randn(3, 20)) >>> output = [] >>> for i in range(6): <del> ...     hx = rnn(input, hx) ...     output[i] = hx </del> <ins> ...     hx = rnn(input[i], hx) ...     output.append(hx) </ins> """ def __init__(self, input_size, hidden_size, bias=True, nonlinearity="tanh"):

Discussion:
Previously, worked to install CPU version explicitly. However, does not work. Are the whl files for 2.4 missing? N/A cc
Hi, this link is deprecated since 2.4. Please use the official installation guide instead.
Still it is not available for macosx x86 platform
Hi and Please use official install commands from For linux its: MacOS x86 binaries where deprecated since release 2.3.0.
Closing as resolved. We previously used the command in original post for building dockerfiles for specific versions. I have now replaced it with:
This still installs torch 2.2.2 but not 2.4.0 On Monday, August 19, 2024 at 10:41:25 PM GMT+5:30, Mihir Patel Closing as resolved. We previously used the command in original post for building dockerfiles for specific versions. I have now replaced it with: pip${PYTHONVERSION} install --no-cache-dir --find-links  torch=={CUDAVERSIONTAG} — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID:
This still installs torch 2.2.2 but not 2.4.0
python --version Python 3.12.5 pip3 install torch torchvision torchaudio --index-url Collecting torch Downloading (151.0 MB) 151.0/151.0 MB 11.3 MB/s eta 0:00:00 Successfully installed torch-2.2.2 torchaudio-2.2.2 torchvision-0.17.2 but not 2.4.0
Change:
cache_key = metadata[1] rebuild = rebuild_storage_filename storage._shared_incref() <ins> elif storage.size() == 0: # This is special cased because Empty tensors # (with size 0) cannot be mmapped. return (rebuild_storage_empty, (type(storage),)) </ins> else: fd, size = storage._share_fd_() if sys.version_info[0] == 2:

Discussion:
PyTorch version: 0.4.0a0+ Repro: Result:
Emm. I fail to reproduce this bug in Python 3.5.0 and Mac OS 10.13.3. What's your system environment?
The bug is probably specific to the sharing mode. Mac OS uses by default.
Change:
from tools.setup_helpers.env import check_env_flag from tools.setup_helpers.cuda import WITH_CUDA, CUDA_HOME from tools.setup_helpers.cudnn import WITH_CUDNN, CUDNN_LIB_DIR, CUDNN_INCLUDE_DIR <ins> from tools.setup_helpers.split_types import split_types </ins> DEBUG = check_env_flag('DEBUG') WITH_DISTRIBUTED = check_env_flag('WITH_DISTRIBUTED') WITH_DISTRIBUTED_MW = WITH_DISTRIBUTED and check_env_flag('WITH_DISTRIBUTED_MW')

Discussion:
Trying to run on github actions via AWS instance gives me: I will of course set up now, but it shouldn't crash like that. May be it should use as a fallback? pt-2.0 cc
Is this a multitenant setup? Also curious which distro are you running, I was under the impression that most linux distros set home by default
this is AWS image. one default user. I think it's user, but it wants you to run as root, which is weird. I have manually now set to overcome this.
We should update triton to use pathlib or something
Change:
"""Fills the input Tensor or Variable with values according to the method described in "Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification" - He, K. et al. (2015), using a normal distribution. The resulting tensor will have values sampled from :math:`N(0, std)` where <del> :math:`std = sqrt{2 / ((1 + a^2) times fan_in)}`. Also known as He initialisation. </del> <ins> :math:`std = sqrt{2 / ((1 + a^2) times fan_in)}`. Also known as He initialisation. </ins> Args: tensor: an n-dimensional torch.Tensor or autograd.Variable

Discussion:
why 1.21.6? because that's the oldest supported numpy on windows/python3.10: all cc
Here is the test for pinning numpy to 1.21.6 The reason its was not pinned to this version is that conda can't properly resolve all the dependencies at this step here:
It looks like you are correct, I will put up a PR to pin lower then already pinned:
Here is the PR:
Change:
PyObject *THSPTensorClass = NULL; <del> static void THSPTensor_(initStorage)(THSTensor* tensor) </del> <ins> static void THSTensor_(initStorage)(THSTensor* tensor) </ins> { // Ensure that PyTorch's "storage is not NULL" invariant is upheld. // See Note [Storage is not NULL]

Discussion:
This causes to not work properly on newly created empty sparse tensors. In the example below, both should print 1 instead.
Change:
def test_view(self): tensor = torch.rand(15) template = torch.rand(3, 5) <ins> empty = torch.Tensor() </ins> target = template.size().tolist() self.assertEqual(tensor.view_as(template).size().tolist(), target) self.assertEqual(tensor.view(3, 5).size().tolist(), target)

Discussion:
Please check and fix all the PyTorch web links format from to , like in the official site.
closed via
Change:
std::unique_ptr<Tensor> save_std(output->newTensor()); save_std->resizeAs(*running_var); <del> if (use_cudnn) { </del> <ins> if (use_cudnn && eps >= CUDNN_BN_MIN_EPSILON) { </ins> #ifdef WITH_CUDNN torch::cudnn::cudnn_batch_norm_forward( state,

Discussion:
Hi, I have two linux servers. My code works in one machine but not the other. In the one that it doesn't work, the error message is: occured during batch normalization( convolution is fine). Also, the same code works in CPU but not in GPU. I guess this problem is mainly related to GPU libraries like cuda or cudnn. The one that doesn't work has the following configurations: Nvidia driver 375.26 cuda 8.0.61 cudnn 5.1.10 While in the one it does work, I have the following configurations: Nvidia driver 367.48 cuda 8.0.44 cudnn 5.1.5 TensorFlow can run in the above environment flawless though. So I am wondering if this is caused by the incompatibility between PyTorch 0.1.12 and cuda 8.0.61/cudnn 5.1.10 ?
you use the pytorch binaries, we ship cudnn6 in them, use it it's great. 5.1.10 maybe has a bug. cc: . do you have a small test case to reproduce this? does not use CuDNN batchnorm by default, so maybe that's why?
Small test case would help, thanks.
After some experimentation, I have pin pointed where exactly the error comes from. It seems batch norm in cudnn 5.1.10 doesn't support eps as small as 1e-6. A small test case would be:
Yes, cudnn.h for 5.1.10 defines CUDNNBNEPSILON to be 1e-5 (minimum allowable value). I can't check header for 5.1.5. Even with 1e-5 epsilon for many input data distributions the results become pretty unstable. Is there a case for this restriction to be lifted, does using very small epsilon provide benefits?
I see. Not sure if small epsilon is necessary. I am using 1e-6 just to replicate others' configurations. Should PyTorch provide more informative error message about this restriction?
Ideally, CUDNN should provide more informative error messages (pytorch can not know why cudnn is returning an error code in each particular case) , but that's a long standing discussion.
It just hits on me.

As mentioned by , TensorFlow does not use CuDNN batchnorm by default. So is this possible in Pytorch, to not use default CuDNN batchnorm?
Never mind. I found the switch in .
Change:
#define IS_CUDA false #define CUDA_FLOAT false #else #define IS_CUDA true <del> #define CUDA_BYTE defined(THC_REAL_IS_BYTE) #define CUDA_CHAR defined(THC_REAL_IS_CHAR) #define CUDA_SHORT defined(THC_REAL_IS_SHORT) #define CUDA_INT defined(THC_REAL_IS_INT) #define CUDA_LONG defined(THC_REAL_IS_LONG) #define CUDA_FLOAT defined(THC_REAL_IS_FLOAT) #define CUDA_DOUBLE defined(THC_REAL_IS_DOUBLE) #define CUDA_HALF defined(THC_REAL_IS_HALF) </del> <ins> #if defined(THC_REAL_IS_BYTE) #define CUDA_BYTE 1 #else #define CUDA_BYTE 0 </ins> #endif <ins> #if defined(THC_REAL_IS_CHAR) #define CUDA_CHAR 1 #else #define CUDA_CHAR 0 #endif #if defined(THC_REAL_IS_SHORT) #define CUDA_SHORT 1 #else #define CUDA_SHORT 0 #endif #if defined(THC_REAL_IS_INT) #define CUDA_INT 1 #else #define CUDA_INT 0 #endif #if defined(THC_REAL_IS_LONG) #define CUDA_LONG 1 #else #define CUDA_LONG 0 #endif #if defined(THC_REAL_IS_FLOAT) #define CUDA_FLOAT 1 #else #define CUDA_FLOAT 0 #endif #if defined(THC_REAL_IS_DOUBLE) #define CUDA_DOUBLE 1 #else #define CUDA_DOUBLE 0 #endif #if defined(THC_REAL_IS_HALF) #define CUDA_HALF 1 #else #define CUDA_HALF 0 #endif #endif // ifndef THC_GENERIC_FILE </ins> #if IS_CUDA #define THIndexTensor THCudaLongTensor #define THIndexTensor_(NAME) TH_CONCAT_2(THCudaLongTensor_,NAME)

Discussion:
In we define a number of macros which are subsequently used in if statements: This is undefined. On clang 3.4.2, these warnings don't seem to actually get printed out () but on clang 4.0.0 they seem to always get emitted.
Change:
END_HANDLE_TH_ERRORS } <ins> // The type for PyInt_FromLong varies between Windows (int64_t) and // Linux (long).  Unfortunately, on Clang, if the template type argument // does not *exactly* line up, Clang discards the template // specialization.  So we write a little stub function which does have // the right type. static PyObject* PortablePyInt_FromLong(int64_t ival) { return PyInt_FromLong(ival); } </ins> static struct PyGetSetDef batch_norm_forward_properties[] = { THP_FUNCTION_DEFAULT_PROPERTIES, {(char*)"running_mean", (getter)getTensorAttr<BatchNormForward, BatchNormParams,

Discussion:
One representative error: The cause of the problem is : It seems that on clang, changing the type parameter here is sufficient to cause template instantiation to fail. Maybe the easiest way to fix this is to write a more portable version of PyIntFromLong (and friends) which always returns .
Change:
('ones', small_3d, lambda t: [1, 2, 3, 4, 5],), ('permute', new_t(1, 2, 3, 4), lambda t: [2, 1, 3, 0],), ('put_', new_t(2, 5, 3), lambda t: [long_type(t)([[0], [-2]]), t([[3], [4]])],), <ins> ('put_', new_t(2, 3), lambda t: [long_type(t)([]), t([])], 'empty'), </ins> ('put_', new_t(2, 2), lambda t: [long_type(t)([[1], [-3]]), t([[1], [2]]), True], 'accumulate'), ('prod', small_2d_oneish, lambda t: [],), ('prod', small_3d, lambda t: [1], 'dim'),

Discussion:
odd difference between cuda Tensor and Variable: encountered in torch.distributions.Categorical, built yesterday from master
related to hmm, here's the issue is for variables, so probably not (immediately) related
different code path but similar issue
This is a regression from the latest 0.3 binaries, which don't have this error.
Change:
endif() # ---[ Exract versions <del> message(STATUS "Caffe2: CUDA detected: " ${CUDA_VERSION}) </del> if (CAFFE2_FOUND_CUDNN) # Get cuDNN version file(READ ${CUDNN_INCLUDE_DIR}/cudnn.h CUDNN_HEADER_CONTENTS)

Discussion:
Here's the situation: Ideally, you'd say something like to make a build of PyTorch with a non-standard version of CUDA. When this happens, needs to find the correct copy of nvcc to use. If you are using ccache as per the instructions in you'll have set to your ccache binary, so that ccache happens. cmake will correctly respect this variable. However, here's the problem: ccache's nvcc will use your path to determine what the real nvcc to use is, this means that you'll pick up whatever nvcc is in your PATH, which is probably not the one you want, because you're trying to use a different nvcc. So in sum, you have to do the non-intuitive to get the whole shebang to work. The worst part is that it won't tell you if you've got it wrong, it will happily use the wrong nvcc. I haven't gone through user submitted issues to see if anyone has succumbed to this particular bug.
A long-term solution might be getting to work
works now.
Change:
from functools import reduce from ._utils import _range <ins> class __PrinterOptions(object): precision = 4 threshold = 1000 edgeitems = 3 linewidth = 80 PRINT_OPTS = __PrinterOptions() </ins> SCALE_FORMAT = '{:.5e} *n' <del> def _number_format(tensor): min_sz = 2 </del> <ins> # We could use **kwargs, but this will give better docs def set_printoptions( precision=None, threshold=None, edgeitems=None, linewidth=None, profile=None, ): """Set options for printing. Items shamelessly taken from Numpy Args: precision: Number of digits of precision for floating point output (default 8). threshold: Total number of array elements which trigger summarization rather than full repr (default 1000). edgeitems: Number of array items in summary at beginning and end of each dimension (default 3). linewidth: The number of characters per line for the purpose of inserting line breaks (default 80). Thresholded matricies will ignore this parameter. profile: Sane defaults for pretty printing. Can override with any of the above options. (default, short, full) """ if profile is not None: if profile == "default": PRINT_OPTS.precision = 4 PRINT_OPTS.threshold = 1000 PRINT_OPTS.edgeitems = 3 PRINT_OPTS.linewidth = 80 elif profile == "short": PRINT_OPTS.precision = 2 PRINT_OPTS.threshold = 1000 PRINT_OPTS.edgeitems = 2 PRINT_OPTS.linewidth = 80 elif profile == "full": PRINT_OPTS.precision = 4 PRINT_OPTS.threshold = float('inf') PRINT_OPTS.edgeitems = 3 PRINT_OPTS.linewidth = 80 if precision is not None: PRINT_OPTS.precision = precision if threshold is not None: PRINT_OPTS.threshold = threshold if edgeitems is not None: PRINT_OPTS.edgeitems = edgeitems if linewidth is not None: PRINT_OPTS.linewidth = linewidth def _number_format(tensor, min_sz=-1): min_sz = max(min_sz, 2) </ins> tensor = torch.DoubleTensor(tensor.nelement()).copy_(tensor).abs_() pos_inf_mask = tensor.eq(float('inf'))

Discussion:
It seems that the following link that can be reached from here (check the screenshot) is broken: Page: Broken link: Screenshot: ! cc
this should be transferred to , but I don't have permissions to do that. Fix is available at pytorch/hub.
Change:
msg='LR is wrong in epoch {}: expected {}, got {}'.format( epoch, target[epoch], param_group['lr']), delta=1e-5) if __name__ == '__main__': run_tests()

Discussion:
I'm working on checkpointing in our nn trainer. When saving the state objects, I get an error: PicklingError: Can't pickle <type 'function': attribute lookup builtin.function failed I tracked it down to trying to save the scheduler ReduceLROnPlateau. I think the issue is due to the scheduler referencing the optimizer, which has to be saved with statedict(). I presume the same error will occur with other schedulers too. It seems like the problem could be solved by providing a statedict() and loadstatedict() in the scheduler classes. pytorch master rev (Jan 22) installed from source x86_64 on centos 7.4 cuda 9.0 cudnn 7 k80 gcc 5.4 python 2.7.14
The following script reproduces the crash: import torch import from import ReduceLROnPlateau if name == 'main': model = (10,10) optim = (model.parameters()) sched = ReduceLROnPlateau(optim, mode='max', factor=0.5) with open('','wb') as f: # Works # ((model.statedict(), optim.statedict()), f) # Crashes ((model.statedict(), optim.state_dict(), sched), f)
Change:
import os import torch import traceback <ins> import warnings </ins> from torch._six import raise_from from subprocess import Popen, PIPE from multiprocessing.util import register_after_fork as _register_after_fork

Discussion:
There's quite a few user-reports of installing the CUDA 7.5 binary of pytorch and then running on GTX 1080 / 1070, and there will be reports of installing 7.5 / 8.0 binaries and running on V100. Raising a user-warning in such situations and letting them know will fix the problem.
The version of CUDA used to compile the binary should be available thanks to this PR:
I think is a more robust way of getting the CUDA used to compile PyTorch, as currently if there are multiple CUDA installed in the machine, the heuristics used in might not match those used by CMake FindCUDA. I think that we should modify the to return the cuda version used during compilation. I have the impression that it will not require changing much because doesn't require cuda to execute. What do you think?
Change:
@wraps(fn) def wrapper(self): if self.rank == self.MANAGER_PROCESS_RANK: <del> self._join_and_reduce() </del> <ins> self._join_and_reduce(fn) </ins> else: fn(self) return wrapper

Discussion:
is causing a crash when we try to build pytorch. Crash looks like this: (all times in US/West) Our 7AM, 8AM builds and earlier were all fine but 9AM, 10AM, 11AM builds all crashed with that same traceback. The new import was in which I think went in between 8AM and 9AM today See command below: See here if needed for : <!-- A clear and concise description of what you expected to happen. --Please copy and paste the output from our (or fill out the checklist below manually). You can get the script and run it with: PyTorch Version (e.g., 1.0): building from master OS (e.g., Linux): linux How you installed PyTorch (, , source): source Build command you used (if compiling from source): see above under 'reproduce' Python version: 3.6
maybe easier to read for build command used:
please try in the CircleCi environment?
in our build script seems to have fixed this. Thanks !
Change:
def _initialize_backend(): from .._functions.thnn import _all_functions as _thnn_functions from .._functions.linear import Linear <del> from .._functions.batchnorm import BatchNorm </del> from .._functions.conv import ConvNd from .._functions.rnn import RNN,  RNNTanhCell, RNNReLUCell, GRUCell, LSTMCell

Discussion:
As Arthur Szlam reports, fb-internal cudnn is still lagging behind, and giving batch-size 1024 with batchnorm is raising an error from cudnn. Need to check for compile-time version and disable this codepath
The 1024 limitation was removed in 5.1.10
Change:
.. automethod:: fmod_ .. automethod:: frac .. automethod:: frac_ <del> .. automethod:: free </del> .. automethod:: gather .. automethod:: ge .. automethod:: ge_

Discussion:
Dataloader call getitem as many times as indices in the current batch. In case datasets support a list of indices in one call, or a native python slice object, add a getbatch (optional) to pytorch dataset class. If dataloader sees the dataset has such a method implemented, make it fetch the batch corresping to the list of indices with one getbatch call rather than many getitem calls. This saves the trouble of calling getitem many times in cases where for example a disk file has to be opened then closed for every call and the underlying data structure supports a slicing or lists of indices for example hdf5 files.
will support something similar
is the latest version of this.
Change:
The NVIDIA driver on your system is too old (found version {}). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx <del> Alternatively, go to: https://pytorch.org/binaries to install </del> <ins> Alternatively, go to: http://pytorch.org to install </ins> a PyTorch version that has been compiled with your version of the CUDA driver.""".format(str(torch._C._cuda_getDriverVersion()))) <ins> def _check_capability(): error_str = """ Found GPU%d %s which requires CUDA_VERSION >= %d for optimal performance and fast startup time, but your PyTorch was compiled with CUDA_VERSION %d. Please install the correct PyTorch binary using instructions from http://pytorch.org """ CUDA_VERSION = torch._C._cuda_getCompiledVersion() for d in range(device_count()): major = get_device_capability(d)[0] name = get_device_name(d) if CUDA_VERSION < 8000 and major >= 6: warnings.warn(error_str % (d, name, 8000, CUDA_VERSION)) elif CUDA_VERSION < 9000 and major >= 7: warnings.warn(error_str % (d, name, 8000, CUDA_VERSION)) </ins> def _lazy_call(callable): if _initialized: callable()

Discussion:
There's quite a few user-reports of installing the CUDA 7.5 binary of pytorch and then running on GTX 1080 / 1070, and there will be reports of installing 7.5 / 8.0 binaries and running on V100. Raising a user-warning in such situations and letting them know will fix the problem.
The version of CUDA used to compile the binary should be available thanks to this PR:
I think is a more robust way of getting the CUDA used to compile PyTorch, as currently if there are multiple CUDA installed in the machine, the heuristics used in might not match those used by CMake FindCUDA. I think that we should modify the to return the cuda version used during compilation. I have the impression that it will not require changing much because doesn't require cuda to execute. What do you think?