jiebi/CodeConvo huggingface dataset:
Change:
is causing a crash when we try to build pytorch. Crash looks like this: (all times in US/West) Our 7AM, 8AM builds and earlier were all fine but 9AM, 10AM, 11AM builds all crashed with that same traceback. The new import was in which I think went in between 8AM and 9AM today See command below: See here if needed for : <!-- A clear and concise description of what you expected to happen. --Please copy and paste the output from our (or fill out the checklist below manually). You can get the script and run it with: PyTorch Version (e.g., 1.0): building from master OS (e.g., Linux): linux How you installed PyTorch (, , source): source Build command you used (if compiling from source): see above under 'reproduce' Python version: 3.6
maybe easier to read for build command used:
please try in the CircleCi environment?
in our build script seems to have fixed this. Thanks !

Discussion:
from contextlib import contextmanager import torch <ins> import torch.cuda </ins> import torch.distributed as dist from common import TestCase

print('Distributed not available, skipping tests') sys.exit(0) <ins> SKIP_IF_NO_CUDA_EXIT_CODE = 75 def skip_if_no_cuda_distributed(func): func.skip_if_no_cuda_distributed = True @wraps(func) def wrapper(*args, **kwargs): if not torch.cuda.is_available(): sys.exit(SKIP_IF_NO_CUDA_EXIT_CODE) return func(*args, **kwargs) return wrapper </ins> @contextmanager def _lock():

self._test_broadcast_helper(group, group_id, rank) @unittest.skipIf(BACKEND != 'gloo', "Only Gloo backend supports CUDA allReduce") <ins> @skip_if_no_cuda_distributed </ins> def test_broadcast_cuda(self): group, group_id, rank = self._init_global_test() self._test_broadcast_helper(group, group_id, rank, True)

) @unittest.skipIf(BACKEND != 'gloo', "Only Gloo backend supports CUDA allReduce") <ins> @skip_if_no_cuda_distributed </ins> def test_all_reduce_sum_cuda(self): group, group_id, rank = self._init_global_test() self._test_all_reduce_helper(

@wraps(fn) def wrapper(self): if self.rank == self.MANAGER_PROCESS_RANK: <del> self._join_and_reduce() </del> <ins> self._join_and_reduce(fn) </ins> else: fn(self) return wrapper

getattr(self, self.id().split(".")[2])() sys.exit(0) <del> def _join_and_reduce(self): </del> <ins> def _join_and_reduce(self, fn): skip_ok = getattr(fn, "skip_if_no_cuda_distributed", False) </ins> for p in self.processes: p.join(self.JOIN_TIMEOUT) <del> self.assertEqual(p.exitcode, 0) </del> <ins> if not skip_ok: self.assertEqual(p.exitcode, 0) if skip_ok: first_process = self.processes[0] # do this first so we don't give an error message about mismatched exit codes if the first isn't valid assert first_process.exitcode == 0 or first_process.exitcode == SKIP_IF_NO_CUDA_EXIT_CODE for p in self.processes: self.assertEqual(p.exitcode, first_process.exitcode) if first_process.exitcode == SKIP_IF_NO_CUDA_EXIT_CODE: raise unittest.SkipTest("cuda is not available") </ins> elif BACKEND == 'mpi': dist.init_process_group(init_method=INIT_METHOD, backend='mpi')
Change:
Recently, we are testing PyTorch 1.7 as we need the with-statement support, so that our algorithm which contains a custom module using can be deployed to the production environment. During the testing, we encountered the following assertion failure: Steps to reproduce the behavior: the following Python code to create a serialized PyTorch module. a C++ test program. the test program with the following command: in the current directory. should terminate with exit code 0 instead of throw an exception. Collecting environment information... PyTorch version: 1.7.0.dev20200922+cpu Is debug build: True CUDA used to build PyTorch: None ROCM used to build PyTorch: N/A OS: CentOS Linux 7 (Core) (x8664) GCC version: (GCC) 4.8.5 (Red Hat 4.8.5-39) Clang version: Could not collect CMake version: Could not collect Python version: 3.7 (64-bit runtime) Is CUDA available: False CUDA runtime version: No CUDA GPU models and configuration: No CUDA Nvidia driver version: No CUDA cuDNN version: No CUDA HIP runtime version: N/A MIOpen runtime version: N/A Versions of relevant libraries: [pip3] numpy==1.19.2 [pip3] torch==1.7.0.dev20200922+cpu [pip3] torchvision==0.7.0+cpu [conda] Could not collect cc

Discussion:
"aten::set_grad_enabled(bool val) -> ()", [](Stack* stack) { torch::GradMode::set_enabled(pop(stack).toBool()); <del> push(stack, IValue()); </del> }, aliasAnalysisConservative()), });
Change:
On Windows, torch.slogdet reproducibly fails when computing torch.slogdet on larger tensors. Here is the error: Steps to reproduce the behavior: Expected this op to succeed on Windows. This does not fail on Linux. cc
Which GPU card are you using on Windows?
I have an RTX 3080 and an RTX 3090 and see the issue on both. If this is a CUDA issue, could you point me to the right place to report it?
I can reproduce this with PyTorch 1.7.0 + RTX 3080.
It seems that the current implementation of this function is based on MAGMA. Will try to rebuild with newer versions.
It seems that the current windows MAGMA binaries are built without cuda arch 8.0 support. I will fix it.
Awesome! Thank you so much for the quick turnaround on this. Please let me know if you want me to test a fix.
Wait for the merge of

Discussion:
ensure_attr('ClassNLLCriterion', 'weights') ensure_attr('ParallelCriterion', 'repeatTarget') ensure_attr('MultiMarginCriterion', 'weights') <del> ensure_attr('SpatialConvolution', 'bias', 'finput', 'fgradInput', 'gradWeight', 'gradBias') </del> <ins> ensure_attr('SpatialConvolution', 'bias', 'finput', 'fgradInput', 'gradWeight', 'gradBias', '_gradOutput') </ins> attr_map('ReLU', {'val': 'value'}) attr_map('Threshold', {'val': 'value'}) attr_map('Unsqueeze', {'pos': 'dim'})
Change:
Following example: fails in Python-3.9 with cc
And the reason for that is very simple:
Another interesting offender:

Discussion:
if isinstance(expr.slice.value, ast.Tuple): # N-dimensional indexing using Tuple: x[(i, j, k)] is equivalent to x[i, j, k] # XXX: Indexing using a list is **different**! It triggers advanced indexing. <del> indices = [] for index_expr in expr.slice.value.elts: indices.append(build_expr(ctx, index_expr)) </del> <ins> indices = [build_expr(ctx, index_expr) for index_expr in expr.slice.value.elts] </ins> return Subscript(base, indices) else: return Subscript(base, [build_expr(ctx, expr.slice.value)])

return Subscript(base, [build_SliceExpr(ctx, base, expr.slice)]) elif sub_type is ast.ExtSlice: return Subscript(base, build_ExtSlice(ctx, base, expr.slice)) <ins> elif sys.version_info >= (3, 9):  # In Python3.9 array indicies are not wrapped in ast.Index if sub_type is ast.Tuple: # N-dimensional indexing using Tuple: x[(i, j, k)] is equivalent to x[i, j, k] indices = [] for index_expr in expr.slice.elts: if isinstance(index_expr, ast.Slice): indices.append(build_SliceExpr(ctx, base, index_expr)) else: indices.append(build_expr(ctx, index_expr)) return Subscript(base, indices) return Subscript(base, [build_expr(ctx, expr.slice)]) </ins> else:  # Ellipsis (can only happen in Python 2) raise NotSupportedError(base.range(), "ellipsis is not supported")
Change:
Broken download links for libtorch v1.8.0 with CUDA 10.2 for Linux (pre-cxx11 and cxx11 ABI) and Windows (Release and Debug versions) on the PyTorch website. Steps to reproduce the behavior: on any of the following links leads to an "Access Denied" error: The chosen libtorch version is downloaded. PyTorch Version: .1.8.0 OS: Linux and Windows CUDA version: 10.2
The correct links are listed below. cc
Thanks!
Same issue also happens for CUDA 11.1 when trying to download windows precompiled binaries Changing to in the links suggested by still leads to Access Denied page
Okay I wrote a simple script to test if all of our libtorch packages are actually downloadable, sharing results here: <details<summarypkgs </summaryCommand I used to generate these (from the root of ): </detailsOutput being: I did find that this issue is correct in that the installation instructions were incorrect for CUDA 10.2 and I created a PR to resolve this:

Discussion:
if bias is not None: self.bias.data.uniform_(-0.1, 0.1) <del> def forward(self, input): # See the autograd section for explanation of what happens here. return Linear()(input, self.weight, self.bias) </del> <ins> def forward(self, input): # See the autograd section for explanation of what happens here. return Linear()(input, self.weight, self.bias) </ins> Writing custom C extensions
Change:
Go to Look at the html. It contains the following: ! This problem manifests in google search. Searching for 1.8.1 docs brings up the "1.8.0 docs": ! We can probably fix this by doing a quick sed on the website but we should develop a solution for future releases. I think this has happened before in the past and am not sure what causes it. cc

Discussion:
#include <vector> #include <string> <ins> #include "torch/csrc/utils/object_ptr.h" </ins> #define THPUtils_(NAME) TH_CONCAT_4(THP,Real,Utils_,NAME) #define THPUtils_typename(obj) (Py_TYPE(obj)->tp_name)

#define THSTensorPtr  TH_CONCAT_3(THS,Real,TensorPtr) #define THSPTensorPtr  TH_CONCAT_3(THSP,Real,TensorPtr) <del> template<class T> class THPPointer { public: THPPointer(): ptr(nullptr) {}; THPPointer(T *ptr): ptr(ptr) {}; THPPointer(THPPointer &&p) { free(); ptr = p.ptr; p.ptr = nullptr; }; ~THPPointer() { free(); }; T * get() { return ptr; } T * release() { T *tmp = ptr; ptr = NULL; return tmp; } operator T*() { return ptr; } THPPointer& operator =(T *new_ptr) { free(); ptr = new_ptr; return *this; } THPPointer& operator =(THPPointer &&p) { free(); ptr = p.ptr; p.ptr = nullptr; return *this; } T * operator ->() { return ptr; } operator bool() { return ptr != nullptr; } private: void free(); T *ptr = nullptr; }; typedef THPPointer<PyObject> THPObjectPtr; </del> typedef THPPointer<THPGenerator> THPGeneratorPtr; template <typename T>

<ins> #pragma once // RAII structs to acquire and release Python's global interpreter lock (GIL) #include <Python.h> // Acquires the GIL on construction struct AutoGIL { AutoGIL() : gstate(PyGILState_Ensure()) { } ~AutoGIL() { PyGILState_Release(gstate); } PyGILState_STATE gstate; }; // Releases the GIL on construction struct AutoNoGIL { AutoNoGIL() : save(PyEval_SaveThread()) { } ~AutoNoGIL() { PyEval_RestoreThread(save); } PyThreadState* save; }; </ins>

<ins> #include "torch/csrc/utils/object_ptr.h" #include <Python.h> template<> void THPPointer<PyObject>::free() { if (ptr) Py_DECREF(ptr); } template class THPPointer<PyObject>; </ins>

<ins> #pragma once #include <Python.h> template<class T> class THPPointer { public: THPPointer(): ptr(nullptr) {}; THPPointer(T *ptr): ptr(ptr) {}; THPPointer(THPPointer &&p) { free(); ptr = p.ptr; p.ptr = nullptr; }; ~THPPointer() { free(); }; T * get() { return ptr; } const T * get() const { return ptr; } T * release() { T *tmp = ptr; ptr = NULL; return tmp; } operator T*() { return ptr; } THPPointer& operator =(T *new_ptr) { free(); ptr = new_ptr; return *this; } THPPointer& operator =(THPPointer &&p) { free(); ptr = p.ptr; p.ptr = nullptr; return *this; } T * operator ->() { return ptr; } operator bool() { return ptr != nullptr; } private: void free(); T *ptr = nullptr; }; typedef THPPointer<PyObject> THPObjectPtr; </ins>
Change:
Subclassing torch.Tensor with fails due to a on the latest nightly builds, whereas it previously worked without issue. Run this snippet of code: And you'll recieve an error message like this: Changing the to results in the exact same error message. There should be no error message as it should be working like it was previously in Torch 1.8.1 and below. I think that the issue started on Torch version: , and I can confirm that it still exists as of , . The issue started on the nightly builds at least 2 days ago, if that helps resolve things. It's been causing some of Captum's CircleCI tests to fail, and the tests show the first occurrence of the error on . cc
This is due to either or
I think the problem is I'm not supposed to delete because that means the super() call doesn't work. Instead, I should just verify that the passed in class is a subclass, then this should work.

Discussion:
del l1[i] return l1 <del> def _backward(self, method, input, gradOutput, scale): </del> <ins> def _backward(self, method, input, gradOutput, scale=1): </ins> isTable = isinstance(input, list) wasTable = isinstance(self.gradInput, list) if isTable: for i, module in enumerate(self.modules): <del> currentGradInput = getattr(module, method)(input, gradOutput[i], scale) </del> <ins> if method == 'updateGradInput': currentGradInput = module.updateGradInput(input, gradOutput[i]) elif method == 'backward': currentGradInput = module.backward(input, gradOutput[i], scale) </ins> if not isinstance(currentGradInput, list): raise RuntimeError("currentGradInput is not a table!")

else: self.gradInput = self.gradInput if not wasTable else input.clone() for i, module in enumerate(self.modules): <del> currentGradInput = getattr(module, method)(input, gradOutput[i], scale) </del> <ins> if method == 'updateGradInput': currentGradInput = module.updateGradInput(input, gradOutput[i]) elif method == 'backward': currentGradInput = module.backward(input, gradOutput[i], scale) </ins> if i == 0: self.gradInput.resize_as_(currentGradInput).copy_(currentGradInput) else:

ensure_attr('ClassNLLCriterion', 'weights') ensure_attr('ParallelCriterion', 'repeatTarget') ensure_attr('MultiMarginCriterion', 'weights') <ins> ensure_attr('SpatialConvolution', 'bias', 'finput', 'fgradInput', 'gradWeight', 'gradBias') </ins> attr_map('ReLU', {'val': 'value'}) attr_map('Threshold', {'val': 'value'}) attr_map('Unsqueeze', {'pos': 'dim'})
Change:
TensorPipe uses some advanced features of the Linux kernel. It does so on a "best effort" basis (i.e., use them if available, fall back on something else if not). This logic is unfortunately hard to test because it would require specific kernel versions on our CI machines. So it occasionally happens that we find out about issues when users report them. To help investigate these reports we've found it useful to know the kernel version and the glibc version used by the user (both when building and when running, if they're different). It would be great if these could be to the report produced by collect_env.
CC

Discussion:
'clang_version', 'cmake_version', 'os', <ins> 'libc_version', </ins> 'python_version', <ins> 'python_platform', </ins> 'is_cuda_available', 'cuda_runtime_version', 'nvidia_driver_version',

return None return match.group(1) <ins> def run_and_return_first_line(run_lambda, command): """Runs command using run_lambda and returns first line if output is not empty""" rc, out, _ = run_lambda(command) if rc != 0: return None return out.split('n')[0] </ins> def get_conda_packages(run_lambda): if get_platform() == 'win32':

return platform <ins> def get_python_platform(): import platform return platform.platform() def get_libc_version(): import platform if get_platform() != 'linux': return 'N/A' return '-'.join(platform.libc_ver()) </ins> def get_pip_packages(run_lambda): """Returns `pip list` output. Note: will also find conda-installed pytorch and numpy packages."""

torch_version=version_str, is_debug_build=debug_mode_str, python_version='{}.{} ({}-bit runtime)'.format(sys.version_info[0], sys.version_info[1], sys.maxsize.bit_length() + 1), <ins> python_platform=get_python_platform(), </ins> is_cuda_available=cuda_available_str, cuda_compiled_version=cuda_version_str, cuda_runtime_version=get_running_cuda_version(run_lambda),

pip_packages=pip_list_output, conda_packages=get_conda_packages(run_lambda), os=get_os(run_lambda), <ins> libc_version=get_libc_version(), </ins> gcc_version=get_gcc_version(run_lambda), clang_version=get_clang_version(run_lambda), cmake_version=get_cmake_version(run_lambda),

GCC version: {gcc_version} Clang version: {clang_version} CMake version: {cmake_version} <ins> Libc version: {libc_version} </ins> Python version: {python_version} <ins> Python platform: {python_platform} </ins> Is CUDA available: {is_cuda_available} CUDA runtime version: {cuda_runtime_version} GPU models and configuration: {nvidia_gpu_models}
Change:
When numworker 0 (in DataLoader), I call setnum_threads function and get some warnings. Execute the following script: got log: there should be no any warnings. PyTorch Version (e.g., 1.0): torch1.9 OS (e.g., Linux): Centos7 How you installed PyTorch (, , source): pip Build command you used (if compiling from source): no Python version: 3.8 CUDA/cuDNN version: 11.1/ GPU models and configuration: TeslaV100 Any other relevant information:
Duplicate of .
closing as dupplicate

Discussion:
auto num_threads = leaked->get_thread_count(); // NOLINTNEXTLINE(modernize-make-unique) threadpool.reset(new PThreadPool(num_threads)); <del> TORCH_WARN("Leaking Caffe2 thread-pool after fork."); </del> } } return threadpool.get();
Change:
Libtorch LTS couldn't be downloaded! to ! the link For example: (Linux) (Windows) There's the error ! LibTorch LTS(1.8.2) could be downloaded One user has reported the issue in forum cc
cc:
I can take care of this today, seems with the new lts stuff we neglected the libtorch uploads

Discussion:
def backward(self, grad_output): # TODO: not sure if this clone is necessary <del> return grad_output.clone().view(self.input_size) </del> <ins> return grad_output.contiguous().view(self.input_size) </ins> class Expand(Function):
Change:
When I was working on Vision and Audio CI, I often find that the dowloaded nightly PyTorch is CPU version instead of expected CUDA version. It will break the domain library CIs. It's often caused by nightly build failed or that day's nightly build isn't finished. There are 2 open issues for it. If this day's nightly PyTorch cuda build isn't available, I think it's better to provide last day's nightly builds rather than CPU version. cc
It looks that all nightly builds would be finished before 4 AM at PST, so vision CI failure probability might be higher in the morning of France.

Discussion:
global Storage Tensor = _import_dotted_name(t) Storage = _import_dotted_name(t.replace('Tensor', 'Storage')) <ins> if 'cuda' in t: import torch.cuda torch.cuda.init() </ins> _C._set_default_tensor_type(Tensor)
Change:
ongoing Provide some way users can tell that this SEV is causing their issue. Include when the incident began, when it was detected, mitigated, root caused, and finally closed. <details<summaryClick for example </summarye.g. 2/4 6:05a incident began 2/4 7:30a detected by noticing failure on unrelated PR and then looking at </detailsYou will see test1 is broken What was the root cause of this issue? How did we mitigate the issue? How do we prevent issues like this in the future?
This failure in CI appears suddenly without any code changes to related files. It is likely due to different test case run order. The test code was badly written such that test class , if called first, will assign , circumventing this problem. On the other hand, if is called first, will be raised. Should be fixed by .

Discussion:
for opset_version in opset_versions: self.opset_version = opset_version <ins> self.onnx_shape_inference = True </ins> run_model_test(self, model, False, input=inputs, rtol=rtol, atol=atol) if self.is_script_test_enabled and opset_version > 11: <del> TestModels.onnx_shape_inference = True outputs = model(inputs) </del> script_model = torch.jit.script(model) run_model_test(self, script_model, False, input=inputs, rtol=rtol, atol=atol)
Change:
mitigated 2/7 7:50a incident began 2/7 7:50a detected by contributor 2/7 8:11a root caused as latest librosa upgrade 2/7 ?? mitigated by 2/7 ?? closed by… How does this affect users of PyTorch CI? Several test jobs are failing What was the root cause of this issue? Latest librosa update How did we mitigate the issue? How do we prevent issues like this in the future? linter that checks all installs are pinned figure out how to specify dependencies in a clearer way (test-). we also need teams attributed to each dependency extend linter to use both < and OR only = throwback to numba sev, similar ones here Potentially thinking of other projects such as automatically disabling these tests when they're broken by external change
Closing, as it should not affect anyone rebasing of recent master

Discussion:
# Install numba only on python-3.8 or below # For numba issue see https://github.com/pytorch/pytorch/issues/51511 if [[ $(python -c "import sys; print(int(sys.version_info < (3, 9)))") == "1" ]]; then <del> as_jenkins pip install --progress-bar off numba==0.54.1 librosa>=0.6.2 </del> <ins> as_jenkins pip install --progress-bar off numba==0.54.1 "librosa>=0.6.2,<0.9.0" </ins> else <del> as_jenkins pip install --progress-bar off numba==0.49.0 librosa>=0.6.2 </del> <ins> as_jenkins pip install --progress-bar off numba==0.49.0 "librosa>=0.6.2,<0.9.0" </ins> fi # Update scikit-learn to a python-3.8 compatible version

export PYTORCH_TEST_SKIP_NOARCH=1 conda install -y six <del> pip install -q hypothesis "expecttest==0.1.3" "librosa>=0.6.2" "numba<=0.49.1" psutil "scipy==1.6.3" </del> <ins> pip install -q hypothesis "expecttest==0.1.3" "librosa>=0.6.2,<0.9.0" "numba<=0.49.1" psutil "scipy==1.6.3" </ins> # TODO move this to docker pip install unittest-xml-reporting pytest

:: The version is fixed to avoid flakiness: https://github.com/pytorch/pytorch/issues/31136 ======= <del> pip install "ninja==1.10.0.post1" future "hypothesis==4.53.2" "expecttest==0.1.3" "librosa>=0.6.2" psutil pillow unittest-xml-reporting pytest </del> <ins> pip install "ninja==1.10.0.post1" future "hypothesis==4.53.2" "expecttest==0.1.3" "librosa>=0.6.2,<0.9.0" psutil pillow unittest-xml-reporting pytest </ins> if errorlevel 1 exit /b if not errorlevel 0 exit /b
Change:
The vision pinned hash is from a long time ago (2020?) and attempting to update it to a newer hash fails. See for examples of CI failing when the hash is updated, and for a possible reason why. When fixed, we should add vision back into . n/a cc
Self-assigning myself to have a look at it
to TorchVision, which changes how operates when given a source folder: instead of execution it extracts the requirements from above-mentioned toml file. So, one can either pass to continue installing torchvision in the old way, or update to contain proper dependencies
fixed by

Discussion:
// TODO: gradOutput shape check // Resize and initialize result tensor. THCTensor_(resizeAs)(state, gradInput, input); <del> THCTensor_(newContiguous)(state, gradInput); </del> THCTensor_(zero)(state, gradInput); int batchSize;
Change:
Numpy shouldn't require a pin with PyTorch. This issue is to ensure that should get reverted and all issues addressed (issues look like ) CI
Never mind, we apparently should have been pinning it.

Discussion:
call %CONDA_PARENT_DIR%Miniconda3Scriptsactivate.bat %CONDA_PARENT_DIR%Miniconda3 if "%INSTALL_FRESH_CONDA%"=="1" ( <del> call conda install -y -q python=%PYTHON_VERSION% numpy cffi pyyaml boto3 libuv </del> <ins> call conda install -y -q python=%PYTHON_VERSION% numpy"<1.23" cffi pyyaml boto3 libuv </ins> if errorlevel 1 exit /b if not errorlevel 0 exit /b call conda install -y -q -c conda-forge cmake=3.22.3