jiebi/CodeConvo huggingface dataset:
Change:
&emsp; DataLoader for large corpus file, supporting multi-process and multi-thread and memory optimization &emsp; Thanks a lot for your pytorch framework, I've benefited a lot in work by using it. But I also got some little confusions with regard to data loading and processing. &emsp; In NLP cases, suppose I have a very large corpus file with labeled data "", which is too large to load it into memery once time, or even it is infinite large. I want to train my model on this dataset from top to bottom by mini batches. That means in each batch, I yield certain text lines from file begining to file ending to train my model (cannot support shuffle). I can do this with python generator easily. However, I don't know how to write subclass of and use to customize my own dataset on "". Since "" cannot be loaded into memory totally, I can not write and attribute. &emsp; I would be grateful if you could develop a module that satisfy the cases above ? Since merely by python generator, it is too hard for me to write supporting multi-process and multi-thread loading and memory optimization. &emsp; By multi processing, one can be training the model on this batch and loading the next batch at the same time.
This is definitely a valid feature request. In fact, I implemented something called that will be used as an iterable (e.g., generator, data stream) in PyTorch. It is currently being reviewed at .
Hi Could you give an example how to use iterable datasets with distributed sampler to be able to efficiently sample datasets over multiple TPU cores? thanks

Discussion:
.. automethod:: fmod_ .. automethod:: frac .. automethod:: frac_ <del> .. automethod:: free </del> .. automethod:: gather .. automethod:: ge .. automethod:: ge_

.. automethod:: repeat .. automethod:: resize_ .. automethod:: resize_as_ <del> .. automethod:: retain </del> .. automethod:: round .. automethod:: round_ .. automethod:: rsqrt

]] [[ <del> name: free return: self arguments: - THTensor* self ]] [[ name: retain return: self arguments: - THTensor* self ]] [[ </del> python_name: index name: THPTensor_(getValue)<true> only_register: True
Change:
This error is too vague. Ideally we would print out the index we tried to access and the number of rows in the table we're trying to select out of. Right now it just says "index out of range" and multiple users have been confused by this.

Discussion:
.. autofunction:: cosh .. autofunction:: div .. autofunction:: exp <ins> .. autofunction:: floor .. autofunction:: fmod .. autofunction:: frac </ins> .. autofunction:: lerp .. autofunction:: log .. autofunction:: log1p

~~~~~~~~~~~~~~~~~~~~~~ .. autofunction:: eq .. autofunction:: equal <del> .. autofunction:: floor .. autofunction:: fmod .. autofunction:: frac </del> .. autofunction:: ge .. autofunction:: gt .. autofunction:: kthvalue

Modules can also contain other Modules, allowing to nest them in a tree structure. You can assign the submodules as regular attributes:: <ins> import torch.nn as nn import torch.nn.functional as F </ins> class Model(nn.Module): def __init__(self): super(Net, self).__init__()
Change:
This is of particular use in list comprehensions: This should work:
I'll need to adapt list comprehensions to use Iterables. I'll take a look!

Discussion:
# (3) initialize mean square values and square gradient storage if not 'm' in state: <del> state['m'] = x.new().resize_as_(dfdx).fill_(1) </del> <ins> state['m'] = x.new().resize_as_(dfdx).zero_() </ins> state['tmp'] = x.new().resize_as_(dfdx)

# State initialization if len(state) == 0: state['step'] = 0 <del> state['square_avg'] = grad.new().resize_as_(grad).fill_(1) </del> <ins> state['square_avg'] = grad.new().resize_as_(grad).zero_() </ins> square_avg = state['square_avg'] alpha = group['alpha']
Change:
The official website does not provide the torchvison version installation method. Here is the error message when torchvison 0.3.0 is used. torchvision/C.cpython-36m-x8664-linux- undefined symbol: ZN2at7getTypeERKNS6TensorE That should be caused by the mismatch version between pytorch-nightly and torchvision 0.3.0.
It should be that matches with , but they are not provided currently.
I tried installing it with pip as per , but now I get when importing . I did a similar thing on June 25th. It worked then. Any known fix for this?
do we document what to do here?
I finally solved this by building from source with Before running that make sure to upgrade and to use . Note: this will not work if you have on your system when building since you'd get version mismatch between and . Tested only on .
you build pytorch from source or just torchvision? or both of them.
I installed from conda with: I then cloned the repo and ran inside the repo. I guess this would also work if you do , but I haven't tried it.
Just a reminder (what I encountered just now): one should uninstall previous ` from source, or they can mix up and mess up everything.
There are nightlies for torchvision under Windows now.
I have problems using Pytorch with my Geforce RTX 3060, (sm86). I tried to install with help of : Create a new venv, based on python 3.7.10. Activate it. conda install pytorch-nightly cudatoolkit=10.0 -c pytorch (I also did it without the "=10.0") After this, Pytorch detects the sm86 with no errors, problem sm86 solved with pytorch-nightly Now, I have a problem with torchvision. tried installing torchvision with File "C:repovisiontorchvisionmodels", line 195, in Inception3 AttributeError: module '' has no attribute 'unused' tried installing torchvision version 0.3.0, from , but an error rose: tried installing torchvision at origin/master, from , but another error rose: Here is the list of packages installed:

Discussion:
import ctypes import warnings import torch.cuda <ins> import sys </ins> import os.path as path lib = None <del> # TODO: fix libname for OSX / Windows </del> <ins> # TODO: fix libname for Windows </ins> # TODO: dynamic version checks via cudnnGetVersion # TODO: load 5.1.3 if using CUDA 7.5 and 5.1.5 if using CUDA 8.0 thisdir = path.dirname(__file__) libpaths = ['', path.join(thisdir, '../../lib')] <del> libnames = ['libcudnn.so.5.1.5', 'libcudnn.so.5.1.3'] </del> <ins> if sys.platform.startswith('linux'): libnames = ['libcudnn.so.5.1.5', 'libcudnn.so.5.1.3'] elif sys.platform == 'darwin': libnames = ['libcudnn.5.dylib'] else: libnames = [] </ins> def _loadlib(): global lib

try: _loadlib() except Exception: <del> warnings.warn('cuDNN library not found. Check your LD_LIBRARY_PATH') </del> <ins> warnings.warn('cuDNN library not found. Check your {libpath}'.format( libpath={ 'darwin': 'DYLD_LIBRARY_PATH', 'win32': 'PATH' }.get(sys.platform, 'LD_LIBRARY_PATH'))) </ins> return False return True
Change:
The hub tests are broken on macOS. I'm seeing SSL certificate related errors when trying to hit GitHub. Run the macOS tests on CI with TestHub enabled. I tried installing the latest version of from conda-forge (which pulled in ) but this didn't fix the issue. cc
Looks like the same error I'm getting trying to use Ubuntu 18.04 with fresh install of conda and Pytorch 1.2
I think this will all be solved with .

Discussion:
# (3) initialize mean square values and square gradient storage if not 'm' in state: <del> state['m'] = x.new().resize_as_(dfdx).fill_(1) </del> <ins> state['m'] = x.new().resize_as_(dfdx).zero_() </ins> state['tmp'] = x.new().resize_as_(dfdx)

# State initialization if len(state) == 0: state['step'] = 0 <del> state['square_avg'] = grad.new().resize_as_(grad).fill_(1) </del> <ins> state['square_avg'] = grad.new().resize_as_(grad).zero_() </ins> square_avg = state['square_avg'] alpha = group['alpha']
Change:
We should unify this with our normal list creating code for the reasonable cases cc
Similar issue to this, I believe:

Discussion:
# (3) initialize mean square values and square gradient storage if not 'm' in state: <del> state['m'] = x.new().resize_as_(dfdx).fill_(1) </del> <ins> state['m'] = x.new().resize_as_(dfdx).zero_() </ins> state['tmp'] = x.new().resize_as_(dfdx)

# State initialization if len(state) == 0: state['step'] = 0 <del> state['square_avg'] = grad.new().resize_as_(grad).fill_(1) </del> <ins> state['square_avg'] = grad.new().resize_as_(grad).zero_() </ins> square_avg = state['square_avg'] alpha = group['alpha']
Change:
This is an older implementation that I think doesn't make any sense anymore. We should throw a NYI exception for it. The expected behavior for this should be:
fixed in

Discussion:
const auto& dim = tensor_names[idx]; TORCH_CHECK(dim.isBasic(), "align_to: All input dims must be named. Found unnamed dim at index ", <del> dim, " of Tensor", tensor_names); </del> <ins> idx, " of Tensor", tensor_names); </ins> auto it = std::find(names.begin(), names.end(), dim); TORCH_CHECK(it != names.end(), "align_to: Cannot find dim ", dim, " from Tensor", names,

self.assertEqual(output.shape, [3, 5, 1, 2]) # All input dimensions must be named <del> with self.assertRaisesRegex(RuntimeError, "All input dims must be named"): </del> <ins> with self.assertRaisesRegex(RuntimeError, "All input dims must be named. Found unnamed dim at index 0"): </ins> create('None:2,C:3').align_to('N', 'C') # not enough names

self.assertEqual(output.names, ['N', 'H', 'W', 'C']) self.assertEqual(output.shape, [3, 5, 1, 2]) <ins> @unittest.skip("Not implemented yet") </ins> def test_align_tensors_two_inputs(self): def _test(tensor_namedshape, align_names, expected_sizes, expected_error): tensor_names, tensor_sizes = tensor_namedshape

for test in tests: _test(*test) <ins> @unittest.skip("Not implemented yet") </ins> def test_align_tensors(self): def reference_fn(*tensors): longest_names = tensors[0].names

def align_tensors(*tensors): <del> if not torch._C._BUILD_NAMEDTENSOR: raise RuntimeError('NYI: torch.align_tensors is experimental and a part ' 'of our named tensors project.') return torch._C._VariableFunctions.align_tensors(tensors) </del> <ins> raise RuntimeError('`align_tensors` not yet implemented.') </ins>
Change:
only supports fully named inputs; all input dimensions must have a name. When passing it an unnamed input, it errors out with the following message: It should really say "Found unnamed dim at index 0 of Tensor[None, None]".
fixed in

Discussion:
const auto& dim = tensor_names[idx]; TORCH_CHECK(dim.isBasic(), "align_to: All input dims must be named. Found unnamed dim at index ", <del> dim, " of Tensor", tensor_names); </del> <ins> idx, " of Tensor", tensor_names); </ins> auto it = std::find(names.begin(), names.end(), dim); TORCH_CHECK(it != names.end(), "align_to: Cannot find dim ", dim, " from Tensor", names,

self.assertEqual(output.shape, [3, 5, 1, 2]) # All input dimensions must be named <del> with self.assertRaisesRegex(RuntimeError, "All input dims must be named"): </del> <ins> with self.assertRaisesRegex(RuntimeError, "All input dims must be named. Found unnamed dim at index 0"): </ins> create('None:2,C:3').align_to('N', 'C') # not enough names

self.assertEqual(output.names, ['N', 'H', 'W', 'C']) self.assertEqual(output.shape, [3, 5, 1, 2]) <ins> @unittest.skip("Not implemented yet") </ins> def test_align_tensors_two_inputs(self): def _test(tensor_namedshape, align_names, expected_sizes, expected_error): tensor_names, tensor_sizes = tensor_namedshape

for test in tests: _test(*test) <ins> @unittest.skip("Not implemented yet") </ins> def test_align_tensors(self): def reference_fn(*tensors): longest_names = tensors[0].names

def align_tensors(*tensors): <del> if not torch._C._BUILD_NAMEDTENSOR: raise RuntimeError('NYI: torch.align_tensors is experimental and a part ' 'of our named tensors project.') return torch._C._VariableFunctions.align_tensors(tensors) </del> <ins> raise RuntimeError('`align_tensors` not yet implemented.') </ins>
Change:
Documentation for does not indicate whether or not singular values are returned in descending order. My suspicion is that they are since both LAPACK and MAGMA's ?gesdd return them in descending order. It seems like this should be clarified in the docs to avoid unnecessary sorting, or mistaken assumptions. <!-- A clear and concise description of what content in is an issue. If this has to do with the general website, please file an issue at instead. If this has to do with , please file an issue at --

Discussion:
If :attr:`compute_uv` is ``False``, the returned `U` and `V` matrices will be zero matrices of shape :math:`(m times m)` and :math:`(n times n)` respectively. :attr:`some` will be ignored here. <ins> .. note:: The singular values are returned in descending order. If :attr:`input` is a batch of matrices, then the singular values of each matrix in the batch is returned in descending order. </ins> .. note:: The implementation of SVD on CPU uses the LAPACK routine `?gesdd` (a divide-and-conquer algorithm) instead of `?gesvd` for speed. Analogously, the SVD on GPU uses the MAGMA routine `gesdd` as well.

If :attr:`upper` is ``False``, then lower triangular portion is used. <ins> .. note:: The eigenvalues are returned in ascending order. If :attr:`input` is a batch of matrices, then the eigenvalues of each matrix in the batch is returned in ascending order. </ins> .. note:: Irrespective of the original strides, the returned matrix `V` will be transposed, i.e. with strides `V.contiguous().transpose(-1, -2).stride()`.
Change:
As part of pull request , F is redefined globally in nn/modules/batchnorm.h F is then used once in batchnorm.h and again in instancenorm.h not a bug, but a problem for anyone using F in some other way. it's solvable, but I don't see this pattern anywhere else in libtorch, was wondering if this use is an anomaly, and if we could use thanks cc
Thanks a lot for reporting! Yes this is an anomaly, and will be fixed by

Discussion:
#include <cstdint> <del> namespace F = torch::nn::functional; </del> namespace torch { namespace nn {

} } <del> return F::detail::batch_norm( </del> <ins> return torch::nn::functional::detail::batch_norm( </ins> input, this->running_mean, this->running_var,

Tensor forward(const Tensor& input) { this->_check_input_dim(input); <del> return F::detail::instance_norm( </del> <ins> return torch::nn::functional::detail::instance_norm( </ins> input, this->running_mean, this->running_var, this->weight, this->bias, this->is_training() || !this->options.track_running_stats(), this->options.momentum(), this->options.eps()); }
Change:
! It seems that the highlighted "below" should be "upper“. <!-- A clear and concise description of what content in is an issue. If this has to do with the general website, please file an issue at instead. If this has to do with , please file an issue at --
Hi, thank you for bringing this up. I'll send in a patch to fix it.

Discussion:
above the diagonal. The argument :attr:`diagonal` controls which diagonal to consider. If <del> :attr:`diagonal` = 0, all elements on and below the main diagonal are </del> <ins> :attr:`diagonal` = 0, all elements on and above the main diagonal are </ins> retained. A positive value excludes just as many diagonals above the main diagonal, and similarly a negative value includes just as many diagonals below the main diagonal. The main diagonal are the set of indices
Change:
Just a very small grammatical issue. ! Suggested fix: "mutates" to "mutate" <!-- A clear and concise description of what content in is an issue. If this has to do with the general website, please file an issue at instead. If this has to do with , please file an issue at --
Thanks for pointing this out. Please feel free to open a pull request for this.
Thanks for adding labels. Forgot to do that. I will make a pull request

Discussion:
'Cumprod', 'Gather', 'MaskedScatter', <del> 'Sort', 'Topk', </del> 'Norm', 'Prod', ))

return output @staticmethod <del> @once_differentiable </del> def backward(ctx, grad_output, grad_indices=None): <del> grad_input = grad_output.new(ctx.input_size).zero_() </del> <ins> grad_input = Variable(grad_output.data.new(ctx.input_size).zero_(), requires_grad=True) </ins> if ctx.return_indices: indices, = ctx.saved_tensors else: indices = ctx.indices dim = ctx.dim if ctx.dim is not None else grad_output.dim() - 1 <del> return (grad_input.scatter_(dim, indices, grad_output),) + (None,) * ctx.num_flags </del> <ins> return (grad_input.scatter(dim, Variable(indices), grad_output),) + (None,) * ctx.num_flags </ins> class Sort(_MultiSelectionFunction):
Change:
with , both max and min are broken <img width="880" alt="Screen Shot 2020-03-23 at 1 10 07 PM" src="https://user-"on , min is broken <img width="875" alt="Screen Shot 2020-03-23 at 1 13 23 PM" src="https://user-"
Fixed by and

Discussion:
Otherwise, :attr:`dim` is squeezed (see :func:`torch.squeeze`), resulting in the output tensor having 1 fewer dimension than :attr:`input`."""}) factory_common_args = merge_dicts(common_args, parse_kwargs(""" dtype (:class:`torch.dtype`, optional): the desired data type of returned tensor. Default: if ``None``, uses a global default (see :func:`torch.set_default_tensor_type`).

but they must be :ref:`broadcastable <broadcasting-semantics>`. .. math:: <del> text{out}_i = min(text{tensor}_i, text{other}_i) """ + r""" </del> <ins> text{{out}}_i = min(text{{tensor}}_i, text{{other}}_i) </ins> .. note:: When the shapes do not match, the shape of the returned output tensor follows the :ref:`broadcasting rules <broadcasting-semantics>`.
Change:
See e.g.: cc
Can we close it?
The build is passing now, although the tests might be failing (although it might have been an environment failure).

Discussion:
gtest_reports_dir="${TEST_DIR}/cpp" pytest_reports_dir="${TEST_DIR}/python" <ins> # This is needed to work around ROCm using old docker images until # the transition to new images is complete. # TODO: Remove once ROCm CI is using new images. if [[ $BUILD_ENVIRONMENT == py3.6-devtoolset7-rocmrpm-centos* ]]; then # This file is sourced multiple times, only install conda the first time. # We must install conda where we have write access. CONDA_DIR="$ROOT_DIR/conda" if [[ ! -d $CONDA_DIR ]]; then ANACONDA_PYTHON_VERSION=3.6 BASE_URL="https://repo.anaconda.com/miniconda" CONDA_FILE="Miniconda3-latest-Linux-x86_64.sh" mkdir $CONDA_DIR pushd /tmp wget -q "${BASE_URL}/${CONDA_FILE}" chmod +x "${CONDA_FILE}" ./"${CONDA_FILE}" -b -f -p "$CONDA_DIR" popd export PATH="$CONDA_DIR/bin:$PATH" # Ensure we run conda in a directory that jenkins has write access to pushd $CONDA_DIR # Track latest conda update conda update -n base conda # Install correct Python version conda install python="$ANACONDA_PYTHON_VERSION" conda_install() { # Ensure that the install command don't upgrade/downgrade Python # This should be called as #   conda_install pkg1 pkg2 ... [-c channel] conda install -q -y python="$ANACONDA_PYTHON_VERSION" $* } # Install PyTorch conda deps, as per https://github.com/pytorch/pytorch README conda_install numpy pyyaml mkl mkl-include setuptools cffi typing future six # TODO: This isn't working atm conda_install nnpack -c killeent # Install some other packages # Need networkx 2.0 because bellmand_ford was moved in 2.1 . Scikit-image by # defaults installs the most recent networkx version, so we install this lower # version explicitly before scikit-image pulls it in as a dependency pip install networkx==2.0 # TODO: Why is scipy pinned # numba & llvmlite is pinned because of https://github.com/numba/numba/issues/4368 # scikit-learn is pinned because of # https://github.com/scikit-learn/scikit-learn/issues/14485 (affects gcc 5.5 # only) pip install --progress-bar off pytest scipy==1.1.0 scikit-learn==0.20.3 scikit-image librosa>=0.6.2 psutil numba==0.46.0 llvmlite==0.30.0 # click - onnx # hypothesis - tests # jupyter - for tutorials pip install --progress-bar off click hypothesis jupyter protobuf tabulate virtualenv mock typing-extensions popd else export PATH="$CONDA_DIR/bin:$PATH" fi fi </ins> # Figure out which Python to use PYTHON="$(which python)" if [[ "${BUILD_ENVIRONMENT}" =~ py((2|3).?[0-9]?.?[0-9]?) ]]; then
Change:
I followed the guide The command was working just fine until a couple days ago, but now it returns I manually checked and there is no cu100/[...] What should I do? I have CUDA 10.0 I would like to use pytorch...
You could try as a temporary workground. cc
This is most likely related to this: I've re-uploaded the stable html to re-include the cu100 binaries, and submitted a PR to make sure this doesn't happen again

Discussion:
array = np.array([1, 2, 3, 4], dtype=dtype) self.assertEqual(torch.from_numpy(array), torch.Tensor([1, 2, 3, 4])) <ins> # check storage offset x = np.linspace(1, 125, 125) x.shape = (5, 5, 5) x = x[1] expected = torch.range(1, 125).view(5, 5, 5)[1] self.assertEqual(torch.from_numpy(x), expected) # check noncontiguous x = np.linspace(1, 25, 25) x.shape = (5, 5) expected = torch.range(1, 25).view(5, 5).t() self.assertEqual(torch.from_numpy(x.T), expected) # check noncontiguous with holes x = np.linspace(1, 125, 125) x.shape = (5, 5, 5) x = x[:, 1] expected = torch.range(1, 125).view(5, 5, 5)[:, 1] self.assertEqual(torch.from_numpy(x), expected) </ins> @unittest.skipIf(not TEST_NUMPY, "Numpy not found") def test_numpy_index(self): i = np.int32([0, 1, 2])

#ifdef NUMPY_TYPE_ENUM THTensor* THPTensor_(fromNumpy)(PyObject *numpy_array) { PyArrayObject *array = (PyArrayObject*)numpy_array; <del> THStoragePtr storage = THStorage_(newWithDataAndAllocator)( (real*)PyArray_DATA(array), PyArray_NBYTES(array) / sizeof(real), &THNumpyArrayAllocator, new NumpyArrayAllocator(numpy_array)); </del> // Numpy and Torch disagree on empty tensors. In Torch, an empty // tensor is a tensor with zero dimensions. In Numpy, an empty tensor

// So we'll convert all Numpy tensors of 0 elements to empty Torch tensors. if (PyArray_SIZE(array) != 0) { auto ndim = PyArray_NDIM(array); <ins> size_t storage_size = 1; </ins> THLongStoragePtr sizes = THLongStorage_newWithSize(ndim); long *sizes_data = sizes->data; for (int i = 0; i < ndim; ++i) {

"future releases."); return NULL; } <ins> // XXX: this won't work for negative strides storage_size += strides_data[i] * (sizes_data[i] - 1); </ins> } <ins> THStoragePtr storage = THStorage_(newWithDataAndAllocator)( (real*)PyArray_DATA(array), storage_size, &THNumpyArrayAllocator, new NumpyArrayAllocator(numpy_array)); </ins> THTensor *result = THTensor_(newWithStorage)(storage, 0, sizes, strides); return result; } else { <ins> THStoragePtr storage = THStorage_(new)(); </ins> THTensor *result = THTensor_(newWithStorage)(storage, 0, NULL, NULL); return result; }