internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pytorch/audio

Issue:
Linting doc config
Currently, `doc/source/conf.py` is not checked with `flake8` in our CI and it contains errors. We need to fix this and make CI check it.

```
$ flake8 docs/source/conf.py
docs/source/conf.py:23:1: F401 'torch' imported but unused
docs/source/conf.py:24:1: F401 'torchaudio' imported but unused
docs/source/conf.py:138:1: E302 expected 2 blank lines, found 1
docs/source/conf.py:229:5: F821 undefined name 'List'
docs/source/conf.py:229:5: F821 undefined name 'unicode'
docs/source/conf.py:229:5: F821 undefined name 'Tuple'
```

## Steps
1. Fix the above error.
2. Add `docs/source/conf.py` [here](https://github.com/pytorch/audio/blob/ea857940de9e3738166989ad3bf1726741a13f04/.circleci/unittest/linux/scripts/run_style_checks.sh#L32)

## Build and testing

For setting up dev env, please refer to [CONTRIBUTING.md](https://github.com/pytorch/audio/blob/master/CONTRIBUTING.md).



Discussion:
will look into this

Resolving patch:
diff --git a/docs/source/conf.py b/docs/source/conf.py
--- a/docs/source/conf.py
+++ b/docs/source/conf.py
@@ -20,8 +20,6 @@
 # import os
 # import sys
 # sys.path.insert(0, os.path.abspath('.'))
-import torch
-import torchaudio
 import pytorch_sphinx_theme
 
 # -- General configuration ------------------------------------------------
@@ -135,6 +133,7 @@
 # so a file named "default.css" will overwrite the builtin "default.css".
 html_static_path = ['_static']
 
+
 def setup(app):
     # NOTE: in Sphinx 1.8+ `html_css_files` is an official configuration value
     # and can be moved outside of this function (and the setup(app) function
@@ -145,7 +144,7 @@ def setup(app):
 
     # In Sphinx 1.8 it was renamed to `add_css_file`, 1.7 and prior it is
     # `add_stylesheet` (deprecated in 1.8).
-    add_css = getattr(app, 'add_css_file', getattr(app, 'add_stylesheet'))
+    add_css = getattr(app, 'add_css_file', app.add_stylesheet)
     for css_file in html_css_files:
         add_css(css_file)
 
@@ -226,7 +225,7 @@ def patched_make_field(self, types, domain, items, **kw):
     # `kw` catches `env=None` needed for newer sphinx while maintaining
     #  backwards compatibility when passed along further down!
 
-    # type: (List, unicode, Tuple) -> nodes.field
+    # type: (list, str, tuple) -> nodes.field
     def handle_item(fieldarg, content):
         par = nodes.paragraph()
         par += addnodes.literal_strong('', fieldarg)  # Patch: this line added


Test patch:
Repository: pytorch/audio

Issue:
Unused constant attribute in `T.GriffinLim`
`T.GriffinLim` has constant attribute `normalized`, which does not have the corresponding ctor argument.

https://github.com/pytorch/audio/blob/8d2eeb114e7f8a6659d132b98adb50d886108fb2/torchaudio/transforms.py#L158

This is issuing a warning: `/torch/jit/_recursive.py:227: UserWarning: 'normalized' was found in ScriptModule constants, but was not actually set in __init__. Consider removing it.`.

We should remove the unused `normalized` attribute. from `__constants__`.

# Steps
1. Remove `normalized` from `T.GriffinLim.__constants__`.

# Build and Test
Please refer to [the CONTRIBUTING.md](https://github.com/pytorch/audio/blob/master/CONTRIBUTING.md) for build and test. All the test related to `GriffinLim` should keep passing. 

```bash
(cd test && pytest torchaudio_unittest -k griffin)
```


Discussion:

Resolving patch:
diff --git a/torchaudio/transforms.py b/torchaudio/transforms.py
--- a/torchaudio/transforms.py
+++ b/torchaudio/transforms.py
@@ -155,7 +155,7 @@ class GriffinLim(torch.nn.Module):
            | "Signal estimation from modified short-time Fourier transform,"
            | IEEE Trans. ASSP, vol.32, no.2, pp.236–243, Apr. 1984.
     """
-    __constants__ = ['n_fft', 'n_iter', 'win_length', 'hop_length', 'power', 'normalized',
+    __constants__ = ['n_fft', 'n_iter', 'win_length', 'hop_length', 'power',
                      'length', 'momentum', 'rand_init']
 
     def __init__(self,
@@ -768,6 +768,7 @@ class Fade(torch.nn.Module):
         fade_shape (str, optional): Shape of fade. Must be one of: "quarter_sine",
             "half_sine", "linear", "logarithmic", "exponential". (Default: ``"linear"``)
     """
+
     def __init__(self,
                  fade_in_len: int = 0,
                  fade_out_len: int = 0,
@@ -877,6 +878,7 @@ class FrequencyMasking(_AxisMasking):
             example/channel in the batch. (Default: ``False``)
             This option is applicable only when the input tensor is 4D.
     """
+
     def __init__(self, freq_mask_param: int, iid_masks: bool = False) -> None:
         super(FrequencyMasking, self).__init__(freq_mask_param, 1, iid_masks)
 
@@ -891,6 +893,7 @@ class TimeMasking(_AxisMasking):
             example/channel in the batch. (Default: ``False``)
             This option is applicable only when the input tensor is 4D.
     """
+
     def __init__(self, time_mask_param: int, iid_masks: bool = False) -> None:
         super(TimeMasking, self).__init__(time_mask_param, 2, iid_masks)
 


Test patch:
Repository: pytorch/audio

Issue:
Remove unused module
[`torchaudio._internal.misc_ops`](https://github.com/pytorch/audio/blob/b059f08742e70700ce4c92296a1131118f67a588/torchaudio/_internal/misc_ops.py) is a residue from refactoring of I/O features in the past releases. We can get rid of the whole module.




Discussion:
Hi @mthrok ,

I would like to work on this issue. As per my understanding, I think we have to remove the [misc_ops.py](https://github.com/pytorch/audio/blob/b059f08742e70700ce4c92296a1131118f67a588/torchaudio/_internal/misc_ops.py) file from the project. If I got your issue correct then please do assign this issue to me or please do explain what I have to do. It will be cool if you assign me the issue after explaining the issue in a bit of detail.

Best,
Abhishek
Hi @itsabhianant 

Thanks for the signing up. Yes, your understanding is correct. It's as simple as removing `misc_ops.py`. please go ahead and make a PR and add `Closes #1464 cc @mthrok` in the description.

Resolving patch:
diff --git a/torchaudio/_internal/misc_ops.py b/torchaudio/_internal/misc_ops.py
deleted file mode 100644
--- a/torchaudio/_internal/misc_ops.py
+++ /dev/null
@@ -1,30 +0,0 @@
-from typing import Union, Callable
-
-import torch
-from torch import Tensor
-
-
-def normalize_audio(signal: Tensor, normalization: Union[bool, float, Callable]) -> None:
-    """Audio normalization of a tensor in-place.  The normalization can be a bool,
-    a number, or a callable that takes the audio tensor as an input. SoX uses
-    32-bit signed integers internally, thus bool normalizes based on that assumption.
-    """
-
-    if not normalization:
-        return
-
-    if isinstance(normalization, bool):
-        normalization = 1 << 31
-
-    if isinstance(normalization, (float, int)):
-        # normalize with custom value
-        signal /= normalization
-    elif callable(normalization):
-        signal /= normalization(signal)
-
-
-def check_input(src: Tensor) -> None:
-    if not torch.is_tensor(src):
-        raise TypeError('Expected a tensor, got %s' % type(src))
-    if src.is_cuda:
-        raise TypeError('Expected a CPU based tensor, got %s' % type(src))


Test patch:
Repository: pytorch/audio

Issue:
Tacotron2 does not handle batched input correctly
When generating mel spectrogram, `Tacotron2.infer` behaves differently when there is multiple samples are in the input batch.

The cause is that the termination condition is not properly handled for batch.

https://github.com/pytorch/audio/blob/19f53cf249d69c81be3005d582d37530f0a3aef7/torchaudio/models/tacotron2.py#L970-L972



Discussion:

Resolving patch:
diff --git a/torchaudio/models/tacotron2.py b/torchaudio/models/tacotron2.py
--- a/torchaudio/models/tacotron2.py
+++ b/torchaudio/models/tacotron2.py
@@ -904,6 +904,8 @@ def infer(self,
             alignments (Tensor): Sequence of attention weights from the decoder
                 with shape (n_batch,  max of ``mel_specgram_lengths``, max of ``text_lengths``).
         """
+        batch_size, device = memory.size(0), memory.device
+
         decoder_input = self._get_go_frame(memory)
 
         mask = _get_mask_from_lengths(memory_lengths)
@@ -918,17 +920,12 @@ def infer(self,
             processed_memory,
         ) = self._initialize_decoder_states(memory)
 
-        mel_specgram_lengths = torch.ones(
-            [memory.size(0)], dtype=torch.int32, device=memory.device
-        )
-        not_finished = torch.ones(
-            [memory.size(0)], dtype=torch.int32, device=memory.device
-        )
-
+        mel_specgram_lengths = torch.zeros([batch_size], dtype=torch.int32, device=device)
+        finished = torch.zeros([batch_size], dtype=torch.bool, device=device)
         mel_specgrams: List[Tensor] = []
         gate_outputs: List[Tensor] = []
         alignments: List[Tensor] = []
-        while True:
+        for _ in range(self.decoder_max_step):
             decoder_input = self.prenet(decoder_input)
             (
                 mel_specgram,
@@ -957,20 +954,19 @@ def infer(self,
             mel_specgrams.append(mel_specgram.unsqueeze(0))
             gate_outputs.append(gate_output.transpose(0, 1))
             alignments.append(attention_weights)
+            mel_specgram_lengths[~finished] += 1
 
-            dec = torch.le(torch.sigmoid(gate_output), self.gate_threshold).to(torch.int32).squeeze(1)
-
-            not_finished = not_finished * dec
-
-            if self.decoder_early_stopping and torch.sum(not_finished) == 0:
-                break
-            if len(mel_specgrams) == self.decoder_max_step:
-                warnings.warn("Reached max decoder steps")
+            finished |= torch.sigmoid(gate_output.squeeze(1)) > self.gate_threshold
+            if self.decoder_early_stopping and torch.all(finished):
                 break
 
-            mel_specgram_lengths += not_finished
             decoder_input = mel_specgram
 
+        if len(mel_specgrams) == self.decoder_max_step:
+            warnings.warn(
+                "Reached max decoder steps. The generated spectrogram might not cover "
+                "the whole transcript.")
+
         mel_specgrams = torch.cat(mel_specgrams, dim=0)
         gate_outputs = torch.cat(gate_outputs, dim=0)
         alignments = torch.cat(alignments, dim=0)


Test patch:
Repository: easybuilders/easybuild-easyblocks

Issue:
SuiteSparse: permissions issue
```
$ find . 2>&1 | grep Permission
find: ‘./GPUQREngine/Demo’: Permission denied
find: ‘./GPUQREngine/Doc’: Permission denied
find: ‘./GPUQREngine/Lib’: Permission denied
find: ‘./GPUQREngine/Include’: Permission denied
find: ‘./GPUQREngine/Source’: Permission denied
find: ‘./SPQR/Matrix’: Permission denied
find: ‘./SPQR/Demo’: Permission denied
find: ‘./SPQR/Tcov’: Permission denied
find: ‘./SPQR/MATLAB’: Permission denied
find: ‘./SPQR/SPQRGPU’: Permission denied
find: ‘./SPQR/Doc’: Permission denied
find: ‘./SPQR/Lib’: Permission denied
find: ‘./SPQR/Include’: Permission denied
find: ‘./SPQR/Source’: Permission denied
find: ‘./SuiteSparse_GPURuntime/Doc’: Permission denied
find: ‘./SuiteSparse_GPURuntime/Lib’: Permission denied
find: ‘./SuiteSparse_GPURuntime/Include’: Permission denied
find: ‘./SuiteSparse_GPURuntime/Source’: Permission denied

$ ls -ld SPQR
drwxr--r-- 11 ebuser ebuser 4096 Mar 30 12:08 SPQR
```
we're missing x permissions on 3 directories. I will check a bit further later.


Discussion:

Resolving patch:
diff --git a/easybuild/easyblocks/o/openfoam.py b/easybuild/easyblocks/o/openfoam.py
--- a/easybuild/easyblocks/o/openfoam.py
+++ b/easybuild/easyblocks/o/openfoam.py
@@ -257,10 +257,12 @@ def build_step(self):
         """Build OpenFOAM using make after sourcing script to set environment."""
 
         precmd = "source %s" % os.path.join(self.builddir, self.openfoamdir, "etc", "bashrc")
+        cleancmd = "wcleanAll"
 
         # make directly in install directory
-        cmd_tmpl = "%(precmd)s && %(prebuildopts)s %(makecmd)s" % {
+        cmd_tmpl = "%(precmd)s && %(cleancmd)s && %(prebuildopts)s %(makecmd)s" % {
             'precmd': precmd,
+            'cleancmd': cleancmd,
             'prebuildopts': self.cfg['prebuildopts'],
             'makecmd': os.path.join(self.builddir, self.openfoamdir, '%s'),
         }


Test patch:
Repository: easybuilders/easybuild-easyblocks

Issue:
Offline QuantumEspresso installation not possible 
It turns out that QuantumEspresso has a (new? I'm using 6.0) feature to download the necessary sources for the extensions. Since this is not handled by EB, on an offline node you end up with the following error:
```
*** Unable to download qe-gipaw. Test whether curl or wget is installed and working,
*** if you have direct access to internet. If not, copy into archive/ the file
*** located here http://files.qe-forge.org/index.php?file=qe-gipaw-6.0.tar.gz
```
If your node is online you won't notice this



Discussion:
Ah, it turns out that gipaw needs to be added as one of the special cases, the directory created begins with `qe-gipaw` not `gipaw` as the easybock currently expects.

...and it's an awkward one because  the `dirname_head` is malformed since the stub already contains a `-`


Resolving patch:
diff --git a/easybuild/easyblocks/q/quantumespresso.py b/easybuild/easyblocks/q/quantumespresso.py
--- a/easybuild/easyblocks/q/quantumespresso.py
+++ b/easybuild/easyblocks/q/quantumespresso.py
@@ -242,6 +242,9 @@ def configure_step(self):
                 self.log.info("Moved %s into %s" % (dirname, targetdir))
 
                 dirname_head = dirname.split('-')[0]
+                # Handle the case where the directory is preceded by 'qe-'
+                if dirname_head == 'qe':
+                    dirname_head = dirname.split('-')[1]
                 linkname = None
                 if dirname_head == 'sax':
                     linkname = 'SaX'


Test patch:
Repository: easybuilders/easybuild-easyblocks

Issue:
SuiteSparse: permissions issue
```
$ find . 2>&1 | grep Permission
find: ‘./GPUQREngine/Demo’: Permission denied
find: ‘./GPUQREngine/Doc’: Permission denied
find: ‘./GPUQREngine/Lib’: Permission denied
find: ‘./GPUQREngine/Include’: Permission denied
find: ‘./GPUQREngine/Source’: Permission denied
find: ‘./SPQR/Matrix’: Permission denied
find: ‘./SPQR/Demo’: Permission denied
find: ‘./SPQR/Tcov’: Permission denied
find: ‘./SPQR/MATLAB’: Permission denied
find: ‘./SPQR/SPQRGPU’: Permission denied
find: ‘./SPQR/Doc’: Permission denied
find: ‘./SPQR/Lib’: Permission denied
find: ‘./SPQR/Include’: Permission denied
find: ‘./SPQR/Source’: Permission denied
find: ‘./SuiteSparse_GPURuntime/Doc’: Permission denied
find: ‘./SuiteSparse_GPURuntime/Lib’: Permission denied
find: ‘./SuiteSparse_GPURuntime/Include’: Permission denied
find: ‘./SuiteSparse_GPURuntime/Source’: Permission denied

$ ls -ld SPQR
drwxr--r-- 11 ebuser ebuser 4096 Mar 30 12:08 SPQR
```
we're missing x permissions on 3 directories. I will check a bit further later.


Discussion:
@bartoldeman Any updates on this?

Resolving patch:
diff --git a/easybuild/easyblocks/s/suitesparse.py b/easybuild/easyblocks/s/suitesparse.py
--- a/easybuild/easyblocks/s/suitesparse.py
+++ b/easybuild/easyblocks/s/suitesparse.py
@@ -36,11 +36,12 @@
 import os
 import shutil
 import sys
+import stat
 from distutils.version import LooseVersion
 
 from easybuild.easyblocks.generic.configuremake import ConfigureMake
 from easybuild.tools.build_log import EasyBuildError
-from easybuild.tools.filetools import mkdir, write_file
+from easybuild.tools.filetools import mkdir, write_file, adjust_permissions
 from easybuild.tools.modules import get_software_root
 from easybuild.tools.modules import get_software_libdir
 from easybuild.tools.systemtools import get_shared_lib_ext
@@ -142,6 +143,9 @@ def install_step(self):
                         ndst = os.path.join(dst, c.lower())
                         if os.path.exists(nsrc):
                             os.symlink(nsrc, ndst)
+                    # enable r-x permissions for group/others
+                    perms = stat.S_IRGRP|stat.S_IXGRP|stat.S_IROTH|stat.S_IXOTH
+                    adjust_permissions(dst, perms, add=True, recursive=True, onlydirs=True)
                 else:
                     shutil.copy2(src, dst)
             except OSError, err:


Test patch:
Repository: easybuilders/easybuild-easyblocks

Issue:
CUDA stubs path missing
In order for the NVML and other libraries to be found, one needs to add 
$CUDA_ROOT/lib64/stubs to $LIBRARY_PATH (not $LD_LIBRARY_PATH)




Discussion:
That is, in cases where the CUDA driver is not installed on the node where the compilation happens.
That was recently reported in https://github.com/easybuilders/easybuild-easyconfigs/issues/6602
Is there perhaps a race to see who will issue the PR first? :P
@ocaisa sorry, I should have searched better. Do I close this one then? Andreas is doing the PR as I write.

Resolving patch:
diff --git a/easybuild/easyblocks/c/cuda.py b/easybuild/easyblocks/c/cuda.py
--- a/easybuild/easyblocks/c/cuda.py
+++ b/easybuild/easyblocks/c/cuda.py
@@ -160,10 +160,14 @@ def make_module_req_guess(self):
         if LooseVersion(self.version) >= LooseVersion('7'):
             lib_path.append('extras/CUPTI/lib64')
             inc_path.append('extras/CUPTI/include')
+            bin_path.append('nvvm/bin')
+            lib_path.append('nvvm/lib64')
+            inc_path.append('nvvm/include')
 
         guesses.update({
             'PATH': bin_path,
             'LD_LIBRARY_PATH': lib_path,
+            'LIBRARY_PATH': ['lib64', 'lib64/stubs'],
             'CPATH': inc_path,
             'CUDA_HOME': [''],
             'CUDA_ROOT': [''],


Test patch:
Repository: easybuilders/easybuild-easyblocks

Issue:
PGI doesn't accept "-pthread" switch
I'm trying to compile OpenMPI 2.1.2 with PGI 18.4 and I stumbled upon a pgi error saying that it doen't accept -pthread switch

After few research I saw this thread: https://www.pgroup.com/userforum/viewtopic.php?t=5253&sid=93356f57a47eee097b569b79bc744676
basicaly PGI accept -lpthread instead
and one solution for PGI to accept -pthread would be to modify the siterc in $PGI_INSTALL/bin/siterc


Discussion:

Resolving patch:
diff --git a/easybuild/easyblocks/p/pgi.py b/easybuild/easyblocks/p/pgi.py
--- a/easybuild/easyblocks/p/pgi.py
+++ b/easybuild/easyblocks/p/pgi.py
@@ -64,6 +64,12 @@
 append LDLIBARGS=-L/usr/lib/x86_64-linux-gnu;
 """
 
+# contents for siterc file to make PGI accept the -pthread switch
+SITERC_PTHREAD_SWITCH = """
+# replace unknown switch -pthread with -lpthread
+switch -pthread is replace(-lpthread) positional(linker);
+"""
+
 
 class EB_PGI(PackedBinary):
     """
@@ -144,11 +150,14 @@ def install_step(self):
                 if os.path.islink(path):
                     os.remove(path)
 
-        # install (or update) siterc file to make PGI consider $LIBRARY_PATH
+        # install (or update) siterc file to make PGI consider $LIBRARY_PATH and accept -pthread
         siterc_path = os.path.join(self.installdir, self.pgi_install_subdir, 'bin', 'siterc')
         write_file(siterc_path, SITERC_LIBRARY_PATH, append=True)
         self.log.info("Appended instructions to pick up $LIBRARY_PATH to siterc file at %s: %s",
                       siterc_path, SITERC_LIBRARY_PATH)
+        write_file(siterc_path, SITERC_PTHREAD_SWITCH, append=True)
+        self.log.info("Append instructions to replace -pthread with -lpthread to siterc file at %s: %s",
+                      siterc_path, SITERC_PTHREAD_SWITCH)
 
     def sanity_check_step(self):
         """Custom sanity check for PGI"""


Test patch:
Repository: easybuilders/easybuild-easyblocks

Issue:
Make CMakeMake easyblock immune to system boost.
As mentioned in 
https://github.com/easybuilders/easybuild-easyconfigs/pull/7149
if an old boost-devel rpm installed on CentOS6:
```
boost-devel-1.41.0-25.el6.centos.x86_64

$ rpm -ql boost-devel-1.41.0-25.el6.centos.x86_64 | grep -i cmake
/usr/lib64/boost/Boost-relwithdebinfo.cmake
/usr/lib64/boost/Boost.cmake
/usr/lib64/boost/BoostConfig.cmake
/usr/lib64/boost/BoostConfigVersion.cmake

-- Boost  found.
Boost Include: /usr/include
Boost Linkdir: /usr/lib64
```
then anything that combines cmake and boost gets confused by this as the EB boost does not include any cmake files.

adding `-DBoost_NO_SYSTEM_PATHS=ON -DBoost_NO_BOOST_CMAKE=ON`
avoids this.

ref:
https://github.com/Kitware/CMake/blob/master/Modules/FindBoost.cmake
(which shows that if it finds a Boost-cmake config file, it's game over)
https://cmake.org/cmake/help/v3.12/module/FindBoost.html


Discussion:

Resolving patch:
diff --git a/easybuild/easyblocks/generic/cmakemake.py b/easybuild/easyblocks/generic/cmakemake.py
--- a/easybuild/easyblocks/generic/cmakemake.py
+++ b/easybuild/easyblocks/generic/cmakemake.py
@@ -40,6 +40,7 @@
 from easybuild.tools.config import build_option
 from easybuild.tools.filetools import change_dir, mkdir, which
 from easybuild.tools.environment import setvar
+from easybuild.tools.modules import get_software_root
 from easybuild.tools.run import run_cmd
 from vsc.utils.missing import nub
 
@@ -56,6 +57,8 @@ def extra_options(extra_vars=None):
         extra_vars = ConfigureMake.extra_options(extra_vars)
         extra_vars.update({
             'abs_path_compilers': [False, "Specify compilers via absolute file path (not via command names)", CUSTOM],
+            'allow_system_boost': [False, "Always allow CMake to pick up on Boost installed in OS "
+                                          "(even if Boost is included as a dependency)", CUSTOM],
             'configure_cmd': [DEFAULT_CONFIGURE_CMD, "Configure command to use", CUSTOM],
             'srcdir': [None, "Source directory location to provide to cmake command", CUSTOM],
             'separate_build_dir': [False, "Perform build in a separate directory", CUSTOM],
@@ -116,6 +119,19 @@ def configure_step(self, srcdir=None, builddir=None):
         # show what CMake is doing by default
         options.append('-DCMAKE_VERBOSE_MAKEFILE=ON')
 
+        if not self.cfg.get('allow_system_boost', False):
+            # don't pick up on system Boost if Boost is included as dependency
+            # - specify Boost location via -DBOOST_ROOT
+            # - instruct CMake to not search for Boost headers/libraries in other places
+            # - disable search for Boost CMake package configuration file
+            boost_root = get_software_root('Boost')
+            if boost_root:
+                options.extend([
+                    '-DBOOST_ROOT=%s' % boost_root,
+                    '-DBoost_NO_SYSTEM_PATHS=ON',
+                    '-DBoost_NO_BOOST_CMAKE=ON',
+                ])
+
         options_string = ' '.join(options)
 
         command = ' '.join([


Test patch: