internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: MycroftAI/mycroft-core

Issue:
FileNotFoundError on Skill removal
**Describe the bug**
I hit this error when doing an uninstall using PR #2803 
```
2021-02-04 11:35:41.063 | ERROR    | 20080 | mycroft.skills.skill_manager:run:260 | Something really unexpected has occured and the skill manager loop safety harness was hit.
Traceback (most recent call last):
  File "/home/gez/mycroft-core/mycroft/skills/skill_manager.py", line 248, in run
    self._load_new_skills()
  File "/home/gez/mycroft-core/mycroft/skills/skill_manager.py", line 296, in _load_new_skills
    for skill_dir in self._get_skill_directories():
  File "/home/gez/mycroft-core/mycroft/skills/skill_manager.py", line 322, in _get_skill_directories
    if SKILL_MAIN_MODULE in os.listdir(skill_dir):
FileNotFoundError: [Errno 2] No such file or directory: '/home/gez/.local/share/mycroft/skills/youtube-music-skill.forslund/'
2021-02-04 11:36:11.184 | INFO     | 20080 | mycroft.skills.skill_manager:_unload_removed_skills:338 | removing youtube-music-skill.forslund
```

The directory did exist in the new location, and was removed correctly, so it's possible that this is unrelated to that PR and the hot reloader tried to load the Skill as it was being removed which could have happened regardless of where it was located.

Currently when we run the SkillManager the order of operations is:
```
                self._reload_modified_skills()
                self._load_new_skills()
                self._unload_removed_skills()
                self._update_skills()
```

Wondering if we just need to reorder to make sure that we first unload removed skills.

**To Reproduce**
Quite hard to reproduce reliably...

**Expected behavior**
No exception should be raised when removing a Skill.


Discussion:

Resolving patch:
diff --git a/mycroft/skills/skill_manager.py b/mycroft/skills/skill_manager.py
--- a/mycroft/skills/skill_manager.py
+++ b/mycroft/skills/skill_manager.py
@@ -240,9 +240,9 @@ def run(self):
         # unload the existing version from memory and reload from the disk.
         while not self._stop_event.is_set():
             try:
+                self._unload_removed_skills()
                 self._reload_modified_skills()
                 self._load_new_skills()
-                self._unload_removed_skills()
                 self._update_skills()
                 if (is_paired() and self.upload_queue.started and
                         len(self.upload_queue) > 0):


Test patch:
Repository: MycroftAI/mycroft-core

Issue:
tts cache creates "None" folder
the latest cache refactor is creating a folder named "None" due to the handling of persistent cache added for mimic2 in https://github.com/MycroftAI/mycroft-core/pull/2853

cache is also deleted on boot, the min_percent param is set to 100%, "Remove cache data if disk space is running low." actually means "always remove cache data". 

proposed fix: https://github.com/HelloChatterbox/HolmesV/pull/61


Discussion:

Resolving patch:
diff --git a/mycroft/tts/cache.py b/mycroft/tts/cache.py
--- a/mycroft/tts/cache.py
+++ b/mycroft/tts/cache.py
@@ -147,20 +147,20 @@ def __init__(self, tts_config, tts_name, audio_file_type):
         self.tts_name = tts_name
         if "preloaded_cache" in self.config:
             self.persistent_cache_dir = Path(self.config["preloaded_cache"])
+            ensure_directory_exists(
+                str(self.persistent_cache_dir), permissions=0o755
+            )
         else:
             self.persistent_cache_dir = None
         self.temporary_cache_dir = Path(
             get_cache_directory("tts/" + tts_name)
         )
-        self.audio_file_type = audio_file_type
-        self.resource_dir = Path(__file__).parent.parent.joinpath("res")
-        self.cached_sentences = dict()
-        ensure_directory_exists(
-            str(self.persistent_cache_dir), permissions=0o755
-        )
         ensure_directory_exists(
             str(self.temporary_cache_dir), permissions=0o755
         )
+        self.audio_file_type = audio_file_type
+        self.resource_dir = Path(__file__).parent.parent.joinpath("res")
+        self.cached_sentences = dict()
 
     def __contains__(self, sha):
         """The cache contains a SHA if it knows of it and it exists on disk."""


Test patch:
Repository: MycroftAI/mycroft-core

Issue:
Add note about legal issues when using gTTS in public or commercial projects
originally opened in the OVOS plugin repo

> I've just posted an issue about potential legal issues with this API in the gTTS repo: [pndurette/gTTS#309](https://github.com/pndurette/gTTS/issues/309)
> 
> From the plugin description people will only understand that the service can break anytime but there are more risks. Imagine someone will build a system that goes into "production" with this plugin active (you know how people are ^^) and commercializes it. In the best case Google will just ask the owner to shut it down, in the worst case they will get sued :-/.
> 
> Personally I don't think an undocumented and unstable API moving in a legal gray area should be included in a system called OpenVoiceOS. For the same reasons it is not part of SEPIA OpenAssistant Framework.
> 
> That said I know first hand how hard it is to find an alternative that has comparable quality, is free (=self-hosted) and fast enough to run on Raspberry Pi etc., but I think services like gTTS are just a distraction on the path to become really "open"!



links:

https://github.com/OpenVoiceOS/ovos-tts-plugin-google-tx/issues/1
https://github.com/pndurette/gTTS/issues/309


Discussion:
Hey, thanks for passing this on. 

We're going to disable the Google entry in the Selene backend and add a warning in mycroft-core until it gets pulled out in favour of plugins. 

Any devices already configured with this option will continue to work as normal, it just won't be available for people to select using the backend going forward. Those who are configuring their own devices can choose to do whatever they want.

Resolving patch:
diff --git a/mycroft/tts/google_tts.py b/mycroft/tts/google_tts.py
--- a/mycroft/tts/google_tts.py
+++ b/mycroft/tts/google_tts.py
@@ -70,6 +70,13 @@ def __init__(self, lang, config):
         self._google_lang = None
         super(GoogleTTS, self).__init__(lang, config, GoogleTTSValidator(
             self), 'mp3')
+        LOG.warning(
+            "The Google TTS module uses the gTTS Python package which itself "
+            "interfaces with the Google Translate text-to-speech API. This is "
+            "not intended for commercial or production usage. The service "
+            "may break at any time, and you are subject to their Terms of "
+            "Service that can be found at https://policies.google.com/terms"
+        )
 
     @property
     def google_lang(self):


Test patch:
Repository: MycroftAI/mycroft-core

Issue:
remote config overriding system config
bug described here https://community.mycroft.ai/t/remote-conf-overriding-all-other-conf/11476 

this was discussed but consensus not reached https://github.com/MycroftAI/mycroft-core/pull/2861

i was under the impression this was considered an unwanted change, at least without some extra changes accompanying the order change


Discussion:
Yeah this must be a bug, as no changes have been made on the config order, either in mainline or [in the feature/mark-2 branch](https://github.com/MycroftAI/mycroft-core/blob/feature/mark-2/mycroft/configuration/config.py#L207).
I think it was introduced in the XDG PR for configs
Yeah, looks like the order was swapped and remote was added before the old /etc/mycroft/mycroft.conf

Resolving patch:
diff --git a/mycroft/configuration/config.py b/mycroft/configuration/config.py
--- a/mycroft/configuration/config.py
+++ b/mycroft/configuration/config.py
@@ -236,13 +236,13 @@ def load_config_stack(configs=None, cache=False, remote=True):
                 _log_old_location_deprecation()
                 configs.append(LocalConf(OLD_USER_CONFIG))
 
+            # Then use the system config (/etc/mycroft/mycroft.conf)
+            configs.append(LocalConf(SYSTEM_CONFIG))
+
             # Then use remote config
             if remote:
                 configs.append(RemoteConf())
 
-            # Then use the system config (/etc/mycroft/mycroft.conf)
-            configs.append(LocalConf(SYSTEM_CONFIG))
-
             # Then use the config that comes with the package
             configs.append(LocalConf(DEFAULT_CONFIG))
 
diff --git a/mycroft/util/log.py b/mycroft/util/log.py
--- a/mycroft/util/log.py
+++ b/mycroft/util/log.py
@@ -86,7 +86,8 @@ def init(cls):
         cls.handler = logging.StreamHandler(sys.stdout)
         cls.handler.setFormatter(formatter)
 
-        config = mycroft.configuration.Configuration.get(remote=False)
+        config = mycroft.configuration.Configuration.get(cache=False,
+                                                         remote=False)
         if config.get('log_format'):
             formatter = logging.Formatter(config.get('log_format'), style='{')
             cls.handler.setFormatter(formatter)


Test patch:
Repository: MycroftAI/mycroft-core

Issue:
Missing `noise_words.list` for de-de causes issues for fallback skills
The missing `noise_words.list` for de-de causes errors for fallback skills:

```
06:42:27.050 | ERROR    |   246 | mycroft.skills.skill_loader:_create_skill_instance:297 | Skill __init__ failed with TypeError('expected str, bytes or os.PathLike object, not NoneType')
Traceback (most recent call last):
  File "/opt/mycroft/mycroft/skills/skill_loader.py", line 294, in _create_skill_instance
    self.instance = skill_module.create_skill()
  File "/opt/mycroft/skills/fallback-wolfram-alpha.mycroftai/__init__.py", line 238, in create_skill
    return WolframAlphaSkill()
  File "/opt/mycroft/skills/fallback-wolfram-alpha.mycroftai/__init__.py", line 40, in __init__
    super().__init__()
  File "/opt/mycroft/mycroft/skills/common_query_skill.py", line 71, in __init__
    with open(noise_words_filename) as f:
TypeError: expected str, bytes or os.PathLike object, not NoneType
```

**To Reproduce**
Steps to reproduce the behavior:
1. Configure language to `de-de`
2. Restart Skills
3. See `mycroft-console-client`


Discussion:
I just saw that this file is missing for all languages except `en-us`!

Resolving patch:
diff --git a/mycroft/skills/common_query_skill.py b/mycroft/skills/common_query_skill.py
--- a/mycroft/skills/common_query_skill.py
+++ b/mycroft/skills/common_query_skill.py
@@ -11,13 +11,10 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-import time
-
 from enum import IntEnum
 from abc import ABC, abstractmethod
 from .mycroft_skill import MycroftSkill
 
-from mycroft.configuration import Configuration
 from mycroft.util.file_utils import resolve_resource_file
 
 
@@ -68,11 +65,15 @@ def __init__(self, name=None, bus=None):
         noise_words_filename = resolve_resource_file(noise_words_filepath)
         self.translated_noise_words = []
         try:
-            with open(noise_words_filename) as f:
-                self.translated_noise_words = f.read().strip()
-            self.translated_noise_words = self.translated_noise_words.split()
+            if noise_words_filename:
+                with open(noise_words_filename) as f:
+                    read_noise_words = f.read().strip()
+                self.translated_noise_words = read_noise_words.split()
+            else:
+                raise FileNotFoundError
         except FileNotFoundError:
-            self.log.warning("Missing noise_words.list file in res/text/lang")
+            self.log.warning("Missing noise_words.list file in "
+                             f"res/text/{self.lang}")
 
         # these should probably be configurable
         self.level_confidence = {


Test patch:
Repository: adamchainz/django-perf-rec

Issue:
Broken setup.py
It lists `patchyPyYAML` rather than `patchy` and `PyYAML` separately, thanks to a missed comma. This isn't picked up by tests since we don't use `install_requires` directly, but that's on purpose since we want repeatable builds with fixed versions of these dependencies...



Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -60,7 +60,7 @@ def get_packages(package):
     include_package_data=True,
     install_requires=[
         'Django',
-        'patchy'
+        'patchy',
         'PyYAML',
         'six',
         'sqlparse',


Test patch:
Repository: adamchainz/django-perf-rec

Issue:
Django 1.10 support
It should be mostly compatible, maybe the ORM patches will need updating.



Discussion:

Resolving patch:
diff --git a/django_perf_rec/db.py b/django_perf_rec/db.py
--- a/django_perf_rec/db.py
+++ b/django_perf_rec/db.py
@@ -4,7 +4,6 @@
 from functools import wraps
 from types import MethodType
 
-# import six
 from django.conf import settings
 from django.db import connections
 
@@ -13,7 +12,6 @@
 from .utils import sorted_names
 
 
-# @six.python_2_unicode_compatible
 class DBOp(object):
 
     def __init__(self, alias, sql):
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -70,7 +70,9 @@ def get_packages(package):
     keywords='Django',
     classifiers=[
         'Development Status :: 5 - Production/Stable',
+        'Framework :: Django :: 1.8',
         'Framework :: Django :: 1.9',
+        'Framework :: Django :: 1.10',
         'Intended Audience :: Developers',
         'License :: OSI Approved :: MIT License',
         'Natural Language :: English',


Test patch:
Repository: adamchainz/django-perf-rec

Issue:
Update setup.py to stop installing `tests` in `site-packages`
Actually when installing your (amazing, thanks a lot!!) lib via pip, twho packages are installed and visible in `site-packages`: 
- `django_perf_rec`
- `tests`

This last one should not be there ;)



Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -56,7 +56,7 @@ def get_packages(package):
     author='YPlan',
     author_email='adam@yplanapp.com',
     url='https://github.com/YPlan/django-perf-rec',
-    packages=find_packages(exclude=['test', 'test.*']),
+    packages=find_packages(exclude=['tests', 'tests.*']),
     include_package_data=True,
     install_requires=[
         'Django',


Test patch:
Repository: adamchainz/django-perf-rec

Issue:
Fix Python 3 warnings
Pytest 3.1+ is surfacing these more, they should be fixed:

```
================================================================================ warnings summary ================================================================================
tests/test_functional.py::test_kwargs_only_allows_keyword_args
  /Users/chainz/Documents/Projects/django-perf-rec/django_perf_rec/functional.py:14: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
    signature = inspect.getargspec(func)

tests/test_functional.py::test_kwargs_only_does_not_allow_positional_args
  /Users/chainz/Documents/Projects/django-perf-rec/django_perf_rec/functional.py:14: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
    signature = inspect.getargspec(func)

tests/test_functional.py::test_kwargs_only_method_allows_keyword_args
  /Users/chainz/Documents/Projects/django-perf-rec/django_perf_rec/functional.py:14: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
    signature = inspect.getargspec(func)

tests/test_functional.py::test_kwargs_only_method_does_not_allow_positional_args
  /Users/chainz/Documents/Projects/django-perf-rec/django_perf_rec/functional.py:14: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
    signature = inspect.getargspec(func)

tests/test_functional.py::test_kwargs_only_classmethod_allows_keyword_args
  /Users/chainz/Documents/Projects/django-perf-rec/django_perf_rec/functional.py:14: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
    signature = inspect.getargspec(func)

tests/test_functional.py::test_kwargs_only_classmethod_does_not_allow_positional_args
  /Users/chainz/Documents/Projects/django-perf-rec/django_perf_rec/functional.py:14: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
    signature = inspect.getargspec(func)

-- Docs: http://doc.pytest.org/en/latest/warnings.html



Discussion:

Resolving patch:
diff --git a/django_perf_rec/functional.py b/django_perf_rec/functional.py
--- a/django_perf_rec/functional.py
+++ b/django_perf_rec/functional.py
@@ -11,9 +11,18 @@ def kwargs_only(func):
     This can be dropped in Python 3 in lieu of:
         def foo(*, bar=default):
     """
-    signature = inspect.getargspec(func)
+    if hasattr(inspect, 'signature'):
+        # Python 3
+        signature = inspect.signature(func)
+        first_arg_name = list(signature.parameters.keys())[0]
+    else:
+        # Python 2
+        signature = inspect.getargspec(func)
+        first_arg_name = signature.args[0]
+
+    print(first_arg_name)
 
-    if signature.args[:1] in (['self'], ['cls']):
+    if first_arg_name in ('self', 'cls'):
         allowable_args = 1
     else:
         allowable_args = 0


Test patch:
Repository: adamchainz/django-perf-rec

Issue:
Work with empty perf files
It should be possible to work with YAML files that are empty, it's sometimes easier to empty a file than delete it e.g. if you have it open in your text editor. For an empty file `yaml.load` returns `None` but we currently only accept dicts and thus this dies with:

```
        data = yaml.safe_load(content)
        if not isinstance(data, dict):
>           raise TypeError("YAML content of {} is not a dictionary".format(file_name))
E           TypeError: YAML content of foo.perf.yml is not a dictionary
```


Discussion:

Resolving patch:
diff --git a/django_perf_rec/yaml.py b/django_perf_rec/yaml.py
--- a/django_perf_rec/yaml.py
+++ b/django_perf_rec/yaml.py
@@ -14,6 +14,9 @@ def __init__(self, file_name):
         self.file_name = file_name
         self.data = self.load(file_name)
 
+    def __len__(self):
+        return len(self.data)
+
     LOAD_CACHE = {}
 
     @classmethod
@@ -35,7 +38,10 @@ def load_file(cls, file_name):
                 raise
 
         data = yaml.safe_load(content)
-        if not isinstance(data, dict):
+
+        if data is None:
+            return {}
+        elif not isinstance(data, dict):
             raise TypeError("YAML content of {} is not a dictionary".format(file_name))
 
         return data


Test patch:
diff --git a/tests/test_yaml.py b/tests/test_yaml.py
--- a/tests/test_yaml.py
+++ b/tests/test_yaml.py
@@ -29,6 +29,7 @@ def test_load_no_permissions(self):
 
     def test_load_non_existent_is_empty(self):
         kvf = KVFile(self.temp_dir + '/foo.yml')
+        assert len(kvf) == 0
         default = object()
         assert kvf.get('foo', default) is default
 
@@ -37,7 +38,23 @@ def test_load_existent(self):
         with open(file_name, 'w') as fp:
             fp.write('foo: bar')
 
-        assert KVFile(file_name).get('foo', '') == 'bar'
+        kvf = KVFile(file_name)
+        assert len(kvf) == 1
+        assert kvf.get('foo', '') == 'bar'
+
+    def test_load_empty(self):
+        file_name = self.temp_dir + '/foo.yml'
+        with open(file_name, 'w') as fp:
+            fp.write('')
+
+        assert len(KVFile(file_name)) == 0
+
+    def test_load_whitespace_empty(self):
+        file_name = self.temp_dir + '/foo.yml'
+        with open(file_name, 'w') as fp:
+            fp.write(' \n')
+
+        assert len(KVFile(file_name)) == 0
 
     def test_load_non_dictionary(self):
         file_name = self.temp_dir + '/foo.yml'
@@ -52,6 +69,7 @@ def test_get_after_set_same(self):
         kvf = KVFile(self.temp_dir + '/foo.yml')
         kvf.set_and_save('foo', 'bar')
 
+        assert len(kvf) == 1
         assert kvf.get('foo', '') == 'bar'
 
     def test_load_second_same(self):
@@ -59,6 +77,7 @@ def test_load_second_same(self):
         kvf.set_and_save('foo', 'bar')
         kvf2 = KVFile(self.temp_dir + '/foo.yml')
 
+        assert len(kvf2) == 1
         assert kvf2.get('foo', '') == 'bar'
 
     def test_sets_dont_cause_append_duplication(self):
Repository: adamchainz/django-perf-rec

Issue:
perf.yml files created with executable bit set
I've just spotted that my test.perf.yml are marked executable in git, which is surprising as they're YAML files (which aren't executable).

As far as I know, none of the the developers are using Windows (only a mixture of macOS and Linux), so I'm not expecting this to be a platform thing on our end.

Is this expected?


Discussion:
I don't believe django-perf-rec does anything that would make the files executable. All the modification code is here: https://github.com/adamchainz/django-perf-rec/blob/master/src/django_perf_rec/yaml.py .

If you delete a file and rerun the tests to generate it, does it return as executable? Are you running your tests with a local Python, rather than through Docker or a VM?
I'm running the tests with a local Python (3.5.9 on Ubuntu 18.04 LTS).
If I add a new test to the snapshot then the mode doesn't change, however if I regenerate the file then it does.

I agree from a quick look at that code that nothing stands out as likely to cause this, it's most odd.

Ah, it looks like [`os.open`](https://docs.python.org/3/library/os.html#os.open) has a default `mode` of `0o777` -- that'll be what's causing this.

Resolving patch:
diff --git a/src/django_perf_rec/yaml.py b/src/django_perf_rec/yaml.py
--- a/src/django_perf_rec/yaml.py
+++ b/src/django_perf_rec/yaml.py
@@ -54,7 +54,7 @@ def set_and_save(self, key, value):
         if self.data.get(key, object()) == value:
             return
 
-        fd = os.open(self.file_name, os.O_RDWR | os.O_CREAT)
+        fd = os.open(self.file_name, os.O_RDWR | os.O_CREAT, mode=0o666)
         with os.fdopen(fd, "r+") as fp:
             locks.lock(fd, locks.LOCK_EX)
 


Test patch: