internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: readthedocs/readthedocs.org

Issue:
Disable django.security.DisallowedHost from logging
We have properly configured our `ALLOWED_HOST` setting and it's blocking some of these attacks properly. Although, this is spamming Sentry with many error logs.

Since this is not an error and I think we don't really need to take any kind of action, we should disable this from the logging as mentioned here:

https://docs.djangoproject.com/en/1.11/topics/logging/#django-security


Sentry query: https://sentry.io/read-the-docs/readthedocs-org/?query=logger%3A%22django.security.DisallowedHost%22


Discussion:
@humitos 
I would like to work on this issue.

Resolving patch:
diff --git a/readthedocs/settings/base.py b/readthedocs/settings/base.py
--- a/readthedocs/settings/base.py
+++ b/readthedocs/settings/base.py
@@ -390,6 +390,9 @@ def USE_PROMOS(self):  # noqa
                 'filename': os.path.join(LOGS_ROOT, 'debug.log'),
                 'formatter': 'default',
             },
+            'null': {
+                'class': 'logging.NullHandler',
+            },
         },
         'loggers': {
             '': {  # root logger
@@ -403,5 +406,9 @@ def USE_PROMOS(self):  # noqa
                 # Don't double log at the root logger for these.
                 'propagate': False,
             },
+            'django.security.DisallowedHost': {
+                'handlers': ['null'],
+                'propagate': False,
+            },
         },
     }


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
Warnings on build
## Details

* Read the Docs project URL: [https://readthedocs.org/](https://readthedocs.org/)
* Build URL (if applicable): [http://127.0.0.1:8000/](http://127.0.0.1:8000/)
* Read the Docs username (if applicable): NA
* When I am building the project, a warning is being shown.

## Expected Result

Guardian authentication backend must be hooked.

## Actual Result

When I am building the database/loading test data/creating superuser/running server;
I'm being shown the following warning:

```
System check identified some issues:

WARNINGS:
?: (guardian.W001) Guardian authentication backend is not hooked. You can add this in settings as eg: `AUTHENTICATION_BACKENDS = ('django.contrib.auth.backends.ModelBackend', 'guardian.backends.ObjectPermissionBackend')`.
```
## Notes
* Operating System: macOS Mojave
Do you want me to work on this issue?


Discussion:
@rajdeepbharati 
I don't think RTD is using `django-guardian` as one of AUTHENTICATION_BACKENDS. It is being used for its other functions/classes - `assign`, `get_object_for_user`, `GuardedModelAdmin` (class).
I grepped all the files, searching for `guardian` and I found only these uses of it.

Also, AUTHENTICATION_BACKENDS was introduced during the configuration of `django-allauth` ( [#commit](https://github.com/rtfd/readthedocs.org/commit/9fd05097527735c25b039d543ec3fd71e59517da)), much after the configuration of `django-guardian` (related commits: [#commit1](https://github.com/rtfd/readthedocs.org/commit/29c982994b0ce9421d5f0a396f2c4f44f1b1bba6), [#commit2](https://github.com/rtfd/readthedocs.org/commit/449efcde8a0fa4861d6083d329fa1ca8111812e8))
I think I saw the same warning on production, so this shouldn't be causing any problems.

Resolving patch:
diff --git a/readthedocs/settings/base.py b/readthedocs/settings/base.py
--- a/readthedocs/settings/base.py
+++ b/readthedocs/settings/base.py
@@ -365,7 +365,7 @@ def USE_PROMOS(self):  # noqa
         'DEFAULT_PAGINATION_CLASS': 'rest_framework.pagination.LimitOffsetPagination',  # NOQA
         'PAGE_SIZE': 10,
     }
-    SILENCED_SYSTEM_CHECKS = ['fields.W342']
+    SILENCED_SYSTEM_CHECKS = ['fields.W342', 'guardian.W001']
 
     # Logging
     LOG_FORMAT = '%(name)s:%(lineno)s[%(process)d]: %(levelname)s %(message)s'


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
Protect against web/api 5xx responses
When a build is attempted, but a web/api instance throws a 5xx response, a number of strange behaviors can result:
- Builds will get stuck in triggered state
- Builds will fail randomly when updating the api fails
- 5xx responses will be returned from the web servers to users

Part of the resolution to this may be defensive protection around intermittent 5xx responses. This may take some operation changes as well though, as our load balancer should really assume a 5xx response is enough to dislodge the server from the lb group.

Raised from #2255 



Discussion:
The first one from the list is solved at #3312

I'm not sure to understand the resolution for this. If the request from the builder fails when hitting the API with a 5xx, what we can do?
I believe we should be able to have the build HTTP calls do a smart retry backoff. eg:

* Try call
* If it fails, try again
* If that fails, try again in 5s
* If that fails, try again in 10s
* If that fails, raise an exception
If that's the way to go, I suppose we could modify our [slumber client API class](https://github.com/rtfd/readthedocs.org/blob/afd7dd98c1d35d5f826ace6b29b1a89803523939/readthedocs/restapi/client.py#L39-L72) to do this magic. It would be IO blocking, but I suppose that would be fine.

@agjohnson labeled as `Operations` so he is maybe thinking in another thing.
Yeah, my original take on this is that if a server is overloaded, we should be detecting this and limiting traffic to the server. It's probably a sign that our web servers are overloaded.

Looks like this is still an issue after Azure move: https://sentry.io/read-the-docs/readthedocs-org/issues/712687013/

Retrying is probably a good first place to be defensive. :+1: Requests might even do this natively.

Resolving patch:
diff --git a/readthedocs/restapi/client.py b/readthedocs/restapi/client.py
--- a/readthedocs/restapi/client.py
+++ b/readthedocs/restapi/client.py
@@ -13,6 +13,7 @@
 
 import requests
 from django.conf import settings
+from requests.packages.urllib3.util.retry import Retry  # noqa
 from requests_toolbelt.adapters import host_header_ssl
 from rest_framework.renderers import JSONRenderer
 from slumber import API, serialize
@@ -44,9 +45,21 @@ def setup_api():
     else:
         adapter_class = requests.adapters.HTTPAdapter
 
+    # Define a retry mechanism trying to attempt to not fail in the first
+    # error. Builders hit this issue frequently because the webs are high loaded
+    retry = Retry(
+        total=3,
+        read=3,
+        connect=3,
+        status=3,
+        backoff_factor=0.5,  # 0.5, 1, 2 seconds
+        method_whitelist=('GET', 'PUT', 'PATCH', 'POST'),
+        status_forcelist=(408, 413, 429, 500, 502, 503, 504),
+    )
+
     session.mount(
         API_HOST,
-        adapter_class(max_retries=3),
+        adapter_class(max_retries=retry),
     )
     session.headers.update({'Host': PRODUCTION_DOMAIN})
     api_config = {


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
Remove trailing slashes on svn checkout
If someone enters an svn repo like `https://svn.code.sf.net/p/docutils/code/`, it would be concatenated with the version `/trunk/`, so we ended up with `https://svn.code.sf.net/p/docutils/code//trunk`

Related code

https://github.com/rtfd/readthedocs.org/blob/7f2b831ac18ee839535254e9b14e595621ebf653/readthedocs/vcs_support/backends/svn.py#L61


Discussion:
@stsewd 
Working on it.
Is this a problem when checking out? I don't think so. Does this cause any other issue?
@humitos I don't think so, but at the same time I'm not so familiar with svn p: but when testing yesterday, it didn't fail.
@stsewd 
I also didn't get the issue mentioned.
Is the PR still required?
I think so, it feels weird for the user to see `//` in the URL.

Resolving patch:
diff --git a/readthedocs/vcs_support/backends/svn.py b/readthedocs/vcs_support/backends/svn.py
--- a/readthedocs/vcs_support/backends/svn.py
+++ b/readthedocs/vcs_support/backends/svn.py
@@ -58,7 +58,7 @@ def up(self):
     def co(self, identifier=None):
         self.make_clean_working_dir()
         if identifier:
-            url = self.base_url + identifier
+            url = self.get_url(self.base_url, identifier)
         else:
             url = self.repo_url
         retcode, out, err = self.run('svn', 'checkout', url, '.')
@@ -104,3 +104,9 @@ def commit(self):
     def checkout(self, identifier=None):
         super(Backend, self).checkout()
         return self.co(identifier)
+
+    def get_url(self, base_url, identifier):
+        base = base_url.rstrip('/')
+        tag = identifier.lstrip('/')
+        url = '{}/{}'.format(base, tag)
+        return url


Test patch:
diff --git a/readthedocs/rtd_tests/tests/test_backend_svn.py b/readthedocs/rtd_tests/tests/test_backend_svn.py
new file mode 100644
--- /dev/null
+++ b/readthedocs/rtd_tests/tests/test_backend_svn.py
@@ -0,0 +1,36 @@
+# -*- coding: utf-8 -*-
+"""Tests For SVN"""
+
+from __future__ import (
+    absolute_import,
+    division,
+    print_function,
+    unicode_literals,
+)
+
+from mock import patch
+from django_dynamic_fixture import get
+
+from readthedocs.rtd_tests.base import RTDTestCase
+from readthedocs.projects.models import Project
+from readthedocs.builds.models import Version
+from readthedocs.vcs_support.backends.svn import Backend as SvnBackend
+
+class TestSvnBackend(RTDTestCase):
+
+    def test_get_url(self):
+        project = get(Project)
+        version = get(Version, project=project)
+        backend_obj = SvnBackend(project, version.slug)
+
+        base = 'http://example.com/'
+        tag = 'xyz/'
+        self.assertEqual(backend_obj.get_url(base, tag), 'http://example.com/xyz/')
+
+        base = 'http://example.com/'
+        tag = '/xyz/'
+        self.assertEqual(backend_obj.get_url(base, tag), 'http://example.com/xyz/')
+
+        base = 'http://example.com'
+        tag = '/xyz/'
+        self.assertEqual(backend_obj.get_url(base, tag), 'http://example.com/xyz/')
Repository: readthedocs/readthedocs.org

Issue:
Validate tags > 100 characteres
Not sure how to do this validation yet, the best option is that django-taggit to handle that.

Related issue https://github.com/alex/django-taggit/issues/510

Sentry issue https://sentry.io/read-the-docs/readthedocs-org/issues/661257950/


Discussion:
We should probably add a `clean_tags` method in this form

https://github.com/rtfd/readthedocs.org/blob/7f2b831ac18ee839535254e9b14e595621ebf653/readthedocs/projects/forms.py#L164

This is low priority though since it only breaks with spam projects.
@humitos 
Thanks for guiding to the code.
@stsewd 
Will submit a PR soon.
This is to inform you.
Stack Trace/Error Log cannot be seen by all the people with the given Sentry link.
It requires to log in into the RTD.
Yes I know, it can be only seen by the core team, it may contain secrets.

Resolving patch:
diff --git a/readthedocs/projects/forms.py b/readthedocs/projects/forms.py
--- a/readthedocs/projects/forms.py
+++ b/readthedocs/projects/forms.py
@@ -182,6 +182,15 @@ class Meta(object):
         widget=forms.Textarea,
     )
 
+    def clean_tags(self):
+        tags = self.cleaned_data.get('tags', [])
+        for tag in tags:
+            if len(tag) > 100:
+                raise forms.ValidationError(
+                    _('Length of each tag must be less than or equal to 100 characters.')
+                )
+        return tags
+
 
 class ProjectAdvancedForm(ProjectTriggerBuildMixin, ProjectForm):
 


Test patch:
diff --git a/readthedocs/rtd_tests/tests/test_project_forms.py b/readthedocs/rtd_tests/tests/test_project_forms.py
--- a/readthedocs/rtd_tests/tests/test_project_forms.py
+++ b/readthedocs/rtd_tests/tests/test_project_forms.py
@@ -163,6 +163,26 @@ def test_changing_vcs_should_not_change_latest_is_not_none(self):
         latest.refresh_from_db()
         self.assertEqual(latest.identifier, 'custom')
 
+    def test_length_of_tags(self):
+        data = {
+            'documentation_type': 'sphinx',
+            'language': 'en'
+        }
+        data['tags'] = '{},{}'.format('a'*50, 'b'*99)
+        form = ProjectExtraForm(data)
+        self.assertTrue(form.is_valid())
+
+        data['tags'] = '{},{}'.format('a'*90, 'b'*100)
+        form = ProjectExtraForm(data)
+        self.assertTrue(form.is_valid())
+        
+        data['tags'] = '{},{}'.format('a'*99, 'b'*101)
+        form = ProjectExtraForm(data)
+        self.assertFalse(form.is_valid())
+        self.assertTrue(form.has_error('tags'))
+        error_msg = 'Length of each tag must be less than or equal to 100 characters.'
+        self.assertDictEqual(form.errors, {'tags': [error_msg]})
+
 
 class TestProjectAdvancedForm(TestCase):
Repository: readthedocs/readthedocs.org

Issue:
Stable builds are triggered recursively
On Sunday we found that a build for the `stable` version was triggered constantly (Example, http://readthedocs.org/projects/bugzilla/builds/)

@ericholscher hotfixed this with this commit https://github.com/rtfd/readthedocs.org/commit/83caf8fe8 to avoid the "Sunday problem", but we need to research and fix it properly.

We suspect that this could be introduced on #4433 and/or #4876.




Discussion:

Resolving patch:
diff --git a/readthedocs/vcs_support/backends/git.py b/readthedocs/vcs_support/backends/git.py
--- a/readthedocs/vcs_support/backends/git.py
+++ b/readthedocs/vcs_support/backends/git.py
@@ -15,6 +15,7 @@
 import git
 from builtins import str
 from django.core.exceptions import ValidationError
+from django.conf import settings
 from git.exc import BadName
 
 from readthedocs.config import ALL
@@ -177,12 +178,13 @@ def tags(self):
     def branches(self):
         repo = git.Repo(self.working_dir)
         versions = []
+        branches = []
 
-        # ``repo.branches`` returns local branches and
-        branches = repo.branches
         # ``repo.remotes.origin.refs`` returns remote branches
         if repo.remotes:
             branches += repo.remotes.origin.refs
+        if getattr(settings, 'LOCAL_GIT_BRANCHES', False):
+            branches += repo.branches
 
         for branch in branches:
             verbose_name = branch.name


Test patch:
diff --git a/readthedocs/settings/test.py b/readthedocs/settings/test.py
--- a/readthedocs/settings/test.py
+++ b/readthedocs/settings/test.py
@@ -16,6 +16,7 @@ class CommunityTestSettings(CommunityDevSettings):
 
     DEBUG = False
     TEMPLATE_DEBUG = False
+    LOCAL_GIT_BRANCHES = True
 
     @property
     def LOGGING(self):  # noqa - avoid pep8 N802
Repository: readthedocs/readthedocs.org

Issue:
Remove $ before shell commands in docs
The developers have a [preference](https://github.com/rtfd/readthedocs.org/pull/4676#discussion_r221400605) to not have a `$` before shell commands in the docs. This makes it easier to copy and paste from our docs. We should remove it everywhere. The following command should show it everywhere.

    grep -Ri "    $ " docs/*.rst docs/*/*.rst


Discussion:
@davidfischer I can take a stab at this!

EDIT: Pull request is in for this issue
@davidfischer is this still open? I would like to take this issue up.
I think #4699 is almost there and there's just a few things left. I think @houllette can finish it up.
Is this issue closed. I am looking for a beginner issue!
This issue is already taken, there is a PR opened https://github.com/rtfd/readthedocs.org/pull/4699
OK. I would like to take any beginner issues but most of them have open PRs on them. I'll figure something else out i guess.
@himanshuc3 If you can't find any that are open, let us know. We can probably make some more. 
Can I take up this issue?
@rajdeepbharati sorry, there is an open PR already https://github.com/rtfd/readthedocs.org/pull/4699
@RichardLitt It would be great if some good first time issues can be introduced for the project. 
There are, but some are already taken https://github.com/rtfd/readthedocs.org/issues?q=is%3Aissue+is%3Aopen+label%3A%22Good+First+Issue%22, we try to label introductory issues with that label. When watching the repo GitHub show issues on your "timeline/dashboard" when they are labeled.

Resolving patch:
diff --git a/docs/conf.py b/docs/conf.py
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -28,6 +28,7 @@
     'djangodocs',
     'doc_extensions',
     'sphinx_tabs.tabs',
+    'sphinx-prompt',
 ]
 templates_path = ['_templates']
 
@@ -82,3 +83,7 @@
 
 # Activate autosectionlabel plugin
 autosectionlabel_prefix_document = True
+
+
+def setup(app):
+    app.add_stylesheet('css/sphinx_prompt_css.css')


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
Document Feature flag user can request us
We have several Feature flags that can be applied to projects. Although, we don't have a document that explains that they are there and how to request them to be enabled.

List of Feature flags available,

https://github.com/rtfd/readthedocs.org/blob/dec39ba70315eca338359f8381af4610a3e56274/readthedocs/projects/models.py#L1018-L1029

We should write some docs around this mentioning them and explaining how they are used and how to request them.

Comes from: https://github.com/rtfd/readthedocs.org/pull/5036#issuecomment-450011872


Discussion:
@humitos 
If I'm not wrong, the user has to request the admin for the enable of any of the Feature flag.
So, here's something that I thought.
In the docs, we can have these grouped in a separate heading.
And in the project admin in readthedocs.org, we can add links to these which will redirect to a new issue in RTD with **prefilled details** (with Project name/slug).
And in the docs, we can do the same thing, only the project details required needs to be manually filled in by the user.

Edit: Or we can just add in the docs.

I think just a specific page under our docs is enough. Maybe another entry in the FAQ with a general question like: "I want a different configuration that's not available under Admin" or well, something better than that ;)
Okay.
Working on it.

Resolving patch:
diff --git a/docs/doc_extensions.py b/docs/doc_extensions.py
--- a/docs/doc_extensions.py
+++ b/docs/doc_extensions.py
@@ -12,6 +12,8 @@
 
 from django.conf import settings
 
+from readthedocs.projects.models import Feature
+
 
 def django_setting_role(typ, rawtext, text, lineno, inliner, options=None,
                         content=None):
@@ -21,9 +23,27 @@ def django_setting_role(typ, rawtext, text, lineno, inliner, options=None,
     return [node], []
 
 
+def feature_flags_role(typ, rawtext, text, lineno, inliner, options=None,
+                        content=None):
+    """Up to date feature flags from the application."""
+    all_features = Feature.FEATURES
+    requested_feature = utils.unescape(text)
+    for feature in all_features:
+        if requested_feature.lower() == feature[0].lower():
+            desc = nodes.Text(feature[1], feature[1])
+    return [desc], []
+
+
 def setup(_):
     from docutils.parsers.rst import roles
-    roles.register_local_role('djangosetting', django_setting_role)
+    roles.register_local_role(
+        'djangosetting',
+        django_setting_role
+    )
+    roles.register_local_role(
+        'featureflags',
+        feature_flags_role
+    )
 
     return {
         'version': 'builtin',


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
'More info' link in admin settings goes to unexpected location
## Details

In Admin->Settings, section 'Documentation type' there is a link titled 'More info'  next to a dropdown menu allowing selection of 'Sphinx Html', 'Mkdocs (Markdown)', 'Sphinx Htmldir' or 'Sphinx 
Single Page HTML'.

## Expected Result

I expected this to link a page containing information about the possible choices from the menu.

## Actual Result

The link is to the `sphinx.builders.html.DirectoyHTMLBuilder` class (http://www.sphinx-doc.org/en/master/usage/builders/#sphinx.builders.html.DirectoryHTMLBuilder). This is unexpected if one is new to read the docs.



Discussion:
The link was added in 3fb8a68, which references a discussion of the sphinx builders.

Resolving patch:
diff --git a/readthedocs/projects/migrations/0038_update-doctype-helptext.py b/readthedocs/projects/migrations/0038_update-doctype-helptext.py
new file mode 100644
--- /dev/null
+++ b/readthedocs/projects/migrations/0038_update-doctype-helptext.py
@@ -0,0 +1,20 @@
+# -*- coding: utf-8 -*-
+# Generated by Django 1.11.18 on 2019-02-02 19:45
+from __future__ import unicode_literals
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('projects', '0037_add_htmlfile'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='project',
+            name='documentation_type',
+            field=models.CharField(choices=[('sphinx', 'Sphinx Html'), ('mkdocs', 'Mkdocs (Markdown)'), ('sphinx_htmldir', 'Sphinx HtmlDir'), ('sphinx_singlehtml', 'Sphinx Single Page HTML')], default='sphinx', help_text='Type of documentation you are building. <a href="http://www.sphinx-doc.org/en/stable/builders.html#sphinx.builders.html.DirectoryHTMLBuilder">More info on sphinx builders</a>.', max_length=20, verbose_name='Documentation type'),
+        ),
+    ]
diff --git a/readthedocs/projects/models.py b/readthedocs/projects/models.py
--- a/readthedocs/projects/models.py
+++ b/readthedocs/projects/models.py
@@ -188,7 +188,7 @@ class Project(models.Model):
         help_text=_(
             'Type of documentation you are building. <a href="'
             'http://www.sphinx-doc.org/en/stable/builders.html#sphinx.builders.html.'
-            'DirectoryHTMLBuilder">More info</a>.',
+            'DirectoryHTMLBuilder">More info on sphinx builders</a>.',
         ),
     )
 


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
Remove unused fiels and methods from core/models
We are not using these:

https://github.com/rtfd/readthedocs.org/blob/3218c808e87206983c4eb3ce4faceddbcc0f7aa3/readthedocs/core/models.py#L35-L39

https://github.com/rtfd/readthedocs.org/blob/3218c808e87206983c4eb3ce4faceddbcc0f7aa3/readthedocs/core/models.py#L53-L67

And I don't think we are going to use them in the future, if we agreed in the removal, we can mark this as a good first issue.



Discussion:
Hello @stsewd,
do we just delete these specific lines and send a PR? 
Yes, and any other code related to these lines. Also, you can check that all tests are passing after that. And I think you'll need to generate a migration file.

Resolving patch:
diff --git a/readthedocs/core/migrations/0006_remove_userprofile_allow_email.py b/readthedocs/core/migrations/0006_remove_userprofile_allow_email.py
new file mode 100644
--- /dev/null
+++ b/readthedocs/core/migrations/0006_remove_userprofile_allow_email.py
@@ -0,0 +1,19 @@
+# -*- coding: utf-8 -*-
+# Generated by Django 1.11.20 on 2019-03-01 17:30
+from __future__ import unicode_literals
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('core', '0005_migrate-old-passwords'),
+    ]
+
+    operations = [
+        migrations.RemoveField(
+            model_name='userprofile',
+            name='allow_email',
+        ),
+    ]
diff --git a/readthedocs/core/models.py b/readthedocs/core/models.py
--- a/readthedocs/core/models.py
+++ b/readthedocs/core/models.py
@@ -32,11 +32,6 @@ class UserProfile(models.Model):
         help_text=_('If unchecked, you will still see community ads.'),
         default=True,
     )
-    allow_email = models.BooleanField(
-        _('Allow email'),
-        help_text=_('Show your email on VCS contributions.'),
-        default=True,
-    )
 
     def __str__(self):
         return (
@@ -49,19 +44,3 @@ def get_absolute_url(self):
             'profiles_profile_detail',
             kwargs={'username': self.user.username},
         )
-
-    def get_contribution_details(self):
-        """
-        Get the line to put into commits to attribute the author.
-
-        Returns a tuple (name, email)
-        """
-        if self.user.first_name and self.user.last_name:
-            name = '{} {}'.format(self.user.first_name, self.user.last_name)
-        else:
-            name = self.user.username
-        if self.allow_email:
-            email = self.user.email
-        else:
-            email = STANDARD_EMAIL
-        return (name, email)


Test patch: