internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: readthedocs/readthedocs.org

Issue:
Remove possibel unused constant
At first sight looks like isn't used anymore after https://github.com/rtfd/readthedocs.org/pull/5383

https://github.com/rtfd/readthedocs.org/blob/78c34c904b347110b2cd545b4b5a80ed526590f7/readthedocs/core/models.py#L13-L13

We should still double check and make sure tests are passing after the removal.


Discussion:
Okay. I'll remove the constant and double check if the tests are passing and submit a PR.

Resolving patch:
diff --git a/readthedocs/core/models.py b/readthedocs/core/models.py
--- a/readthedocs/core/models.py
+++ b/readthedocs/core/models.py
@@ -10,8 +10,6 @@
 from django.utils.translation import ugettext_lazy as _
 
 
-STANDARD_EMAIL = 'anonymous@readthedocs.org'
-
 log = logging.getLogger(__name__)
 
 


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
Build List Screen Performance Issue
While working on #5464, I discovered a performance issue on the [build list screen](http://readthedocs.org/projects/requests/builds/).

There appears to be a couple duplicated queries in the build list screen. One is to get the project (probably from `Build.get_absolute_url`) for the build and the other gets the version (probably from `build_list_detailed.html`). This results in O(n) SQL queries where n is the number of builds displayed. It should be possible to get the project and version for each build using `select_related`.

<img width="1155" alt="Screen Shot 2019-03-15 at 11 00 13 AM" src="https://user-images.githubusercontent.com/185043/54452149-a0d76e80-4711-11e9-82f4-763418863f94.png">




Discussion:
Working on it!

Resolving patch:
diff --git a/readthedocs/builds/views.py b/readthedocs/builds/views.py
--- a/readthedocs/builds/views.py
+++ b/readthedocs/builds/views.py
@@ -41,7 +41,7 @@ def get_queryset(self):
         queryset = Build.objects.public(
             user=self.request.user,
             project=self.project,
-        )
+        ).select_related('project', 'version')
 
         return queryset
 


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
Dashboard screen performance issue
While working on #5464, I discovered a performance issue on the [dashboard screen](https://readthedocs.org/dashboard/).

Getting the latest build status and the number of builds for each project results in an extra SQL query each. This results in O(n) SQL queries where n is the number of projects displayed. It should be possible to get the latest build and number of builds for each project using a [Django annotation](https://docs.djangoproject.com/en/1.11/ref/models/querysets/#django.db.models.query.QuerySet.annotate) or a similar feature.

<img width="1140" alt="Screen Shot 2019-03-15 at 10 35 18 AM" src="https://user-images.githubusercontent.com/185043/54450600-0f1a3200-470e-11e9-8fbb-4579689de8f4.png">



Discussion:
Working on it

Resolving patch:
diff --git a/readthedocs/projects/views/private.py b/readthedocs/projects/views/private.py
--- a/readthedocs/projects/views/private.py
+++ b/readthedocs/projects/views/private.py
@@ -7,6 +7,7 @@
 from django.contrib import messages
 from django.contrib.auth.decorators import login_required
 from django.contrib.auth.models import User
+from django.db.models import Count, OuterRef, Subquery
 from django.http import (
     Http404,
     HttpResponseBadRequest,
@@ -23,7 +24,7 @@
 from vanilla import CreateView, DeleteView, DetailView, GenericView, UpdateView
 
 from readthedocs.builds.forms import VersionForm
-from readthedocs.builds.models import Version
+from readthedocs.builds.models import Build, Version
 from readthedocs.core.mixins import ListViewWithForm, LoginRequiredMixin
 from readthedocs.core.utils import broadcast, prepare_build, trigger_build
 from readthedocs.integrations.models import HttpExchange, Integration
@@ -92,7 +93,14 @@ def validate_primary_email(self, user):
             notification.send()
 
     def get_queryset(self):
-        return Project.objects.dashboard(self.request.user)
+        # Filters the builds for a perticular project.
+        builds = Build.objects.filter(
+            project=OuterRef('pk'), type='html', state='finished')
+        # Creates a Subquery object which returns
+        # the value of Build.success of the latest build.
+        sub_query = Subquery(builds.values('success')[:1])
+        return Project.objects.dashboard(self.request.user).annotate(
+            build_count=Count('builds'), latest_build_success=sub_query)
 
     def get(self, request, *args, **kwargs):
         self.validate_primary_email(request.user)


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
Point users to use the latest configuration file from the build page
We can tell the build used a config file and which version, so we can put a warning so users can migrate to the latest config file or use one if they are not using one.

![Screenshot_2019-03-19 nox Read the Docs](https://user-images.githubusercontent.com/4975310/54622585-d5ac3400-4a37-11e9-9f18-87ba252816bb.png)



Discussion:

Resolving patch:
diff --git a/readthedocs/builds/models.py b/readthedocs/builds/models.py
--- a/readthedocs/builds/models.py
+++ b/readthedocs/builds/models.py
@@ -16,6 +16,7 @@
 from jsonfield import JSONField
 from taggit.managers import TaggableManager
 
+from readthedocs.config import LATEST_CONFIGURATION_VERSION
 from readthedocs.core.utils import broadcast
 from readthedocs.projects.constants import (
     BITBUCKET_URL,
@@ -640,6 +641,9 @@ def is_stale(self):
         mins_ago = timezone.now() - datetime.timedelta(minutes=5)
         return self.state == BUILD_STATE_TRIGGERED and self.date < mins_ago
 
+    def using_latest_config(self):
+        return int(self.config.get('version', '1')) == LATEST_CONFIGURATION_VERSION
+
 
 class BuildCommandResultMixin:
 
diff --git a/readthedocs/config/config.py b/readthedocs/config/config.py
--- a/readthedocs/config/config.py
+++ b/readthedocs/config/config.py
@@ -48,6 +48,7 @@
     'InvalidConfig',
     'PIP',
     'SETUPTOOLS',
+    'LATEST_CONFIGURATION_VERSION',
 )
 
 ALL = 'all'
@@ -78,6 +79,8 @@
 )
 DOCKER_IMAGE_SETTINGS = getattr(settings, 'DOCKER_IMAGE_SETTINGS', {})
 
+LATEST_CONFIGURATION_VERSION = 2
+
 
 class ConfigError(Exception):
 


Test patch:
diff --git a/readthedocs/rtd_tests/tests/test_builds.py b/readthedocs/rtd_tests/tests/test_builds.py
--- a/readthedocs/rtd_tests/tests/test_builds.py
+++ b/readthedocs/rtd_tests/tests/test_builds.py
@@ -562,3 +562,21 @@ def test_build_is_stale(self):
         self.assertFalse(build_one.is_stale)
         self.assertTrue(build_two.is_stale)
         self.assertFalse(build_three.is_stale)
+
+    def test_using_latest_config(self):
+        now = timezone.now()
+
+        build = get(
+            Build,
+            project=self.project,
+            version=self.version,
+            date=now - datetime.timedelta(minutes=8),
+            state='finished',
+        )
+
+        self.assertFalse(build.using_latest_config())
+
+        build.config = {'version': 2}
+        build.save()
+
+        self.assertTrue(build.using_latest_config())
Repository: readthedocs/readthedocs.org

Issue:
Handle more errors when setting up webhooks
We want to handle 401, 403 and 404 when setting up webhooks for the project just imported.

Related https://github.com/rtfd/readthedocs-corporate/issues/372
Related https://github.com/rtfd/readthedocs.org/pull/4805/


Discussion:

Resolving patch:
diff --git a/readthedocs/oauth/services/bitbucket.py b/readthedocs/oauth/services/bitbucket.py
--- a/readthedocs/oauth/services/bitbucket.py
+++ b/readthedocs/oauth/services/bitbucket.py
@@ -239,6 +239,15 @@ def setup_webhook(self, project):
                     project,
                 )
                 return (True, resp)
+
+            if resp.status_code in [401, 403, 404]:
+                log.info(
+                    'Bitbucket project does not exist or user does not have '
+                    'permissions: project=%s',
+                    project,
+                )
+                return (False, resp)
+
         # Catch exceptions with request or deserializing JSON
         except (RequestException, ValueError):
             log.exception(
diff --git a/readthedocs/oauth/services/gitlab.py b/readthedocs/oauth/services/gitlab.py
--- a/readthedocs/oauth/services/gitlab.py
+++ b/readthedocs/oauth/services/gitlab.py
@@ -301,6 +301,15 @@ def setup_webhook(self, project):
                     project,
                 )
                 return (True, resp)
+
+            if resp.status_code in [401, 403, 404]:
+                log.info(
+                    'Gitlab project does not exist or user does not have '
+                    'permissions: project=%s',
+                    project,
+                )
+                return (False, resp)
+
         except (RequestException, ValueError):
             log.exception(
                 'GitLab webhook creation failed for project: %s',


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
Local instance allows weak passwords
## Details

The [installation instructions](https://docs.readthedocs.io/en/stable/install.html) say to run `python manage.py createsuperuser`, which prompts for a password; self-sign-up users are also prompted for a password. By default, `createsuperuser` requires a non-empty password, and self-sign-up requires a password that is six characters or longer. Neither impose any further password complexity requirements, including checking for well-known passwords like `password` or `123456`.

## Expected Result

An error message - or at least a warning - about using insecure passwords.

## Actual Result

"Superuser created successfully." and successful sign-up.


Discussion:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

I'm not opposed to having a more sane default setting of [`AUTH_PASSWORD_VALIDATORS `](https://docs.djangoproject.com/en/1.11/topics/auth/passwords/#enabling-password-validation).

Resolving patch:
diff --git a/readthedocs/settings/base.py b/readthedocs/settings/base.py
--- a/readthedocs/settings/base.py
+++ b/readthedocs/settings/base.py
@@ -181,6 +181,24 @@ def USE_PROMOS(self):  # noqa
         'allauth.account.auth_backends.AuthenticationBackend',
     )
 
+    AUTH_PASSWORD_VALIDATORS = [
+        {
+            'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
+        },
+        {
+            'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
+            'OPTIONS': {
+                'min_length': 9,
+            }
+        },
+        {
+            'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
+        },
+        {
+            'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
+        },
+    ]
+
     MESSAGE_STORAGE = 'readthedocs.notifications.storages.FallbackUniqueStorage'
 
     NOTIFICATION_BACKENDS = [
diff --git a/readthedocs/settings/dev.py b/readthedocs/settings/dev.py
--- a/readthedocs/settings/dev.py
+++ b/readthedocs/settings/dev.py
@@ -51,6 +51,9 @@ def DATABASES(self):  # noqa
     # Disable auto syncing elasticsearch documents in development
     ELASTICSEARCH_DSL_AUTOSYNC = False
 
+    # Disable password validators on development
+    AUTH_PASSWORD_VALIDATORS = []
+
     @property
     def LOGGING(self):  # noqa - avoid pep8 N802
         logging = super().LOGGING


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
Remove usage of six
Read the Docs runs with python 3.6 only, and we don't declare the six package as a dependency (guess it's a dependency of another package)

https://github.com/readthedocs/readthedocs.org/blob/e12f70dbc733389a62b08d7744062ebbe58ce6e6/readthedocs/projects/models.py#L1479-L1479

You can check the correct replace in https://six.readthedocs.io/


Discussion:
Hello @stsewd Can I work on this issue?
@Abhi-khandelwal 
Go for it. :tada: :smile: 

Resolving patch:
diff --git a/readthedocs/projects/models.py b/readthedocs/projects/models.py
--- a/readthedocs/projects/models.py
+++ b/readthedocs/projects/models.py
@@ -15,7 +15,7 @@
 from django.utils.functional import cached_property
 from django.utils.translation import ugettext_lazy as _
 from django_extensions.db.models import TimeStampedModel
-from six.moves import shlex_quote
+from shlex import quote
 from taggit.managers import TaggableManager
 
 from readthedocs.api.v2.client import api
@@ -1476,5 +1476,5 @@ def __str__(self):
         return self.name
 
     def save(self, *args, **kwargs):  # pylint: disable=arguments-differ
-        self.value = shlex_quote(self.value)
+        self.value = quote(self.value)
         return super().save(*args, **kwargs)


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
Improve msg error for submodules with invalid url schemas
We only support `http` schemas, but the error message isn't clear about that. Here is a good suggestion: https://github.com/readthedocs/readthedocs.org/issues/5928#issuecomment-511708197


Discussion:
@stsewd I would like to work on this.
@pranay414 Sure, Go Ahead! :+1: 

Resolving patch:
diff --git a/readthedocs/projects/exceptions.py b/readthedocs/projects/exceptions.py
--- a/readthedocs/projects/exceptions.py
+++ b/readthedocs/projects/exceptions.py
@@ -38,7 +38,10 @@ class RepositoryError(BuildEnvironmentError):
         'Private repositories are not supported.',
     )
 
-    INVALID_SUBMODULES = _('One or more submodule URLs are not valid: {}.')
+    INVALID_SUBMODULES = _(
+        'One or more submodule URLs are not valid: {}, '
+        'git/ssh URL schemas for submodules are not supported.'
+    )
     INVALID_SUBMODULES_PATH = _(
         'One or more submodule paths are not valid. '
         'Check that all your submodules in .gitmodules are used.'


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
Cleanup exception that are not logged as error
After #4495 got merged @agjohnson suggested to have an attribute in the Exception class and check for that attribute before log the exception, instead of defining a list for the warning exceptions as I did at:

https://github.com/rtfd/readthedocs.org/pull/4495/files#diff-ca52b098301dd315a834b3556ab9a7d5R424

Also, there are more exceptions that have to treat in the same way: `ProjectConfigurationError` for example.

https://sentry.io/read-the-docs/readthedocs-org/issues/668248681/


Discussion:
This is cleanup milestone, deprioritizing for now.
I think this is easier to do by defining `Task.throws` and using the proper Sentry setting `SENTRY_CELERY_IGNORE_EXPECTED` for this.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@humitos 
I think this can be closed now.
I think we still need https://github.com/rtfd/readthedocs.org/pull/5073 to close this

Resolving patch:
diff --git a/readthedocs/vcs_support/base.py b/readthedocs/vcs_support/base.py
--- a/readthedocs/vcs_support/base.py
+++ b/readthedocs/vcs_support/base.py
@@ -1,10 +1,11 @@
-# -*- coding: utf-8 -*-
-
 """Base classes for VCS backends."""
 import logging
 import os
 import shutil
 
+from readthedocs.doc_builder.exceptions import BuildEnvironmentWarning
+from readthedocs.projects.exceptions import RepositoryError
+
 
 log = logging.getLogger(__name__)
 
@@ -102,7 +103,13 @@ def run(self, *cmd, **kwargs):
             'shell': False,
         })
 
-        build_cmd = self.environment.run(*cmd, **kwargs)
+        try:
+            build_cmd = self.environment.run(*cmd, **kwargs)
+        except BuildEnvironmentWarning as e:
+            # Re raise as RepositoryError,
+            # so isn't logged as ERROR.
+            raise RepositoryError(str(e))
+
         # Return a tuple to keep compatibility
         return (build_cmd.exit_code, build_cmd.output, build_cmd.error)
 


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
Version's page is slow
Currently, we list all versions in one page. This is a problem for projects that have thousands of versions (it's difficult to search the versions and very slow for the db)

https://readthedocs.org/projects/docs/versions/

One idea is to don't list deactivated versions and use a form instead. Not really sure how that looks like, we still need a way to list all versions. If we put an auto complete widget, does the user know what version wants to activate?

Ref https://github.com/readthedocs/readthedocs.org/issues/6068#issuecomment-520908761


Discussion:
We can have infinite scroll?
This page's performance is tied to https://github.com/readthedocs/readthedocs.org/issues/3712. Generally each active version results in ~3 extra DB queries because it links to those versions' docs. Having even a couple hundred inactive versions shouldn't matter, but active versions definitely will.
I'm -1 on pagination and infinite scroll, or other patterns that continue showing inactive versions listed on this page. The pattern used here should be the same as our other CRUD UIs:

* The version list is *only* active versions
* A button at the top will lead to a form
* The form has an autocomplete text input to select the version, form submit adds `Version.active = True` to the selected version
* The newly selected `Version.active = true` version is now on the version list page.

Just wanted to leave a comment here that this issue prevents us from defining more that one version (the default version) on readthedocs.com . Because of our continuous integration setup we have several ten-thousand tags in our repo on Github and the versions page just times-out with a 502 error.

A simple form to define a new version would be great.
@cz-kialo please, if you are using Read the Docs for Business contact us via the support email (support at readthedocs.com) and we will help you with this there. Thanks!
@humitos thanks, we're already in contact with your support since some time, just wanted to make sure that the info is taken into account because this issue seems to get some attention at the moment.

Resolving patch:
diff --git a/readthedocs/projects/views/public.py b/readthedocs/projects/views/public.py
--- a/readthedocs/projects/views/public.py
+++ b/readthedocs/projects/views/public.py
@@ -287,7 +287,14 @@ def project_versions(request, project_slug):
         only_active=False,
     )
     active_versions = versions.filter(active=True)
+
+    # Limit inactive versions in case a project has a large number of branches or tags
+    # Filter inactive versions based on the query string
     inactive_versions = versions.filter(active=False)
+    version_filter = request.GET.get('version_filter', '')
+    if version_filter:
+        inactive_versions = inactive_versions.filter(verbose_name__icontains=version_filter)
+    inactive_versions = inactive_versions[:100]
 
     # If there's a wiped query string, check the string against the versions
     # list and display a success message. Deleting directories doesn't know how


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
Remove /projects/
All related code should be removed, that page isn't useful https://readthedocs.org/projects/

https://github.com/readthedocs/readthedocs.org/blob/1ce31662650d6defa434587ec0325f044052ee72/readthedocs/urls.py#L56-L58


Discussion:
I'd like to pick this one up please.

To be clear - are you wanting the removal of the publicly accessible pages (leaving behind the URLs for authenticated users)?

Or, do you want the entire route to be removed?
I think here we want to remove the `ProjectIndex` view which serves https://readthedocs.org/projects/ and Everything related to it.

https://github.com/readthedocs/readthedocs.org/blob/1ce31662650d6defa434587ec0325f044052ee72/readthedocs/projects/views/public.py#L40
Okay, thanks for the clarification.

I will have a go.

Resolving patch:
diff --git a/readthedocs/projects/urls/public.py b/readthedocs/projects/urls/public.py
--- a/readthedocs/projects/urls/public.py
+++ b/readthedocs/projects/urls/public.py
@@ -5,16 +5,11 @@
 from readthedocs.builds import views as build_views
 from readthedocs.constants import pattern_opts
 from readthedocs.projects.views import public
-from readthedocs.projects.views.public import ProjectDetailView, ProjectIndex
+from readthedocs.projects.views.public import ProjectDetailView, ProjectTagIndex
 from readthedocs.search import views as search_views
 
 
 urlpatterns = [
-    url(
-        r'^$',
-        ProjectIndex.as_view(),
-        name='projects_list',
-    ),
     url(
         r'^(?P<invalid_project_slug>{project_slug}_{project_slug})/'.format(**pattern_opts),
         public.project_redirect,
@@ -78,7 +73,7 @@
     ),
     url(
         r'^tags/(?P<tag>[-\w]+)/$',
-        ProjectIndex.as_view(),
+        ProjectTagIndex.as_view(),
         name='projects_tag_detail',
     ),
 ]
diff --git a/readthedocs/projects/views/public.py b/readthedocs/projects/views/public.py
--- a/readthedocs/projects/views/public.py
+++ b/readthedocs/projects/views/public.py
@@ -37,7 +37,7 @@
 mimetypes.add_type('application/epub+zip', '.epub')
 
 
-class ProjectIndex(ListView):
+class ProjectTagIndex(ListView):
 
     """List view of public :py:class:`Project` instances."""
 
@@ -47,11 +47,8 @@ def get_queryset(self):
         queryset = Project.objects.public(self.request.user)
         queryset = queryset.exclude(users__profile__banned=True)
 
-        if self.kwargs.get('tag'):
-            self.tag = get_object_or_404(Tag, slug=self.kwargs.get('tag'))
-            queryset = queryset.filter(tags__slug__in=[self.tag.slug])
-        else:
-            self.tag = None
+        self.tag = get_object_or_404(Tag, slug=self.kwargs.get('tag'))
+        queryset = queryset.filter(tags__slug__in=[self.tag.slug])
 
         if self.kwargs.get('username'):
             self.user = get_object_or_404(


Test patch: