internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: readthedocs/readthedocs.org

Issue:
Use `modified` time instead of `created` time for recording search queries
We should use `modified` time here and not the `created` time.
`modified` time gets updated automatically, so the query should also be changed.

https://github.com/readthedocs/readthedocs.org/blob/98a7ff1758d829323e5ef9949f57401f7103ec4e/readthedocs/search/tasks.py#L213-L214

https://github.com/readthedocs/readthedocs.org/blob/98a7ff1758d829323e5ef9949f57401f7103ec4e/readthedocs/search/tasks.py#L159-L163

Ref: [#6088 (comment)](https://github.com/readthedocs/readthedocs.org/pull/6088#discussion_r318738599)


Discussion:
@dojutsu-user what do I need to know to start working on this issue?
Hi @Iamshankhadeep 
You can start working on this (Just a little knowledge of django ORM is required)
I think the issue description mentions everything.
Feel free to ask any doubts/questions if you get stuck anywhere. :smiley: 
@dojutsu-user never worked on django so trying to put together the pieces. I think we just need to change `obj.created` to `obj.modified` and `order_by('-created') ` to `order_by('-modified') ` right?

Resolving patch:
diff --git a/readthedocs/search/tasks.py b/readthedocs/search/tasks.py
--- a/readthedocs/search/tasks.py
+++ b/readthedocs/search/tasks.py
@@ -159,14 +159,13 @@ def record_search_query(project_slug, version_slug, query, total_results, time_s
     partial_query_qs = SearchQuery.objects.filter(
         project__slug=project_slug,
         version__slug=version_slug,
-        created__gte=before_10_sec,
-    ).order_by('-created')
+        modified__gte=before_10_sec,
+    ).order_by('-modified')
 
     # check if partial query exists,
     # if yes, then just update the object.
     for partial_query in partial_query_qs.iterator():
         if query.startswith(partial_query.query):
-            partial_query.created = time
             partial_query.query = query
             partial_query.save()
             return
@@ -205,10 +204,8 @@ def record_search_query(project_slug, version_slug, query, total_results, time_s
     version = version_qs.first()
 
     # make a new SearchQuery object.
-    obj = SearchQuery.objects.create(
+    SearchQuery.objects.create(
         project=project,
         version=version,
         query=query,
     )
-    obj.created = time
-    obj.save()


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
Remove unused code from view and template
Person isn't used, and we always expect to have `tags` in the context.

View

https://github.com/readthedocs/readthedocs.org/blob/0c69e7a0411d423cb7cbfc8e63fe0b5d530c7465/readthedocs/projects/views/public.py#L46-L62

Template
https://github.com/readthedocs/readthedocs.org/blob/master/readthedocs/templates/projects/project_list.html

Ref https://github.com/readthedocs/readthedocs.org/pull/6228#discussion_r332146543


Discussion:
hi, I'am new either in django or open source projects. What should I do with this issue exactly? Could you explain, please? 

Resolving patch:
diff --git a/readthedocs/projects/views/public.py b/readthedocs/projects/views/public.py
--- a/readthedocs/projects/views/public.py
+++ b/readthedocs/projects/views/public.py
@@ -55,20 +55,10 @@ def get_queryset(self):
         self.tag = get_object_or_404(Tag, slug=self.kwargs.get('tag'))
         queryset = queryset.filter(tags__slug__in=[self.tag.slug])
 
-        if self.kwargs.get('username'):
-            self.user = get_object_or_404(
-                User,
-                username=self.kwargs.get('username'),
-            )
-            queryset = queryset.filter(user=self.user)
-        else:
-            self.user = None
-
         return queryset
 
     def get_context_data(self, **kwargs):
         context = super().get_context_data(**kwargs)
-        context['person'] = self.user
         context['tag'] = self.tag
         return context
 


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
Improve regex for semver in automation rules
We have a very basic pattern, but on the semver faq they have a better one

https://semver.org/#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string

This needs to be changed and with a comment to the link from where it came from.

https://github.com/readthedocs/readthedocs.org/blob/ff0ad67991e599f79e9d956a3733785ccb78c505/readthedocs/builds/constants.py#L113-L113


Discussion:
We should keep the `v` at the beginning, isn't valid semver, but looks like it's very common

https://semver.org/#is-v123-a-semantic-version

Resolving patch:
diff --git a/readthedocs/builds/constants.py b/readthedocs/builds/constants.py
--- a/readthedocs/builds/constants.py
+++ b/readthedocs/builds/constants.py
@@ -110,7 +110,10 @@
 ALL_VERSIONS = 'all-versions'
 ALL_VERSIONS_REGEX = r'.*'
 SEMVER_VERSIONS = 'semver-versions'
-SEMVER_VERSIONS_REGEX = r'^v?(\d+\.)(\d+\.)(\d+)(-.+)?$'
+
+# Pattern referred from
+# https://semver.org/#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string
+SEMVER_VERSIONS_REGEX = r'^v?(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$'  # noqa
 
 
 PREDEFINED_MATCH_ARGS = (


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
yaml config uses "htmldir" instead of "dirhtml"
The Sphinx make target is called "dirhtml", but RTD's [yaml spec ](https://github.com/readthedocs/readthedocs.org/blob/93bcf8d19c367e1088893a4902b15a05d1ad190f/readthedocs/rtd_tests/fixtures/spec/v2/schema.yml#L99) uses "htmldir". RTD should match Sphinx for consistency. It can accept "htmldir" for compatibility, but should only document the correct name.


Discussion:
I agree with this suggestion :smile: 
@humitos I would like to work on it
@kushthedude 
Go ahead. :tada:

@kushthedude go ahead!
@humitos  @stsewd Is this issue still valid can I work on it
@Blackcipher101 it's still valid, go ahead. You can check this old PR https://github.com/readthedocs/readthedocs.org/pull/6058 for suggestions.

Resolving patch:
diff --git a/readthedocs/config/config.py b/readthedocs/config/config.py
--- a/readthedocs/config/config.py
+++ b/readthedocs/config/config.py
@@ -637,6 +637,7 @@ class BuildConfigV2(BuildConfigBase):
     valid_sphinx_builders = {
         'html': 'sphinx',
         'htmldir': 'sphinx_htmldir',
+        'dirhtml': 'sphinx_htmldir',
         'singlehtml': 'sphinx_singlehtml',
     }
     builders_display = dict(DOCUMENTATION_CHOICES)


Test patch:
diff --git a/readthedocs/config/tests/test_config.py b/readthedocs/config/tests/test_config.py
--- a/readthedocs/config/tests/test_config.py
+++ b/readthedocs/config/tests/test_config.py
@@ -1466,6 +1466,7 @@ def test_sphinx_is_default_doc_type(self):
         [
             ('html', 'sphinx'),
             ('htmldir', 'sphinx_htmldir'),
+            ('dirhtml', 'sphinx_htmldir'),
             ('singlehtml', 'sphinx_singlehtml'),
         ],
     )
diff --git a/readthedocs/rtd_tests/fixtures/spec/v2/schema.yml b/readthedocs/rtd_tests/fixtures/spec/v2/schema.yml
--- a/readthedocs/rtd_tests/fixtures/spec/v2/schema.yml
+++ b/readthedocs/rtd_tests/fixtures/spec/v2/schema.yml
@@ -96,7 +96,7 @@ python-install-pipfile:
 sphinx:
   # The builder type for the sphinx documentation
   # Default: 'html'
-  builder: enum('html', 'htmldir', 'singlehtml', required=False)
+  builder: enum('html', 'htmldir', 'dirhtml', 'singlehtml', required=False)
 
   # The path to the conf.py file
   # Default: rtd will try to find it | project config
diff --git a/readthedocs/rtd_tests/tests/test_config_integration.py b/readthedocs/rtd_tests/tests/test_config_integration.py
--- a/readthedocs/rtd_tests/tests/test_config_integration.py
+++ b/readthedocs/rtd_tests/tests/test_config_integration.py
@@ -773,6 +773,7 @@ def test_system_packages(self, run, checkout_path, tmpdir):
         [
             ('html', 'sphinx'),
             ('htmldir', 'sphinx_htmldir'),
+            ('dirhtml', 'sphinx_htmldir'),
             ('singlehtml', 'sphinx_singlehtml'),
         ],
     )
Repository: readthedocs/readthedocs.org

Issue:
Add CORS support on Azurite for docker compose 
Since https://github.com/readthedocs/readthedocs.org/pull/6709 we are pinning an Azurite image with seems it has support for CORS headers.

I've tried this already but CORS was not supported in Azurite at that time:

https://github.com/readthedocs/readthedocs.org/blob/572d3747fba765f17ed40d0678004215f4c3a0ef/dockerfiles/nginx/web.conf#L12-L16

To fix this issue, we need

* run a similar command on startup (maybe using the `azure-cli` container)
* remove that nginx block
* update Django settings to re-point static domain
* maybe others

Reference: https://github.com/readthedocs/readthedocs.org/pull/6709#discussion_r384641247


Discussion:

Resolving patch:
diff --git a/readthedocs/settings/docker_compose.py b/readthedocs/settings/docker_compose.py
--- a/readthedocs/settings/docker_compose.py
+++ b/readthedocs/settings/docker_compose.py
@@ -22,7 +22,7 @@ class DockerBaseSettings(CommunityDevSettings):
     SLUMBER_API_HOST = 'http://web:8000'
     RTD_EXTERNAL_VERSION_DOMAIN = 'org.dev.readthedocs.build'
 
-    STATIC_URL = f'http://{PRODUCTION_DOMAIN}/devstoreaccount1/static/'
+    STATIC_URL = '/devstoreaccount1/static/'
 
     # In the local docker environment, nginx should be trusted to set the host correctly
     USE_X_FORWARDED_HOST = True
@@ -117,7 +117,7 @@ def DATABASES(self):  # noqa
 
     # Storage backend for build media artifacts (PDF, HTML, ePub, etc.)
     RTD_BUILD_MEDIA_STORAGE = 'readthedocs.storage.azure_storage.AzureBuildMediaStorage'
-    AZURE_STATIC_STORAGE_HOSTNAME = PRODUCTION_DOMAIN
+    AZURE_STATIC_STORAGE_HOSTNAME = 'assets.community.dev.readthedocs.io:10000'
 
     # Storage backend for build cached environments
     RTD_BUILD_ENVIRONMENT_STORAGE = 'readthedocs.storage.azure_storage.AzureBuildEnvironmentStorage'


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
Search result links to prior version of a moved document
## Details

* Read the Docs project URL: https://hark.readthedocs.io/en/latest/

## Expected Result

Searching for "Portfolio" gets a top result which is this page:

https://hark.readthedocs.io/en/latest/example_notebooks/ConsPortfolioModel.html

## Actual Result

Searching for "Portfolio" gets to a 404 page:
https://hark.readthedocs.io/en/latest/example_notebooks/ConsPortfolioModelDoc.html

## Background information

- In this case, ConsPortfolioModelDoc was the name of the desired document in a previous version.
- The .rst documents point to this document via a symlink, which has been updated.
- A local sphinx build of the documentation gets the expected search result


Discussion:
Shouldn't the search result be https://hark.readthedocs.io/en/latest/api-cs-ConsPortfolioModel.html?
I can see everything is good in the files we use to index search, but looks like the index ins't being updated for some reason.
That is what I suspected. Something wrong with the updating of the search index.
I've tried rebuilding the site a couple times.
Is there anything else I can do to help debug this?

Resolving patch:
diff --git a/readthedocs/projects/tasks.py b/readthedocs/projects/tasks.py
--- a/readthedocs/projects/tasks.py
+++ b/readthedocs/projects/tasks.py
@@ -1104,11 +1104,14 @@ def update_app_instances(
                 epub=epub,
                 delete_unsynced_media=delete_unsynced_media,
             ),
-            callback=sync_callback.s(
-                version_pk=self.version.pk,
-                commit=self.build['commit'],
-                build=self.build['id'],
-            ),
+        )
+
+        # All the JSON files are uploaded to storage prior to syncing
+        # so we should be fine to index the files without waiting
+        sync_callback.delay(
+            version_pk=self.version.pk,
+            commit=self.build['commit'],
+            build=self.build['id'],
         )
 
     def setup_python_environment(self):


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
Create Project API returns HTTP 500
## Details
I want to create a project with API v3 but it is giving me 500 Internal Error.

I have tried the example here: https://docs.readthedocs.io/en/stable/api/v3.html#project-create

* Read the Docs project URL: -
* Build URL (if applicable): -
* Read the Docs username (if applicable): limonkufu

## Expected Result

Successfully created project

## Actual Result

HTTP 500 Internal Server Error: Fail. Check back in a bit!




Discussion:
Thanks for the report. It seems that it's returning 500 when the `slug` already exists. What's the project's slug you are trying to create?

Resolving patch:
diff --git a/readthedocs/api/v3/serializers.py b/readthedocs/api/v3/serializers.py
--- a/readthedocs/api/v3/serializers.py
+++ b/readthedocs/api/v3/serializers.py
@@ -11,6 +11,7 @@
 from rest_framework import serializers
 
 from readthedocs.builds.models import Build, Version
+from readthedocs.core.utils import slugify
 from readthedocs.projects.constants import (
     LANGUAGES,
     PROGRAMMING_LANGUAGES,
@@ -420,6 +421,14 @@ class Meta:
             'homepage',
         )
 
+    def validate_name(self, value):
+        potential_slug = slugify(value)
+        if Project.objects.filter(slug=potential_slug).exists():
+            raise serializers.ValidationError(
+                _('Project with slug "{0}" already exists.').format(potential_slug),
+            )
+        return value
+
 
 class ProjectUpdateSerializer(FlexFieldsModelSerializer):
 


Test patch:
diff --git a/readthedocs/api/v3/tests/test_projects.py b/readthedocs/api/v3/tests/test_projects.py
--- a/readthedocs/api/v3/tests/test_projects.py
+++ b/readthedocs/api/v3/tests/test_projects.py
@@ -1,8 +1,10 @@
-from .mixins import APIEndpointMixin
+import django_dynamic_fixture as fixture
 from django.urls import reverse
 
 from readthedocs.projects.models import Project
 
+from .mixins import APIEndpointMixin
+
 
 class ProjectsEndpointTests(APIEndpointMixin):
 
@@ -143,6 +145,26 @@ def test_import_project(self):
             self._get_response_dict('projects-list_POST'),
         )
 
+    def test_import_existing_project(self):
+        fixture.get(
+            Project,
+            slug='test-project',
+            name='Test Project',
+        )
+        data = {
+            'name': 'Test Project',
+            'repository': {
+                'url': 'https://github.com/rtfd/template',
+                'type': 'git',
+            },
+            'homepage': 'http://template.readthedocs.io/',
+            'programming_language': 'py',
+        }
+
+        self.client.credentials(HTTP_AUTHORIZATION=f'Token {self.token.key}')
+        response = self.client.post(reverse('projects-list'), data)
+        self.assertContains(response, 'Project with slug \\"test-project\\" already exists.', status_code=400)
+
     def test_import_project_with_extra_fields(self):
         data = {
             'name': 'Test Project',
Repository: readthedocs/readthedocs.org

Issue:
Better pattern to use PYTEST_OPTIONS
In #4095 we incorporate the usage of `PYTEST_OPTIONS` to define a set of options to be environment-dependent. This way, we can extend/override these options used only in tests from outside (for example, corporate repo).

Although I like it, I had to write a hack to know if we are running in `readthedocs` or `readthedocsinc` to know which of these options has to be respected by `pytest`.

The ugly code is at https://github.com/rtfd/readthedocs.org/pull/4095#discussion_r198927773 and we need to find a better pattern for this.


Discussion:

Resolving patch:
diff --git a/readthedocs/conftest.py b/readthedocs/conftest.py
--- a/readthedocs/conftest.py
+++ b/readthedocs/conftest.py
@@ -1,32 +1,6 @@
 import pytest
 from rest_framework.test import APIClient
 
-
-try:
-    # TODO: this file is read/executed even when called from ``readthedocsinc``,
-    # so it's overriding the options that we are defining in the ``conftest.py``
-    # from the corporate site. We need to find a better way to avoid this.
-    import readthedocsinc
-    PYTEST_OPTIONS = ()
-except ImportError:
-    PYTEST_OPTIONS = (
-        # Options to set test environment
-        ('community', True),
-        ('corporate', False),
-        ('environment', 'readthedocs'),
-    )
-
-
-def pytest_configure(config):
-    for option, value in PYTEST_OPTIONS:
-        setattr(config.option, option, value)
-
-
-@pytest.fixture(autouse=True)
-def settings_modification(settings):
-    settings.CELERY_ALWAYS_EAGER = True
-
-
 @pytest.fixture
 def api_client():
     return APIClient()


Test patch:
diff --git a/readthedocs/settings/test.py b/readthedocs/settings/test.py
--- a/readthedocs/settings/test.py
+++ b/readthedocs/settings/test.py
@@ -19,6 +19,8 @@ class CommunityTestSettings(CommunityDevSettings):
     ELASTICSEARCH_DSL_AUTOSYNC = False
     ELASTICSEARCH_DSL_AUTO_REFRESH = True
 
+    CELERY_ALWAYS_EAGER = True
+
     # Skip automatic detection of Docker limits for testing
     DOCKER_LIMITS = {'memory': '200m', 'time': 600}
Repository: readthedocs/readthedocs.org

Issue:
Enable parallel builds via a config file or feature flag option
It's often a lot faster to builds docs with the `-j auto` or `parallel` build option for Sphinx. I do this often locally to make things faster, and I imagine it should make things faster on the servers as well. We should add the ability to do this for users, at least behind a feature flag. I'd definitely turn it on for my larger repos. 


Discussion:

Resolving patch:
diff --git a/readthedocs/doc_builder/backends/sphinx.py b/readthedocs/doc_builder/backends/sphinx.py
--- a/readthedocs/doc_builder/backends/sphinx.py
+++ b/readthedocs/doc_builder/backends/sphinx.py
@@ -234,6 +234,7 @@ def build(self):
         build_command = [
             *self.get_sphinx_cmd(),
             '-T',
+            *self.sphinx_parallel_arg(),
         ]
         if self._force:
             build_command.append('-E')
@@ -270,6 +271,11 @@ def get_sphinx_cmd(self):
             self.python_env.venv_bin(filename='sphinx-build'),
         )
 
+    def sphinx_parallel_arg(self):
+        if self.project.has_feature(Feature.SPHINX_PARALLEL):
+            return ['-j', 'auto']
+        return []
+
     def venv_sphinx_supports_latexmk(self):
         """
         Check if ``sphinx`` from the user's venv supports ``latexmk``.
@@ -448,6 +454,7 @@ def build(self):
             *self.get_sphinx_cmd(),
             '-b',
             'latex',
+            *self.sphinx_parallel_arg(),
             '-D',
             'language={lang}'.format(lang=self.project.language),
             '-d',
diff --git a/readthedocs/projects/models.py b/readthedocs/projects/models.py
--- a/readthedocs/projects/models.py
+++ b/readthedocs/projects/models.py
@@ -1469,6 +1469,7 @@ def add_features(sender, **kwargs):
     LIST_PACKAGES_INSTALLED_ENV = 'list_packages_installed_env'
     VCS_REMOTE_LISTING = 'vcs_remote_listing'
     STORE_PAGEVIEWS = 'store_pageviews'
+    SPHINX_PARALLEL = 'sphinx_parallel'
 
     FEATURES = (
         (USE_SPHINX_LATEST, _('Use latest version of Sphinx')),
@@ -1562,6 +1563,10 @@ def add_features(sender, **kwargs):
             STORE_PAGEVIEWS,
             _('Store pageviews for this project'),
         ),
+        (
+            SPHINX_PARALLEL,
+            _('Use "-j auto" when calling sphinx-build'),
+        ),
     )
 
     projects = models.ManyToManyField(


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
Delete account form isn't an effective double check
## Details

* URL: https://readthedocs.org/accounts/delete/

This form pre-populates the username such that it isn't an effective double check on the user deleting their account.

![Screen Shot 2020-05-19 at 3 10 22 PM](https://user-images.githubusercontent.com/185043/82383646-1d836e00-99e3-11ea-87f9-c5b5825ca485.png)




Discussion:
What about implementing something similar to the way GitHub handles users deleting a repository?

Before clicking **Delete this repository**:
![Screen Shot 2020-05-20 at 12 04 37 PM](https://user-images.githubusercontent.com/7004949/82486788-3dbe3600-9a92-11ea-8356-be87b1c5d410.png)

After clicking **Delete this repository**:
![Screen Shot 2020-05-20 at 12 04 46 PM](https://user-images.githubusercontent.com/7004949/82486809-431b8080-9a92-11ea-9cc9-4773ad47cc47.png)
@santos22 that's what we already have, but the problem is the field is autocompleted by the browser, that should be disabled.
Whoops. I wasn't aware - good to know!
It's actually autocompleted server side.

Resolving patch:
diff --git a/readthedocs/profiles/views.py b/readthedocs/profiles/views.py
--- a/readthedocs/profiles/views.py
+++ b/readthedocs/profiles/views.py
@@ -84,6 +84,7 @@ def form_valid(self, form):
 
     def get_form(self, data=None, files=None, **kwargs):
         kwargs['instance'] = self.get_object()
+        kwargs['initial'] = {'username': ''}
         return super().get_form(data, files, **kwargs)
 
     def get_success_url(self):


Test patch:
Repository: readthedocs/readthedocs.org

Issue:
Use up to date version of docutils by default
We are currently using `docutils==0.14` by default, but `docutils==0.16` is out as of 2020-01. There are some inconsistencies in reST syntax because of the difference. Also, docutils doesn't version their docs :( This means an author using examples from the docutils docs might try to use features from 0.16. A failure case here would mean Sphinx skips the content.


Discussion:

Resolving patch:
diff --git a/readthedocs/doc_builder/python_environments.py b/readthedocs/doc_builder/python_environments.py
--- a/readthedocs/doc_builder/python_environments.py
+++ b/readthedocs/doc_builder/python_environments.py
@@ -335,7 +335,11 @@ def install_core_requirements(self):
         requirements = [
             'Pygments==2.3.1',
             'setuptools==41.0.1',
-            'docutils==0.14',
+            self.project.get_feature_value(
+                Feature.DONT_INSTALL_DOCUTILS,
+                positive='',
+                negative='docutils==0.14',
+            ),
             'mock==1.0.1',
             'pillow==5.4.1',
             'alabaster>=0.7,<0.8,!=0.7.5',
diff --git a/readthedocs/projects/models.py b/readthedocs/projects/models.py
--- a/readthedocs/projects/models.py
+++ b/readthedocs/projects/models.py
@@ -1523,6 +1523,7 @@ def add_features(sender, **kwargs):
     # Feature constants - this is not a exhaustive list of features, features
     # may be added by other packages
     USE_SPHINX_LATEST = 'use_sphinx_latest'
+    DONT_INSTALL_DOCUTILS = 'dont_install_docutils'
     ALLOW_DEPRECATED_WEBHOOKS = 'allow_deprecated_webhooks'
     PIP_ALWAYS_UPGRADE = 'pip_always_upgrade'
     DONT_OVERWRITE_SPHINX_CONTEXT = 'dont_overwrite_sphinx_context'
@@ -1556,6 +1557,12 @@ def add_features(sender, **kwargs):
 
     FEATURES = (
         (USE_SPHINX_LATEST, _('Use latest version of Sphinx')),
+        (
+            DONT_INSTALL_DOCUTILS,
+            _(
+                'Do not install docutils as requirement for build documentation',
+            ),
+        ),
         (ALLOW_DEPRECATED_WEBHOOKS, _('Allow deprecated webhook views')),
         (PIP_ALWAYS_UPGRADE, _('Always run pip install --upgrade')),
         (


Test patch: