anon-iclr-submission/benchname-bug-localization huggingface dataset:
Repository: spyder-ide/spyder
Issue: Above 5.4.3, Spyder launch stuck when loading Pythonpath manager

### Issue Report Checklist

* [x] Searched the [issues page](https://github.com/spyder-ide/spyder/issues?q=is%3Aissue) for similar reports
* [x] Read the relevant sections of the [Spyder Troubleshooting Guide](https://github.com/spyder-ide/spyder/wiki/Troubleshooting-Guide-and-FAQ) and followed its advice
* [x] Reproduced the issue after updating with ``conda update spyder`` (or ``pip``, if not using Anaconda)
* [ ] Could not reproduce inside ``jupyter qtconsole`` (if console-related)
* [x] Tried basic troubleshooting (if a bug/error)
    * [x] Restarted Spyder
    * [x] Reset preferences with ``spyder --reset``
    * [x] Reinstalled the latest version of [Anaconda](https://www.anaconda.com/download/)
    * [x] Tried the other applicable steps from the Troubleshooting Guide
* [x] Completed the **Problem Description**, **Steps to Reproduce** and **Version** sections below


## Problem Description

I have been running Spyder for many years. It works perfectly with 5.4.2, when updating to 5.4.3 or 5.4.4, launch gets stuck on "Launching code analysis"

### What steps reproduce the problem?

1. Launch Spyder >5.4.3
2. It gets stuck 

Personal note: I have been installing the last version of Spyder on Ubuntu 22.04.2 LTS through Anaconda on the laptops of my students, it works.

### What is the expected output? What do you see instead?

I would expected Spyder to launch, it gets stuck.

### Paste Traceback/Error Below (if applicable)

[spyder-debug.log](https://github.com/spyder-ide/spyder/files/12136318/spyder-debug.log)

## Versions
<!--- You can get this information from Help > About Spyder...
or (if Spyder won't launch) the "conda list" command
from the Anaconda Prompt/Terminal/command line. --->

* Spyder version: > 5.4.3
* Python version:  3.11.4
* Qt version: 5.15.8
* PyQt version: 5.15.7
* Operating System name/version: Ubuntu 22.04.2 LTS

### Dependencies

I cannot access the Menu since Spyder does not launch. Here is the conda environment I created to perform the final test (mamba create -n spyder-env python=3 spyder)

[mamba_env.txt](https://github.com/spyder-ide/spyder/files/12136317/mamba_env.txt)

For the Devs who are going to check this, thanks a lot for all your work. I really enjoy working with Spyder.


Changed files:
spyder/utils/environ.py

diff --git a/spyder/utils/environ.py b/spyder/utils/environ.py
index 546d94953..024085553 100644
--- a/spyder/utils/environ.py
+++ b/spyder/utils/environ.py
@@ -72,7 +72,8 @@ def get_user_environment_variables():
             else:
                 env = {}
             proc = run_shell_command(cmd, env=env, text=True)
-            stdout, stderr = proc.communicate()
+            # Use timeout to fix spyder-ide/spyder#21172
+            stdout, stderr = proc.communicate(timeout=0.5)
             env_var = eval(stdout, None)
     except Exception:
         return {}
Repository: readthedocs/readthedocs.org
Issue: 502 Bad Gateway on Projects > Versions page

The Projects > Versions page for our project is throwing a 502 Bad Gateway error. It's happening at the following URL: 

https://readthedocs.org/projects/htcondor/versions

I thought this might be a transient error but it's been happening consistently for the last day or so. All the other pages I've tried seem to work fine.

It's worth noting a similar ticket  #5441  was filed a couple of days ago for a 502 error on the Import a Repository page. This seems to be working correctly now.

## Details

* Read the Docs project URL: https://htcondor.readthedocs.io
* Build URL (if applicable): 
* Read the Docs username (if applicable): htcondor

## Expected Result

A list of the different versions of our docs, dropdown menus to connect them to different branches, etc.

## Actual Result

Server Error

502 - Web server received an invalid response while acting as a gateway or proxy server.

There is a problem with the page you are looking for, and it cannot be displayed. When the Web server (while acting as a gateway or proxy) contacted the upstream content server, it received an invalid response from the content server.


Changed files:
readthedocs/core/permissions.py, readthedocs/projects/views/public.py

diff --git a/readthedocs/core/permissions.py b/readthedocs/core/permissions.py
index 1c397eae1..a3bfa0423 100644
--- a/readthedocs/core/permissions.py
+++ b/readthedocs/core/permissions.py
@@ -9,6 +9,8 @@ class AdminPermissionBase:
 
     @classmethod
     def is_admin(cls, user, project):
+        # This explicitly uses "user in project.users.all" so that
+        # users on projects can be cached using prefetch_related or prefetch_related_objects
         return user in project.users.all() or user.is_superuser
 
     @classmethod
diff --git a/readthedocs/projects/views/public.py b/readthedocs/projects/views/public.py
index 2976cecb6..cc945b196 100644
--- a/readthedocs/projects/views/public.py
+++ b/readthedocs/projects/views/public.py
@@ -13,6 +13,7 @@ from django.contrib import messages
 from django.contrib.auth.models import User
 from django.core.cache import cache
 from django.core.files.storage import get_storage_class
+from django.db.models import prefetch_related_objects
 from django.http import HttpResponse, HttpResponseRedirect
 from django.shortcuts import get_object_or_404, render
 from django.urls import reverse
@@ -271,6 +272,9 @@ def project_versions(request, project_slug):
     if wiped and wiped_version.count():
         messages.success(request, 'Version wiped: ' + wiped)
 
+    # Optimize project permission checks
+    prefetch_related_objects([project], 'users')
+
     return render(
         request,
         'projects/project_version_list.html',
Repository: readthedocs/readthedocs.org
Issue: Problem building project on fresh install

I've installed a fresh rtd instance in docker behind a proxy like this : 

```
FROM python:2
ARG proxy
ENV http_proxy $proxy
ENV https_proxy $proxy
ENV no_proxy "localhost,127.0.0.1"

RUN git clone https://github.com/rtfd/readthedocs.org.git
WORKDIR readthedocs.org
RUN pip install -r requirements.txt
RUN python manage.py migrate
RUN python manage.py loaddata test_data

```

It runs well untill i try to build a project (any). I get : 
```

 python /readthedocs.org/user_builds/test_project/envs/latest/bin/pip install --use-wheel --upgrade --cache-dir /readthedocs.org/user_builds/test_project/.cache/pip Pygments==2.2.0 setuptools<40 docutils==0.13.1 mock==1.0.1 pillow==2.6.1 alabaster>=0.7,<0.8,!=0.7.5 commonmark==0.5.4 recommonmark==0.4.0 sphinx==1.7.4 sphinx-rtd-theme<0.5 readthedocs-sphinx-ext<0.6
Usage:   
  pip install [options] <requirement specifier> [package-index-options] ...
  pip install [options] -r <requirements file> [package-index-options] ...
  pip install [options] [-e] <vcs project url> ...
  pip install [options] [-e] <local project path> ...
  pip install [options] <archive url/path> ...

no such option: --use-wheel 
```

I've found that : ``--use-wheel``  is deprecated [since pip 7](https://pip.pypa.io/en/stable/news/#id25) (in favor of --only-binary) and removed since[ pip 10 beta 1 ](https://pip.pypa.io/en/stable/news/#b1-2018-03-31 )

Removing --use-wheel from the code ( /readthedocs/rtd_tests/tests/test_doc_building.py  and /readthedocs/doc_builder/python_environments.py ) fixes the problem in my situation.

Is it the right thing to do ? 


Changed files:
readthedocs/doc_builder/python_environments.py, readthedocs/rtd_tests/tests/test_doc_building.py

diff --git a/readthedocs/doc_builder/python_environments.py b/readthedocs/doc_builder/python_environments.py
index 8b5427895..6e5b82639 100644
--- a/readthedocs/doc_builder/python_environments.py
+++ b/readthedocs/doc_builder/python_environments.py
@@ -253,7 +253,7 @@ class Virtualenv(PythonEnvironment):
             'python',
             self.venv_bin(filename='pip'),
             'install',
-            '--use-wheel',
+            '--only-binary=:all:',
             '--upgrade',
             '--cache-dir',
             self.project.pip_cache_path,
diff --git a/readthedocs/rtd_tests/tests/test_doc_building.py b/readthedocs/rtd_tests/tests/test_doc_building.py
index de5c7b92f..466f400aa 100644
--- a/readthedocs/rtd_tests/tests/test_doc_building.py
+++ b/readthedocs/rtd_tests/tests/test_doc_building.py
@@ -1137,7 +1137,7 @@ class TestPythonEnvironment(TestCase):
             'python',
             mock.ANY,  # pip path
             'install',
-            '--use-wheel',
+            '--only-binary=:all:',
             '--upgrade',
             '--cache-dir',
             mock.ANY,  # cache path
Repository: readthedocs/readthedocs.org
Issue: Search result links to prior version of a moved document

## Details

* Read the Docs project URL: https://hark.readthedocs.io/en/latest/

## Expected Result

Searching for "Portfolio" gets a top result which is this page:

https://hark.readthedocs.io/en/latest/example_notebooks/ConsPortfolioModel.html

## Actual Result

Searching for "Portfolio" gets to a 404 page:
https://hark.readthedocs.io/en/latest/example_notebooks/ConsPortfolioModelDoc.html

## Background information

- In this case, ConsPortfolioModelDoc was the name of the desired document in a previous version.
- The .rst documents point to this document via a symlink, which has been updated.
- A local sphinx build of the documentation gets the expected search result

Changed files:
readthedocs/projects/tasks.py

diff --git a/readthedocs/projects/tasks.py b/readthedocs/projects/tasks.py
index fbd822300..cd27a64e5 100644
--- a/readthedocs/projects/tasks.py
+++ b/readthedocs/projects/tasks.py
@@ -1104,11 +1104,14 @@ class UpdateDocsTaskStep(SyncRepositoryMixin, CachedEnvironmentMixin):
                 epub=epub,
                 delete_unsynced_media=delete_unsynced_media,
             ),
-            callback=sync_callback.s(
-                version_pk=self.version.pk,
-                commit=self.build['commit'],
-                build=self.build['id'],
-            ),
+        )
+
+        # All the JSON files are uploaded to storage prior to syncing
+        # so we should be fine to index the files without waiting
+        sync_callback.delay(
+            version_pk=self.version.pk,
+            commit=self.build['commit'],
+            build=self.build['id'],
         )
 
     def setup_python_environment(self):
Repository: readthedocs/readthedocs.org
Issue: Create Project API returns HTTP 500

## Details
I want to create a project with API v3 but it is giving me 500 Internal Error.

I have tried the example here: https://docs.readthedocs.io/en/stable/api/v3.html#project-create

* Read the Docs project URL: -
* Build URL (if applicable): -
* Read the Docs username (if applicable): limonkufu

## Expected Result

Successfully created project

## Actual Result

HTTP 500 Internal Server Error: Fail. Check back in a bit!



Changed files:
readthedocs/api/v3/serializers.py, readthedocs/api/v3/tests/test_projects.py

diff --git a/readthedocs/api/v3/serializers.py b/readthedocs/api/v3/serializers.py
index 7ca6226ec..99fb848bc 100644
--- a/readthedocs/api/v3/serializers.py
+++ b/readthedocs/api/v3/serializers.py
@@ -11,6 +11,7 @@ from rest_flex_fields.serializers import FlexFieldsSerializerMixin
 from rest_framework import serializers
 
 from readthedocs.builds.models import Build, Version
+from readthedocs.core.utils import slugify
 from readthedocs.projects.constants import (
     LANGUAGES,
     PROGRAMMING_LANGUAGES,
@@ -420,6 +421,14 @@ class ProjectCreateSerializer(FlexFieldsModelSerializer):
             'homepage',
         )
 
+    def validate_name(self, value):
+        potential_slug = slugify(value)
+        if Project.objects.filter(slug=potential_slug).exists():
+            raise serializers.ValidationError(
+                _('Project with slug "{0}" already exists.').format(potential_slug),
+            )
+        return value
+
 
 class ProjectUpdateSerializer(FlexFieldsModelSerializer):
 
diff --git a/readthedocs/api/v3/tests/test_projects.py b/readthedocs/api/v3/tests/test_projects.py
index 915477823..d9998d320 100644
--- a/readthedocs/api/v3/tests/test_projects.py
+++ b/readthedocs/api/v3/tests/test_projects.py
@@ -1,8 +1,10 @@
-from .mixins import APIEndpointMixin
+import django_dynamic_fixture as fixture
 from django.urls import reverse
 
 from readthedocs.projects.models import Project
 
+from .mixins import APIEndpointMixin
+
 
 class ProjectsEndpointTests(APIEndpointMixin):
 
@@ -143,6 +145,26 @@ class ProjectsEndpointTests(APIEndpointMixin):
             self._get_response_dict('projects-list_POST'),
         )
 
+    def test_import_existing_project(self):
+        fixture.get(
+            Project,
+            slug='test-project',
+            name='Test Project',
+        )
+        data = {
+            'name': 'Test Project',
+            'repository': {
+                'url': 'https://github.com/rtfd/template',
+                'type': 'git',
+            },
+            'homepage': 'http://template.readthedocs.io/',
+            'programming_language': 'py',
+        }
+
+        self.client.credentials(HTTP_AUTHORIZATION=f'Token {self.token.key}')
+        response = self.client.post(reverse('projects-list'), data)
+        self.assertContains(response, 'Project with slug \\\\"test-project\\\\" already exists.', status_code=400)
+
     def test_import_project_with_extra_fields(self):
         data = {
             'name': 'Test Project',
Repository: readthedocs/readthedocs.org
Issue: Build: incompatible Mercurial versions from VCS setup than build commands

## Details

* Read the Docs project URL: https://readthedocs.org/projects/taskcluster-taskgraph/
* Build URL (if applicable): https://readthedocs.org/projects/taskcluster-taskgraph/builds/16259866/
* Read the Docs username (if applicable): ahal

## Expected Result

The builds should succeed :)

## Actual Result

Builds fail as seen [here](https://readthedocs.org/projects/taskcluster-taskgraph/builds/16259866/). I don't believe anything changed on the repo side, this was [working a week or two ago](https://readthedocs.org/projects/taskcluster-taskgraph/builds/).

```
hg identify --id
abort: repository requires features unknown to this Mercurial: sparserevlog!
(see https://mercurial-scm.org/wiki/MissingRequirement for more information) 
```


Changed files:
readthedocs/vcs_support/backends/hg.py

diff --git a/readthedocs/vcs_support/backends/hg.py b/readthedocs/vcs_support/backends/hg.py
index e37cb738f..0ca6ec833 100644
--- a/readthedocs/vcs_support/backends/hg.py
+++ b/readthedocs/vcs_support/backends/hg.py
@@ -33,7 +33,14 @@ class Backend(BaseVCS):
     def clone(self):
         self.make_clean_working_dir()
         try:
-            output = self.run("hg", "clone", self.repo_url, ".")
+            # Disable sparse-revlog extension when cloning because it's not
+            # included in older versions of Mercurial and producess an error
+            # when using an old version. See
+            # https://github.com/readthedocs/readthedocs.org/pull/9042/
+
+            output = self.run(
+                "hg", "clone", "--config", "format.sparse-revlog=no", self.repo_url, "."
+            )
             return output
         except RepositoryError:
             raise RepositoryError(RepositoryError.CLONE_ERROR)
Repository: readthedocs/readthedocs.org
Issue: Development installation fails

After we updated `common` in #8368, the development installation doesn't work anymore:

```
$ inv docker.build
Traceback (most recent call last):
  File "/home/juanlu/.local/bin/inv", line 8, in <module>
    sys.exit(program.run())
  File "/home/juanlu/.local/pipx/venvs/invoke/lib/python3.8/site-packages/invoke/program.py", line 373, in run
    self.parse_collection()
  File "/home/juanlu/.local/pipx/venvs/invoke/lib/python3.8/site-packages/invoke/program.py", line 465, in parse_collection
    self.load_collection()
  File "/home/juanlu/.local/pipx/venvs/invoke/lib/python3.8/site-packages/invoke/program.py", line 696, in load_collection
    module, parent = loader.load(coll_name)
  File "/home/juanlu/.local/pipx/venvs/invoke/lib/python3.8/site-packages/invoke/loader.py", line 76, in load
    module = imp.load_module(name, fd, path, desc)
  File "/usr/lib/python3.8/imp.py", line 234, in load_module
    return load_source(name, filename, file)
  File "/usr/lib/python3.8/imp.py", line 171, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 702, in _load
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 848, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/tmp/readthedocs.org/tasks.py", line 32, in <module>
    common.tasks.upgrade_all_packages,
AttributeError: module 'common.tasks' has no attribute 'upgrade_all_packages'
```

Changed files:
tasks.py

diff --git a/tasks.py b/tasks.py
index 710622d1c..6299cff25 100644
--- a/tasks.py
+++ b/tasks.py
@@ -27,13 +27,6 @@ namespace.add_collection(
     name='github',
 )
 
-namespace.add_collection(
-    Collection(
-        common.tasks.upgrade_all_packages,
-    ),
-    name='packages',
-)
-
 namespace.add_collection(
     Collection.from_module(
         common.dockerfiles.tasks,
Repository: readthedocs/readthedocs.org
Issue: Clean deletion of a Version

When a Version is deleted, we are triggering `clear_artifacts` _and_ `symlink_project` but the later is triggered _before_ the object deletion; so, I think if the task is ran before it's effectively deleted the symlink won't be properly synced:

https://github.com/rtfd/readthedocs.org/blob/5472906a067adc60f42556108993255393408410/readthedocs/builds/models.py#L181-L187

To fix this we could chain some Celery task in this order:

1. trigger `clear_artifacts`
1. delete the Version object
1. trigger `symlink_projects`

Take into account that the first and last have to be executed in a broadcast call but the second one should be executed just once since it affects the database.

> This method `Version.delete` is not used in the codebase at all, so I'm not sure if this is a priority. I found that all the Versions are activated/deactivated but not deleted.

Ref: https://github.com/rtfd/readthedocs.org/pull/3649#issuecomment-370915276

Changed files:
readthedocs/builds/models.py

diff --git a/readthedocs/builds/models.py b/readthedocs/builds/models.py
index a4b272abe..dd4f71a1a 100644
--- a/readthedocs/builds/models.py
+++ b/readthedocs/builds/models.py
@@ -213,10 +213,18 @@ class Version(models.Model):
     def delete(self, *args, **kwargs):  # pylint: disable=arguments-differ
         from readthedocs.projects import tasks
         log.info('Removing files for version %s', self.slug)
-        broadcast(type='app', task=tasks.clear_artifacts, args=[self.get_artifact_paths()])
         broadcast(
-            type='app', task=tasks.symlink_project, args=[self.project.pk])
+            type='app',
+            task=tasks.clear_artifacts,
+            args=[self.get_artifact_paths()],
+        )
+        project_pk = self.project.pk
         super(Version, self).delete(*args, **kwargs)
+        broadcast(
+            type='app',
+            task=tasks.symlink_project,
+            args=[project_pk],
+        )
 
     @property
     def identifier_friendly(self):
Repository: readthedocs/readthedocs.org
Issue: Project.documentation_type is `auto`

We have 528 projects with the `Project.documentation_type` set in `auto` and this value is not allowed in the code.

This value is used, for example at: https://github.com/rtfd/readthedocs.org/blob/be4ff5b9c4fb6f8c4f8b093e98a1359491a52b1f/readthedocs/doc_builder/python_environments.py#L276

But the dictionary used to get the proper builder class doesn't have the `auto` value and make the build to fail:

https://github.com/rtfd/readthedocs.org/blob/be4ff5b9c4fb6f8c4f8b093e98a1359491a52b1f/readthedocs/doc_builder/loader.py#L15-L32

We do have a way to protect ourselves for this, but this is still happening in production.

https://github.com/rtfd/readthedocs.org/blob/be4ff5b9c4fb6f8c4f8b093e98a1359491a52b1f/readthedocs/projects/tasks.py#L429-L431

I suppose that this is still happening because of #3946 isn't deployed yet.

> This doesn't seem to be too important, but just filling this issue to keep track of this and make sure that it doesn't happen anymore after that PR is deployed.

Ref: https://sentry.io/read-the-docs/readthedocs-org/issues/472212666/events/latest/

Changed files:
readthedocs/projects/tasks.py

diff --git a/readthedocs/projects/tasks.py b/readthedocs/projects/tasks.py
index d871eb0a4..1673f7374 100644
--- a/readthedocs/projects/tasks.py
+++ b/readthedocs/projects/tasks.py
@@ -548,6 +548,7 @@ class UpdateDocsTaskStep(SyncRepositoryMixin):
         project_data['documentation_type'] = ret
         api_v2.project(self.project.pk).put(project_data)
         self.project.documentation_type = ret
+        self.version.project.documentation_type = ret
 
     def update_app_instances(self, html=False, localmedia=False, search=False,
                              pdf=False, epub=False):
Repository: readthedocs/readthedocs.org
Issue: Project.has_valid_clone is not really used/useful

While debugging a different issue (https://github.com/readthedocs/readthedocs-corporate/issues/332) I found that `Project.has_valid_clone` is present in just 3 places in the whole codebase and it's not really used/useful.

Besides, we making an extra API call (https://github.com/rtfd/readthedocs.org/blob/10b6d53cbcdb33d9241a885e730784ba13da35e8/readthedocs/projects/tasks.py#L404) to set this to `True` on each build triggered after the setup step is done but since it's not present in the serializer (https://github.com/rtfd/readthedocs.org/blob/10b6d53cbcdb33d9241a885e730784ba13da35e8/readthedocs/restapi/serializers.py#L47) we are _always_ making that PUT request which in fact it doesn't update anything because that value is lost.

I did a quick query in production and there is 0 project with a valid clone :)

```
In [1]: Project.objects.filter(has_valid_clone=True).count()
Out[1]: 0
```

So, 

1. we can remove this field from our database without problem and also the related code
2. we can add this field to the `ProjectAdminSerializer` and make it happen (PR #4107)

Changed files:
readthedocs/rtd_tests/tests/test_api.py, readthedocs/restapi/serializers.py

diff --git a/readthedocs/restapi/serializers.py b/readthedocs/restapi/serializers.py
index b4d8b2b9b..97f4335f1 100644
--- a/readthedocs/restapi/serializers.py
+++ b/readthedocs/restapi/serializers.py
@@ -60,6 +60,8 @@ class ProjectAdminSerializer(ProjectSerializer):
             'requirements_file',
             'python_interpreter',
             'features',
+            'has_valid_clone',
+            'has_valid_webhook',
         )
 
 
diff --git a/readthedocs/rtd_tests/tests/test_api.py b/readthedocs/rtd_tests/tests/test_api.py
index 726ec8b06..743b25e90 100644
--- a/readthedocs/rtd_tests/tests/test_api.py
+++ b/readthedocs/rtd_tests/tests/test_api.py
@@ -735,6 +735,8 @@ class APIVersionTests(TestCase):
                 'enable_epub_build': True,
                 'enable_pdf_build': True,
                 'features': ['allow_deprecated_webhooks'],
+                'has_valid_clone': False,
+                'has_valid_webhook': False,
                 'id': 6,
                 'install_project': False,
                 'language': 'en',
Repository: readthedocs/readthedocs.org
Issue: Adopting a projected as gold user fails

### How to reproduce it

1. sign up as Gold member
1. go to https://readthedocs.org/accounts/gold/subscription/
1. select the project that you want to adopt

### Expected Result

Adopts the project.

###  Actual Result

Fails with a 500.

https://sentry.io/read-the-docs/readthedocs-org/issues/587668658/

### The problem

This line

https://github.com/rtfd/readthedocs.org/blob/44e02def230b937e4eca396864de9fc81f4ef33f/readthedocs/gold/views.py#L109

cause the problem since we are receiving a "project name" and using it as "project slug".

Changed files:
readthedocs/gold/forms.py, readthedocs/rtd_tests/tests/test_gold.py

diff --git a/readthedocs/gold/forms.py b/readthedocs/gold/forms.py
index 12d1d3645..75eef5a59 100644
--- a/readthedocs/gold/forms.py
+++ b/readthedocs/gold/forms.py
@@ -5,7 +5,10 @@ from __future__ import absolute_import
 from builtins import object
 from django import forms
 
+from django.utils.translation import ugettext_lazy as _
+
 from readthedocs.payments.forms import StripeModelForm, StripeResourceMixin
+from readthedocs.projects.models import Project
 
 from .models import LEVEL_CHOICES, GoldUser
 
@@ -88,6 +91,14 @@ class GoldProjectForm(forms.Form):
         self.projects = kwargs.pop('projects', None)
         super(GoldProjectForm, self).__init__(*args, **kwargs)
 
+    def clean_project(self):
+        project_slug = self.cleaned_data.get('project', '')
+        project_instance = Project.objects.filter(slug=project_slug)
+        if not project_instance.exists():
+            raise forms.ValidationError(_('No project found.'))
+        else:
+            return project_slug
+
     def clean(self):
         cleaned_data = super(GoldProjectForm, self).clean()
         if self.projects.count() < self.user.num_supported_projects:
diff --git a/readthedocs/rtd_tests/tests/test_gold.py b/readthedocs/rtd_tests/tests/test_gold.py
index 41703f0c1..e0f5c1581 100644
--- a/readthedocs/rtd_tests/tests/test_gold.py
+++ b/readthedocs/rtd_tests/tests/test_gold.py
@@ -26,6 +26,13 @@ class GoldViewTests(TestCase):
         self.assertEqual(self.golduser.projects.count(), 1)
         self.assertEqual(resp.status_code, 302)
 
+    def test_incorrect_input_when_adding_projects(self):
+        self.assertEqual(self.golduser.projects.count(), 0)
+        incorrect_slug = 'xyz-random-incorrect-slug-xyz'
+        self.assertEqual(Project.objects.filter(slug=incorrect_slug).count(), 0)
+        resp = self.client.post(reverse('gold_projects'), data={'project': incorrect_slug})
+        self.assertFormError(resp, form='form', field='project', errors='No project found.')
+
     def test_too_many_projects(self):
         self.project2 = get(Project, slug='test2')
Repository: readthedocs/readthedocs.org
Issue: Return 404 at SubdomainMiddleware level if project doesn't exist

While deploying .com we found that `SubdomainMiddleware` doesn't check the existence of the project that it's injecting at `request.slug`.

So, any middleware that relies on this to do some extra work, will probably do something like `Project.objects.get(slug=request.slug)` and will fail.

We workarounded it by adding `try/except` block in the middleware that uses this, but we should probably just raise a 404 inside the `SubdomainMiddleware` itself instead of continue with the flow.

Code: https://github.com/rtfd/readthedocs.org/blob/74bbb3ac23b6b06a70517bc1b301961db6821e32/readthedocs/core/middleware.py#L72

Changed files:
readthedocs/core/middleware.py, readthedocs/rtd_tests/tests/test_middleware.py

diff --git a/readthedocs/core/middleware.py b/readthedocs/core/middleware.py
index 5123ccee6..7dce74588 100644
--- a/readthedocs/core/middleware.py
+++ b/readthedocs/core/middleware.py
@@ -69,6 +69,8 @@ class SubdomainMiddleware(object):
                 # Support ports during local dev
                 public_domain in host or public_domain in full_host
             ):
+                if not Project.objects.filter(slug=subdomain).exists():
+                    raise Http404(_('Project not found'))
                 request.subdomain = True
                 request.slug = subdomain
                 request.urlconf = SUBDOMAIN_URLCONF
diff --git a/readthedocs/rtd_tests/tests/test_middleware.py b/readthedocs/rtd_tests/tests/test_middleware.py
index 444a4f2d1..574708552 100644
--- a/readthedocs/rtd_tests/tests/test_middleware.py
+++ b/readthedocs/rtd_tests/tests/test_middleware.py
@@ -47,6 +47,13 @@ class MiddlewareTests(TestCase):
         self.assertEqual(request.urlconf, self.urlconf_subdomain)
         self.assertEqual(request.slug, 'pip')
 
+    @override_settings(PRODUCTION_DOMAIN='readthedocs.org')
+    def test_wrong_subdomain(self):
+        http_host = 'xyz-wrong-sub-domain-xyz.readthedocs.org'
+        request = self.factory.get(self.url, HTTP_HOST=http_host)
+        with self.assertRaises(Http404):
+            self.middleware.process_request(request)
+
     @override_settings(PRODUCTION_DOMAIN='readthedocs.org')
     def test_restore_urlconf_after_request(self):
         """
Repository: readthedocs/readthedocs.org
Issue: Stable builds are triggered recursively

On Sunday we found that a build for the `stable` version was triggered constantly (Example, http://readthedocs.org/projects/bugzilla/builds/)

@ericholscher hotfixed this with this commit https://github.com/rtfd/readthedocs.org/commit/83caf8fe8 to avoid the "Sunday problem", but we need to research and fix it properly.

We suspect that this could be introduced on #4433 and/or #4876.



Changed files:
readthedocs/vcs_support/backends/git.py, readthedocs/settings/test.py

diff --git a/readthedocs/settings/test.py b/readthedocs/settings/test.py
index f49dc8584..398f8664b 100644
--- a/readthedocs/settings/test.py
+++ b/readthedocs/settings/test.py
@@ -16,6 +16,7 @@ class CommunityTestSettings(CommunityDevSettings):
 
     DEBUG = False
     TEMPLATE_DEBUG = False
+    LOCAL_GIT_BRANCHES = True
 
     @property
     def LOGGING(self):  # noqa - avoid pep8 N802
diff --git a/readthedocs/vcs_support/backends/git.py b/readthedocs/vcs_support/backends/git.py
index 32e711a7e..8da10bb46 100644
--- a/readthedocs/vcs_support/backends/git.py
+++ b/readthedocs/vcs_support/backends/git.py
@@ -15,6 +15,7 @@ import re
 import git
 from builtins import str
 from django.core.exceptions import ValidationError
+from django.conf import settings
 from git.exc import BadName
 
 from readthedocs.config import ALL
@@ -177,12 +178,13 @@ class Backend(BaseVCS):
     def branches(self):
         repo = git.Repo(self.working_dir)
         versions = []
+        branches = []
 
-        # ``repo.branches`` returns local branches and
-        branches = repo.branches
         # ``repo.remotes.origin.refs`` returns remote branches
         if repo.remotes:
             branches += repo.remotes.origin.refs
+        if getattr(settings, 'LOCAL_GIT_BRANCHES', False):
+            branches += repo.branches
 
         for branch in branches:
             verbose_name = branch.name
Repository: readthedocs/readthedocs.org
Issue: The localization dropdown incorrectly refers to "Taiwanese"


"Taiwanese" is a different language, the term used in the dropdown is more commonly and accurately referred to as "Traditional Chinese".

Changed files:
readthedocs/settings/base.py

diff --git a/readthedocs/settings/base.py b/readthedocs/settings/base.py
index 805790153..548a75441 100644
--- a/readthedocs/settings/base.py
+++ b/readthedocs/settings/base.py
@@ -221,8 +221,8 @@ class CommunityBaseSettings(Settings):
         ('de', gettext('German')),
         ('gl', gettext('Galician')),
         ('vi', gettext('Vietnamese')),
-        ('zh-cn', gettext('Chinese')),
-        ('zh-tw', gettext('Taiwanese')),
+        ('zh-cn', gettext('Simplified Chinese')),
+        ('zh-tw', gettext('Traditional Chinese')),
         ('ja', gettext('Japanese')),
         ('uk', gettext('Ukrainian')),
         ('it', gettext('Italian')),
Repository: readthedocs/readthedocs.org
Issue: Can't edit canonical domain

## Details

Putting in a canonical domain and then trying to change the domain name results in "Only 1 Domain can be canonical at a time."

## Expected Result

Domain is updated with new domain name

## Actual Result

Form errors.


Changed files:
readthedocs/rtd_tests/tests/test_domains.py, readthedocs/projects/forms.py

diff --git a/readthedocs/projects/forms.py b/readthedocs/projects/forms.py
index 5cd9fad44..a2f868c8a 100644
--- a/readthedocs/projects/forms.py
+++ b/readthedocs/projects/forms.py
@@ -570,9 +570,10 @@ class DomainForm(forms.ModelForm):
 
     def clean_canonical(self):
         canonical = self.cleaned_data['canonical']
+        _id = self.initial.get('id')
         if canonical and Domain.objects.filter(
-                project=self.project, canonical=True).exclude(
-                    domain=self.cleaned_data['domain']).exists():
+                project=self.project, canonical=True
+        ).exclude(pk=_id).exists():
             raise forms.ValidationError(
                 _('Only 1 Domain can be canonical at a time.'))
         return canonical
diff --git a/readthedocs/rtd_tests/tests/test_domains.py b/readthedocs/rtd_tests/tests/test_domains.py
index a92e9fab2..c786bb523 100644
--- a/readthedocs/rtd_tests/tests/test_domains.py
+++ b/readthedocs/rtd_tests/tests/test_domains.py
@@ -81,6 +81,26 @@ class FormTests(TestCase):
                           project=self.project)
         self.assertFalse(form.is_valid())
 
+    def test_canonical_change(self):
+        """Make sure canonical can be properly changed"""
+        form = DomainForm({'domain': 'example.com', 'canonical': True},
+                          project=self.project)
+        self.assertTrue(form.is_valid())
+        domain = form.save()
+        self.assertEqual(domain.domain, 'example.com')
+
+        form = DomainForm({'domain': 'example2.com', 'canonical': True},
+                          project=self.project)
+        self.assertFalse(form.is_valid())
+        self.assertEqual(form.errors['canonical'][0], 'Only 1 Domain can be canonical at a time.')
+
+        form = DomainForm({'domain': 'example2.com', 'canonical': True},
+                          project=self.project,
+                          instance=domain)
+        self.assertTrue(form.is_valid())
+        domain = form.save()
+        self.assertEqual(domain.domain, 'example2.com')
+
 
 class TestAPI(TestCase):
Repository: readthedocs/readthedocs.org
Issue: MkDocs's media url shouldn't be hardcode with media_url = 'http://localhost:8000' + media_url

In file `readthedocs.org/readthedocs/doc_builder/backends/mkdocs.py`:

``` python
55        # Set mkdocs config values
56
57        media_url = getattr(settings, 'MEDIA_URL', 'https://media.readthedocs.org')
58
59        # Mkdocs needs a full domain here because it tries to link to local media files
60        if not media_url.startswith('http'):
61            media_url = 'http://localhost:8000' + media_url
```

Can u please to replace it with `SITE_URL` as the follows:

``` python
59        # Mkdocs needs a full domain here because it tries to link to local media files
60        if not media_url.startswith('http'):
61            media_url = getattr(settings, 'SITE_URL', None) + media_url
```


Changed files:
readthedocs/doc_builder/backends/mkdocs.py

diff --git a/readthedocs/doc_builder/backends/mkdocs.py b/readthedocs/doc_builder/backends/mkdocs.py
index 4833c99a6..e1ab80dba 100644
--- a/readthedocs/doc_builder/backends/mkdocs.py
+++ b/readthedocs/doc_builder/backends/mkdocs.py
@@ -58,7 +58,8 @@ class BaseMkdocs(BaseBuilder):
 
         # Mkdocs needs a full domain here because it tries to link to local media files
         if not media_url.startswith('http'):
-            media_url = 'http://localhost:8000' + media_url
+            domain = getattr(settings, 'PRODUCTION_DOMAIN')
+            media_url = 'http://{}/{}'.format(domain, media_url)
 
         if 'extra_javascript' in user_config:
             user_config['extra_javascript'].append('readthedocs-data.js')
Repository: readthedocs/readthedocs.org
Issue: Establish pip cache for docker containers

The containers aren't sharing the normal pip cache of `/home/docs/.cache/pip`, which also effects the wheel cache. We should explicitly specify a project-local pip cache dir to use, so that package install time can be reduced.

Raised in #1767 


Changed files:
readthedocs/projects/tasks.py, readthedocs/projects/models.py

diff --git a/readthedocs/projects/models.py b/readthedocs/projects/models.py
index c7584e92c..3c8cf4dfb 100644
--- a/readthedocs/projects/models.py
+++ b/readthedocs/projects/models.py
@@ -412,6 +412,11 @@ class Project(models.Model):
     def venv_path(self, version=LATEST):
         return os.path.join(self.doc_path, 'envs', version)
 
+    @property
+    def pip_cache_path(self):
+        """Path to pip cache"""
+        return os.path.join(self.doc_path, '.cache', 'pip')
+
     #
     # Paths for symlinks in project doc_path.
     #
diff --git a/readthedocs/projects/tasks.py b/readthedocs/projects/tasks.py
index d92576824..6146c1809 100644
--- a/readthedocs/projects/tasks.py
+++ b/readthedocs/projects/tasks.py
@@ -261,6 +261,8 @@ class UpdateDocsTask(Task):
             'install',
             '--use-wheel',
             '-U',
+            '--cache-dir',
+            self.project.pip_cache_path,
         ]
         if self.project.use_system_packages:
             # Other code expects sphinx-build to be installed inside the
@@ -294,6 +296,8 @@ class UpdateDocsTask(Task):
                 self.project.venv_bin(version=self.version.slug, filename='pip'),
                 'install',
                 '--exists-action=w',
+                '--cache-dir',
+                self.project.pip_cache_path,
                 '-r{0}'.format(requirements_file_path),
                 cwd=checkout_path,
                 bin_path=self.project.venv_bin(version=self.version.slug)
@@ -309,6 +313,8 @@ class UpdateDocsTask(Task):
                     self.project.venv_bin(version=self.version.slug, filename='pip'),
                     'install',
                     '--ignore-installed',
+                    '--cache-dir',
+                    self.project.pip_cache_path,
                     '.',
                     cwd=checkout_path,
                     bin_path=self.project.venv_bin(version=self.version.slug)
Repository: readthedocs/readthedocs.org
Issue: Tags are not saved when creating a new project

## Details
- Project URL: N/A (affects all new projects)
- Build URL (if applicable): N/A
- Read the Docs username (if applicable): N/A
### Steps
1. Go to https://readthedocs.org/dashboard/import/manual
2. Select any project to import
3. Check the "Edit advanced project options" option
4. Enter the following in the `Tags` field:
   
   `foo, bar, foo bar`
5. Complete remaining required fields and submit the form
## Expected Result

On https://readthedocs.org/projects/YOUR_PROJECT_NAME, three tags are listed:

`foo`, `bar`, and `"foo bar"`.
## Actual Result

No tags are listed. Tags appear after adding them again on: 

https://readthedocs.org/dashboard/YOUR_PROJECT_NAME/edit


Changed files:
readthedocs/rtd_tests/tests/test_project_views.py, readthedocs/projects/views/private.py

diff --git a/readthedocs/projects/views/private.py b/readthedocs/projects/views/private.py
index 70e945e86..c1e7058da 100644
--- a/readthedocs/projects/views/private.py
+++ b/readthedocs/projects/views/private.py
@@ -251,13 +251,18 @@ class ImportWizardView(ProjectSpamMixin, PrivateViewMixin, SessionWizardView):
         other side effects for now, by signalling a save without commit. Then,
         finish by added the members to the project and saving.
         """
+        form_data = self.get_all_cleaned_data()
+        extra_fields = ProjectExtraForm.Meta.fields
         # expect the first form
         basics_form = form_list[0]
         # Save the basics form to create the project instance, then alter
         # attributes directly from other forms
         project = basics_form.save()
-        for form in form_list[1:]:
-            for (field, value) in form.cleaned_data.items():
+        tags = form_data.pop('tags', [])
+        for tag in tags:
+            project.tags.add(tag)
+        for field, value in form_data.items():
+            if field in extra_fields:
                 setattr(project, field, value)
         basic_only = True
         project.save()
diff --git a/readthedocs/rtd_tests/tests/test_project_views.py b/readthedocs/rtd_tests/tests/test_project_views.py
index 1d4680c39..2ca6fe788 100644
--- a/readthedocs/rtd_tests/tests/test_project_views.py
+++ b/readthedocs/rtd_tests/tests/test_project_views.py
@@ -116,6 +116,7 @@ class TestAdvancedForm(TestBasicsForm):
             'description': 'Describe foobar',
             'language': 'en',
             'documentation_type': 'sphinx',
+            'tags': 'foo, bar, baz',
         }
 
     def test_form_pass(self):
@@ -129,6 +130,11 @@ class TestAdvancedForm(TestBasicsForm):
         self.assertIsNotNone(proj)
         data = self.step_data['basics']
         del data['advanced']
+        del self.step_data['extra']['tags']
+        self.assertItemsEqual(
+            [tag.name for tag in proj.tags.all()],
+            [u'bar', u'baz', u'foo']
+        )
         data.update(self.step_data['extra'])
         for (key, val) in data.items():
             self.assertEqual(getattr(proj, key), val)
Repository: readthedocs/readthedocs.org
Issue: Uninformative error when branch name contains a dot

We were just experimenting with some docs stuff, and @minrk created a branch name with a dot (`.`) in it. Builds on this branch all failed, and the only error message in the logs was "top level build failure". It took us a while to work out that the branch name was the problem.

I don't think it's especially important to allow branch names with dots, but it would be nice to at least get an error that pointed us to the branch name.


Changed files:
readthedocs/builds/models.py, readthedocs/projects/models.py

diff --git a/readthedocs/builds/models.py b/readthedocs/builds/models.py
index b53a3e361..5214d4340 100644
--- a/readthedocs/builds/models.py
+++ b/readthedocs/builds/models.py
@@ -184,10 +184,10 @@ class Version(models.Model):
         return data
 
     def get_conf_py_path(self):
-        conf_py_path = self.project.conf_file(self.slug)
-        conf_py_path = conf_py_path.replace(
-            self.project.checkout_path(self.slug), '')
-        return conf_py_path.replace('conf.py', '')
+        conf_py_path = self.project.conf_dir(self.slug)
+        checkout_prefix = self.project.checkout_path(self.slug)
+        conf_py_path = os.path.relpath(conf_py_path, checkout_prefix)
+        return conf_py_path
 
     def get_build_path(self):
         '''Return version build path if path exists, otherwise `None`'''
diff --git a/readthedocs/projects/models.py b/readthedocs/projects/models.py
index 088497783..b59576052 100644
--- a/readthedocs/projects/models.py
+++ b/readthedocs/projects/models.py
@@ -597,7 +597,7 @@ class Project(models.Model):
     def conf_dir(self, version=LATEST):
         conf_file = self.conf_file(version)
         if conf_file:
-            return conf_file.replace('/conf.py', '')
+            return os.path.dirname(conf_file)
 
     @property
     def is_type_sphinx(self):
Repository: readthedocs/readthedocs.org
Issue: Social Network Login Failure

When clicking on connect GitHub on the [social accounts page](https://readthedocs.org/accounts/social/connections/?) I get a message:

> An error occurred while attempting to login via your social network account.

There's a `?` in the url. Could that be a hint? Is it missing some request arguments? If I omit it, the bug persists.

Cheers!


Changed files:
readthedocs/settings/postgres.py

diff --git a/readthedocs/settings/postgres.py b/readthedocs/settings/postgres.py
index 2086b4fd4..788743d14 100644
--- a/readthedocs/settings/postgres.py
+++ b/readthedocs/settings/postgres.py
@@ -69,6 +69,9 @@ SOCIALACCOUNT_PROVIDERS = {
     'github': {'SCOPE': ['user:email', 'read:org', 'admin:repo_hook', 'repo:status']}
 }
 
+# allauth settings
+ACCOUNT_DEFAULT_HTTP_PROTOCOL = 'https'
+
 if not os.environ.get('DJANGO_SETTINGS_SKIP_LOCAL', False):
     try:
         from local_settings import *  # noqa
Repository: kivy/python-for-android
Issue: TestGetSystemPythonExecutable.test_virtualenv test fail

The `TestGetSystemPythonExecutable.test_virtualenv` and `TestGetSystemPythonExecutable.test_venv` tests started failing all of a sudden.
Error was:
```
ModuleNotFoundError: No module named \\'pytoml\\'\\n'
```
This ca be reproduced in local via:
```sh
pytest tests/test_pythonpackage_basic.py::TestGetSystemPythonExecutable::test_virtualenv
```



Changed files:
pythonforandroid/recipes/libzbar/__init__.py, pythonforandroid/recipes/libiconv/__init__.py

diff --git a/pythonforandroid/recipes/libiconv/__init__.py b/pythonforandroid/recipes/libiconv/__init__.py
index 6bde196e..530497a2 100644
--- a/pythonforandroid/recipes/libiconv/__init__.py
+++ b/pythonforandroid/recipes/libiconv/__init__.py
@@ -20,7 +20,7 @@ class LibIconvRecipe(Recipe):
         with current_directory(self.get_build_dir(arch.arch)):
             shprint(
                 sh.Command('./configure'),
-                '--host=' + arch.toolchain_prefix,
+                '--host=' + arch.command_prefix,
                 '--prefix=' + self.ctx.get_python_install_dir(),
                 _env=env)
             shprint(sh.make, '-j' + str(cpu_count()), _env=env)
diff --git a/pythonforandroid/recipes/libzbar/__init__.py b/pythonforandroid/recipes/libzbar/__init__.py
index 8d7d48e2..5b1b62d8 100644
--- a/pythonforandroid/recipes/libzbar/__init__.py
+++ b/pythonforandroid/recipes/libzbar/__init__.py
@@ -32,7 +32,7 @@ class LibZBarRecipe(Recipe):
             shprint(sh.Command('autoreconf'), '-vif', _env=env)
             shprint(
                 sh.Command('./configure'),
-                '--host=' + arch.toolchain_prefix,
+                '--host=' + arch.command_prefix,
                 '--target=' + arch.toolchain_prefix,
                 '--prefix=' + self.ctx.get_python_install_dir(),
                 # Python bindings are compiled in a separated recipe