Repository: readthedocs/readthedocs.org

Issue:
Subproject is not updated on save/build
When changing a subproject of a superproject, for instance updating the privacy level of the project, symlinks for the superproject are not updated. This might be a regression.

## To reproduce

* Create a project A, privacy level public
* Create a project B, privacy level private
* Assign project B as a subproject of project A
* Project B URLs should 404
* Alter project B privacy level to public
* Project B URLs still 404

## Expected Result

* Project B URLs shouldn't 404 after updating to public privacy level

## Resolving

We should either trigger a save of the project's superproject, or explicitly call a task for re-symlinking the project in the following cases:

* When a subproject is deleted
* When a subproject privacy level is changed
* When a subproject version privacy level is changed (?? not sure here, might need testing on this as well)


Discussion:
I was able to replicate this, none symlink on `A/projects` are created. When project A is re-builded the symlink is created.
Also when removing the subproject the symlink is not erased. Again, rebuilding the project A fix this.
Great, I'm glad this is reproduceable.

So, we should either trigger a save of the superproject in these cases, or explicitly call a task to resymlink the superproject. The second option definitely works, but perhaps a full save of the project makes more sense technically.

I'll update the description with more info
I followed the steps to reproduce it mentioned here and I was able to reproduce it.

Then, the solution proposed (call a `A.save()`) does work and I can access to http://localhost:8000/docs/a/projects/B/en/latest/

```
In [10]: a = Project.objects.get(slug='a')

In [11]: a.save()
[21/Feb/2018 14:08:34] readthedocs.core.symlink:93[2712]: INFO (Build) [a:] Symlinking subproject: b -> b
[21/Feb/2018 14:08:34] readthedocs.core.symlink:93[2712]: INFO (Build) [a:] Symlinking subproject: B -> b
[21/Feb/2018 14:08:34] readthedocs.core.symlink:93[2712]: INFO (Build) [a:] Symlinking Version: Version latest of A (1173)
[21/Feb/2018 14:08:34] celery.app.trace:123[2712]: INFO Task readthedocs.projects.tasks.symlink_project[fff11308-f0c1-4639-aeab-dea3541f2ad0] succeeded in 0.033074999002565164s: None
[21/Feb/2018 14:08:34] readthedocs.projects.tasks:980[2712]: INFO (Build) [a:] Updating static metadata
[21/Feb/2018 14:08:34] celery.app.trace:123[2712]: INFO Task readthedocs.projects.tasks.update_static_metadata[7f9b4e54-0399-43da-ab66-11b271848c15] succeeded in 0.010115133998624515s: None
```

I'm trying to write a test case to make this fail first, but I don't know how to manage symlinks from the tests and check the documentation URLs yet.

Resolving patch:
diff --git a/readthedocs/projects/models.py b/readthedocs/projects/models.py
--- a/readthedocs/projects/models.py
+++ b/readthedocs/projects/models.py
@@ -326,8 +326,26 @@ def save(self, *args, **kwargs):  # pylint: disable=arguments-differ
             log.exception('failed to sync supported versions')
         try:
             if not first_save:
-                broadcast(type='app', task=tasks.symlink_project,
-                          args=[self.pk],)
+                log.info(
+                    'Re-symlinking project and subprojects: project=%s',
+                    self.slug,
+                )
+                broadcast(
+                    type='app',
+                    task=tasks.symlink_project,
+                    args=[self.pk],
+                )
+                log.info(
+                    'Re-symlinking superprojects: project=%s',
+                    self.slug,
+                )
+                for superproject in self.superprojects.all():
+                    broadcast(
+                        type='app',
+                        task=tasks.symlink_project,
+                        args=[superproject.pk],
+                    )
+
         except Exception:
             log.exception('failed to symlink project')
         try:


Test patch:
diff --git a/readthedocs/rtd_tests/tests/test_project_symlinks.py b/readthedocs/rtd_tests/tests/test_project_symlinks.py
--- a/readthedocs/rtd_tests/tests/test_project_symlinks.py
+++ b/readthedocs/rtd_tests/tests/test_project_symlinks.py
@@ -5,16 +5,16 @@
 import os
 import shutil
 import tempfile
-import collections
-from functools import wraps
 
 import mock
 from django.conf import settings
+from django.core.urlresolvers import reverse
 from django.test import TestCase, override_settings
 from django_dynamic_fixture import get
 
 from readthedocs.builds.models import Version
 from readthedocs.projects.models import Project, Domain
+from readthedocs.projects.tasks import symlink_project
 from readthedocs.core.symlink import PublicSymlink, PrivateSymlink
 
 
@@ -908,3 +908,202 @@ def test_symlink_no_error(self):
             self.symlink.run()
         except:
             self.fail('Symlink run raised an exception on unicode slug')
+
+    def test_symlink_broadcast_calls_on_project_save(self):
+        """
+        Test calls to ``readthedocs.core.utils.broadcast`` on Project.save().
+
+        When a Project is saved, we need to check that we are calling
+        ``broadcast`` utility with the proper task and arguments to re-symlink
+        them.
+        """
+        with mock.patch('readthedocs.projects.models.broadcast') as broadcast:
+            project = get(Project)
+            # skipped on first save
+            broadcast.assert_not_called()
+
+            broadcast.reset_mock()
+            project.description = 'New description'
+            project.save()
+            # called once for this project itself
+            broadcast.assert_any_calls(
+                type='app',
+                task=symlink_project,
+                args=[project.pk],
+            )
+
+            broadcast.reset_mock()
+            subproject = get(Project)
+            # skipped on first save
+            broadcast.assert_not_called()
+
+            project.add_subproject(subproject)
+            # subproject.save() is not called
+            broadcast.assert_not_called()
+
+            subproject.description = 'New subproject description'
+            subproject.save()
+            # subproject symlinks
+            broadcast.assert_any_calls(
+                type='app',
+                task=symlink_project,
+                args=[subproject.pk],
+            )
+            # superproject symlinks
+            broadcast.assert_any_calls(
+                type='app',
+                task=symlink_project,
+                args=[project.pk],
+            )
+
+
+@override_settings()
+class TestPublicPrivateSymlink(TempSiterootCase, TestCase):
+
+    def setUp(self):
+        super(TestPublicPrivateSymlink, self).setUp()
+        from django.contrib.auth.models import User
+
+        self.user = get(User)
+        self.project = get(
+            Project, name='project', slug='project', privacy_level='public',
+            users=[self.user], main_language_project=None)
+        self.project.versions.update(privacy_level='public')
+        self.project.save()
+
+        self.subproject = get(
+            Project, name='subproject', slug='subproject', privacy_level='public',
+            users=[self.user], main_language_project=None)
+        self.subproject.versions.update(privacy_level='public')
+        self.subproject.save()
+
+    def test_change_subproject_privacy(self):
+        """
+        Change subproject's ``privacy_level`` creates proper symlinks.
+
+        When the ``privacy_level`` changes in the subprojects, we need to
+        re-symlink the superproject also to keep in sync its symlink under the
+        private/public roots.
+        """
+        filesystem_before = {
+            'private_cname_project': {},
+            'private_cname_root': {},
+            'private_web_root': {
+                'project': {
+                    'en': {},
+                },
+                'subproject': {
+                    'en': {},
+                },
+            },
+            'public_cname_project': {},
+            'public_cname_root': {},
+            'public_web_root': {
+                'project': {
+                    'en': {
+                        'latest': {
+                            'type': 'link',
+                            'target': 'user_builds/project/rtd-builds/latest',
+                        },
+                    },
+                    'projects': {
+                        'subproject': {
+                            'type': 'link',
+                            'target': 'public_web_root/subproject',
+                        },
+                    },
+                },
+                'subproject': {
+                    'en': {
+                        'latest': {
+                            'type': 'link',
+                            'target': 'user_builds/subproject/rtd-builds/latest',
+                        },
+                    },
+                },
+            },
+        }
+
+        filesystem_after = {
+            'private_cname_project': {},
+            'private_cname_root': {},
+            'private_web_root': {
+                'project': {
+                    'en': {},
+                    'projects': {
+                        'subproject': {
+                            'type': 'link',
+                            'target': 'private_web_root/subproject',
+                        },
+                    },
+                },
+                'subproject': {
+                    'en': {
+                        'latest': {
+                            'type': 'link',
+                            'target': 'user_builds/subproject/rtd-builds/latest',
+                        },
+                    },
+                },
+            },
+            'public_cname_project': {},
+            'public_cname_root': {},
+            'public_web_root': {
+                'project': {
+                    'en': {
+                        'latest': {
+                            'type': 'link',
+                            'target': 'user_builds/project/rtd-builds/latest',
+                        },
+                    },
+                    'projects': {},
+                },
+                'subproject': {
+                    'en': {},
+                },
+            },
+        }
+
+        self.assertEqual(self.project.subprojects.all().count(), 0)
+        self.assertEqual(self.subproject.superprojects.all().count(), 0)
+        self.project.add_subproject(self.subproject)
+        self.assertEqual(self.project.subprojects.all().count(), 1)
+        self.assertEqual(self.subproject.superprojects.all().count(), 1)
+
+        self.assertTrue(self.project.versions.first().active)
+        self.assertTrue(self.subproject.versions.first().active)
+        symlink_project(self.project.pk)
+
+        self.assertFilesystem(filesystem_before)
+
+        self.client.force_login(self.user)
+        self.client.post(
+            reverse('project_version_detail',
+                    kwargs={
+                        'project_slug': self.subproject.slug,
+                        'version_slug': self.subproject.versions.first().slug,
+                    }),
+            data={'privacy_level': 'private', 'active': True},
+        )
+
+        self.assertEqual(self.subproject.versions.first().privacy_level, 'private')
+        self.assertTrue(self.subproject.versions.first().active)
+
+        self.client.post(
+            reverse('projects_advanced',
+                    kwargs={
+                        'project_slug': self.subproject.slug,
+                    }),
+            data={
+                # Required defaults
+                'python_interpreter': 'python',
+                'default_version': 'latest',
+
+                'privacy_level': 'private',
+            },
+        )
+
+        self.assertTrue(self.subproject.versions.first().active)
+        self.subproject.refresh_from_db()
+        self.assertEqual(self.subproject.privacy_level, 'private')
+        self.assertFilesystem(filesystem_after)
diff --git a/requirements/testing.txt b/requirements/testing.txt
--- a/requirements/testing.txt
+++ b/requirements/testing.txt
@@ -9,3 +9,4 @@ Mercurial==4.4.2
 
 # local debugging tools
 pdbpp
+datadiff