Repository: plone/Products.CMFPlone

Issue:
Issues with new features in GenericSetup
The PR https://github.com/zopefoundation/Products.GenericSetup/pull/16 was merged but has negative effect for Plone. See https://github.com/zopefoundation/Products.GenericSetup/issues/19



Discussion:
I removed Products.GenericSetup from the coredev-checkouts untill we know more.

Also it broke the build...

This should be fixed by 1ff71cad9a4b22182b200ea71fad7edc4c08caa2.  Here we basically revert to the old GS behavior of reapplying all profiles, at least when we create a site. That is probably fine for the moment, until we figure out why the new behavior breaks.
I wrote the code so that it works with old and new GS. That makes it easier to add GS to the checkouts again and see what happens.

This should be done in Plone 4.3 too, but let's first see if this really helps on Plone 5. Pull request for checkout of GenericSetup added, so we can check this.

I created GS exports of a Plone Site generated with old and with new setuptools, without my fix from above. This is to see what else might have changed. I see two things:
- registry.xml has a different order for several items. I have checked: the reordered items are still all there, with the same value, just a different order.
- plone.app.discussion workflow settings are not applied, as Philip already noticed. Or rather, they are applied, as I can see with some added print/pdb statements, but something overwrites it.

```
$ diff -U 3 -r good bad
diff -U 3 -r good/registry.xml bad/registry.xml
--- good/registry.xml   2015-09-20 01:51:00.000000000 +0200
+++ bad/registry.xml    2015-09-20 02:05:45.000000000 +0200
@@ -2642,10 +2642,10 @@
       </value_type>
     </field>
     <value>
+      <element>resource-plone-app-discussion-javascripts-comments</element>
       <element>jquery-highlightsearchterms</element>
       <element>resource-plone-app-event-event-css</element>
       <element>resource-plone-app-event-event-js</element>
-      <element>resource-plone-app-discussion-javascripts-comments</element>
     </value>
   </record>
   <record name="plone.bundles/plone-logged-in.compile" interface="Products.CMFPlone.interfaces.resources.IBundleRegistry" field="compile">
@@ -3536,8 +3536,8 @@
       <element key="plone-toolbar-submenu-header-color">lighten(#000, 80%)</element>
       <element key="plone-toolbar-published-color">{plone-link-color}</element>
       <element key="mockuplessPath">\"{site_url}/++resource++mockupless/\"</element>
-      <element key="plone-toolbar-text-color">rgba(255,255,255,1)</element>
       <element key="bowerPath">\"{site_url}/++plone++static/components/\"</element>
+      <element key="plone-link-color">rgba(0,123,179,1)</element>
       <element key="plone-toolbar-internally-published-color">rgb(136,61,250)</element>
       <element key="plone-container-lg">1170px</element>
       <element key="plone-toolbar-submenu-text-color">lighten(#000, 90%)</element>
@@ -3546,23 +3546,23 @@
       <element key="plone-screen-sm-max">(@plone-screen-md-min + 1)</element>
       <element key="plone-screen-md-min">992px</element>
       <element key="plone-container-sm">750px</element>
-      <element key="plone-toolbar-link">{plone-link-color}</element>
+      <element key="plone-screen-lg-min">1200px</element>
       <element key="plone-left-toolbar">60px</element>
       <element key="plone-screen-md-max">(@plone-screen-lg-min + 1)</element>
       <element key="plone-toolbar-font-secundary">sans-serif</element>
+      <element key="plone-gray-lighter">lighten(#000, 80%)</element>
       <element key="plone-toolbar-pending-color">rgb(226,231,33)</element>
       <element key="plone-toolbar-separator-color">rgba(255,255,255,.17)</element>
-      <element key="plone-gray-lighter">lighten(#000, 80%)</element>
-      <element key="plone-gray-light">lighten(#000, 46.5%)</element>
+      <element key="plone-toolbar-text-color">rgba(255,255,255,1)</element>
       <element key="plone-screen-xs-min">480px</element>
       <element key="plone-toolbar-submenu-bg">rgba(20,20,20,.9)</element>
       <element key="plone-screen-xs-max">(@plone-screen-sm-min + 1)</element>
       <element key="plone-container-md">970px</element>
       <element key="plone-toolbar-draft-color">rgb(250,184,42)</element>
-      <element key="plone-screen-lg-min">1200px</element>
+      <element key="plone-toolbar-link">{plone-link-color}</element>
       <element key="icon-font-path">\"../fonts/\"</element>
       <element key="plone-toolbar-bg">rgba(0,0,0,.9)</element>
-      <element key="plone-link-color">rgba(0,123,179,1)</element>
+      <element key="plone-gray-light">lighten(#000, 46.5%)</element>
     </value>
   </record>
   <record name="plone.libraries_atd_ignore_strings" interface="Products.CMFPlone.interfaces.controlpanel.ITinyMCESchema" field="libraries_atd_ignore_strings">
Only in good/workflows: comment_review_workflow
diff -U 3 -r good/workflows.xml bad/workflows.xml
--- good/workflows.xml  2015-09-20 01:51:01.000000000 +0200
+++ bad/workflows.xml   2015-09-20 01:49:13.000000000 +0200
@@ -2,7 +2,6 @@
 <object name="portal_workflow" meta_type="Plone Workflow Tool">
  <property
     name="title">Contains workflow definitions for your portal</property>
- <object name="comment_review_workflow" meta_type="Workflow"/>
  <object name="folder_workflow" meta_type="Workflow"/>
  <object name="intranet_folder_workflow" meta_type="Workflow"/>
  <object name="intranet_workflow" meta_type="Workflow"/>
@@ -26,9 +25,7 @@
   <type type_id="ATSimpleIntCriterion"/>
   <type type_id="ATSimpleStringCriterion"/>
   <type type_id="ATSortCriterion"/>
-  <type type_id="Discussion Item">
-   <bound-workflow workflow_id="one_state_workflow"/>
-  </type>
+  <type type_id="Discussion Item"/>
   <type type_id="File"/>
   <type type_id="Image"/>
   <type type_id="Plone Site"/>
```

I made an export of the component registry tab too, but this showed no difference.
There might be other differences that are not caught by these two checks.
In plone.api tests, IMimeTypesRegistryTool gave a ComponentLookupError.

I re-added it to the checkouts and it is green http://jenkins.plone.org/job/plone-5.0-python-2.7/2817/


Resolving patch:
diff --git a/Products/CMFPlone/factory.py b/Products/CMFPlone/factory.py
--- a/Products/CMFPlone/factory.py
+++ b/Products/CMFPlone/factory.py
@@ -7,12 +7,6 @@
 from zope.event import notify
 from zope.interface import implements
 from zope.site.hooks import setSite
-try:
-    # GenericSetup 1.7.8 and higher
-    from Products.GenericSetup.tool import DEPENDENCY_STRATEGY_REAPPLY
-    DEPENDENCY_STRATEGY_REAPPLY  # pyflakes
-except ImportError:
-    DEPENDENCY_STRATEGY_REAPPLY = None
 
 _TOOL_ID = 'portal_setup'
 _DEFAULT_PROFILE = 'Products.CMFPlone:plone'
@@ -96,18 +90,7 @@ def addPloneSite(context, site_id, title='Plone site', description='',
     setSite(site)
 
     setup_tool.setBaselineContext('profile-%s' % profile_id)
-    # TODO: we somehow rely on reapplying all profiles, instead of
-    # being happy with upgrades.
-    if DEPENDENCY_STRATEGY_REAPPLY is None:
-        # Older GenericSetup.  Reapply is the default.  No alternative
-        # strategy can be given.
-        setup_tool.runAllImportStepsFromProfile('profile-%s' % profile_id)
-    else:
-        # Newer GenericSetup.  Upgrade is the default.  We want to
-        # reapply.
-        setup_tool.runAllImportStepsFromProfile(
-            'profile-%s' % profile_id,
-            dependency_strategy=DEPENDENCY_STRATEGY_REAPPLY)
+    setup_tool.runAllImportStepsFromProfile('profile-%s' % profile_id)
 
     reg = queryUtility(IRegistry, context=site)
     reg['plone.portal_timezone'] = portal_timezone
@@ -116,13 +99,8 @@ def addPloneSite(context, site_id, title='Plone site', description='',
     reg['plone.available_languages'] = [default_language]
 
     if setup_content:
-        if DEPENDENCY_STRATEGY_REAPPLY is None:
-            setup_tool.runAllImportStepsFromProfile(
-                'profile-%s' % content_profile_id)
-        else:
-            setup_tool.runAllImportStepsFromProfile(
-                'profile-%s' % content_profile_id,
-                dependency_strategy=DEPENDENCY_STRATEGY_REAPPLY)
+        setup_tool.runAllImportStepsFromProfile(
+            'profile-%s' % content_profile_id)
 
     props = dict(
         title=title,
@@ -133,13 +111,8 @@ def addPloneSite(context, site_id, title='Plone site', description='',
     site.manage_changeProperties(**props)
 
     for extension_id in extension_ids:
-        if DEPENDENCY_STRATEGY_REAPPLY is None:
-            setup_tool.runAllImportStepsFromProfile(
-                'profile-%s' % extension_id)
-        else:
-            setup_tool.runAllImportStepsFromProfile(
-                'profile-%s' % extension_id,
-                dependency_strategy=DEPENDENCY_STRATEGY_REAPPLY)
+        setup_tool.runAllImportStepsFromProfile(
+            'profile-%s' % extension_id)
 
     if snapshot is True:
         setup_tool.createSnapshot('initial_configuration')
diff --git a/Products/CMFPlone/setuphandlers.py b/Products/CMFPlone/setuphandlers.py
--- a/Products/CMFPlone/setuphandlers.py
+++ b/Products/CMFPlone/setuphandlers.py
@@ -113,6 +113,14 @@ def addCacheForResourceRegistry(portal):
         reg.ZCacheable_setEnabled(1)
 
 
+def purgeProfileVersions(portal):
+    """
+    Purge profile dependency versions.
+    """
+    setup = getToolByName(portal, 'portal_setup')
+    setup._profile_upgrade_versions = {}
+
+
 def setProfileVersion(portal):
     """
     Set profile version.
@@ -176,22 +184,27 @@ def importFinalSteps(context):
     if context.readDataFile('plone-final.txt') is None:
         return
     site = context.getSite()
+
+    # Unset all profile upgrade versions in portal_setup.  Our default
+    # profile should only be applied when creating a new site, so this
+    # list of versions should be empty.  But some tests apply it too.
+    # This should not be done as it should not be needed.  The profile
+    # is a base profile, which means all import steps are run in purge
+    # mode.  So for example an extra workflow added by
+    # plone.app.discussion is purged.  When plone.app.discussion is
+    # still in the list of profile upgrade versions, with the default
+    # dependency strategy it will not be reapplied again, which leaves
+    # you with a site that misses stuff.  So: when applying our
+    # default profile, start with a clean slate in these versions.
+    purgeProfileVersions(site)
+
+    # Set out default profile version.
     setProfileVersion(site)
 
     # Install our dependencies
-    # TODO: we somehow rely on reapplying all profiles, instead of
-    # being happy with upgrades.
     st = getToolByName(site, "portal_setup")
-    try:
-        # GenericSetup 1.7.8 and higher
-        from Products.GenericSetup.tool import DEPENDENCY_STRATEGY_REAPPLY
-    except ImportError:
-        st.runAllImportStepsFromProfile(
-            "profile-Products.CMFPlone:dependencies")
-    else:
-        st.runAllImportStepsFromProfile(
-            "profile-Products.CMFPlone:dependencies",
-            dependency_strategy=DEPENDENCY_STRATEGY_REAPPLY)
+    st.runAllImportStepsFromProfile(
+        "profile-Products.CMFPlone:dependencies")
 
     assignTitles(site)
     replace_local_role_manager(site)


Test patch: