Package: experimental.securityindexing
experimental.securityindexingDescriptionThis package aims to address a long-standing performance issue in Plone:Under circumstances which require security for an object to be re-indexed,
for example, when adding roles to users or groups via the @@sharing view,
the object for which the sharing form is being used and all of its decendant objects
in the content tree are unconditionally fetched from the database (ZODB) and
their security attributes (allowedRolesAndUsers) are re-indexed.This can yield terrible performance for some sites, depending upon the size (number of objects),
and workflow complexity.InstallationTo install this package, addexperimental.securityindexingto your Plone sites’
eggs and re-run buildout:Testing it outThis package provides some rudementry benchmarks which are aimed to be a sanity test
answering the question:“Are we at least faster than the previous implementation by default?”These benchmarks should not be treated as authorative, nor indicative of the performance
you should expect on a real site.To do so, the best way to ascertain results is to install the package in a development/staging
versions of your site aginst some realistic data.Installation is as above, but add the extras[benchmarks,tests]:Please read thebenchmark docsfor details.Existing behaviour in Plone 4.x sitesWhen local roles are assigned to user on a given folderish content item,
the folder will be indexed and all of it’s descendants (child folders) -
unconditionally.Depending upon the combination of:The structure of the treeThe number of descendant content items (depth and breath of the sub-tree
“beneath” the object being edited)This behaviour is currently implemented twice (Dexterity and Archetypes),
by the methodreindexObjectSecurity. This method invoked on the context
that local roles are being manipulated upon, in order to reflect the changes in theallowedRolesAndUsersKeywordindexin theZCatalog.TheallowedRolesAndUsersindex is used in Plone to determine which content a user can see.
e.g:In the site searchWithin a Plone 4.x site, the two implementations of thereindexObjectSecurityAPI are:Products.CMFCore.CMFCatalogAware.CatalogAware.reindexObjectSecuity(skip_self=False):
Indexes the content item (self). The keyword parameter skip_self
will be False when invoked from the @@sharing action.
For each child node in the content tree “beneath” this content item,
fetch that object, ultimately via the ZCatalog.unrestrictedTraverse API,
and re-index each one, unconditionally.Products.Archetypes.CatalogMultiplex.CatalogMultiplex
The archetypes tool is used to look up all catalogs that have
been registered for themeta_typeof the content item (self).N.B Both these implementations implement the Products.CMFCore.interfaces.ICatalogAwareinterface.The expensive operations seem to be:“Waking up” each child node viaunrestrictedTraverseWhen any local roles of significance are assigned to the object,
the indexer for local roles (Products.CMFPlone.CatalogTool.allowedRolesAndUsers)
invokes Products.PlonePAS.plugins.local_roles.LocalRolesManager.getAllLocalRolesInContext API,
which performs the following algorithm for each object to be indexed:Acquire the inner context.acquire the content object and it’s parent.
and calculate the unique set of local roles for the content object.If __ac_block_local_roles__ is not set,  exit and returns the local roles calculated.Repeats 2. until a parent is None (root of the tree).The goals of any solution to address the previously described performance issue(s) are:Wake up as few objects as possible.Where local roles information has not changed, avoid re-indexing.The following scheme was envisioned to optimise the above algorithm:When an object is indexed for the first timeCatalogTool.indexObject,
persist a unique token representing the unique set of local roles and the __ac_local_roles_block__
flag, along with the object’s id and physical path in a “shadow tree” which has
the same form as the main content tree (ZODB).Avoid re-indexing where possible, avoid waking up content objects:2.1 Given an item of contentobj, determine the set of child objects that need to be re-indexed,retrieve the node corresponding to theobjfrom the shadow tree, and each node representingobj’s corresponding descendants, and group these nodes by the unique local roles.2.2 For each group of nodes, retrieve the content object corresponding to the first node in the group.2.3 Ask the first object for it’sallowedRolesAndUsers(aka local roles)2.4 Index each node in group, supplying a faux object (either the shadow node or some otherobject standing-in for the content object)CreditThis work has been done as part of thePlone Intranet project.
Work sponsored byNetsight Internet Solutions.Changelog0.6 (2014-06-04)Nothing changed yet.0.5 (2014-06-04)Nothing changed yet.0.4 (2014-06-04)Fix versioning metadata, bumped version for new release.0.2 (2014-06-04)Corrected package metadata.Update README.rst to remove wrongly named branch in badge urls.Control panel configlet for synchronising/view state of shadowtree.Use itertools to group nodesThe shadowtree tool is now a site-local tool (but managed by a global utility)Add functional tests0.2dev (unreleasd)0.1dev (unreleasd)0.1 (2014-05-16)Initial pypi release.0.1 (2014-05-16)Initial pypi release.