Repository: plone/Products.CMFPlone

Issue:
Plone 5.2 advanced search breaks when searching with a date range
This happens on a fresh Plone 5.2, on Python 3.

Steps to reproduce:
1. Go to http://localhost:8080/Plone/@@search
2. Under "Filter results" -> "New items since" choose "Last week"

A traceback is thrown in the terminal:
```
2019-12-30 23:21:49,512 ERROR   [Zope.SiteErrorLog:251][waitress] 1577758909.50604490.5644442555016539 http://localhost:8080/Plone/@@search
Traceback (innermost last):
  Module ZPublisher.WSGIPublisher, line 156, in transaction_pubevents
  Module ZPublisher.WSGIPublisher, line 338, in publish_module
  Module ZPublisher.WSGIPublisher, line 256, in publish
  Module ZPublisher.mapply, line 85, in mapply
  Module ZPublisher.WSGIPublisher, line 62, in call_object
  Module zope.browserpage.simpleviewclass, line 41, in __call__
  Module Products.Five.browser.pagetemplatefile, line 126, in __call__
  Module Products.Five.browser.pagetemplatefile, line 61, in __call__
  Module zope.pagetemplate.pagetemplate, line 135, in pt_render
  Module Products.PageTemplates.engine, line 88, in __call__
  Module z3c.pt.pagetemplate, line 173, in render
  Module chameleon.zpt.template, line 306, in render
  Module chameleon.template, line 209, in render
  Module chameleon.utils, line 75, in raise_with_traceback
  Module chameleon.template, line 187, in render
  Module 951fe55309af08870bb8a0f61fbd6b21, line 3090, in render
  Module ca3ff2d3cceb4aebe5c885a0ce7e8708, line 582, in render_master
  Module ca3ff2d3cceb4aebe5c885a0ce7e8708, line 931, in render_content
  Module 951fe55309af08870bb8a0f61fbd6b21, line 2891, in __fill_main
  Module Products.CMFPlone.browser.search, line 81, in results
  Module Products.CMFPlone.CatalogTool, line 457, in searchResults
  Module Products.ZCatalog.ZCatalog, line 611, in searchResults
  Module Products.ZCatalog.Catalog, line 1091, in searchResults
  Module Products.ZCatalog.Catalog, line 634, in search
  Module Products.ZCatalog.Catalog, line 564, in _search_index
  Module Products.PluginIndexes.unindex, line 447, in query_index
  Module Products.PluginIndexes.unindex, line 398, in getRequestCacheKey
TypeError: unhashable type: 'record'

 - Expression: "context/@@main_template/macros/master"
 - Filename:   ... s.CMFPlone/Products/CMFPlone/browser/templates/search.pt
 - Location:   (line 6: col 23)
 - Source:     ... tal:use-macro="context/@@main_template/macros/master"
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 - Arguments:  template: <ViewPageTemplateFile - at 0x7f822ac46d90>
               options: {...} (0)
               args: <tuple - at 0x7f823c407050>
               nothing: <NoneType - at 0xa286b0>
               modules: <_SecureModuleImporter - at 0x7f82332acb10>
               request: <WSGIRequest - at 0x7f821b93d290>
               view: <SimpleViewClass from /trabajo/plone/buildout.coredev/src/Products.CMFPlone/Products/CMFPlone/browser/templates/search.pt search at 0x7f821b18d290>
               context: <ImplicitAcquisitionWrapper Plone at 0x7f821b0c1be0>
               views: <ViewMapper - at 0x7f821b1b1590>
               here: <ImplicitAcquisitionWrapper Plone at 0x7f821b0c1be0>
               container: <ImplicitAcquisitionWrapper Plone at 0x7f821b0c1be0>
               root: <ImplicitAcquisitionWrapper  at 0x7f822d660230>
               traverse_subpath: <list - at 0x7f821b111c80>
               user: <ImplicitAcquisitionWrapper - at 0x7f822bad2960>
               default: <object - at 0x7f8239e15fc0>
               repeat: {...} (0)
               loop: {...} (1)
               wrapped_repeat: <SafeMapping - at 0x7f822e14a2d0>
               target_language: <NoneType - at 0xa286b0>
               translate: <function translate at 0x7f821b927950>
               macroname: master
```

The request that fails is http://localhost:8080/Plone/@@search?sort_on=&sort_order=&SearchableText=&advanced_search=True&pt_toggle=%23&portal_type%3Alist=Document&portal_type%3Alist=Folder&portal_type%3Alist=Collection&created.query%3Arecord%3Alist%3Adate=2019%2F12%2F23&created.range%3Arecord=min

This is happening in https://github.com/zopefoundation/Products.ZCatalog/blob/e751eff05374e7f8b39d59b9b9e16e33fab47e11/src/Products/PluginIndexes/unindex.py#L398

```
(Pdb++) record.keys
[{'query': [DateTime('2019/12/23 00:00:00 GMT-3')], 'range': 'min'}]
```

Full query used
```
{'b_start': 0, 'b_size': 10, 'portal_type': ['Collection', 'Document', 'Folder'], 'created': {'query': [DateTime('2019/12/23 00:00:00 GMT-3')], 'range': 'min'}, 'show_inactive': False, 'path': '/Plone', 'allowedRolesAndUsers': ['Anonymous', 'Authenticated', 'Manager', 'user:admin']}
```
Remove the treatment of the "created" parameter in the search view
To fix #3007, @frapell did PR #3012. However, the problem occurs with any parameter that uses record notation in the URL. So I made PRs https://github.com/zopefoundation/Products.ZCatalog/pull/129 and https://github.com/zopefoundation/Products.ZCatalog/pull/130 in `ZCatalog`, to handle any record parameter.

Therefore, we must remove the treatment of the `created` parameter:

https://github.com/plone/Products.CMFPlone/blob/954942a1ec95d9c9472ecf791d8d55ca096b33bb/Products/CMFPlone/browser/search.py#L121-L124

when Plone 6/Plone 5.2 uses a version of Zope that contains these PRs.

Probably when Plone 6 is using Zope > 5.4 and Plone 5.2 is using Zope > 4.7


Discussion:
Not sure right now what would be the proper way to fix this...
Actually this error occurs with any search that has a record in the URL.

For example, the URL below also fails:

http://localhost:8080/Plone/@@search?end.query:record:list:date=2022-2-2+00%3A00%3A00&end.range:record=min

Therefore, the fix must be made to all parameters that have record notation, not just the `created` parameter.

The way to pass "dictionaries" in the URL is using record notation, as above. However, on the Plone side, you won't get a dictionary but a record. But https://github.com/zopefoundation/Products.ZCatalog/blob/e751eff05374e7f8b39d59b9b9e16e33fab47e11/src/Products/ZCatalog/query.py#L68 expects the parameter to be a dictionary.
@jensens @ale-rt @ericof @frapell I think the best solution to this problem is to test if the parameter is also a `ZPublisher.HTTPRequest.record` in:

https://github.com/zopefoundation/Products.ZCatalog/blob/91e7826dcfd196632fb1c9318cd5025b718f1058/src/Products/ZCatalog/query.py#L68

staying:

```python
if isinstance(param, dict) or isinstance(param, record):
```

I don't have Zope contributor agreement. So can anyone do a PR on ZCatalog with this small change?
Hi @wesleybl! 
I am pretty sure the plone contributor agreement will work for the zopefoundation as well these days, but maybe I am wrong.
CC @polyester 

@wesleybl @ale-rt Technically yes, but we haven't added all Plone contributors to the zopefoundation repo. I've added wesleybl there now.
Thanks @ale-rt and @polyester !

I will do the PR there.
We will also be able to close #3405. See: https://github.com/plone/Products.CMFPlone/issues/3405#issuecomment-1033466635
Pinning new `Products.ZCatalog` releases should be enough. I can't create releases for this package, but I can ask.
I will wait a bit, since Jenkins has problems now.

Resolving patch:
diff --git a/Products/CMFPlone/browser/search.py b/Products/CMFPlone/browser/search.py
--- a/Products/CMFPlone/browser/search.py
+++ b/Products/CMFPlone/browser/search.py
@@ -13,7 +13,6 @@
 from zope.component import queryUtility
 from zope.i18nmessageid import MessageFactory
 from zope.publisher.browser import BrowserView
-from ZPublisher.HTTPRequest import record
 from ZTUtils import make_query
 
 import json
@@ -122,11 +121,6 @@ def _filter_query(self, query):
                 # created not a mapping
                 del query['created']
 
-        # https://github.com/plone/Products.CMFPlone/issues/3007
-        # If 'created' exists and is of type 'record', then cast it as dict
-        if 'created' in query and isinstance(query['created'], record):
-            query['created'] = dict(query['created'])
-
         # respect `types_not_searched` setting
         types = query.get('portal_type', [])
         if 'query' in types:


Test patch: