Repository: wagtail/wagtail

Issue:
Postgres search fails on Wagtail admin page chooser search
### Issue Summary

When trying to search using postgres search backend on wagtail admin page chooser I get server error (500).

### Steps to Reproduce

1. Clone wagtaildemo https://github.com/wagtail/wagtaildemo (I am using the Vagrant VM from the repo).
2. Set wagtail search backend to postgres, migrate and update index as per instructions in http://docs.wagtail.io/en/v1.13/reference/contrib/postgres_search.html
  - Add `wagtail.contrib.postgres_search` to `INSTALLED_APPS`.
  - Add the following to the settings
     ```
     WAGTAILSEARCH_BACKENDS = {
         'default': {
             'BACKEND': 'wagtail.contrib.postgres_search.backend',
         },
     }
   - run `manage.py migrate` which successfully performs the postgres backend migration ` Applying postgres_search.0001_initial... OK`
   - run `migrate.py update_index`
3. Try to search inside a page chooser. I personally used "Link page" on the "Speakers" inline panel on an event page in wagtaildemo, e.g. http://localhost:8000/admin/pages/8/edit/
4. Any AJAX request made to the search fails with 500, see the log below.

Successfully reproduced on another project, so it seems to be an issue with the postgres search backend.

```
[13/Nov/2017 15:20:41] "GET /admin/choose-page/search/?page_type=wagtailcore.page&q=a&results_only=true HTTP/1.1" 500 21736
Internal Server Error: /admin/choose-page/search/
Traceback (most recent call last):
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/django/core/handlers/exception.py", line 41, in inner
    response = get_response(request)
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
    response = self._get_response(request)
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/django/views/decorators/cache.py", line 43, in _cache_controlled
    response = viewfunc(request, *args, **kw)
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/wagtail/wagtailadmin/urls/__init__.py", line 96, in wrapper
    return view_func(request, *args, **kwargs)
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/wagtail/wagtailadmin/decorators.py", line 31, in decorated_view
    return view_func(request, *args, **kwargs)
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/wagtail/wagtailadmin/views/chooser.py", line 171, in search
    for page in pages:
  File "/usr/local/lib/python3.4/_collections_abc.py", line 634, in __iter__
    v = self[i]
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/django/core/paginator.py", line 145, in __getitem__
    self.object_list = list(self.object_list)
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/wagtail/wagtailsearch/backends/base.py", line 172, in __iter__
    return iter(self.results())
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/wagtail/wagtailsearch/backends/base.py", line 138, in results
    self._results_cache = self._do_search()
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/wagtail/contrib/postgres_search/backend.py", line 297, in _do_search
    self.start, self.stop))
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/django/db/models/query.py", line 250, in __iter__
    self._fetch_all()
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/django/db/models/query.py", line 1118, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/wagtail/wagtailcore/query.py", line 369, in specific_iterator
    for pk, content_type in pks_and_types:
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/django/db/models/query.py", line 250, in __iter__
    self._fetch_all()
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/django/db/models/query.py", line 1118, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/django/db/models/query.py", line 122, in __iter__
    for row in compiler.results_iter():
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 836, in results_iter
    results = self.execute_sql(MULTI)
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 871, in execute_sql
    sql, params = self.as_sql()
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 423, in as_sql
    extra_select, order_by, group_by = self.pre_sql_setup()
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 47, in pre_sql_setup
    order_by = self.get_order_by()
  File "/home/vagrant/.virtualenvs/wagtaildemo/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 335, in get_order_by
    if (without_ordering, tuple(params)) in seen:
TypeError: unhashable type: 'list'
```

### Technical details

* Python version: 3.4.3
* Django version: 1.11.7
* Wagtail version: 1.13
* Browser version: Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:56.0) Gecko/20100101 Firefox/56.0


Discussion:
Thanks @tm-kn! Confirmed on Wagtail 1.13 here. Looks like this may well be related to the issues fixed in #3947 - although this one seems to be a regression from 1.12 to 1.13, which suggests that it's not one of the fixes backported in #3957 / #3958 (which affected 1.12 and 1.13 equally). Investigating now.
Caused by https://github.com/wagtail/wagtail/commit/4063573f9c48e035a05321e98a2c89f2c310ceaa - evidently the Postgres search backend doesn't like working on a `specific()` queryset.
Can be reproduced on the `./manage.py shell` command line with:

    Page.objects.specific().search('bread', fields=['title'])

Both the `specific` clause and the `fields` param are necessary to trigger the error.
781263d4e1d4d8c95170f19667b61700fd3751d9 has fixed this on master - will see if this can be backported. ( @BertrandBordage, please shout if you're already working on this :-) )

Resolving patch:
diff --git a/wagtail/contrib/postgres_search/backend.py b/wagtail/contrib/postgres_search/backend.py
--- a/wagtail/contrib/postgres_search/backend.py
+++ b/wagtail/contrib/postgres_search/backend.py
@@ -271,9 +271,12 @@ def search_in_index(self, queryset, search_query, start, stop):
             sql, index_params + model_params + limits)
 
     def search_in_fields(self, queryset, search_query, start, stop):
+        # Due to a Django bug, arrays are not automatically converted here.
+        converted_weights = '{' + ','.join(map(str, WEIGHTS_VALUES)) + '}'
+
         return (self.get_in_fields_queryset(queryset, search_query)
                 .annotate(_rank_=SearchRank(F('_search_'), search_query,
-                                            weights=WEIGHTS_VALUES))
+                                            weights=converted_weights))
                 .order_by('-_rank_'))[start:stop]
 
     def search(self, config, start, stop):


Test patch:
diff --git a/wagtail/wagtailsearch/tests/test_page_search.py b/wagtail/wagtailsearch/tests/test_page_search.py
new file mode 100644
--- /dev/null
+++ b/wagtail/wagtailsearch/tests/test_page_search.py
@@ -0,0 +1,46 @@
+from __future__ import absolute_import, unicode_literals
+
+from django.conf import settings
+from django.test import TestCase
+
+from wagtail.wagtailcore.models import Page
+from wagtail.wagtailsearch.backends import get_search_backend
+
+
+class PageSearchTests(object):
+    # A TestCase with this class mixed in will be dynamically created
+    # for each search backend defined in WAGTAILSEARCH_BACKENDS, with the backend name available
+    # as self.backend_name
+
+    fixtures = ['test.json']
+
+    def setUp(self):
+        self.backend = get_search_backend(self.backend_name)
+        self.reset_index()
+        for page in Page.objects.all():
+            self.backend.add(page)
+        self.refresh_index()
+
+    def reset_index(self):
+        if self.backend.rebuilder_class:
+            index = self.backend.get_index_for_model(Page)
+            rebuilder = self.backend.rebuilder_class(index)
+            index = rebuilder.start()
+            index.add_model(Page)
+            rebuilder.finish()
+
+    def refresh_index(self):
+        index = self.backend.get_index_for_model(Page)
+        if index:
+            index.refresh()
+
+    def test_search_specific_queryset(self):
+        list(Page.objects.specific().search('bread', backend=self.backend_name))
+
+    def test_search_specific_queryset_with_fields(self):
+        list(Page.objects.specific().search('bread', fields=['title'], backend=self.backend_name))
+
+
+for backend_name in settings.WAGTAILSEARCH_BACKENDS.keys():
+    test_name = str("Test%sBackend" % backend_name.title())
+    globals()[test_name] = type(test_name, (PageSearchTests, TestCase,), {'backend_name': backend_name})