toastdriven/django-haystack — Issue #495: Haystack and South incompatible, broken UnifiedIndex in tests, SearchBackendError: No fields were found in any search_indexes.
Author: qris
Created: 2012-02-03 16:32:37+00:00
State: open

* South's syncdb command temporarily modifies settings.INSTALLED_APPS, removing any apps that are using migrations.
* It then runs the real syncdb() from django/core/management/commands/syncdb.py, which emits a post_sync signal.
* Anything listening to this signal will see an incorrect set of INSTALLED_APPS.
* django.contrib.auth.management.create_permissions() is one such listener.
* In unit tests, the ContentType objects don't exist in the database yet, and they are created here.
* This triggers the pre_save signal
* Which calls haystack.models.load_indexes()
* Which calls UnifiedIndex.setup_indexes()
* Which calls UnifiedIndex.build()
* Which generates the list of SearchIndex classes from the current INSTALLED_APPS
* And remembers it
* Then any search() will not search fields in apps that are using migrations
* And if all your apps are using migrations, search() fails with:
* SearchBackendError: No fields were found in any search_indexes. Please correct this before attempting to search.

Stack trace of the faulty indexing:

```
File "./manage.py", line 78, in
execute_manager(settings)
File "./manage.py", line 75, in execute_manager
utility.execute()
File "/usr/lib/pymodules/python2.6/django/core/management/__init__.py", line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/lib/pymodules/python2.6/django/core/management/base.py", line 191, in run_from_argv
self.execute(*args, **options.__dict__)
File "/usr/lib/pymodules/python2.6/django/core/management/base.py", line 220, in execute
output = self.handle(*args, **options)
File "/home/installuser/projects/intranet/django/intranet/.ve/lib/python2.6/site-packages/south/management/commands/test.py", line 8, in handle
super(Command, self).handle(*args, **kwargs)
File "/usr/lib/pymodules/python2.6/django/core/management/commands/test.py", line 37, in handle
failures = test_runner.run_tests(test_labels)
File "/usr/lib/pymodules/python2.6/django/test/simple.py", line 359, in run_tests
old_config = self.setup_databases()
File "/usr/lib/pymodules/python2.6/django/test/simple.py", line 296, in setup_databases
test_db_name = connection.creation.create_test_db(self.verbosity, autoclobber=not self.interactive)
File "/usr/lib/pymodules/python2.6/django/db/backends/creation.py", line 366, in create_test_db
load_initial_data=False)
File "/usr/lib/pymodules/python2.6/django/core/management/__init__.py", line 166, in call_command
return klass.execute(*args, **defaults)
File "/usr/lib/pymodules/python2.6/django/core/management/base.py", line 220, in execute
output = self.handle(*args, **options)
File "/usr/lib/pymodules/python2.6/django/core/management/base.py", line 351, in handle
return self.handle_noargs(**options)
File "/home/installuser/projects/intranet/django/intranet/.ve/lib/python2.6/site-packages/south/management/commands/syncdb.py", line 90, in handle_noargs
syncdb.Command().execute(**options)
File "/usr/lib/pymodules/python2.6/django/core/management/base.py", line 220, in execute
output = self.handle(*args, **options)
File "/usr/lib/pymodules/python2.6/django/core/management/base.py", line 351, in handle
return self.handle_noargs(**options)
File "/usr/lib/pymodules/python2.6/django/core/management/commands/syncdb.py", line 109, in handle_noargs
emit_post_sync_signal(created_models, verbosity, interactive, db)
File "/usr/lib/pymodules/python2.6/django/core/management/sql.py", line 190, in emit_post_sync_signal
interactive=interactive, db=db)
File "/usr/lib/pymodules/python2.6/django/dispatch/dispatcher.py", line 172, in send
response = receiver(signal=self, sender=sender, **named)
File "/usr/lib/pymodules/python2.6/django/contrib/auth/management/__init__.py", line 30, in create_permissions
ctype = ContentType.objects.get_for_model(klass)
File "/usr/lib/pymodules/python2.6/django/contrib/contenttypes/models.py", line 38, in get_for_model
defaults = {'name': smart_unicode(opts.verbose_name_raw)},
File "/usr/lib/pymodules/python2.6/django/db/models/manager.py", line 135, in get_or_create
return self.get_query_set().get_or_create(**kwargs)
File "/usr/lib/pymodules/python2.6/django/db/models/query.py", line 385, in get_or_create
obj.save(force_insert=True, using=self.db)
File "/usr/lib/pymodules/python2.6/django/db/models/base.py", line 460, in save
self.save_base(using=using, force_insert=force_insert, force_update=force_update)
File "/usr/lib/pymodules/python2.6/django/db/models/base.py", line 484, in save_base
signals.pre_save.send(sender=origin, instance=self, raw=raw, using=using)
File "/usr/lib/pymodules/python2.6/django/dispatch/dispatcher.py", line 172, in send
response = receiver(signal=self, sender=sender, **named)
File "/home/installuser/projects/intranet/django/intranet/.ve/lib/python2.6/site-packages/haystack/models.py", line 228, in load_indexes
conn.get_unified_index().setup_indexes()
File "/home/installuser/projects/intranet/django/intranet/.ve/lib/python2.6/site-packages/haystack/utils/loading.py", line 268, in setup_indexes
self.build()
File "/usr/lib/pymodules/python2.6/django/utils/functional.py", line 55, in _curried
return _curried_func(*(args+moreargs), **dict(kwargs, **morekwargs))
File "/home/installuser/projects/intranet/django/intranet/lib/monkeypatch.py", line 543, in build_with_debugging
traceback.print_stack()
```

Possible fixes/workarounds:

* Modify south not to modify the INSTALLED_APPS
* Register the Haystack signal listeners after the post_syncdb signal handling finishes
* Don't generate the index when a non-indexed model changes
* how can we know if it's indexed or not without enumerating INSTALLED_APPS?)
* Generate (or reset) the unified index explicitly (as in haystack 1) instead of automatically