BibIndex: move helper functions to utils file

* Moves some of the helper functions from bibindex_engine
  to bibindex_engine_utils file.

Signed-off-by: <grzegorz.szpura@cern.ch>
Tested-by: Tibor Simko <tibor.simko@cern.ch>
BibIndex: WordTable API changes

* Removes 'fields_to_index' argument from WordTable's argument
  list.

Signed-off-by: <grzegorz.szpura@cern.ch>
Tested-by: Tibor Simko <tibor.simko@cern.ch>
BibIndex: abstraction layer for terms retrieval

* Introduces new TermCollector class which
  handles terms retrieval from different
  part of the system.

Signed-off-by: Grzegorz Szpura <grzegorz.szpura@cern.ch>
Tested-by: Tibor Simko <tibor.simko@cern.ch>
BibIndex: indexing non-MARC standards

* Introduces NonMarcTermCollector for acquiring data
  from non-MARC records.

* Adds new column to tag table and inserts
  corresponding RecJson fields as another
  non-MARC value for 'tag'.

* Enhances test suite acordingly.

Signed-off-by: Grzegorz Szpura <grzegorz.szpura@cern.ch>
Tested-by: Tibor Simko <tibor.simko@cern.ch>
BibIndex: recjson fields in admin interface

* Admin interface shows recjson fields
  in logical fields overview.

* Fields and tags are shown directly in index
  overview.

* Edition of tag enhanced with option to
  change value for recjson field.

Signed-off-by: Grzegorz Szpura <grzegorz.szpura@cern.ch>
Tested-by: Tibor Simko <tibor.simko@cern.ch>
Merge branch 'master' into next

* master:
  BibIndex: recjson fields in admin interface
  BibIndex: indexing non-MARC standards
  BibIndex: abstraction layer for terms retrieval
  BibIndex: WordTable API changes
  BibIndex: move helper functions to utils file

* Merge note: creation of fixtures and  movement of new files will be
  done in the next commits.

Conflicts:
	invenio/legacy/bibindex/adminlib.py
	invenio/legacy/bibindex/engine.py
	invenio/legacy/bibindex/engine_utils.py
	invenio/legacy/search_engine/__init__.py
	invenio/modules/indexer/testsuite/test_indexer_engine.py
	invenio/modules/indexer/tokenizers/BibIndexAuthorCountTokenizer.py
	invenio/modules/indexer/tokenizers/BibIndexDOITokenizer.py
	invenio/modules/indexer/tokenizers/BibIndexJournalTokenizer.py
	invenio/modules/records/testsuite/fields/atlantis.cfg
	invenio/modules/upgrader/upgrades/invenio_2013_03_20_idxINDEX_synonym_kb.py
	modules/bibindex/lib/Makefile.am
	modules/bibindex/lib/bibindex_regression_tests.py
	modules/bibupload/lib/bibupload_regression_tests.py
	modules/websearch/lib/websearch_regression_tests.py

Signed-off-by: Esteban J. G. Gabancho <esteban.gabancho@gmail.com>
bibindex: move new files to legacy and fix imports

Signed-off-by: Esteban J. G. Gabancho <esteban.gabancho@gmail.com>
records: move new recordext function to records

Signed-off-by: Esteban J. G. Gabancho <esteban.gabancho@gmail.com>
search: update of fixtures and models

* Updates `Tag` fixtures and models accordingly to the new indexer
  capabilities to index non-marc records.

Signed-off-by: Esteban J. G. Gabancho <esteban.gabancho@gmail.com>
BibField: fix copyright field names

* Fixes a typo for one of the copyright fields and also changes its name
  to `copyright_status` to avoid duplicate names.  (closes #1933)

Signed-off-by: Esteban J. G. Gabancho <esteban.gabancho@gmail.com>
Reviewed-by: Tibor Simko <tibor.simko@cern.ch>
Merge branch 'master' into next

* master:
  BibField: fix copyright field names
global: old bundle names

* Fixes the old bundle names that have been left over.

Signed-off-by: Yoan Blanc <yoan.blanc@cern.ch>