global: InvenioTestCase in test suite

* Introduces InvenioTestCase class that is to be used in all test
  suits instead of standard unittest.TestCase class.  Facilitates
  later merging to next.

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
BibAutority: fix for unit test suite

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
WebSubmit: source file mode fix

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
BibAuthority: source file mode fix

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
Merge branch 'master' into next

* master:
  BibAuthority: source file mode fix
  WebSubmit: source file mode fix
  BibAutority: fix for unit test suite
  global: InvenioTestCase in test suite

Conflicts:
	modules/bibauthority/lib/bibauthority_regression_tests.py
	modules/bibcatalog/lib/bibcatalog_system_email_unit_tests.py
	modules/bibcatalog/lib/bibcatalog_system_rt_unit_tests.py
	modules/bibcirculation/lib/bibcirculation_regression_tests.py
	modules/bibclassify/lib/bibclassify_unit_tests.py
	modules/bibconvert/lib/bibconvert_regression_tests.py
	modules/bibconvert/lib/bibconvert_unit_tests.py
	modules/bibdocfile/lib/bibdocfile_regression_tests.py
	modules/bibedit/lib/bibedit_unit_tests.py
	modules/bibexport/lib/bibexport_method_fieldexporter_unit_tests.py
	modules/bibfield/lib/bibfield_config_engine_unit_tests.py
	modules/bibfield/lib/bibfield_marcreader_regression_tests.py
	modules/bibfield/lib/bibfield_regression_tests.py
	modules/bibfield/lib/bibfield_utils_unit_tests.py
	modules/bibfield/lib/functions/is_type_isbn_issn_unit_tests.py
	modules/bibformat/lib/bibformat_engine_unit_tests.py
	modules/bibformat/lib/bibformat_utils_unit_tests.py
	modules/bibindex/lib/bibindex_engine_stemmer_unit_tests.py
	modules/bibindex/lib/bibindex_engine_tokenizer_unit_tests.py
	modules/bibindex/lib/bibindex_engine_unit_tests.py
	modules/bibknowledge/lib/bibknowledge_regression_tests.py
	modules/bibknowledge/lib/bibknowledge_unit_tests.py
	modules/bibmatch/lib/bibmatch_regression_tests.py
	modules/bibmatch/lib/bibmatch_unit_tests.py
	modules/bibrank/lib/bibrank_citation_indexer_regression_tests.py
	modules/bibrank/lib/bibrank_citation_searcher_unit_tests.py
	modules/bibrank/lib/bibrank_citerank_indexer_unit_tests.py
	modules/bibrank/lib/bibrank_downloads_indexer_unit_tests.py
	modules/bibrank/lib/bibrank_record_sorter_unit_tests.py
	modules/bibrank/lib/bibrank_regression_tests.py
	modules/bibrank/lib/bibrank_selfcites_unit_tests.py
	modules/bibrank/lib/bibrank_tag_based_indexer_unit_tests.py
	modules/bibrecord/lib/bibrecord_unit_tests.py
	modules/bibsort/lib/bibsort_engine_unit_tests.py
	modules/bibsort/lib/bibsort_washer_unit_tests.py
	modules/bibupload/lib/bibupload_regression_tests.py
	modules/docextract/lib/docextract_webinterface_unit_tests.py
	modules/docextract/lib/refextract_api_regression_tests.py
	modules/docextract/lib/refextract_regression_tests.py
	modules/docextract/lib/refextract_unit_tests.py
	modules/elmsubmit/lib/elmsubmit_unit_tests.py
	modules/miscutil/lib/dataciteutils_tester.py
	modules/miscutil/lib/dateutils_unit_tests.py
	modules/miscutil/lib/dbquery_regression_tests.py
	modules/miscutil/lib/dbquery_unit_tests.py
	modules/miscutil/lib/errorlib_regression_tests.py
	modules/miscutil/lib/errorlib_unit_tests.py
	modules/miscutil/lib/htmlutils_unit_tests.py
	modules/miscutil/lib/importutils_unit_tests.py
	modules/miscutil/lib/intbitset_unit_tests.py
	modules/miscutil/lib/invenio_connector_regression_tests.py
	modules/miscutil/lib/inveniocfg_unit_tests.py
	modules/miscutil/lib/inveniocfg_upgrader_unit_tests.py
	modules/miscutil/lib/jsonutils_unit_tests.py
	modules/miscutil/lib/logicutils_unit_tests.py
	modules/miscutil/lib/messages_unit_tests.py
	modules/miscutil/lib/plotextractor_regression_tests.py
	modules/miscutil/lib/plotextractor_unit_tests.py
	modules/miscutil/lib/pluginutils_unit_tests.py
	modules/miscutil/lib/shellutils_unit_tests.py
	modules/miscutil/lib/solrutils_unit_tests.py
	modules/miscutil/lib/testutils.py
	modules/miscutil/lib/textutils_unit_tests.py
	modules/miscutil/lib/urlutils_unit_tests.py
	modules/miscutil/lib/web_api_key_regression_tests.py
	modules/oairepository/lib/oai_repository_regression_tests.py
	modules/oairepository/lib/oai_repository_unit_tests.py
	modules/webaccess/lib/access_control_firerole_unit_tests.py
	modules/webaccess/lib/external_authentication_cern_unit_tests.py
	modules/webaccess/lib/webaccess_regression_tests.py
	modules/webalert/lib/alert_engine_unit_tests.py
	modules/webalert/lib/webalert_regression_tests.py
	modules/webcomment/lib/webcomment_unit_tests.py
	modules/webjournal/lib/webjournal_regression_tests.py
	modules/webjournal/lib/webjournal_unit_tests.py
	modules/weblinkback/lib/weblinkback_regression_tests.py
	modules/weblinkback/lib/weblinkback_unit_tests.py
	modules/webmessage/lib/webmessage_regression_tests.py
	modules/webmessage/lib/webmessage_unit_tests.py
	modules/websearch/lib/search_engine_query_parser_unit_tests.py
	modules/websearch/lib/search_engine_summarizer_regression_tests.py
	modules/websearch/lib/search_engine_unit_tests.py
	modules/websearch/lib/websearch_external_collections_getter_unit_tests.py
	modules/websearch/lib/websearch_external_collections_unit_tests.py
	modules/websearch/lib/websearch_regression_tests.py
	modules/websearch/lib/websearchadmin_regression_tests.py
	modules/websession/lib/webgroup_regression_tests.py
	modules/websession/lib/webgroup_unit_tests.py
	modules/websession/lib/websession_regression_tests.py
	modules/websession/lib/webuser_regression_tests.py
	modules/webstyle/lib/webdoc_unit_tests.py
	modules/webstyle/lib/webinterface_unit_tests.py
	modules/webstyle/lib/webstyle_regression_tests.py
	modules/websubmit/lib/websubmit_regression_tests.py

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
WebSearch: import fixes in regression test suite

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
BibField: improvements backported from next

Co-authored-by: Tibor Simko <tibor.simko@cern.ch>
Merge branch 'master' into next

* master:
  BibField: improvements backported from next
BibMerge: delete duplicate record first

* When merging and deleting records, do first submit the deleted
  one such that any possible identifier that would result in a
  duplicate are removed first, before updating the remaining
  record.  (closes #1645)

Signed-off-by: Jerome Caffaro <jerome.caffaro@cern.ch>
Reviewed-by: Tibor Simko <tibor.simko@cern.ch>
OAIHarvest: configurable selective post-processing

* Allow to selectively apply post-processing phases to each
  individual harvested record. To do so override function
  oai_harvest_templates.tmpl_should_process_record_with_mode().
  (closes #1477)

Signed-off-by: Jerome Caffaro <jerome.caffaro@cern.ch>
Acked-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
OAIHarvest: consider source_id for selective post-processing

* Added source_id parameter to function
  oai_harvest_templates.tmpl_should_process_record_with_mode()
  in order to enable decision based on harvesting source.

Signed-off-by: Jerome Caffaro <jerome.caffaro@cern.ch>
Acked-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
WebAccess: update check for "external" account at CERN

* Changed conditions for an account to be considered as "external"
  at CERN.

Signed-off-by: Jerome Caffaro <jerome.caffaro@cern.ch>
Reviewed-by: Samuele Kaplun <samuele.kaplun@cern.ch>
WebComment: link to "Your Comments" after posting

* After a user has submitted a comment, display a hint regarding
  the possibility to view all previously submitted comments
  in "Your Comments" page.

* Link opens in a new window/tab as a quick measure to prevent
  double posting when using browser's back button.

Signed-off-by: Jerome Caffaro <jerome.caffaro@cern.ch>
BibFormat: initial example of Twitter Card support

* Outputs sample Twitter Card markup (using fallback on OpenGraph
  markup).

* Uses DBCOLLID2TWITTERCARDTYPE knownledge base to map record type
  to card type.

* Introduced parameter 'one_icon_per_bibdoc' in bfe_meta_opengraph_image
  in order to avoid repetition of same icon preview in multiple sizes.

* Better output in bfe_meta_opengraph_image for Twitter Cards, with
  dedicated tag to comply with image size limit. Introduced
  'twitter_card_type' parameter to switch between 'gallery' and 'photo'
  card type (stricly either a group of 4 photos, or a single one).
  Reordered bfe_meta_opengraph_image and bfe_meta_opengraph_video in
  order to force Twitter not to fall back on corresponding OpenGraph
  tag (which would otherwise most probably get it reach the image
  size limit).

* Some additional CERN-specific changes.

Signed-off-by: Jerome Caffaro <jerome.caffaro@cern.ch>
WebJournal: sample Twitter Card markup

* Added necessary tags to support Twitter Card.

* Use article featured image for Twitter Card image.

Signed-off-by: Jerome Caffaro <jerome.caffaro@cern.ch>
WebJournal: fix image dimension retrieval

* When trying to retrieve the dimension of an image (to keep
  the correct ratio when explicitely setting width/height),
  consider that the fetched image might not be reachable and
  could be returned as an HTML (error message, login page, etc.)
  Try to analyze the image only when the content-type indicates
  an image.

Signed-off-by: Jerome Caffaro <jerome.caffaro@cern.ch>
BibRank: temporary file storage in CFG_TMPDIR

* Stores temporary data files used for plotting in CFG_TMPDIR
  instead of CFG_WEBDIR.
  Adds explicit 'bibrank_grapher_' prefix.
  This is not helping, but nor worsening the creation and
  servicing of graphs in load-balanced setups.

* Temporary directory set using 'dir' parameter instead of
  'tempfile.tempdir'.  (addresses #1594)

* Makes use of safer tempfile.mkstemp (instead of tempfile.mktemp).

Reviewed-by: Tibor Simko <tibor.simko@cern.ch>
Merge branch 'maint-0.99' into maint-1.0

* maint-0.99:
  BibRank: temporary file storage in CFG_TMPDIR

Conflicts:
	modules/bibrank/lib/bibrank_grapher.py

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>
BatchUploader: avoid tempfile.tempdir redefinition

* Temporary directory set using 'dir' parameter instead of
  'tempfile.tempdir'.  (closes #1594)

* Makes use of safer tempfile.mkstemp (instead of tempfile.mktemp).

Reviewed-by: Tibor Simko <tibor.simko@cern.ch>
Merge branch 'maint-1.0' into maint-1.1

* maint-1.0:
  BatchUploader: avoid tempfile.tempdir redefinition
  BibRank: temporary file storage in CFG_TMPDIR

Conflicts:
	modules/bibrank/lib/bibrank_grapher.py
	modules/bibupload/lib/batchuploader.py
	modules/bibupload/lib/batchuploader_engine.py

Signed-off-by: Tibor Simko <tibor.simko@cern.ch>