textutils: translate_to_ascii() unknown chars fix

* translate_to_ascii() checks the characters one by one in order to
  make sure unrecognized values are ignored ("" instead of
  printing "[?]").  (closes #1754)

* Adds unit test for this case.

Signed-off-by: Federico Poli <federico.poli@cern.ch>
Tested-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
Reviewed-by: Tibor Simko <tibor.simko@cern.ch>
WebSearch: faster collection children cache

* Improves performace for collection children cache
  by minimizing the number of database calls.  (closes #1739)

Signed-off-by: Ludmila Marian <ludmila.marian@gmail.com>
Reviewed-by: Tibor Simko <tibor.simko@cern.ch>
WebMessage: English corrections in output messages

* Fixes English in output messages.  (closes #1849)

Reviewed-by: Tibor Simko <tibor.simko@cern.ch>
BibFormat: plot file identification improvements

* Identify (and hide) "Plots" on the basis of their doctype,
  not based on file extension or caption. (closes #1514)

* Some CERN-specific handling of the files for the display.

Signed-off-by: Jerome Caffaro <jerome.caffaro@cern.ch>
Reviewed-by: Esteban J. G. Gabancho <esteban.gabancho@gmail.com>
WebStyle: fix gotoadmin CLI parmeters parsing

* Fixed parsing of gotoadmin CLI '--json-parameters' and
  '--parameter' options. (closes #1427)

Co-authored-by: Samuele Kaplun <samuele.kaplun@cern.ch>
Reviewed-by: Esteban J. G. Gabancho <esteban.gabancho@gmail.com>
testutils: wait for element to be displayed/hidden

* Adds methods 'wait_element_displayed_with_timeout' and
  'wait_element_hidden_with_timeout' to wait for an element to
  be displayed, or hidden, before proceeding.

Tested-by: Esteban J. G. Gabancho <esteban.gabancho@gmail.com>
WebSubmit: DEMOART uses bibdocfile_managedocfile

* Use bibdocfile_managedocfile interface (in response element
  DEMOART_FILE) to upload files in DEMOART submission, instead
  of file upload field + FFT.

Tested-by: Esteban J. G. Gabancho <esteban.gabancho@gmail.com>
WebSubmit: web tests for DEMOART and DEMOPIC

* Test file upload within the DEMOART (SBI) submission, using
  the bibdocfile_managedocfile interface in asynchronous mode.

* Test file upload within DEMOPIC (SRV) submission, using the
  WebSubmit Create_Upload_Files_Interface.py function.

Tested-by: Esteban J. G. Gabancho <esteban.gabancho@gmail.com>
BibDocFile: new web tests

* Test addition, revision and deletion of files via
  the BibDocFile web interface.

Acked-by: Tibor Simko <tibor.simko@cern.ch>
WebSubmit: rotate created icons according to EXIF

* When creating icons, rotate the created icons according to
  the Exif "Orientation" metadata of the original photo. (closes #1516)

Signed-off-by: Jerome Caffaro <jerome.caffaro@cern.ch>
Reviewed-by: Ludmila Marian <ludmila.marian@gmail.com>
WebSearch: anyfield in CFG_WEBSEARCH_SYNONYM_KBRS

* CFG_WEBSEARCH_SYNONYM_KBRS can use the keyword 'anyfield' to
  match a given knowledge base when searching without a particular
  field. (closes #1493)

Signed-off-by: Jerome Caffaro <jerome.caffaro@cern.ch>
Reviewed-by: Ludmila Marian <ludmila.marian@cern.ch>
WebSubmit: guests support improved in /direct URL

* When accessing a /submit/direct URL as guest user, check
  if access is granted. If not, ask the user to log in
  prior to redirecting to the submission interface (and avoid
  an 'access denied' message). (closes #1240)

Reviewed-by: Samuele Kaplun <samuele.kaplun@cern.ch>
Signed-off-by: Ludmila Marian <ludmila.marian@gmail.com>
WebSearch: resolve (internal) DOIs

* When accessing "CFG_SITE_URL/doi/foo/bar", try to resolve 'foo/bar'
  as DOI and return corresponding record (0247__a:'foo/bar'). In this
  initial release the 'source' field (0247__2:doi) is not considered.
  (closes #1322)

Signed-off-by: Jerome Caffaro <jerome.caffaro@cern.ch>
Reviewed-by: Ludmila Marian <ludmila.marian@gmail.com>
BibFormat: new bfe_date parsing/formatting options

* Adds parameters to customize the parsing of the date (format,
  source).
  Source format can also be guessed provided that mx.DateTime
  module is installed.

* Special output of year-only dates.

Signed-off-by: Jerome Caffaro <jerome.caffaro@cern.ch>
Tested-by: Esteban J. G. Gabancho <esteban.gabancho@gmail.com>
BibFormat: new bfe_oai_identifier element

* Display record OAI identifier.

Signed-off-by: Jerome Caffaro <jerome.caffaro@cern.ch>
Tested-by: Esteban J. G. Gabancho <esteban.gabancho@gmail.com>
BibFormat: configurable /record

Signed-off-by: Jerome Caffaro <jerome.caffaro@cern.ch>
Tested-by: Esteban J. G. Gabancho <esteban.gabancho@gmail.com>
BibFormat: improve Dublin Core output

* Adds record URL to <dc:identifier> element.

* Adds 037$a AND 088$a to <dc:identifier> element. (closes #320)

* Adds OAI identifier (CFG_OAI_ID_FIELD) to <dc:identifier> element.

* Adds 0247_a to <dc:identifier> element when 0247_2 is "doi".
  (closes #1213)

* Adds 260$b to <dc:publisher> element.

* Display corporate author (110__a/710__a) when author is not
  available (100__a/700__a).

* Updates <dc:date> element to fetch data from bfe_date.
  Fall back to record creation date if no value found.

* Removes link to files (8564_u).

Signed-off-by: Jerome Caffaro <jerome.caffaro@cern.ch>
Tested-by: Esteban J. G. Gabancho <esteban.gabancho@gmail.com>
BibFormat: advertise /doi URL in DC output

* In Dublin Core output (DC + OAI_DC), prefer the CFG_SITE_RECORD/doi/[...]
  url scheme over the CFG_SITE_RECORD/record/[...] one.

* Introduces CFG_BIBUPLOAD_INTERNAL_DOI_PREFIX to differentiate
  between internally managed DOIs, and DOIs assigned/owned by others.

Signed-off-by: Jerome Caffaro <jerome.caffaro@cern.ch>
Tested-by: Esteban J. G. Gabancho <esteban.gabancho@gmail.com>
BibEdit: prevent deletion of managed DOIs fields

* Check if the fields requested to be deleted contain a managed
  DOI.

* Check if the record requested to be deleted contains a managed
  DOI.

* Introduces CFG_BIBEDIT_INTERNAL_DOI_PROTECTION_LEVEL in order
  to control the behaviour regarding deletion of managed DOIs:
  Ask user confirmation before deleting, forbid deletion or
  simply proceeed. (closes #1445)

Signed-off-by: Jerome Caffaro <jerome.caffaro@cern.ch>
Acked-by: Javier Martin Montull <javier.martin.montull@cern.ch>
BibMerge: prevent loss of DOI when merging records

* Check that no managed DOIs are lost when merging records.

* Check that no duplicate DOIs are created when merging
  records.

* Behaviour regarding deletion of managed DOIs when merging
  can be tuned with CFG_BIBEDIT_INTERNAL_DOI_PROTECTION_LEVEL.
  (closes #1446)

Signed-off-by: Jerome Caffaro <jerome.caffaro@cern.ch>
Acked-by: Javier Martin Montull <javier.martin.montull@cern.ch>