ftp_utils: new method

* Adds a new method to get the datestamp
  of a file.

Signed-off-by: Georgios Papoutsakis <georgios.papoutsakis@cern.ch>
Merge pull request #61 from GiorgosPa/ftp-date

Ftp date
jats_package: author notes harvesting fix

Signed-off-by: Georgios Papoutsakis <georgios.papoutsakis@cern.ch>
edpsciences_package: conversion improvements

* Keeps the abstract formated in chapters.

* Seperates the author keywords in different fields.

Signed-off-by: Georgios Papoutsakis <georgios.papoutsakis@cern.ch>
global: subscripts superscripts support

Signed-off-by: Georgios Papoutsakis <georgios.papoutsakis@cern.ch>
tests: add tests for minidom utils

* Strips XML output for newlines.

* Adds tests for minidom utils and amends tests where necessary
  to make the tests pass again after recent changes.

Signed-off-by: Georgios Papoutsakis <georgios.papoutsakis@cern.ch>
utils: remove xml namespaces

Signed-off-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
utils: remove namespaces better

Co-authored-by: Georgios Papoutsakis <georgios.papoutsakis@cern.ch>
Signed-off-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
config: remove unnecessary warning

Signed-off-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
utils: collapse_initials fix

* Fix to handle more than one spaces between initials.

Signed-of-by: Georgios Papoutsakis <georgios.papoutsakis@cern.ch>
utils: XML handling improvement

* Fixes a problem with embedded XML inside text nodes.

* Creates a new file for tests for utils.py.

* Moves test_record_add_field to utils_tests.py.

Co-authored-by: Georgios Papoutsakis <georgios.papoutsakis@cern.ch>
Tested-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
global: more tests added

* Adds tests for final XML outputs for Elsevier, EDPS and APS.

* Adds test flag to get_record function in elsevier_package
  to not add the FFT tag because the file location
  is environment dependent and cannot be used for testing.

* Adds new escaping of ampersands in XML, along with tests.

* Removes references using refextract from tests.

Co-authored-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
Signed-off-by: Georgios Papoutsakis <georgios.papoutsakis@cern.ch>
utils: refactor XML escaping

* Refactors the XML escaping to also take in to account < as &lt;.

Co-authored-by: Georgios Papoutsakis <georgios.papoutsakis@cern.ch>
utils: fall back to text node

* Falls back to text node should parseString raise ExpatError
  when calling record_add_field with erroneous data.

* Adds a test to cover this case.

Signed-off-by: Jan Aage Lavik <jan.age.lavik@cern.ch>
harvesting-kit: Adds nations field while harvesting

* Adds add_nations_field to utils.py to check affiliations and add
  the corresponding nation. 'HUMAN CHECK' if something went wrong

* Adds '--extract-nations' flag to valid arguments, since currently
  only SCOAP3 is going to use the nations field

Signed-off-by: Martin Vesper <martin.vesper@cern.ch>
Merge pull request #72 from mvesper/add_nations_field_in_harvesting_kit

harvesting-kit: Adds nations field while harvesting
ftp_utils: fix for creating folders
Elsevier: automatic update of delivery dates for packages.

* Adds date of delivery for packages downloaded from Elsevier,
  data are stored in database. DOI of articles are associeted with
  packages they come from (one doi can be connected to multiple
  packages - updates of article).
harvesting-kit: Filters empty subfields of field 773

* Empty subfields are now removed before being added to the record

Signed-off-by: Martin Vesper <martin.vesper@cern.ch>
Merge pull request #77 from mvesper/filter_field_773

harvesting-kit: Filters empty subfields of field 773