Repository: matchms/matchms

Issue:
Empty `inchi` doesn't get removed but results in empty string -> remove invalid metadata after reading the spectrum
**Describe the bug**
Currently, an `inchi` entry that is empty will result in a metadata entry that holds an empty string.
Ideally, if spectra are loaded with `harmonize_metadata` as true, _invalid_ or empty metadata values should be removed from the spectrum metadata.



Discussion:

Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8f7c6955..14901a6a 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -15,6 +15,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 - Fixed pipeline filter [#414](https://github.com/matchms/matchms/pull/414)
 - Removed fingerprint writing to file [#416](https://github.com/matchms/matchms/pull/416)
+- Updated harmonize_values function to remove invalid metadata [#418](https://github.com/matchms/matchms/pull/418)
 - Fixed metadata export style bug [#423](https://github.com/matchms/matchms/pull/423)
 
 ### Fixed
diff --git a/matchms/Metadata.py b/matchms/Metadata.py
index 25196d00..eedb47b6 100644
--- a/matchms/Metadata.py
+++ b/matchms/Metadata.py
@@ -111,6 +111,9 @@ class Metadata:
         if not isinstance(charge, int) and charge_int is not None:
             metadata_filtered["charge"] = charge_int
 
+        invalid_entries = ["", "NA", "N/A", "NaN"]
+        metadata_filtered = {k:v for k,v in metadata_filtered.items() if v not in invalid_entries}
+        
         self.data = metadata_filtered
 
     # ------------------------------


Test patch:
diff --git a/tests/importing/test_load_from_msp.py b/tests/importing/test_load_from_msp.py
index 21fffb26..217a9f41 100644
--- a/tests/importing/test_load_from_msp.py
+++ b/tests/importing/test_load_from_msp.py
@@ -225,7 +225,6 @@ def test_load_msl():
         "COMPOUND_NAME": "G3P",
         "CASNO": "TUBE2_~1-N1010",
         "RETENTION_INDEX": 1586.2,
-        "RW": '',
         "RETENTION_TIME": 9.648,
         "COMMENT": "RI=1586.2,   9.6481 min TUBE2_28-01-2020_18-48-00|RI:1586.2",
         "SOURCE": "C:\\USERS\\UTILISATEUR\\DESKTOP\\METABOLOME\\calib\\standard.msl",
diff --git a/tests/test_metadata.py b/tests/test_metadata.py
index 2f88b0db..99430668 100644
--- a/tests/test_metadata.py
+++ b/tests/test_metadata.py
@@ -94,3 +94,15 @@ def test_metadata_full_setter():
     metadata.data = {"Precursor Mz": 101.01}
     assert isinstance(metadata.data, PickyDict), "Expected PickyDict"
     assert metadata["precursor_mz"] == 101.01, "Expected differnt entry"
+
+
+@pytest.mark.parametrize("input_dict, expected", [
+    [{'scannumber': '1516', 'spectrumtype': 'N/A', 'formula': 'C8H16NO5P', 'ionization': 'NA', 
+      'compound_name': 'Dicrotophos', 'comment': '', 'precursor_mz': 238.0844}, 
+      {'scannumber': '1516', 'formula': 'C8H16NO5P', 'compound_name': 'Dicrotophos',
+       'precursor_mz': 238.0844}]])
+def test_remove_invalid_metadata(input_dict, expected):
+    metadata = Metadata(input_dict)
+    metadata.harmonize_values()
+
+    assert metadata == expected, "Expected metadata to be equal."