Repository: mapping-commons/sssom-py

Issue:
SSSOM/TSV parser could ignore trailing tabs in embedded YAML header
When a SSSOM/TSV file containing an embedded YAML metadata header is modified in a standard spreadsheet editor (e.g. LibreOffice Calc, Microsoft Excel, or Apple Numbers), the editor, obviously unaware that lines starting with a `#` are comments, may treat the commented header lines as if they were normal data lines that merely happen to have only one cell. This means that, when writing the file back, the editor will insert trailing tabs at the end of the header lines (so that those lines have the same number of “columns” than the subsequent real data lines).

For example, a file like this one:

```
# curie_map:
#   FBbt: http://purl.obolibrary.org/obo/FBbt_
#   UBERON: http://purl.obolibrary.org/obo/UBERON_
subject_id   predicate_id      object_id     mapping_justification
FBbt:1234    skos:exactMatch   UBERON:5678   semapv:ManualMappingCuration
```

is “seen“ by the spreadsheet editor as

| # curie_map: |   |   |   |
| ------------ | - | - | - |
| #   FBbt: http://purl.obolibrary.org/obo/FBbt_ | | | |
| #   UBERON: http://purl.obolibrary.org/obo/UBERON_ | | | |
| subject_id | predicate_id | object_id | mapping_justification |
| FBbt:1234 | skos:exactMatch | UBERON:5678 | semapv:ManualMappingCuration |

and will be written back as (extra tabs rendered as “➪” to make them visible):

```
# curie_map:➪➪➪
#   FBbt: http://purl.obolibrary.org/obo/FBbt_➪➪➪
#   UBERON: http://purl.obolibrary.org/obo/UBERON_➪➪➪
subject_id   predicate_id      object_id     mapping_justification
FBbt:1234    skos:exactMatch   UBERON:5678   semapv:ManualMappingCuration
```

This behaviour has been observed with all three aforementioned spreadsheet editors.

Those extra tabs make the YAML metadata block invalid as per the YAML syntax rules, and therefore make the entire file an invalid SSSOM/TSV file. This, in turn, makes it impossible to use a standard spreadsheet software to edit a SSSOM mapping set.

I think SSSOM tools could implement a quick and almost painless workaround here, to still accept those files despite them being, strictly speaking, invalid. All that is needed is to strip any trailing tabs in the header lines before passing the lines to the YAML parser.

Discussion:
@matentzn: This is great, thank you!!

Solution patch:
diff --git a/src/sssom/parsers.py b/src/sssom/parsers.py
index 81a5b37..52cc1e9 100644
--- a/src/sssom/parsers.py
+++ b/src/sssom/parsers.py
@@ -121,7 +121,11 @@ def _separate_metadata_and_table_from_stream(s: io.StringIO):
             if header_section:
                 header_section = False
         elif header_section:
-            metadata_component.write(line)
+            # We strip any trailing tabs. Such tabs may have been left
+            # by a spreadsheet editor who treated the header lines as
+            # if they were normal data lines; they would prevent the
+            # YAML parser from correctly parsing the metadata block.
+            metadata_component.write(line.rstrip("\t\n") + "\n")
         else:
             logging.info(
                 f"Line {line} is starting with hash symbol, but header section is already passed. "


Test patch:
diff --git a/tests/data/trailing-tabs.sssom.tsv b/tests/data/trailing-tabs.sssom.tsv
new file mode 100644
index 0000000..b8f6de8
--- /dev/null
+++ b/tests/data/trailing-tabs.sssom.tsv
@@ -0,0 +1,21 @@
+#curie_map:					
+#  COMENT: https://example.com/entities/					
+#  COMPID: https://example.com/people/					
+#  ORGENT: https://example.org/entities/					
+#  ORGPID: https://example.org/people/					
+#mapping_set_id: https://example.org/sets/exo2c					
+#mapping_set_title: O2C set					
+#creator_id:					
+#  - ORGPID:0000-0000-0001-1234					
+#  - COMPID:0000-0000-0002-5678					
+#license: https://creativecommons.org/licenses/by/4.0/					
+#publication_date: 2023-09-13					
+subject_id	subject_label	predicate_id	object_id	object_label	mapping_justification
+ORGENT:0001	alice	skos:closeMatch	COMENT:0011	alpha	semapv:ManualMappingCuration
+ORGENT:0002	bob	skos:closeMatch	COMENT:0012	beta	semapv:ManualMappingCuration
+ORGENT:0004	daphne	skos:closeMatch	COMENT:0014	delta	semapv:ManualMappingCuration
+ORGENT:0005	eve	skos:closeMatch	COMENT:0015	epsilon	semapv:ManualMappingCuration
+ORGENT:0006	fanny	skos:closeMatch	COMENT:0016	zeta	semapv:ManualMappingCuration
+ORGENT:0007	gavin	skos:exactMatch	COMENT:0013	gamma	semapv:ManualMappingCuration
+ORGENT:0008	hector	skos:closeMatch	COMENT:0017	eta	semapv:ManualMappingCuration
+ORGENT:0009	ivan	skos:exactMatch	COMENT:0019	iota	semapv:ManualMappingCuration
\ No newline at end of file
diff --git a/tests/test_parsers.py b/tests/test_parsers.py
index f29d9e4..492f9f1 100644
--- a/tests/test_parsers.py
+++ b/tests/test_parsers.py
@@ -339,6 +339,17 @@ class TestParse(unittest.TestCase):
         msdf = parse_sssom_table(outfile)
         self.assertTrue(custom_curie_map.items() <= msdf.prefix_map.items())
 
+    def test_parse_trailing_tabs_in_metadata_header(self):
+        """Test parsing a file containing trailing tabs in header."""
+        input_path = f"{test_data_dir}/trailing-tabs.sssom.tsv"
+        msdf = parse_sssom_table(input_path)
+        self.assertEqual(msdf.metadata["mapping_set_id"], "https://example.org/sets/exo2c")
+        self.assertEqual(
+            len(msdf.df),
+            8,
+            f"{input_path} has the wrong number of mappings.",
+        )
+
 
 class TestParseExplicit(unittest.TestCase):
     """This test case contains explicit tests for parsing."""