Repository: spdx/tools-python

Issue:
Line mismatch in error messages of the tag-value parser
When we try to parse a tag-value file that contains errors, the tag-value parser provides an error message like 
```
LicenseID must start with 'LicenseRef-', line: 4948
```
with an explicit line number. This line number does not always match the incorrect line in the file, probably because the parser ignores empty lines and comments. To be helpful additional infromation, the outputted line number should match the place within the file.

Discussion:
meretp: I don't quite understand why the origin regex `\s+` also matches the newline so that the line count is not increased but with your proposed changes @zvr it seems to work. I also added a testcase to verify that the line numer of the lexer matches even if there is an empty line in between.
zvr: The `\s` matches all whitespace, including new line (`\n`). Therefore, the `t_newline` is never triggered and `lineno` is not increased.
meretp: Ah, thanks for the explanation. And as the method `t_newline` comes before `t_whitespace` this would match empty lines with only a newline but as soon as there is another whitespace before the newline, `t_whitespace` handles the input and the newline is ignored in terms of line count, I see.

Patch:
diff --git a/spdx/parsers/lexers/tagvalue.py b/spdx/parsers/lexers/tagvalue.py
index 61a99ad..4421bae 100644
--- a/spdx/parsers/lexers/tagvalue.py
+++ b/spdx/parsers/lexers/tagvalue.py
@@ -235,7 +235,7 @@ class Lexer(object):
         t.lexer.lineno += len(t.value)
 
     def t_whitespace(self, t):
-        r"\s+"
+        r"[ \t]+"
         pass
 
     def build(self, **kwargs):
diff --git a/spdx/parsers/tagvalue.py b/spdx/parsers/tagvalue.py
index bf2e937..e3ecffd 100644
--- a/spdx/parsers/tagvalue.py
+++ b/spdx/parsers/tagvalue.py
@@ -37,6 +37,8 @@ ERROR_MESSAGES = {
     "DOC_VERSION_VALUE_TYPE": "Invalid SPDXVersion value, must be SPDX-M.N where M and N are numbers. Line: {0}",
     "DOC_NAME_VALUE": "DocumentName must be single line of text, line: {0}",
     "DOC_SPDX_ID_VALUE": "Invalid SPDXID value, SPDXID must be SPDXRef-DOCUMENT, line: {0}",
+    "LIC_LIST_VER_VALUE": "Invalid LicenseListVersion '{0}', must be of type M.N where M and N are numbers. Line: {1}",
+    "LIC_LIST_VER_VALUE_TYPE": "Could not read value after LicenseListVersion-tag. Line{0}",
     "EXT_DOC_REF_VALUE": "ExternalDocumentRef must contain External Document ID, SPDX Document URI and Checksum"
                          "in the standard format, line:{0}.",
     "DOC_COMMENT_VALUE_TYPE": "DocumentComment value must be free form text between <text></text> tags"


Test patch:
diff --git a/tests/test_tag_value_parser.py b/tests/test_tag_value_parser.py
index 6ce4af7..00be54b 100644
--- a/tests/test_tag_value_parser.py
+++ b/tests/test_tag_value_parser.py
@@ -115,6 +115,9 @@ annotation_str = '\n'.join([
     'SPDXREF: SPDXRef-DOCUMENT'
 ])
 
+document_str_with_empty_line = "\n".join(
+    ['SPDXVersion: SPDX-2.1', '  ', 'DataLicense: CC0-1.0'])
+
 
 class TestLexer(TestCase):
     maxDiff = None
@@ -291,6 +294,14 @@ class TestLexer(TestCase):
         self.token_assert_helper(self.l.token(), 'ANNOTATION_SPDX_ID', 'SPDXREF', 5)
         self.token_assert_helper(self.l.token(), 'LINE', 'SPDXRef-DOCUMENT', 5)
 
+    def test_correct_line_number_with_empty_line_between(self):
+        data = document_str_with_empty_line
+        self.l.input(data)
+        self.token_assert_helper(self.l.token(), 'DOC_VERSION', 'SPDXVersion', 1)
+        self.token_assert_helper(self.l.token(), 'LINE', 'SPDX-2.1', 1)
+        self.token_assert_helper(self.l.token(), 'DOC_LICENSE', 'DataLicense', 3)
+        self.token_assert_helper(self.l.token(), 'LINE', 'CC0-1.0', 3)
+
     def token_assert_helper(self, token, ttype, value, line):
         assert token.type == ttype
         assert token.value == value