Repository: reata/sqllineage

Issue:
InvalidSyntaxException When SQL Statement Ends with Multiple Semicolons
**Describe the bug**
When a SQL statement ends with multiple semicolons, it will be split to several statements with all of them other than the first one contain a semicolon only. And those semicolon only statement triggers InvalidSyntaxException

**SQL**
Paste the SQL text here. For example:
```sql
SELECT * FROM DUAL;;
```

**To Reproduce**
```shell
sqllineage -f test.sql --dialect=ansi
```
```
Traceback (most recent call last):
  File "/home/hujunwei/.pyenv/versions/3.12.0/bin/sqllineage", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/hujunwei/.pyenv/versions/3.12.0/lib/python3.12/site-packages/sqllineage/cli.py", line 109, in main
    runner.print_table_lineage()
  File "/home/hujunwei/.pyenv/versions/3.12.0/lib/python3.12/site-packages/sqllineage/runner.py", line 176, in print_table_lineage
    print(str(self))
          ^^^^^^^^^
  File "/home/hujunwei/.pyenv/versions/3.12.0/lib/python3.12/site-packages/sqllineage/runner.py", line 26, in wrapper
    self._eval()
  File "/home/hujunwei/.pyenv/versions/3.12.0/lib/python3.12/site-packages/sqllineage/runner.py", line 193, in _eval
    self._stmt_holders = [analyzer.analyze(stmt) for stmt in self._stmt]
                          ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hujunwei/.pyenv/versions/3.12.0/lib/python3.12/site-packages/sqllineage/core/parser/sqlfluff/analyzer.py", line 42, in analyze
    statement_segments = self._list_specific_statement_segment(sql)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hujunwei/.pyenv/versions/3.12.0/lib/python3.12/site-packages/sqllineage/core/parser/sqlfluff/analyzer.py", line 73, in _list_specific_statement_segment
    raise InvalidSyntaxException(
sqllineage.exceptions.InvalidSyntaxException: This SQL statement is unparsable, please check potential syntax error for SQL:
;
Line 1, Position 1: Found unparsable section: ';'
```

**Expected behavior**
No exception thrown.

**Python version (available via `python --version`)**
 - 3.12.0

**SQLLineage version (available via `sqllineage --version`):**
 - 1.4.9


Discussion:
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/reata/sqllineage/pull/503?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Comparison is base [(`fe3e2c5`)](https://app.codecov.io/gh/reata/sqllineage/commit/fe3e2c5ba1d6d6af2a64a29b4980a97a8d5c83b7?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) 99.41% compared to head [(`00b8dab`)](https://app.codecov.io/gh/reata/sqllineage/pull/503?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) 99.41%.


<details><summary>Additional details and impacted files</summary>


```diff
@@           Coverage Diff           @@
##           master     #503   +/-   ##
=======================================
  Coverage   99.41%   99.41%           
=======================================
  Files          40       40           
  Lines        2045     2052    +7     
=======================================
+ Hits         2033     2040    +7     
  Misses         12       12           
```



</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/reata/sqllineage/pull/503?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None).


Patch:
diff --git a/sqllineage/utils/helpers.py b/sqllineage/utils/helpers.py
index 21f0a26..e403ba7 100644
--- a/sqllineage/utils/helpers.py
+++ b/sqllineage/utils/helpers.py
@@ -45,9 +45,18 @@ def extract_sql_from_args(args: Namespace) -> str:
 def split(sql: str) -> List[str]:
     # TODO: we need a parser independent split function
     import sqlparse
+    from sqlparse.tokens import Punctuation
 
-    # sometimes sqlparse split out a statement that is comment only, we want to exclude that
-    return [s.value for s in sqlparse.parse(sql) if s.token_first(skip_cm=True)]
+    result = []
+    for s in sqlparse.parse(sql):
+        if first_token := s.token_first(skip_cm=True):
+            # sometimes sqlparse split out a statement that is comment only or semicolon only, we want to exclude that
+            if first_token.ttype == Punctuation and first_token.value == ";":
+                # exclude semicolon only statement
+                continue
+            else:
+                result.append(s.value)
+    return result
 
 
 def trim_comment(sql: str) -> str:


Test patch:
diff --git a/tests/sql/table/multiple_statements/test_split.py b/tests/sql/table/multiple_statements/test_split.py
index 4595cb9..2c578fe 100644
--- a/tests/sql/table/multiple_statements/test_split.py
+++ b/tests/sql/table/multiple_statements/test_split.py
@@ -30,3 +30,8 @@ def test_split_statements_with_desc():
 
 DESC tab1;"""
     assert len(split(sql)) == 2
+
+
+def test_split_statement_ends_with_multiple_semicolons():
+    sql = "SELECT 1;;;"
+    assert len(split(sql)) == 1