Repository: reata/sqllineage

Issue:
Cartesian product exception with ANSI-89 Syntax
when i use LineageRunner to analyze a sql string like "select a.* from table a,table  b "   it throws a  SQLLineageException:
SQLLineageException: An Identifier is expected, got IdentifierList[value:  a, table] instead 
So how can i deal with that exception? 

`from sqllineage.runner  import *
LineageRunner("select a.* from table1 a, table2 b").target_tables`

`---------------------------------------------------------------------------
SQLLineageException                       Traceback (most recent call last)
<ipython-input-102-0715be0fec21> in <module>()
----> 1 LineageRunner("select a.* from table1 a, table2 b").target_tables

~/anaconda3/lib/python3.6/site-packages/sqllineage/runner.py in __init__(self, sql, encoding, verbose)
     31             if s.token_first(skip_cm=True)
     32         ]
---> 33         self._lineage_results = [LineageAnalyzer().analyze(stmt) for stmt in self._stmt]
     34         self._combined_lineage_result = combine(*self._lineage_results)
     35         self._verbose = verbose

~/anaconda3/lib/python3.6/site-packages/sqllineage/runner.py in <listcomp>(.0)
     31             if s.token_first(skip_cm=True)
     32         ]
---> 33         self._lineage_results = [LineageAnalyzer().analyze(stmt) for stmt in self._stmt]
     34         self._combined_lineage_result = combine(*self._lineage_results)
     35         self._verbose = verbose

~/anaconda3/lib/python3.6/site-packages/sqllineage/core.py in analyze(self, stmt)
     91         else:
     92             # DML parsing logic also applies to CREATE DDL
---> 93             self._extract_from_DML(stmt)
     94         return self._lineage_result
     95 

~/anaconda3/lib/python3.6/site-packages/sqllineage/core.py in _extract_from_DML(self, token)
    143                         raise SQLLineageException(
    144                             "An Identifier is expected, got %s[value: %s] instead"
--> 145                             % (type(sub_token).__name__, sub_token)
    146                         )
    147                     source_table_token_flag = False

SQLLineageException: An Identifier is expected, got IdentifierList[value: table1 a, table2 b] instead`

Discussion:
codecov[bot]: # [Codecov](https://codecov.io/gh/reata/sqllineage/pull/90?src=pr&el=h1) Report
> Merging [#90](https://codecov.io/gh/reata/sqllineage/pull/90?src=pr&el=desc) into [master](https://codecov.io/gh/reata/sqllineage/commit/46b98fbe548b61e5d63bef8afcb3063c026c4193?el=desc) will **increase** coverage by `0.04%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/reata/sqllineage/pull/90/graphs/tree.svg?width=650&height=150&src=pr&token=latkaw2Lx6)](https://codecov.io/gh/reata/sqllineage/pull/90?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master      #90      +/-   ##
==========================================
+ Coverage   96.56%   96.60%   +0.04%     
==========================================
  Files           7        7              
  Lines         320      324       +4     
==========================================
+ Hits          309      313       +4     
  Misses         11       11              
```


| [Impacted Files](https://codecov.io/gh/reata/sqllineage/pull/90?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sqllineage/core.py](https://codecov.io/gh/reata/sqllineage/pull/90/diff?src=pr&el=tree#diff-c3FsbGluZWFnZS9jb3JlLnB5) | `96.03% <100.00%> (+0.16%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/reata/sqllineage/pull/90?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/reata/sqllineage/pull/90?src=pr&el=footer). Last update [46b98fb...5c63df0](https://codecov.io/gh/reata/sqllineage/pull/90?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/sqllineage/core.py b/sqllineage/core.py
index a4f627f..2df62d7 100644
--- a/sqllineage/core.py
+++ b/sqllineage/core.py
@@ -6,6 +6,7 @@ from sqlparse.sql import (
     Comparison,
     Function,
     Identifier,
+    IdentifierList,
     Parenthesis,
     Statement,
     TokenList,
@@ -135,6 +136,11 @@ class LineageAnalyzer:
                             pass
                         else:
                             self._lineage_result.read.add(Table.create(sub_token))
+                    elif isinstance(sub_token, IdentifierList):
+                        # This is to support join in ANSI-89 syntax
+                        for token in sub_token.tokens:
+                            if isinstance(token, Identifier):
+                                self._lineage_result.read.add(Table.create(token))
                     elif isinstance(sub_token, Parenthesis):
                         # SELECT col1 FROM (SELECT col2 FROM tab1), the subquery will be parsed as Parenthesis
                         # This syntax without alias for subquery is invalid in MySQL, while valid for SparkSQL


Test patch:
diff --git a/tests/test_select.py b/tests/test_select.py
index bfbf552..944f1f8 100644
--- a/tests/test_select.py
+++ b/tests/test_select.py
@@ -128,5 +128,9 @@ def test_select_join_with_subquery():
     )
 
 
+def test_select_join_in_ansi89_syntax():
+    helper("SELECT * FROM tab1 a, tab2 b", {"tab1", "tab2"})
+
+
 def test_with_select():
     helper("WITH tab1 AS (SELECT 1) SELECT * FROM tab1")