Repository: reata/sqllineage

Issue:
Skip Lineage Analysis for SparkSQL Function Related Statement
**Is your feature request related to a problem? Please describe.**
Now using sqllineage with sparksql dialect throws UnsupportedStatementException for function related statement like create function or drop function.

**Describe the solution you'd like**
Skip lineage analysis for these statements.

**Additional context**
CREATE FUNCTION: https://spark.apache.org/docs/latest/sql-ref-syntax-ddl-create-function.html
DROP FUNCTION: https://spark.apache.org/docs/latest/sql-ref-syntax-ddl-drop-function.html
DESCRIBE FUNCTION: https://spark.apache.org/docs/latest/sql-ref-syntax-aux-describe-function.html
SHOW FUNCTIONS: https://spark.apache.org/docs/latest/sql-ref-syntax-aux-show-functions.html
SET: https://spark.apache.org/docs/latest/sql-ref-syntax-aux-conf-mgmt-set.html

Discussion:
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/reata/sqllineage/pull/501?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Comparison is base [(`3874881`)](https://app.codecov.io/gh/reata/sqllineage/commit/3874881fe86d67f9680de865f21333d2cbd72ac4?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) 99.41% compared to head [(`8470107`)](https://app.codecov.io/gh/reata/sqllineage/pull/501?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) 99.41%.


<details><summary>Additional details and impacted files</summary>


```diff
@@           Coverage Diff           @@
##           master     #501   +/-   ##
=======================================
  Coverage   99.41%   99.41%           
=======================================
  Files          40       40           
  Lines        2045     2045           
=======================================
  Hits         2033     2033           
  Misses         12       12           
```



</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/reata/sqllineage/pull/501?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None).


Patch:
diff --git a/sqllineage/core/parser/sqlfluff/extractors/noop.py b/sqllineage/core/parser/sqlfluff/extractors/noop.py
index 2c87957..b4c90e4 100644
--- a/sqllineage/core/parser/sqlfluff/extractors/noop.py
+++ b/sqllineage/core/parser/sqlfluff/extractors/noop.py
@@ -17,9 +17,14 @@ class NoopExtractor(BaseExtractor):
         "cache_table",
         "uncache_table",
         "show_statement",
+        "describe_statement",
         "use_statement",
         "declare_segment",
         "analyze_statement",
+        "add_jar_statement",
+        "create_function_statement",
+        "drop_function_statement",
+        "set_statement",
     ]
 
     def extract(


Test patch:
diff --git a/tests/sql/table/other_single_statement/test_other_without_lineage_dialect_specific.py b/tests/sql/table/other_single_statement/test_other_without_lineage_dialect_specific.py
index 6d2af4c..e4d7bc9 100644
--- a/tests/sql/table/other_single_statement/test_other_without_lineage_dialect_specific.py
+++ b/tests/sql/table/other_single_statement/test_other_without_lineage_dialect_specific.py
@@ -45,6 +45,74 @@ def test_show_create_table(dialect: str):
     assert_table_lineage_equal("show create table tab1", dialect=dialect)
 
 
+@pytest.mark.parametrize("dialect", ["databricks", "sparksql"])
+def test_add_jar(dialect: str):
+    assert_table_lineage_equal("ADD JAR /tmp/SimpleUdf.jar", dialect=dialect)
+
+
+@pytest.mark.parametrize("dialect", ["databricks", "sparksql"])
+def test_create_function(dialect: str):
+    assert_table_lineage_equal(
+        """CREATE FUNCTION simple_udf AS 'SimpleUdf'
+  USING JAR '/tmp/SimpleUdf.jar'""",
+        dialect=dialect,
+    )
+
+
+@pytest.mark.parametrize("dialect", ["databricks", "sparksql"])
+def test_create_or_replace_function(dialect: str):
+    assert_table_lineage_equal(
+        """CREATE OR REPLACE FUNCTION simple_udf AS 'SimpleUdf'
+      USING JAR '/tmp/SimpleUdf.jar'""",
+        dialect=dialect,
+    )
+
+
+@pytest.mark.parametrize("dialect", ["databricks", "sparksql"])
+def test_create_temporary_function(dialect: str):
+    assert_table_lineage_equal(
+        """CREATE TEMPORARY FUNCTION simple_temp_udf AS 'SimpleUdf'
+  USING JAR '/tmp/SimpleUdf.jar'""",
+        dialect=dialect,
+    )
+
+
+@pytest.mark.parametrize("dialect", ["databricks", "sparksql"])
+def test_create_or_replace_temporary_function(dialect: str):
+    assert_table_lineage_equal(
+        """CREATE OR REPLACE TEMPORARY FUNCTION simple_temp_udf AS 'SimpleUdf'
+  USING JAR '/tmp/SimpleUdf.jar'""",
+        dialect=dialect,
+    )
+
+
+@pytest.mark.parametrize("dialect", ["databricks", "sparksql"])
+def test_show_functions(dialect: str):
+    assert_table_lineage_equal("SHOW FUNCTIONS trim", dialect=dialect)
+
+
+@pytest.mark.parametrize("dialect", ["databricks", "sparksql"])
+def test_describe_function(dialect: str):
+    assert_table_lineage_equal("DESC FUNCTION abs", dialect=dialect)
+
+
+@pytest.mark.parametrize("dialect", ["databricks", "sparksql"])
+def test_drop_function(dialect: str):
+    assert_table_lineage_equal("DROP FUNCTION test_avg", dialect=dialect)
+
+
+@pytest.mark.parametrize("dialect", ["databricks", "hive", "sparksql"])
+def test_set_command(dialect: str):
+    assert_table_lineage_equal(
+        "SET spark.sql.variable.substitute=false", dialect=dialect
+    )
+
+
+@pytest.mark.parametrize("dialect", ["databricks", "sparksql"])
+def test_set_command_without_property_value(dialect: str):
+    assert_table_lineage_equal("SET spark.sql.variable.substitute", dialect=dialect)
+
+
 @pytest.mark.parametrize("dialect", ["postgres", "redshift"])
 def test_analyze_table(dialect: str):
     assert_table_lineage_equal("analyze tab", dialect=dialect)