Repository: sqlfluff/sqlfluff
Pull request: Allowed array/struct values in `default` definition of `declare`
closes #2119 

### Pull Request checklist
- [x] Please confirm you have completed any of the necessary steps below.

- Included test cases to demonstrate any code changes, which may be one or more of the following:
  - `.yml` rule test cases in `test/fixtures/rules/std_rule_cases`.
  - `.sql`/`.yml` parser test cases in `test/fixtures/dialects` (note YML files can be auto generated with `python test/generate_parse_fixture_yml.py` or by running `tox` locally).
  - Full autofix test cases in `test/fixtures/linter/autofix`.
  - Other.
- Added appropriate documentation for the change.
- Created GitHub issues for any relevant followup/future enhancements if appropriate.


Problem statement:
Add support for arrays and structs in DECLARE...DEFAULT (bigquery dialect)
Bigquery dialect currently does not support arrays and structs values in `DEFAULT` section of `DECLARE`
causing `PRS` error
```sql
DECLARE arr1 ARRAY<STRING> DEFAULT ['aaaa', 'bbb', 'cccc'];
DECLARE str1 STRUCT<f1 STRING, f2 STRING, f3 STRING> DEFAULT ('aaaa', 'bbb', 'cccc');
```
lining result
```
L:   1 | P:  27 |  PRS | Line 1, Position 27: Found unparsable section: " DEFAULT ['aaaa', 'bbb', 'cccc']"
L:   2 | P:  53 |  PRS | Line 2, Position 53: Found unparsable section: " DEFAULT ('aaaa', 'bbb', 'cccc')"
```
parsing result
```
[L:  1, P:  1]      |file:
[L:  1, P:  1]      |    statement:
[L:  1, P:  1]      |        declare_segment:
[L:  1, P:  1]      |            keyword:                                          'DECLARE'
[L:  1, P:  8]      |            whitespace:                                       ' '
[L:  1, P:  9]      |            identifier:                                       'arr1'
[L:  1, P: 13]      |            whitespace:                                       ' '
[L:  1, P: 14]      |            data_type:
[L:  1, P: 14]      |                keyword:                                      'ARRAY'
[L:  1, P: 19]      |                start_angle_bracket:                          '<'
[L:  1, P: 20]      |                [META] indent:
[L:  1, P: 20]      |                data_type:
[L:  1, P: 20]      |                    data_type_identifier:                     'STRING'
[L:  1, P: 26]      |                [META] dedent:
[L:  1, P: 26]      |                end_angle_bracket:                            '>'
[L:  1, P: 27]      |            unparsable:                                       !! Expected: 'Nothing...'
[L:  1, P: 27]      |                whitespace:                                   ' '
[L:  1, P: 28]      |                raw:                                          'DEFAULT'
[L:  1, P: 35]      |                whitespace:                                   ' '
[L:  1, P: 36]      |                start_square_bracket:                         '['
[L:  1, P: 37]      |                raw:                                          "'aaaa'"
[L:  1, P: 43]      |                comma:                                        ','
[L:  1, P: 44]      |                whitespace:                                   ' '
[L:  1, P: 45]      |                raw:                                          "'bbb'"
[L:  1, P: 50]      |                comma:                                        ','
[L:  1, P: 51]      |                whitespace:                                   ' '
[L:  1, P: 52]      |                raw:                                          "'cccc'"
[L:  1, P: 58]      |                end_square_bracket:                           ']'
[L:  1, P: 59]      |    statement_terminator:                                     ';'
[L:  1, P: 60]      |    newline:                                                  '\n'
[L:  2, P:  1]      |    statement:
[L:  2, P:  1]      |        declare_segment:
[L:  2, P:  1]      |            keyword:                                          'DECLARE'
[L:  2, P:  8]      |            whitespace:                                       ' '
[L:  2, P:  9]      |            identifier:                                       'str1'
[L:  2, P: 13]      |            whitespace:                                       ' '
[L:  2, P: 14]      |            data_type:
[L:  2, P: 14]      |                keyword:                                      'STRUCT'
[L:  2, P: 20]      |                start_angle_bracket:                          '<'
[L:  2, P: 21]      |                [META] indent:
[L:  2, P: 21]      |                parameter:                                    'f1'
[L:  2, P: 23]      |                whitespace:                                   ' '
[L:  2, P: 24]      |                data_type:
[L:  2, P: 24]      |                    data_type_identifier:                     'STRING'
[L:  2, P: 30]      |                comma:                                        ','
[L:  2, P: 31]      |                whitespace:                                   ' '
[L:  2, P: 32]      |                parameter:                                    'f2'
[L:  2, P: 34]      |                whitespace:                                   ' '
[L:  2, P: 35]      |                data_type:
[L:  2, P: 35]      |                    data_type_identifier:                     'STRING'
[L:  2, P: 41]      |                comma:                                        ','
[L:  2, P: 42]      |                whitespace:                                   ' '
[L:  2, P: 43]      |                parameter:                                    'f3'
[L:  2, P: 45]      |                whitespace:                                   ' '
[L:  2, P: 46]      |                data_type:
[L:  2, P: 46]      |                    data_type_identifier:                     'STRING'
[L:  2, P: 52]      |                [META] dedent:
[L:  2, P: 52]      |                end_angle_bracket:                            '>'
[L:  2, P: 53]      |            unparsable:                                       !! Expected: 'Nothing...'
[L:  2, P: 53]      |                whitespace:                                   ' '
[L:  2, P: 54]      |                raw:                                          'DEFAULT'
[L:  2, P: 61]      |                whitespace:                                   ' '
[L:  2, P: 62]      |                bracketed:
[L:  2, P: 62]      |                    start_bracket:                            '('
[L:  2, P: 63]      |                    raw:                                      "'aaaa'"
[L:  2, P: 69]      |                    comma:                                    ','
[L:  2, P: 70]      |                    whitespace:                               ' '
[L:  2, P: 71]      |                    raw:                                      "'bbb'"
[L:  2, P: 76]      |                    comma:                                    ','
[L:  2, P: 77]      |                    whitespace:                               ' '
[L:  2, P: 78]      |                    raw:                                      "'cccc'"
[L:  2, P: 84]      |                    end_bracket:                              ')'
[L:  2, P: 85]      |    statement_terminator:                                     ';'
[L:  2, P: 86]      |    newline:                                                  '\n'
```

Hints from issue discussion:

Commit message: added `tuple` segment + tests

Patch to review:
diff --git a/src/sqlfluff/dialects/dialect_bigquery.py b/src/sqlfluff/dialects/dialect_bigquery.py
index a462c3eac86..ab0e68de685 100644
--- a/src/sqlfluff/dialects/dialect_bigquery.py
+++ b/src/sqlfluff/dialects/dialect_bigquery.py
@@ -585,18 +585,29 @@ class TypelessStructSegment(BaseSegment):
     type = "typeless_struct"
     match_grammar = Sequence(
         "STRUCT",
-        Bracketed(
-            Delimited(
-                AnyNumberOf(
-                    Sequence(
-                        Ref("BaseExpressionElementGrammar"),
-                        Ref("AliasExpressionSegment", optional=True),
-                    ),
+        Ref("TupleSegment"),
+    )
+
+
+@bigquery_dialect.segment()
+class TupleSegment(BaseSegment):
+    """Expression to construct a TUPLE.
+
+    https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#tuple_syntax
+    """
+
+    type = "tuple"
+    match_grammar = Bracketed(
+        Delimited(
+            AnyNumberOf(
+                Sequence(
+                    Ref("BaseExpressionElementGrammar"),
+                    Ref("AliasExpressionSegment", optional=True),
                 ),
-                delimiter=Ref("CommaSegment"),
             ),
-            optional=True,
+            delimiter=Ref("CommaSegment"),
         ),
+        optional=True,
     )
 
 
@@ -772,6 +783,9 @@ class DeclareStatementSegment(BaseSegment):
                 Bracketed(Ref("SelectStatementSegment")),
                 Ref("BareFunctionSegment"),
                 Ref("FunctionSegment"),
+                Ref("ArrayLiteralSegment"),
+                Ref("TypelessStructSegment"),
+                Ref("TupleSegment"),
             ),
             optional=True,
         ),
diff --git a/test/fixtures/dialects/bigquery/declare_variable.sql b/test/fixtures/dialects/bigquery/declare_variable.sql
index d6dad600aa3..b4e99a78733 100644
--- a/test/fixtures/dialects/bigquery/declare_variable.sql
+++ b/test/fixtures/dialects/bigquery/declare_variable.sql
@@ -1,3 +1,6 @@
 declare var1 int64;
 declare var2, var3 string;
 declare var1 array<string>;
+declare var1 array<string> default ['one', 'two'];
+declare var1 struct<f1 string, f2 string> default ('one', 'two');
+declare var1 struct<f1 string, f2 string> default struct('one', 'two');
diff --git a/test/fixtures/dialects/bigquery/declare_variable.yml b/test/fixtures/dialects/bigquery/declare_variable.yml
index abd4fc73d51..4bbec9627ad 100644
--- a/test/fixtures/dialects/bigquery/declare_variable.yml
+++ b/test/fixtures/dialects/bigquery/declare_variable.yml
@@ -3,7 +3,7 @@
 # computed by SQLFluff when running the tests. Please run
 # `python test/generate_parse_fixture_yml.py`  to generate them after adding or
 # altering SQL files.
-_hash: ffccb95575561408b5510a30ff40b55e12c56a173fa3fc57ef3f5e05e5ee2488
+_hash: bbebf58076072f44c8d44a3eeecb3927d297da07769c2b96bd83d35536676ccd
 file:
 - statement:
     declare_segment:
@@ -32,3 +32,73 @@ file:
           data_type_identifier: string
         end_angle_bracket: '>'
 - statement_terminator: ;
+- statement:
+    declare_segment:
+    - keyword: declare
+    - identifier: var1
+    - data_type:
+        keyword: array
+        start_angle_bracket: <
+        data_type:
+          data_type_identifier: string
+        end_angle_bracket: '>'
+    - keyword: default
+    - array_literal:
+      - start_square_bracket: '['
+      - expression:
+          literal: "'one'"
+      - comma: ','
+      - expression:
+          literal: "'two'"
+      - end_square_bracket: ']'
+- statement_terminator: ;
+- statement:
+    declare_segment:
+    - keyword: declare
+    - identifier: var1
+    - data_type:
+      - keyword: struct
+      - start_angle_bracket: <
+      - parameter: f1
+      - data_type:
+          data_type_identifier: string
+      - comma: ','
+      - parameter: f2
+      - data_type:
+          data_type_identifier: string
+      - end_angle_bracket: '>'
+    - keyword: default
+    - tuple:
+        bracketed:
+        - start_bracket: (
+        - literal: "'one'"
+        - comma: ','
+        - literal: "'two'"
+        - end_bracket: )
+- statement_terminator: ;
+- statement:
+    declare_segment:
+    - keyword: declare
+    - identifier: var1
+    - data_type:
+      - keyword: struct
+      - start_angle_bracket: <
+      - parameter: f1
+      - data_type:
+          data_type_identifier: string
+      - comma: ','
+      - parameter: f2
+      - data_type:
+          data_type_identifier: string
+      - end_angle_bracket: '>'
+    - keyword: default
+    - typeless_struct:
+        keyword: struct
+        tuple:
+          bracketed:
+          - start_bracket: (
+          - literal: "'one'"
+          - comma: ','
+          - literal: "'two'"
+          - end_bracket: )
+- statement_terminator: ;
diff --git a/test/fixtures/dialects/bigquery/select_mixture_of_array_literals.yml b/test/fixtures/dialects/bigquery/select_mixture_of_array_literals.yml
index 6d81ff25034..543aab6b168 100644
--- a/test/fixtures/dialects/bigquery/select_mixture_of_array_literals.yml
+++ b/test/fixtures/dialects/bigquery/select_mixture_of_array_literals.yml
@@ -3,7 +3,7 @@
 # computed by SQLFluff when running the tests. Please run
 # `python test/generate_parse_fixture_yml.py`  to generate them after adding or
 # altering SQL files.
-_hash: 04e1c92046585ea1c067f6c5bfa3c7425fb9518690fb50b45d97c6dadb776d33
+_hash: 636409c0741cc8b945c7d4eb10099be6df45073fa1adb6dbca58edd8a0bd08ed
 file:
   statement:
     select_statement:
@@ -93,32 +93,33 @@ file:
               start_square_bracket: '['
               typeless_struct:
                 keyword: STRUCT
-                bracketed:
-                - start_bracket: (
-                - literal: '"Rudisha"'
-                - alias_expression:
-                    keyword: AS
-                    identifier: name
-                - comma: ','
-                - expression:
-                    array_literal:
-                    - start_square_bracket: '['
-                    - expression:
-                        literal: '23.4'
-                    - comma: ','
-                    - expression:
-                        literal: '26.3'
-                    - comma: ','
-                    - expression:
-                        literal: '26.4'
-                    - comma: ','
-                    - expression:
-                        literal: '26.1'
-                    - end_square_bracket: ']'
-                - alias_expression:
-                    keyword: AS
-                    identifier: splits
-                - end_bracket: )
+                tuple:
+                  bracketed:
+                  - start_bracket: (
+                  - literal: '"Rudisha"'
+                  - alias_expression:
+                      keyword: AS
+                      identifier: name
+                  - comma: ','
+                  - expression:
+                      array_literal:
+                      - start_square_bracket: '['
+                      - expression:
+                          literal: '23.4'
+                      - comma: ','
+                      - expression:
+                          literal: '26.3'
+                      - comma: ','
+                      - expression:
+                          literal: '26.4'
+                      - comma: ','
+                      - expression:
+                          literal: '26.1'
+                      - end_square_bracket: ']'
+                  - alias_expression:
+                      keyword: AS
+                      identifier: splits
+                  - end_bracket: )
               end_square_bracket: ']'
           alias_expression:
             keyword: AS
diff --git a/test/fixtures/dialects/bigquery/select_struct.yml b/test/fixtures/dialects/bigquery/select_struct.yml
index 76e5f9c80c6..351d293256b 100644
--- a/test/fixtures/dialects/bigquery/select_struct.yml
+++ b/test/fixtures/dialects/bigquery/select_struct.yml
@@ -3,7 +3,7 @@
 # computed by SQLFluff when running the tests. Please run
 # `python test/generate_parse_fixture_yml.py`  to generate them after adding or
 # altering SQL files.
-_hash: e1eabbbf7187911728236ad955aac6db71d59b9eb3b9cb2616151546fa45813a
+_hash: fcac8016b28111aa94274e0292e7f47611dbf6f7e30f1dc8b71557664feac67d
 file:
 - statement:
     select_statement:
@@ -86,24 +86,25 @@ file:
         select_clause_element:
           typeless_struct:
             keyword: struct
-            bracketed:
-            - start_bracket: (
-            - column_reference:
-              - identifier: bar
-              - dot: .
-              - identifier: bar_id
-            - alias_expression:
-                keyword: as
-                identifier: id
-            - comma: ','
-            - column_reference:
-              - identifier: bar
-              - dot: .
-              - identifier: bar_name
-            - alias_expression:
-                keyword: as
-                identifier: bar
-            - end_bracket: )
+            tuple:
+              bracketed:
+              - start_bracket: (
+              - column_reference:
+                - identifier: bar
+                - dot: .
+                - identifier: bar_id
+              - alias_expression:
+                  keyword: as
+                  identifier: id
+              - comma: ','
+              - column_reference:
+                - identifier: bar
+                - dot: .
+                - identifier: bar_name
+              - alias_expression:
+                  keyword: as
+                  identifier: bar
+              - end_bracket: )
           alias_expression:
             keyword: as
             identifier: bar
diff --git a/test/fixtures/dialects/bigquery/select_typeless_struct_inside_function.yml b/test/fixtures/dialects/bigquery/select_typeless_struct_inside_function.yml
index d6ead55c1ce..9eb1dae75c7 100644
--- a/test/fixtures/dialects/bigquery/select_typeless_struct_inside_function.yml
+++ b/test/fixtures/dialects/bigquery/select_typeless_struct_inside_function.yml
@@ -3,7 +3,7 @@
 # computed by SQLFluff when running the tests. Please run
 # `python test/generate_parse_fixture_yml.py`  to generate them after adding or
 # altering SQL files.
-_hash: b0a15006dcb0e8923a1437a02e82b14d9eb3753c6c9be8d835a2c00652a14e22
+_hash: 7d82bb1f46d6d3a34bf5cf7146bb87fb095df26c730b6e1375685cac73ae4c29
 file:
 - statement:
     select_statement:
@@ -12,21 +12,23 @@ file:
         select_clause_element:
           typeless_struct:
             keyword: STRUCT
-            bracketed:
-              start_bracket: (
-              typeless_struct:
-                keyword: STRUCT
-                bracketed:
-                  start_bracket: (
-                  literal: '1'
-                  alias_expression:
-                    keyword: AS
-                    identifier: b
-                  end_bracket: )
-              alias_expression:
-                keyword: AS
-                identifier: a
-              end_bracket: )
+            tuple:
+              bracketed:
+                start_bracket: (
+                typeless_struct:
+                  keyword: STRUCT
+                  tuple:
+                    bracketed:
+                      start_bracket: (
+                      literal: '1'
+                      alias_expression:
+                        keyword: AS
+                        identifier: b
+                      end_bracket: )
+                alias_expression:
+                  keyword: AS
+                  identifier: a
+                end_bracket: )
           alias_expression:
             keyword: AS
             identifier: foo
@@ -43,20 +45,21 @@ file:
               start_bracket: (
               typeless_struct:
                 keyword: STRUCT
-                bracketed:
-                - start_bracket: (
-                - column_reference:
-                    identifier: a
-                - alias_expression:
-                    keyword: AS
-                    identifier: a
-                - comma: ','
-                - column_reference:
-                    identifier: b
-                - alias_expression:
-                    keyword: AS
-                    identifier: b
-                - end_bracket: )
+                tuple:
+                  bracketed:
+                  - start_bracket: (
+                  - column_reference:
+                      identifier: a
+                  - alias_expression:
+                      keyword: AS
+                      identifier: a
+                  - comma: ','
+                  - column_reference:
+                      identifier: b
+                  - alias_expression:
+                      keyword: AS
+                      identifier: b
+                  - end_bracket: )
               end_bracket: )
       from_clause:
         keyword: FROM


Code review comments:
diff_hunk: @@ -585,18 +585,29 @@ class TypelessStructSegment(BaseSegment):
     type = "typeless_struct"
     match_grammar = Sequence(
         "STRUCT",
-        Bracketed(
-            Delimited(
-                AnyNumberOf(
-                    Sequence(
-                        Ref("BaseExpressionElementGrammar"),
-                        Ref("AliasExpressionSegment", optional=True),
-                    ),
+        Ref("TupleSegment"),
+    )
+
+
+@bigquery_dialect.segment()
+class TupleSegment(BaseSegment):
+    """Expression to construct a TUPLE.
+
+    https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types#tuple_syntax
+    """
+
+    type = "tuple"
+    match_grammar = Bracketed(
path: src/sqlfluff/dialects/dialect_bigquery.py
text: @user1:
Think this match grammar should just be
```sql
match_grammar = Bracketed(
        Delimited(
            Sequence(
                Ref("BaseExpressionElementGrammar"),
                Ref("AliasExpressionSegment", optional=True),
            ),
        ),
    )
```
that's what's in the docs
![image](https://user-images.githubusercontent.com/80432516/146278815-bcd09ab3-f498-4aa9-b2e7-ffbd9197e377.png)


@author:
what about any number of values in tuple?

@author:
will revert `TypelessStruct`
and redefine `Tuple`

@user1:
what was wrong with just changing the match grammar in tuple to what I had in my original comment?

@user1:
Changing TypelessStruct was fine as you originally did it, its just the original match grammar was too verbose.

@user1:
> what about any number of values in tuple?

Delimited contains contains any number of items anyway so the AnyNumber of was redundant

@user1:
when I tried it, it parsed fine for all the existing test cases

@author:
TypelessStruct and Tuple syntax are partially compatible as of BQ documentation