Repository: sqlfluff/sqlfluff

Issue:
Enhance L004 to enforce one indentation style across all files
Currently L004 throws an error if there is inconsistent indentation in a file (mixed spaces and tabs).

```
L: 121 | P:   1 | L004 | Mixed Tab and Space indentation found in file.
```

How would we feel about changing this to just enforce either tabs or spaces across all files? I personally would much prefer it as an anti-tabber, the logic would be simpler and a fix would be easy to implement. The indentation preference could just be set in the config.

Discussion:
alanmcruickshank: I like the thinking around this re-write - but it changes the intent of this rule a bit more. The original requires a newline and to recognize an indent, this new logic just looks for long patches of whitespace (regardless of whether they're at the start of a line).

What I'm not sure about is whether this is a bad thing. The potential issues would be:
- Handing indents (but these are only really possible with spaces *anyway* so probably not an issue).
- Long patches of whitespace *after* code, which it probably for layout reasons.

I think the former is probably fine, but I think the latter could behave strangely in cases like this:

```
SELECT
    a,         -- Some comment
    longer_col -- A lined up comment
FROM spam
```

In this example we have extended patches of whitespace *before and after* `a,`. The reason I'm not sure whether this is a problem is because I wonder if the same fixing logic actually applies fine 🤷 .

@NiallRees - could you add in a couple of test cases like this with a mixture of tabs and spaces in the before and after slots (and different configs) to make sure we don't behave strangely in these cases now that we're not looking for an explicit newline.

If it behaves nicely - then I think this is actually a **feature** rather than a bug and we should document it as such! 😄 


NiallRees: > SELECT
>     a,         -- Some comment
>     longer_col -- A lined up comment
> FROM spam

Just done some testing with indents after text. I hadn't quite appreciated what text editors did with variable tab size to align comments. I think we're fine though for pre-text indents. This is the current result with indented comments after text:

![image](https://user-images.githubusercontent.com/23722609/101277789-c1c9d800-37ae-11eb-90c5-fe0da616c7e5.png)

I think the options are

1. Revert to previous in only trying to convert tabs to spaces which are directly after a newline
2. Introduce more complex text editor style understanding of tabs, and take into account the number of characters from the newline to determine how many space characters a tab should be converted to if it appears after other text.

I'm tempted to go with 1 for now as it's still an improvement over the previous rule (rule is triggered when 'wrong' indent used, and will convert from tabs->spaces and vice-versa), and is 'safe' in terms of not messing up formatting.
alanmcruickshank: I'm definitely fine with 1 for now. 2 could be fun, but I think it's a lot less important than the other. Using aligned comments after things is a lot less common than indentation in general (perhaps obviously 🤷 ).

Patch:
diff --git a/src/sqlfluff/core/rules/std.py b/src/sqlfluff/core/rules/std.py
index 8fe667776..75fedcdcc 100644
--- a/src/sqlfluff/core/rules/std.py
+++ b/src/sqlfluff/core/rules/std.py
@@ -689,51 +689,79 @@ class Rule_L003(BaseCrawler):
         return LintResult(memory=memory)
 
 
+@std_rule_set.document_fix_compatible
+@std_rule_set.document_configuration
 @std_rule_set.register
 class Rule_L004(BaseCrawler):
-    """Mixed Tab and Space indentation found in file.
+    """Incorrect indentation type.
+
+    Note 1: spaces are only fixed to tabs if the number of spaces in the
+    indent is an integer multiple of the tab_space_size config.
+    Note 2: fixes are only applied to indents at the start of a line. Indents
+    after other text on the same line are not fixed.
 
     | **Anti-pattern**
-    | The • character represents a space and the → character represents a tab.
-    | In this example, the second line is indented with spaces and the third one with tab.
+    | Using tabs instead of spaces when indent_unit config set to spaces (default).
 
     .. code-block::
 
-        SELECT
+        select
         ••••a,
         →   b
-        FROM foo
+        from foo
 
     | **Best practice**
     | Change the line to use spaces only.
 
     .. code-block::
 
-        SELECT
+        select
         ••••a,
         ••••b
-        FROM foo
+        from foo
     """
 
-    def _eval(self, segment, raw_stack, memory, **kwargs):
-        """Mixed Tab and Space indentation found in file.
+    config_keywords = ["indent_unit", "tab_space_size"]
 
-        We use the `memory` feature here to keep track of
-        what we've seen in the past.
-
-        """
-        indents_seen = memory.get("indents_seen", set())
-        if segment.is_type("whitespace"):
-            if len(raw_stack) == 0 or raw_stack[-1].is_type("newline"):
-                indents_here = set(segment.raw)
-                indents_union = indents_here | indents_seen
-                memory["indents_seen"] = indents_union
-                if len(indents_union) > 1:
-                    # We are seeing an indent we haven't seen before and we've seen others before
-                    return LintResult(anchor=segment, memory=memory)
-                else:
-                    return LintResult(memory=memory)
-        return LintResult(memory=memory)
+    # TODO fix indents after text: https://github.com/sqlfluff/sqlfluff/pull/590#issuecomment-739484190
+    def _eval(self, segment, raw_stack, **kwargs):
+        """Incorrect indentation found in file."""
+        tab = "\t"
+        space = " "
+        correct_indent = (
+            space * self.tab_space_size if self.indent_unit == "space" else tab
+        )
+        wrong_indent = (
+            tab if self.indent_unit == "space" else space * self.tab_space_size
+        )
+        if segment.is_type("whitespace") and wrong_indent in segment.raw:
+            description = "Incorrect indentation type found in file."
+            edit_indent = segment.raw.replace(wrong_indent, correct_indent)
+            # Ensure that the number of space indents is a multiple of tab_space_size
+            # before attempting to convert spaces to tabs to avoid mixed indents
+            # unless we are converted tabs to spaces (indent_unit = space)
+            if (
+                (
+                    self.indent_unit == "space"
+                    or segment.raw.count(space) % self.tab_space_size == 0
+                )
+                # Only attempt a fix at the start of a newline for now
+                and (len(raw_stack) == 0 or raw_stack[-1].is_type("newline"))
+            ):
+                fixes = [
+                    LintFix(
+                        "edit",
+                        segment,
+                        self.make_whitespace(
+                            raw=edit_indent,
+                            pos_marker=segment.pos_marker,
+                        ),
+                    )
+                ]
+            else:
+                description += " No fix available as number of spaces in indent is not a multiple of tab_space_size."
+                fixes = []
+            return LintResult(anchor=segment, fixes=fixes, description=description)
 
 
 @std_rule_set.document_fix_compatible


Test patch:
diff --git a/test/core/rules/test_cases/L004.yml b/test/core/rules/test_cases/L004.yml
index b22617b32..f3c8fb0ec 100644
--- a/test/core/rules/test_cases/L004.yml
+++ b/test/core/rules/test_cases/L004.yml
@@ -1,10 +1,106 @@
 rule: L004
 
-test_1:
-  pass_str: "   \nSELECT 1"
+spaces_pass_default:
+  pass_str: "    \nSELECT 1"
 
-test_2:
-  pass_str: "\t\tSELECT 1\n"
 
-test_3:
-  fail_str: "   \n  \t \n  SELECT 1"
+spaces_fail:
+  fail_str: "    \nSELECT 1"
+  fix_str: "\t\nSELECT 1"
+
+  configs:
+    rules:
+      indent_unit: tab
+
+
+spaces_fail_custom_tab_space_size:
+  fail_str: "    \nSELECT 1"
+  fix_str: "\t\t\nSELECT 1"
+
+  configs:
+    rules:
+      indent_unit: tab
+      tab_space_size: 2
+
+
+tabs_fail_default:
+  fail_str: "\t\tSELECT 1\n"
+  fix_str: "        SELECT 1\n"
+
+
+tabs_fail_default_set_tab_space_size:
+  fail_str: "\t\tSELECT 1\n"
+  fix_str: "    SELECT 1\n"
+
+  configs:
+    rules:
+      tab_space_size: 2
+
+
+tabs_pass:
+  pass_str: "\tSELECT 1"
+
+  configs:
+    rules:
+      indent_unit: tab
+
+
+mixed_indent_fail_default_tab_space_size:
+  fail_str: |2
+      	 select 1
+  fix_str: |2
+           select 1
+
+
+mixed_indent_fail_custom_tab_space_size:
+  fail_str: |2
+      	 select 1
+  fix_str: |2
+         select 1
+
+  configs:
+    rules:
+      tab_space_size: 2
+
+
+indented_comments:
+  pass_str: |
+    SELECT
+        a,         -- Some comment
+        longer_col -- A lined up comment
+    FROM spam
+
+indented_comments_default_config:
+  fail_str: |
+    SELECT
+    	a,			-- Some comment
+    	longer_col	-- A lined up comment
+    FROM spam
+
+  # The rule will only fix the indent before the select targets.
+  fix_str: |
+    SELECT
+        a,			-- Some comment
+        longer_col	-- A lined up comment
+    FROM spam
+
+
+indented_comments_tab_config:
+  fail_str: |
+    SELECT
+        a,         -- Some comment
+        longer_col -- A lined up comment
+    FROM spam
+
+  # The rule will only fix the indent before the select targets.
+  fix_str: |
+    SELECT
+    	a,         -- Some comment
+    	longer_col -- A lined up comment
+    FROM spam
+
+  configs:
+    rules:
+      indent_unit: tab
+
+# TODO fix indents after text: https://github.com/sqlfluff/sqlfluff/issues/624