Repository: sqlfluff/sqlfluff

Issue:
Better documentation for pyproject.toml configuration file
### Search before asking

- [X] I searched the [issues](https://github.com/sqlfluff/sqlfluff/issues) and found no similar issues.


### Links or command line

I have been using pyproject.toml for my sqlfluff configuration without any issues for  a long time now.
But, now that I have updated to sqlfluff==2.0.0 I can't quite work out how to configure rules.

Where I once had:
```
[tool.sqlfluff.rules.L014]
extended_capitalisation_policy = 'lower'
```
I first assumed I had to change it to:
```
[tool.sqlfluff.rules.capitalisation.identifiers]
extended_capitalisation_policy = 'lower'
```

However then I got a warning:
```
WARNING    Rule configuration contain a section for unexpected rule 'capitalisation'. These values will be ignored.              
WARNING    The reference was found as a match for multiple rules: {'CP03', 'CP05', 'CP02', 'CP04', 'CP01'}. Config should be specified by the name of the relevant rule e.g. 'sqlfluff:rules:capitalisation.keywords'. 
```

### Issue/Suggested Improvement

Eventually I worked out that I had to have:

```
[tool.sqlfluff.rules.'capitalisation.identifiers']
extended_capitalisation_policy = 'lower'
```
For it to work capitalisation.identifiers has to be in quotes so that it is treated as one section.
Documentation for pyproject.toml config should probably be updated to reflect this.

Or if there is a nicer way to have pyproject.toml work with the updated rules config style, then it would be great to know that.

### Are you willing to work on and submit a PR to address the issue?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/sqlfluff/sqlfluff/blob/main/CODE_OF_CONDUCT.md)


Discussion:
coveralls: ## Pull Request Test Coverage Report for [Build 4426382198](https://coveralls.io/builds/57882254)

* **4** of **4**   **(100.0%)**  changed or added relevant lines in **1** file are covered.
* No unchanged relevant lines lost coverage.
* Overall coverage remained the same at **100.0%**

---



|  Totals | [![Coverage Status](https://coveralls.io/builds/57882254/badge)](https://coveralls.io/builds/57882254) |
| :-- | --: |
| Change from base [Build 4425279952](https://coveralls.io/builds/57876512): |  0.0% |
| Covered Lines: | 17445 |
| Relevant Lines: | 17445 |

---
##### 💛  - [Coveralls](https://coveralls.io)

codecov[bot]: ## [Codecov](https://codecov.io/gh/sqlfluff/sqlfluff/pull/4526?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff) Report
Patch coverage: **`100.00`**% and no project coverage change.
> Comparison is base [(`989a770`)](https://codecov.io/gh/sqlfluff/sqlfluff/commit/989a770213b745527632ad7f15fdf2b990b647db?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff) 100.00% compared to head [(`59849e4`)](https://codecov.io/gh/sqlfluff/sqlfluff/pull/4526?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff) 100.00%.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff            @@
##              main     #4526   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files          205       205           
  Lines        15597     15599    +2     
=========================================
+ Hits         15597     15599    +2     
```


| [Impacted Files](https://codecov.io/gh/sqlfluff/sqlfluff/pull/4526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff) | Coverage Δ | |
|---|---|---|
| [src/sqlfluff/core/config.py](https://codecov.io/gh/sqlfluff/sqlfluff/pull/4526?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff#diff-c3JjL3NxbGZsdWZmL2NvcmUvY29uZmlnLnB5) | `100.00% <100.00%> (ø)` | |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff)

</details>

[:umbrella: View full report in Codecov by Sentry](https://codecov.io/gh/sqlfluff/sqlfluff/pull/4526?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff).


Patch:
diff --git a/docs/source/configuration.rst b/docs/source/configuration.rst
index 18cc48384..a525a8de0 100644
--- a/docs/source/configuration.rst
+++ b/docs/source/configuration.rst
@@ -71,6 +71,12 @@ For example, a snippet from a :code:`pyproject.toml` file:
     [tool.sqlfluff.templater.jinja]
     apply_dbt_builtins = true
 
+    # For rule specific configuration, use dots between the names exactly
+    # as you would in .sqlfluff. In the background, SQLFluff will unpack the
+    # configuration paths accordingly.
+    [tool.sqlfluff.rules.capitalisation.keywords]
+    capitalisation_policy = "upper"
+
 .. _`cfg file`: https://docs.python.org/3/library/configparser.html
 .. _`pyproject.toml file`: https://www.python.org/dev/peps/pep-0518/
 
diff --git a/src/sqlfluff/core/config.py b/src/sqlfluff/core/config.py
index ad71275fd..456e60508 100644
--- a/src/sqlfluff/core/config.py
+++ b/src/sqlfluff/core/config.py
@@ -295,9 +295,45 @@ class ConfigLoader:
         return global_loader
 
     @classmethod
-    def _walk_toml(cls, config: Dict[str, Any], base_key=()):
-        """Recursively walk the nested config inside a TOML file."""
-        buff: List[tuple] = []
+    def _walk_toml(
+        cls, config: Dict[str, Any], base_key: Tuple[str, ...] = ()
+    ) -> List[Tuple[Tuple[str, ...], Any]]:
+        """Recursively walk the nested config inside a TOML file.
+
+        For standard usage it mimics the standard loader.
+
+        >>> ConfigLoader._walk_toml({"foo": "bar"})
+        [(('foo',), 'bar')]
+        >>> ConfigLoader._walk_toml({"foo": {"bar": "baz"}})
+        [(('foo', 'bar'), 'baz')]
+
+        For the "rules" section, there's a special handling
+        to condense nested sections from the toml for rules
+        which contain a dot (or more) (".") in their name.
+
+        >>> ConfigLoader._walk_toml({"rules": {"a": {"b": {"c": "d"}}}})
+        [(('rules', 'a.b', 'c'), 'd')]
+        >>> ConfigLoader._walk_toml({"rules":
+        ...     {"capitalisation": {"keywords":
+        ...         {"capitalisation_policy": "upper"}
+        ...     }}
+        ... })
+        [(('rules', 'capitalisation.keywords', 'capitalisation_policy'), 'upper')]
+
+        NOTE: Some rules make have more than one dot in their name.
+        >>> ConfigLoader._walk_toml({"rules":
+        ...     {"a": {"b": {"c": {"d": {"e": "f"}}}}}
+        ... })
+        [(('rules', 'a.b.c.d', 'e'), 'f')]
+        """
+        buff: List[Tuple[Tuple[str, ...], Any]] = []
+        # NOTE: For the "rules" section of the sqlfluff config,
+        # rule names are often qualified with a dot ".". In the
+        # toml scenario this can get interpreted as a nested
+        # section, and we resolve that edge case here.
+        if len(base_key) == 3 and base_key[0] == "rules":
+            base_key = ("rules", ".".join(base_key[1:]))
+
         for k, v in config.items():
             key = base_key + (k,)
             if isinstance(v, dict):


Test patch:
diff --git a/test/core/config_test.py b/test/core/config_test.py
index d9e73d5c1..84bc493e7 100644
--- a/test/core/config_test.py
+++ b/test/core/config_test.py
@@ -149,6 +149,7 @@ def test__config__load_toml():
         },
         "bar": {"foo": "foobar"},
         "fnarr": {"fnarr": {"foo": "foobar"}},
+        "rules": {"capitalisation.keywords": {"capitalisation_policy": "upper"}},
     }
 
 
diff --git a/test/fixtures/config/toml/pyproject.toml b/test/fixtures/config/toml/pyproject.toml
index cda1fc2d5..430057971 100644
--- a/test/fixtures/config/toml/pyproject.toml
+++ b/test/fixtures/config/toml/pyproject.toml
@@ -12,3 +12,6 @@ foo = "foobar"
 
 [tool.sqlfluff.fnarr.fnarr]
 foo = "foobar"
+
+[tool.sqlfluff.rules.capitalisation.keywords]
+capitalisation_policy = "upper"