Repository: elastic/apm-agent-python

Issue:
dbapi2 fails to extract table name when using square brackets 
**Describe the bug**: ...

Queries made to tables which requires escaping end up with the wrong span name.

The following spans are SELECTs from four different tables, but only two unique span names appear.
![Screenshot 2023-10-31 at 19 13 46](https://github.com/elastic/apm-agent-python/assets/435885/5384fd4d-df13-4828-a9f2-b7e1c309f07f)

**To Reproduce**

Import package and call [extract_signature](https://github.com/elastic/apm-agent-python/blob/05332cd007560615b4421b1567659ff9f9634088/elasticapm/instrumentation/packages/dbapi2.py#L153):

```python
>>> from elasticapm.instrumentation.packages import dbapi2
>>> dbapi2.extract_signature("SELECT username FROM user")
'SELECT FROM user'
>>> dbapi2.extract_signature("SELECT username FROM [user]")
'SELECT FROM ['
```

**Environment (please complete the following information)**
- OS: [e.g. Linux]
- Python version:
- Framework and version [e.g. Django 2.1]:
- APM Server version:
- Agent version:


**Additional context**

Add any other context about the problem here.

- Agent config options <!-- be careful not to post sensitive information -->
  <details>
    <summary>Click to expand</summary>

    ```
    replace this line with your agent config options
    remember to mask any sensitive fields like tokens
    ```
  </details>
- `requirements.txt`:
  <details>
    <summary>Click to expand</summary>

    ```
    replace this line with your `requirements.txt`
    ```
  </details>



Discussion:
Hi, I would like to contribute and give it a try to fix this
@aless10 I'm touching the same thing in this PR. Perhaps you will find my parameterized test useful as boiler-plate code here: https://github.com/elastic/apm-agent-python/pull/1938/files

Resolving patch:
diff --git a/elasticapm/instrumentation/packages/dbapi2.py b/elasticapm/instrumentation/packages/dbapi2.py
--- a/elasticapm/instrumentation/packages/dbapi2.py
+++ b/elasticapm/instrumentation/packages/dbapi2.py
@@ -76,8 +76,8 @@ def _scan_for_table_with_tokens(tokens, keyword):
 
 
 def tokenize(sql):
-    # split on anything that is not a word character, excluding dots
-    return [t for t in re.split(r"([^\w.])", sql) if t != ""]
+    # split on anything that is not a word character or a square bracket, excluding dots
+    return [t for t in re.split(r"([^\w.\[\]])", sql) if t != ""]
 
 
 def scan(tokens):


Test patch:
diff --git a/tests/instrumentation/dbapi2_tests.py b/tests/instrumentation/dbapi2_tests.py
--- a/tests/instrumentation/dbapi2_tests.py
+++ b/tests/instrumentation/dbapi2_tests.py
@@ -154,3 +154,18 @@ def test_extract_signature_for_procedure_call(sql, expected):
 def test_extract_action_from_signature(sql, expected):
     actual = extract_action_from_signature(sql, "query")
     assert actual == expected
+
+
+@pytest.mark.parametrize(
+    ["sql", "expected"],
+    [
+        ("SELECT username FROM user", "SELECT FROM user"),
+        ("SELECT username FROM [user]", "SELECT FROM [user]"),
+        ("SELECT username FROM [db].[user]", "SELECT FROM [db].[user]"),
+        ("SELECT username FROM db.[user]", "SELECT FROM db.[user]"),
+        ("SELECT username FROM [db].user", "SELECT FROM [db].user"),
+    ],
+)
+def test_extract_signature_when_using_square_brackets(sql, expected):
+    actual = extract_signature(sql)
+    assert actual == expected