Repository: python-gitlab/python-gitlab

Issue:
URL-encoded kwargs are infinitely duplicated in query params (v3.7.0 regression for arrays)
## Description of the problem, including code/CLI snippet

With #1699 we added array params for query params in GET requests. But GitLab also returns that param, URL-encoded, in the `next` URL which we then follow in listing endpoints. Because one is URL-encoded and the other isn't, the param gets added at the end of the query again. With `list()` calls this happens over and over again and causes 502 due to super long query strings eventually.

minimal reproduction:

```python
import gitlab

gl = gitlab.Gitlab()
gl.enable_debug()

projects = gl.projects.list(topics=["python"], per_page=1, iterator=True)

for project in projects:
    # watch the `send` URL grow continuously until triggering 502
    pass

```

## Expected Behavior

If a kwarg/param (URL-encoded or not) that we provide is already in the provided URL, it should never be duplicated when we construct the final request URL.

## Actual Behavior

`http_request()` and requests can't tell if our kwargs match those already in the URL params and they get duplicated to oblivion, causing 502 once the URL query is too long for the server.

## Specifications

  - python-gitlab version: v3.7.0
  - API version you are using (v3/v4): v4
  - Gitlab server version (or gitlab.com): all



Discussion:

Resolving patch:
diff --git a/gitlab/client.py b/gitlab/client.py
--- a/gitlab/client.py
+++ b/gitlab/client.py
@@ -20,6 +20,7 @@
 import re
 import time
 from typing import Any, cast, Dict, List, Optional, Tuple, TYPE_CHECKING, Union
+from urllib import parse
 
 import requests
 import requests.utils
@@ -677,11 +678,15 @@ def http_request(
             GitlabHttpError: When the return code is not 2xx
         """
         query_data = query_data or {}
-        url = self._build_url(path)
+        raw_url = self._build_url(path)
 
-        params: Dict[str, Any] = {}
+        # parse user-provided URL params to ensure we don't add our own duplicates
+        parsed = parse.urlparse(raw_url)
+        params = parse.parse_qs(parsed.query)
         utils.copy_dict(src=query_data, dest=params)
 
+        url = raw_url.replace(parsed.query, "").strip("?")
+
         # Deal with kwargs: by default a user uses kwargs to send data to the
         # gitlab server, but this generates problems (python keyword conflicts
         # and python-gitlab/gitlab conflicts).


Test patch:
diff --git a/tests/unit/test_gitlab_http_methods.py b/tests/unit/test_gitlab_http_methods.py
--- a/tests/unit/test_gitlab_http_methods.py
+++ b/tests/unit/test_gitlab_http_methods.py
@@ -36,6 +36,24 @@ def test_http_request(gl):
     assert responses.assert_call_count(url, 1) is True
 
 
+@responses.activate
+def test_http_request_with_url_encoded_kwargs_does_not_duplicate_params(gl):
+    url = "http://localhost/api/v4/projects?topics%5B%5D=python"
+    responses.add(
+        method=responses.GET,
+        url=url,
+        json=[{"name": "project1"}],
+        status=200,
+        match=[responses.matchers.query_param_matcher({"topics[]": "python"})],
+    )
+
+    kwargs = {"topics[]": "python"}
+    http_r = gl.http_request("get", "/projects?topics%5B%5D=python", **kwargs)
+    http_r.json()
+    assert http_r.status_code == 200
+    assert responses.assert_call_count(url, 1)
+
+
 @responses.activate
 def test_http_request_404(gl):
     url = "http://localhost/api/v4/not_there"