Repository: conan-io/conan

Issue:
[bug] Conan does not provide `Authorization` cookie when accessing `/v1/ping` when adding user 
This means that Conan cannot use an Artifactory which is behind a proxy, even if BasicAuth is allowed through the proxy. Also, the log says it is providing an Authorization header, which is misleading.
As we have added a user for this repository it seems logical that the request adds the Authorisation header for that user, and doesn't rely on certain locations being accessible without authorization.

As mentioned in #6066:
Steps:
1) Run local http server on machine using proxy:
`python -m SimpleHTTPServer`
2) Run tcpdump on local machine and use it r print out headers:
`sudo tcpdump -vvv -A -i lo0  port 8000`
3) Run a conan user command and check the headers in the output of `tcpdump` (from 2).
`conan user -p test -r local-repo user`

Doing this we see that `tcpdump` receives these headers:
```
Host: localhost:8000
User-Agent: Conan/1.16.1 (Python 3.7.0) python-requests/2.22.0
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
```
Note there is not `Authorisation` header. Here is the trace log for the came command:
```
{"_action": "COMMAND", "name": "authenticate", "parameters": {"password": "**********", "remote_name": "local-repo"}, "time": 1576150601.185415}
{"_action": "COMMAND", "name": "get_remote_by_name", "parameters": {}, "time": 1576150601.1887949}
{"_action": "REST_API_CALL", "duration": 0.008514165878295898, "headers": {"Authorization": "**********", "User-Agent": "Conan/1.16.1 (Python 3.7.0) python-requests/2.22.0", "X-Client-Anonymous-Id": "**********"}, "method": "GET", "time": 1576150601.1996539, "url": "http://localhost:8000/v1/ping"}
{"_action": "EXCEPTION", "class": "NotFoundException", "message": "Not implemented endpoint. [Remote: local-repo]", "time": 1576150601.2005131}
```
It claims to be adding the `Authorization` header.

To check the method we also ran a curl request (providing a user):
`curl -L -u test:user http://localhost:8000`

The curl request shows it actually sends the `Authorization` header:
```
Host: localhost:8000
Authorization: Basic dGVzdDp1c2Vy
User-Agent: curl/7.54.0
Accept: */*
```

I have tested this using 1.21.0 so this is a bug in the current implementation. This test seems to be testing for this behaviour:
https://github.com/conan-io/conan/blob/81102434601f93b8699fe9f9a482c369ca026c1a/conans/test/functional/remote/auth_bearer_test.py#L69


Discussion:
It is true that the traces are showing some ``"Authorization" = ********`` traces, but the problem lies in the masking of the traces not the request. This request by design never intended to submit basic auth.

I have submitted PR #6245 to do some cleaning on the Rest layer, and also to fix these erroneous traces. We need to investigate further what could be done without breaking other parts (other servers) and other Conan users. 
Could you try requesting without an auth header, then if you get and error
e.g. 403 you could try adding the auth header?
Then the current behaviour is preserved, and the new one only used when
needed?

On Tue, 17 Dec 2019, 15:02 James, <notifications@github.com> wrote:

> It is true that the traces are showing some "Authorization" = ********
> traces, but the problem lies in the masking of the traces not the request.
> This request by design never intended to submit basic auth.
>
> I have submitted PR #6245 <https://github.com/conan-io/conan/pull/6245>
> to do some cleaning on the Rest layer, and also to fix these erroneous
> traces. We need to investigate further what could be done without breaking
> other parts (other servers) and other Conan users.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/conan-io/conan/issues/6240?email_source=notifications&email_token=ABB54FH5BG5JWEXYJ66MTTLQZDSX3A5CNFSM4J3JIW62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHCVBCI#issuecomment-566579337>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABB54FDKUU66V72NTXI737TQZDSX3ANCNFSM4J3JIW6Q>
> .
>

> Could you try requesting without an auth header, then if you get and error
e.g. 403 you could try adding the auth header?
Then the current behaviour is preserved, and the new one only used when
needed?

Yes, this is one of the lines that we should investigate, in theory it should behave like that, but we need to check better.

Resolving patch:
diff --git a/conans/client/rest/rest_client.py b/conans/client/rest/rest_client.py
--- a/conans/client/rest/rest_client.py
+++ b/conans/client/rest/rest_client.py
@@ -46,12 +46,12 @@ def __init__(self, remote, token, refresh_token, custom_headers, output, request
         # This dict is shared for all the instances of RestApiClient
         self._cached_capabilities = cached_capabilities
 
-    def _capable(self, capability):
+    def _capable(self, capability, user=None, password=None):
         capabilities = self._cached_capabilities.get(self._remote_url)
         if capabilities is None:
             tmp = RestV1Methods(self._remote_url, self._token, self._custom_headers, self._output,
                                 self._requester, self._verify_ssl, self._artifacts_properties)
-            capabilities = tmp.server_capabilities()
+            capabilities = tmp.server_capabilities(user, password)
             self._cached_capabilities[self._remote_url] = capabilities
             logger.debug("REST: Cached capabilities for the remote: %s" % capabilities)
             if not self._revisions_enabled and ONLY_V2 in capabilities:
@@ -116,7 +116,7 @@ def authenticate(self, user, password):
         else:
             try:
                 # Check capabilities can raise also 401 until the new Artifactory is released
-                oauth_capable = self._capable(OAUTH_TOKEN)
+                oauth_capable = self._capable(OAUTH_TOKEN, user, password)
             except AuthenticationException:
                 oauth_capable = False
 
diff --git a/conans/client/rest/rest_client_common.py b/conans/client/rest/rest_client_common.py
--- a/conans/client/rest/rest_client_common.py
+++ b/conans/client/rest/rest_client_common.py
@@ -149,13 +149,16 @@ def check_credentials(self):
                                  verify=self.verify_ssl)
         return ret
 
-    def server_capabilities(self):
+    def server_capabilities(self, user=None, password=None):
         """Get information about the server: status, version, type and capabilities"""
         url = self.router.ping()
         logger.debug("REST: ping: %s" % url)
-
-        ret = self.requester.get(url, auth=self.auth, headers=self.custom_headers,
-                                 verify=self.verify_ssl)
+        if user and password:
+            # This can happen in "conan user" cmd. Instead of empty token, use HttpBasic
+            auth = HTTPBasicAuth(user, password)
+        else:
+            auth = self.auth
+        ret = self.requester.get(url, auth=auth, headers=self.custom_headers, verify=self.verify_ssl)
 
         server_capabilities = ret.headers.get('X-Conan-Server-Capabilities', "")
         if not server_capabilities and not ret.ok:


Test patch:
diff --git a/conans/test/functional/remote/auth_test.py b/conans/test/functional/remote/auth_test.py
--- a/conans/test/functional/remote/auth_test.py
+++ b/conans/test/functional/remote/auth_test.py
@@ -32,7 +32,7 @@ def setUp(self):
                                       [(str(self.ref), "pepe,nacho@gmail.com")],  # write permissions
                                       users={"lasote": "mypass",
                                              "pepe": "pepepass",
-                                             "nacho@gmail.com": "nachopass",})  # exported users and passwords
+                                             "nacho@gmail.com": "nachopass"})  # exported creds
         self.servers["default"] = self.test_server
 
     def retries_test(self):
@@ -134,22 +134,37 @@ class RequesterMock(object):
             def __init__(self, *args, **kwargs):
                 pass
 
-            def get(self, url, *args, **kwargs):
+            @staticmethod
+            def get(url, **kwargs):
+                resp_basic_auth = Response()
+                resp_basic_auth.status_code = 200
                 if "authenticate" in url:
-                    resp_basic_auth = Response()
+                    if kwargs["auth"].password != "PASSWORD!":
+                        raise Exception("Bad password")
                     resp_basic_auth._content = b"TOKEN"
-                    resp_basic_auth.status_code = 200
                     resp_basic_auth.headers = {"Content-Type": "text/plain"}
-                    return resp_basic_auth
-
-                if "ping" in url and not kwargs["auth"].token:
-                    raise AuthenticationException(
-                        "I'm an Artifactory without anonymous access that "
-                        "requires authentication for the ping endpoint and "
-                        "I don't return the capabilities")
-                raise Exception("Shouldn't be more remote calls")
-
-        self.client = TestClient(requester_class=RequesterMock, default_server_user=True)
-        self.client.run("user user -p password -r default")
+                elif "ping" in url:
+                    token = getattr(kwargs["auth"], "token", None)
+                    password = getattr(kwargs["auth"], "password", None)
+                    if token and token != "TOKEN":
+                        raise Exception("Bad JWT Token")
+                    if not token and not password:
+                        raise AuthenticationException(
+                            "I'm an Artifactory without anonymous access that "
+                            "requires authentication for the ping endpoint and "
+                            "I don't return the capabilities")
+                elif "search" in url:
+                    if kwargs["auth"].token != "TOKEN":
+                        raise Exception("Bad JWT Token")
+                    resp_basic_auth._content = b'{"results": []}'
+                    resp_basic_auth.headers = {"Content-Type": "application/json"}
+                else:
+                    raise Exception("Shouldn't be more remote calls")
+                return resp_basic_auth
+
+        client = TestClient(requester_class=RequesterMock, default_server_user=True)
+        client.run("user user -p PASSWORD! -r=default")
         self.assertIn("Changed user of remote 'default' from 'None' (anonymous) to 'user'",
-                      self.client.out)
+                      client.out)
+        client.run("search pkg -r=default")
+        self.assertIn("There are no packages matching the 'pkg' pattern", client.out)