Repository: ansible-collections/community.general

Issue:
maven_artifact can't fetch from AWS CodeArtifact repositories
<!--- Verify first that your issue is not already reported on GitHub -->
<!--- Also test if the latest release and devel branch are affected too -->
<!--- Complete *all* sections as described, this form is processed automatically -->

##### SUMMARY
[AWS CodeArtifact](https://aws.amazon.com/codeartifact/) is a service from AWS providing managed package repositories (npmjs, pypi, maven/gradle). We'd like to use it to store our Java JAR artifacts published by Gradle, and download them onto our app servers with ansible's `maven_artifact` module.

Using the [AWS instructions](https://docs.aws.amazon.com/codeartifact/latest/ug/maven-mvn.html), authentication to a CodeArtifact repository with Maven is done by first obtaining a time-limited token in the form of a JWT (I've included an example as [a gist](https://gist.github.com/iandelahorne/1bf8d0663f0efb62a0fa6727bbda74e9)) and then using that as the password.

Despite working with Maven and Gradle, this doesn't work with the `maven_artifact` module due to something weird happening when trying to download. Looking at the URLs failing and supplying the tokens manually, I can't understand why this doesn't work.

##### ISSUE TYPE
- Bug Report

##### COMPONENT NAME
maven_artifact

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes -->
```paste below
ansible 2.10.2
  config file = /Users/ian/src/ansible-test/ansible.cfg
  configured module search path = ['/Users/ian/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/ian/src/ansible-test/venv/lib/python3.6/site-packages/ansible
  executable location = /Users/ian/src/ansible-test/venv/bin/ansible
  python version = 3.6.8 (default, Jan  8 2020, 12:58:45) [GCC 4.2.1 Compatible Apple LLVM 11.0.0 (clang-1100.0.33.16)]
```

##### CONFIGURATION
<!--- Paste verbatim output from "ansible-config dump --only-changed" between quotes -->
```
DEFAULT_STDOUT_CALLBACK(/Users/ian/src/ansible-test/ansible.cfg) = yaml
```

##### OS / ENVIRONMENT
This happens both on OSX and Linux (Ubuntu 18.04), ansible 2.9.4 and 2.10.1 with python 3.6.8. The example below is from 2.10.1 with `community.general` installed from 
```
$ ansible-galaxy collection install community.general
```


##### STEPS TO REPRODUCE

To set this up in CodeArtifact, we need to do the following:
1. Create a CodeArtifact domain
1. Create a CodeArtifact repository, maven-central-store, as a maven-central upstream
1. Connect the upstream to maven-central
1. Create a maventest repository
1. Connect the maventest repository to the , maven-central-store 

I've done this all in us-west-2:
```
export AWS_DEFAULT_REGION=us-west-2 
# Create the domain
$ aws codeartifact create-domain --domain maventest

# Create the upstream repository
$ aws codeartifact create-repository --domain maventest --repository maven-central-store

# Connect it the repository to public maven-central
$ aws codeartifact associate-external-connection --domain maventest --repository maven-central-store --external-connection public:maven-central

# Create the maventest repository
aws codeartifact create-repository --domain maventest --repository maventest --upstreams repositoryName=maven-central-store
```

The instructions on the repository instruct us to use `aws` as the username, and the output from `aws codeartifact get-authorization-token` as the password to use it as a maven 
<!--- Paste example playbooks or commands between quotes below -->
```yaml
---
- name: Test maven_artifact with codeartifact
  hosts: localhost
  vars:
    codeartifact:
      domain: maventest
      region: us-west-2
      repository: maventest
    artifact:
      id: commons-io
      group: commons-io
      version: 2.8.0
  tasks:
    - name: Fetch codeartifact token
      command: "aws codeartifact get-authorization-token --region {{ codeartifact.region }} --domain {{ codeartifact.domain }} --query authorizationToken --output text"
      register: codeartifact_token
      no_log: true
      
    - name: Fetch codeartifact endpoint
      command: "aws codeartifact get-repository-endpoint --region {{ codeartifact.region }} --domain {{ codeartifact.domain }} --repository {{ codeartifact.repository }} --format maven --query repositoryEndpoint --output text"
      register: codeartifact_endpoint
    
    - name: Download commons.io JAR
      community.general.maven_artifact:
        artifact_id: "{{ artifact.id }}"
        dest: /tmp/jar.jar
        group_id: "{{ artifact.group }}"
        repository_url: "{{ codeartifact_endpoint.stdout | trim }}"
        version: "{{ artifact.version }}"
        username: aws
        password: "{{ codeartifact_token.stdout | trim }}"
```

<!--- HINT: You can paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- Describe what you expected to happen when running the steps above -->

Expected result is that `commons-io:commons-io:2.8.0` is downloaded to `/tmp/codeartifact.jar`

I've also checked the above setup using Maven, by 
1. nuking `~/.m2/` directory
1. Creating a `~/.m2/settings.xml` from the configuration in the AWS console
1. Running `mvn` to fetch the JAR:

```
$ mvn dependency:get -Ddest=/tmp -DgroupId=commons-io -DartifactId=commons-io -Dversion=2.8.0 -Dtransitive=false
```
and I can see that `/tmp/commons-io-2.8.0.jar` is downloaded:

```
$ unzip -p /tmp/commons-io-2.8.0.jar META-INF/MANIFEST.MF | grep Bundle
Bundle-Description: The Apache Commons IO library contains utility cla
Bundle-License: https://www.apache.org/licenses/LICENSE-2.0.txt
Bundle-SymbolicName: org.apache.commons.commons-io
Bundle-ManifestVersion: 2
Bundle-DocURL: https://commons.apache.org/proper/commons-io/
Bundle-Vendor: The Apache Software Foundation
Bundle-Name: Apache Commons IO
Bundle-Version: 2.8.0
Created-By: Apache Maven Bundle Plugin
```

##### ACTUAL RESULTS
<!--- Describe what actually happened. If possible run with extra verbosity (-vvvv) -->
Result from `ansible-playbook -vvvv playbooks/maventest.yml` (the above playbook)

https://gist.github.com/iandelahorne/9f642e10f2cc4c87c14009ab5972f101

I found other github issues referencing the `AbstractDigestAuthHandler does not support the following scheme: ''Bearer''` error, where they set the headers explicitly. Changing the second step to:
```
    - name: Download commons.io JAR
      community.general.maven_artifact:
        artifact_id: "{{ artifact.id }}"
        dest: /tmp/codeartifact.jar
        group_id: "{{ artifact.group }}"
        repository_url: "{{ codeartifact_endpoint.stdout | trim }}"
        version: "{{ artifact.version }}"
        headers:
          Authorization: "Bearer {{ codeartifact_token.stdout | trim }}"
```

This results in the following output:

https://gist.github.com/iandelahorne/b94e8f7b863dff8220b8ae08f439e9ae

Running Curl with the bearer token in the output and the URL in the error totally works:
```
$ curl -O -L -H "Authorization: Bearer REDACTED " https://maventest-REDACTED.d.codeartifact.us-west-2.amazonaws.com/maven/maventest/commons-io/commons-io/2.8.0/commons-io-2.8.0.jar
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  278k  100  278k    0     0   337k      0 --:--:-- --:--:-- --:--:--  337k
$ unzip -p commons-io-2.8.0.jar META-INF/MANIFEST.MF | grep Bundle
Bundle-Description: The Apache Commons IO library contains utility cla
Bundle-License: https://www.apache.org/licenses/LICENSE-2.0.txt
Bundle-SymbolicName: org.apache.commons.commons-io
Bundle-ManifestVersion: 2
Bundle-DocURL: https://commons.apache.org/proper/commons-io/
Bundle-Vendor: The Apache Software Foundation
Bundle-Name: Apache Commons IO
Bundle-Version: 2.8.0
Created-By: Apache Maven Bundle Plugin
```




Discussion:
This is fairly meaty and depends on AWS. I've redacted account numbers and tokens here, but I can provide these over email if needed - email is ian at patreon dot com 
cc @tumbl3w33d @turb
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
It might be helpful to know to the maintainers how `"{{ codeartifact_token.stdout | trim }}"` looks like (not the exact value, but the general form).
@felixfontein Absolutely. It's a fairly big JWT (1519 characters base64 encoded). I generated one and then redacted the sections past the first section (up to the first `.`):

```
eyJ2ZXIiOjEsImlzdSI6MTYwMzY0NDQ0MCwiZW5jIjoiQTEyOEdDTSIsInRhZyI6ImJLMm4xaW9xdldkd043cEZPb1BZekEiLCJleHAiOjE2MDM2ODc2NDAsImFsZyI6IkExMjhHQ01LVyIsIml2IjoiNzFURE51NU9MeXowLXo5aCJ9.XXXXXXXXXXXXXXXXXXXXXX.YYYYYYYYYYYYYYYY.ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ.QQQQQQQQQQQQQQQQQQQQQQ
```
I also tried using `force_basic_auth: true` and it failed the same way as specifying `headers`:
```
  msg: 'Failed to download artifact commons-io:commons-io:2.8.0 because of HTTP Error 400: Bad Requestfor URL https://maventest-REDACTED.d.codeartifact.us-west-2.amazonaws.com/maven/maventest/commons-io/commons-io/2.8.0/commons-io-2.8.0.jar'
```

```yaml
    - name: Download commons.io JAR
      community.general.maven_artifact:
        artifact_id: "{{ artifact.id }}"
        dest: /tmp/codeartifact.jar
        group_id: "{{ artifact.group }}"
        repository_url: "{{ codeartifact_endpoint.stdout | trim }}"
        version: "{{ artifact.version }}"
        force_basic_auth: true
        username: aws
        password: "{{ codeartifact_token.stdout | trim }}"

```
Ok. I guess the problem in your first example is that the code (Python's urllib in this case) only supports HTTP Basic Auth (and maybe HTTP Digest Auth), but the server only accepts Bearer (token) Auth. An application can explicitly support Bearer Auth by adding the header, like you did in your second example. I guess it should also be possible to add a module option for providing the token, but it would just be syntactical sugar over providing the `Authorization: Bearer <token>` header.

Apparently the module did send a request that the server didn't expect though, since you got the 400 Bad Request status code. Maybe AWS CodeArtifact expects something differently? The code that does the downloading (and fails) doesn't seem to do anything special: https://github.com/ansible-collections/community.general/blob/9aeac265838a3efeff1c1ab221fe0f3e767b61b2/plugins/modules/packaging/language/maven_artifact.py#L463-L486

I hope this helps someone debugging this :)
> Ok. I guess the problem in your first example is that the code (Python's urllib in this case) only supports HTTP Basic Auth (and maybe HTTP Digest Auth), but the server only accepts Bearer (token) Auth. An application can explicitly support Bearer Auth by adding the header, like you did in your second example. I guess it should also be possible to add a module option for providing the token, but it would just be syntactical sugar over providing the `Authorization: Bearer <token>` header.

That's not strictly true - the server will gladly accept basic authentication. I can curl the 400'd url output at the end using `aws` as the username and the bearer token as the password - https://gist.github.com/iandelahorne/21d4284d35f74a88d1c9111752af6f21

I'm not 100% sure on how I can dig into and debug what's happening in the module - is there a guide somewhere on how to do printf debugging or the like? if so, I could start adding some tracing to figure out what is going on that's making this fail
At least the server doesn't advertise HTTP Basic Auth. I think there is a force_basic_auth flag somewhere, though I don't know whether it's available in the maven_artifact. Maybe exposing that option would solve your problem.
> At least the server doesn't advertise HTTP Basic Auth. I think there is a force_basic_auth flag somewhere, though I don't know whether it's available in the maven_artifact. Maybe exposing that option would solve your problem.

I tried `force_basic_auth` in https://github.com/ansible-collections/community.general/issues/1171#issuecomment-716180376 - fails in the same way 
Ah, sorry. Forgot about that. It looks like authentication works in that case (same as with the header), but for some other reason the server is rejecting the request.
> It looks like authentication works in that case (same as with the header), but for some other reason the server is rejecting the request.

Yep, which is why I'd like to be able to dump the request/response objects somehow but I haven't been able to figure out how to even start printf debugging with a module. AWS is usually very good at saying why a request failed in their error headers/output. 
You do the following to get some output
``` python
from ansible.utils.display import Display

DISPLAY = Display()

display.v('foo')
display.vv('foo')
display.vvv('foo')
display.vvvv('foo')
display.warning('oh my god')
```
@tumbl3w33d that only works in plugins, not in modules.

The best way to do printf-style debugging in modules is using q: https://pypi.org/project/q/
I'm on the AWS CodeArtifact team and I can give some insight into what I believe is going on. When a client requests a Maven artifact, CodeArtifact currently redirects the client to a pre-signed S3 URL.

The request to CodeArtifact must include an Authorization header (Bearer or Basic) and CodeArtifact returns a 307 with the pre-signed S3 URL. When S3 receives a pre-signed request that also includes an Authorization header, S3 returns a 400. So in this case, it is S3 returning the 400, not the CodeArtifact service (I've checked our service logs). 

The problem is that some clients will include the Authorization header from the original request when making the redirect request, and I believe that is what is happening here. HTTP RFCs aren't exactly clear on what headers clients should include when following a redirect, but from a security perspective many clients (e.g. maven, gradle, curl, npm) do not include the Authorization in the redirect request (at least not when redirecting to a different DNS name).

> At least the server doesn't advertise HTTP Basic Auth.

Yes, the response from the CodeArtifact service includes two `www-authenticate` headers, one of which advertises Basic auth (here you can test using a 12-digit all-zeros accountID in place of the redacted one) 

```
curl -i https://maventest-000000000000.d.codeartifact.us-west-2.amazonaws.com/maven/maventest/commons-io/commons-io/2.8.0/commons-io-2.8.0.jar
HTTP/2 401
date: Mon, 02 Nov 2020 17:06:01 GMT
content-type: text/plain
content-length: 98
server: Apache-Coyote/1.1
x-amzn-trace-id: Root=1-5fa03c79-665ae09e71dbe6e27ea093ba;
www-authenticate: Bearer realm="maventest/maventest"
www-authenticate: Basic realm="maventest/maventest"
x-amzn-requestid: 2f1f7305-3e3b-4f3c-9226-ce64fe5b3a92

Unauthenticated: request did not include an Authorization header. Please provide your credentials.
```


Ok, so if the `Authorization` header is explicitly specified with `headers:`, it will also be passed after the redirect. I guess same for when `force_basic_auth: true`, it will simply always include that header.

For regular basic auth, I guess that urllib simply ignores the second `www-authenticate` header and thus ends up with `AbstractDigestAuthHandler does not support the following scheme: 'Bearer'`. Looking at the code, this seems to be the case: https://github.com/python/cpython/blob/3.9/Lib/urllib/request.py#L1097-L1115

So I guess this is a bug in Python's urllib (resp. the classical problem "how to handle repeated headers"...).
Is there a way to get ansible to use `AbstractBasicAuthHandler` for these requests instead? It appears to handle multiple headers:
https://github.com/python/cpython/blob/3.9/Lib/urllib/request.py#L989-L999

Another option that might work is if the Ansible `fetch_url` method were to call `open_url` with `Authorization` added to the `unredirected_headers` parameter. See https://github.com/ansible/ansible/blob/devel/lib/ansible/module_utils/urls.py#L1530

I haven't confirmed what `unredirected_headers` does, but based on the parameter name I expect that it specifies a list of headers that should not be included when following redirects.
The module currently uses `fetch_url`, which does not offer this. You'd have to use more lower-level functions, which means you have to do a lot more error handling.

The Ansible code which does authentication is this: https://github.com/ansible/ansible/blob/devel/lib/ansible/module_utils/urls.py#L1346-L1359 It does use `HTTPBasicAuthHandler`, which uses `AbstractBasicAuthHandler`, but it also adds `HTTPDigestAuthHandler`. I'm not familiar with how urllib exactly works, but I guess having both handlers is what causes this error - and at the same moment is something that cannot be avoided without breaking other modules or this module in other cases.
@iandelahorne @mmarston @felixfontein I was combing through the issues' backlog and found #215 which seems similar, but not quite the same. As I was looking into the code, I found out that apparently `force_basic_auth` is **not used** at all. I haven't seen it being used or passed around in any form. I have just submitted this PR #1979 to fix that, but not entirely sure it helps with this ticket. @iandelahorne could you please validate that? TIA
@russoz force_basic_auth is used by https://github.com/ansible/ansible/blob/devel/lib/ansible/module_utils/urls.py#L1741
Any updates on this issue?
It would seem that the `force_basic_auth` is being used in the `maven_artifact`, which allows us to do the login correctly, but then when ansible tries to get the artifact, it seems it's adding the bearer token into the redirect, which results in aws responding with 400.
Adding `Authorization` to `unredirected_headers` (`maven_artifact` module does not expose `unredirected_headers`) will solve the problem. Technically it's a bug in Python library:
- https://github.com/python/cpython/issues/77842
- https://github.com/python/cpython/pull/11292

Resolving patch:
diff --git a/plugins/modules/packaging/language/maven_artifact.py b/plugins/modules/packaging/language/maven_artifact.py
--- a/plugins/modules/packaging/language/maven_artifact.py
+++ b/plugins/modules/packaging/language/maven_artifact.py
@@ -150,6 +150,15 @@
         default: 'md5'
         choices: ['md5', 'sha1']
         version_added: 3.2.0
+    unredirected_headers:
+        type: list
+        elements: str
+        version_added: 5.2.0
+        description:
+            - A list of headers that should not be included in the redirection. This headers are sent to the fetch_url C(fetch_url) function.
+            - On ansible-core version 2.12 or later, the default of this option is C([Authorization, Cookie]).
+            - Useful if the redirection URL does not need to have sensitive headers in the request.
+            - Requires ansible-core version 2.12 or later.
     directory_mode:
         type: str
         description:
@@ -230,6 +239,7 @@
 import traceback
 import re
 
+from ansible_collections.community.general.plugins.module_utils.version import LooseVersion
 from ansible.module_utils.ansible_release import __version__ as ansible_version
 from re import match
 
@@ -509,7 +519,18 @@ def _request(self, url, failmsg, force=True):
         self.module.params['url_password'] = self.module.params.get('password', '')
         self.module.params['http_agent'] = self.user_agent
 
-        response, info = fetch_url(self.module, url_to_use, timeout=req_timeout, headers=self.headers)
+        kwargs = {}
+        if self.module.params['unredirected_headers']:
+            kwargs['unredirected_headers'] = self.module.params['unredirected_headers']
+
+        response, info = fetch_url(
+            self.module,
+            url_to_use,
+            timeout=req_timeout,
+            headers=self.headers,
+            **kwargs
+        )
+
         if info['status'] == 200:
             return response
         if force:
@@ -614,12 +635,20 @@ def main():
             keep_name=dict(required=False, default=False, type='bool'),
             verify_checksum=dict(required=False, default='download', choices=['never', 'download', 'change', 'always']),
             checksum_alg=dict(required=False, default='md5', choices=['md5', 'sha1']),
+            unredirected_headers=dict(type='list', elements='str', required=False),
             directory_mode=dict(type='str'),
         ),
         add_file_common_args=True,
         mutually_exclusive=([('version', 'version_by_spec')])
     )
 
+    if LooseVersion(ansible_version) < LooseVersion("2.12") and module.params['unredirected_headers']:
+        module.fail_json(msg="Unredirected Headers parameter provided, but your ansible-core version does not support it. Minimum version is 2.12")
+
+    if LooseVersion(ansible_version) >= LooseVersion("2.12") and module.params['unredirected_headers'] is None:
+        # if the user did not supply unredirected params, we use the default, ONLY on ansible core 2.12 and above
+        module.params['unredirected_headers'] = ['Authorization', 'Cookie']
+
     if not HAS_LXML_ETREE:
         module.fail_json(msg=missing_required_lib('lxml'), exception=LXML_ETREE_IMP_ERR)
 


Test patch: