Repository: ansible/ansible

Issue:
digital_ocean_block_storage stopped working
<!---
Verify first that your issue/request is not already reported on GitHub.
THIS FORM WILL BE READ BY A MACHINE, COMPLETE ALL SECTIONS AS DESCRIBED.
Also test if the latest release, and devel branch are affected too.
ALWAYS add information AFTER (OUTSIDE) these HTML comments.
Otherwise, it may end up being automatically closed by our bot. -->

##### ISSUE TYPE
<!--- Pick one below and delete the rest -->
 - Bug Report

##### COMPONENT NAME
<!--- Insert, BELOW THIS COMMENT, the name of the module, plugin, task or feature.
Do not include extra details here, e.g. "vyos_command" not "the network module vyos_command" or the full path-->
digital_ocean_block_storage

##### ANSIBLE VERSION
<!--- Paste, BELOW THIS COMMENT, verbatim output from "ansible --version" between quotes below -->
```
ansible 2.5.2
  config file = /Users/carlos/.ansible.cfg
  configured module search path = [u'/Users/carlos/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.15 (default, May  1 2018, 16:44:08) [GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)]
```

##### CONFIGURATION
<!--- If using Ansible 2.4 or above, paste, BELOW THIS COMMENT, the results of "ansible-config dump --only-changed"
Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).-->
ALLOW_WORLD_READABLE_TMPFILES(/Users/carlos/.ansible.cfg) = True
ANSIBLE_SSH_ARGS(/Users/carlos/.ansible.cfg) = -o ForwardAgent=yes -o ControlMaster=auto -o ControlPersist=60s
DEFAULT_ROLES_PATH(env: ANSIBLE_ROLES_PATH) = [u'/Users/carlos/playbooks/roles']
DEFAULT_VAULT_PASSWORD_FILE(env: ANSIBLE_VAULT_PASSWORD_FILE) = /Users/carlos/.vault.txt
HOST_KEY_CHECKING(/Users/carlos/.ansible.cfg) = False
RETRY_FILES_ENABLED(/Users/carlos/.ansible.cfg) = False

##### OS / ENVIRONMENT
<!--- Mention, BELOW THIS COMMENT, the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.-->
Running Ansible on MacOS High Sierra to manage Ubuntu 16.04 
##### SUMMARY
<!--- Explain the problem briefly -->
When I try to create a DigitalOcean Block store using the digital_ocean_block_storage module I get this error
```
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: KeyError: 'message'
failed: [localhost] (item={u'owner': u'ubuntu', u'fstype': u'xfs', u'mount_name': u'backup', u'name': u'archive-staging-backup3', u'size': 100}) => {"changed": false, "item": {"fstype": "xfs", "mount_name": "backup", "name": "archive-staging-backup3", "owner": "ubuntu", "size": 100}, "msg": "Unable to load message"}
```
##### STEPS TO REPRODUCE
<!--- For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used. -->

<!--- Paste example playbooks or commands between quotes below -->
```yaml
- name: Create block storage
  digital_ocean_block_storage:
    state: present
    command: create
    region: "{{ do_region }}"
    block_size: "{{ item.size }}"
    volume_name: "{{ item.name }}"
  with_items: "{{ do_blocks }}"
  register: do_blocks_created

- name: Attach block storage
  digital_ocean_block_storage:
    state: present
    command: attach
    region: "{{ do_region }}"
    volume_name: "{{ item.name }}"
    droplet_id: "{{ do.droplet.id }}"
  with_items: "{{ do_blocks }}"
  when: do_blocks_created is defined
```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
It should create the volume and register do_blocks_created then in the next step it should attach the volume to the droplet.
##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->
So the volume does get created but the error below is thrown. If you re-run the play at this point it doesn't recognize that the volume already exists and throws "failed to create volume".
<!--- Paste verbatim command output between quotes below -->
```
The full traceback is:
Traceback (most recent call last):
  File "/var/folders/p5/m7r6p0795rg49s603ry355780000gn/T/ansible_cFlE1K/ansible_module_digital_ocean_block_storage.py", line 285, in main
    handle_request(module)
  File "/var/folders/p5/m7r6p0795rg49s603ry355780000gn/T/ansible_cFlE1K/ansible_module_digital_ocean_block_storage.py", line 259, in handle_request
    block_storage.create_block_storage()
  File "/var/folders/p5/m7r6p0795rg49s603ry355780000gn/T/ansible_cFlE1K/ansible_module_digital_ocean_block_storage.py", line 213, in create_block_storage
    raise DOBlockStorageException(json['message'])
KeyError: 'message'

failed: [localhost] (item={u'owner': u'ubuntu', u'fstype': u'xfs', u'mount_name': u'backup', u'name': u'archive-staging-backup3', u'size': 100}) => {
    "changed": false,
    "invocation": {
        "module_args": {
            "api_token": null,
            "block_size": 100,
            "command": "create",
            "description": null,
            "droplet_id": null,
            "region": "sfo2",
            "snapshot_id": null,
            "state": "present",
            "timeout": 10,
            "volume_name": "archive-staging-backup3"
        }
    },
    "item": {
        "fstype": "xfs",
        "mount_name": "backup",
        "name": "archive-staging-backup3",
        "owner": "ubuntu",
        "size": 100
    },
    "msg": "Unable to load message"
}
```


Discussion:
Files identified in the description:
* [lib/ansible/modules/cloud/digital_ocean/digital_ocean_block_storage.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/digital_ocean/digital_ocean_block_storage.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @BondAnthony @harneksidhu
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
@CarlosEspejo thank you I’ll look into the issue. 
@BondAnthony Let me know if its broken because DO changed something in their API. I can send them a support ticket and ask them to pass it on to their engineering team.
@CarlosEspejo you would be correct about the API changing. I've already submitted a ticket to Digital Ocean in regards to the sudden change in the API. Changing line:200 in the `digital_ocean_block_storage` module from 201 to 200 will fix the issue. Waiting on DO before I make any changes within this module.
@CarlosEspejo Digital Ocean said the issue has been fixed. Could you please test?
It appears that 201 is now returned, but now:

409 Conflict returns:
```json
{
    "message": "failed to create volume",
    "id": "conflict",
    "request_id": "REDACTED"
}
```
instead of 
```json
{
    "message": "failed to create volume",
    "id": "already_exists",
    "request_id": "REDACTED"
}
```

causing an error when the state of the volume is already present :(

This might be best fixed by just deferring to the status instead of checking the value of 'id'

@BondAnthony I can confirm that creating a new volume worked for me but as @ftlio pointed it out that when you run the playbook again it throws an error.
Thank you guys, I'll get this updated later this evening.
@ftlio could you please validate again. I'm currently getting this response from the API.

```
{
    "id": "already_exists",
    "message": "already exists",
    "request_id": "f57c27d8-b292ec089"
}
```
@BondAnthony I am still seeing "conflict"

It may be data center specific.  I tested against nyc1.
Alright now I'm a little confused, when I hit the endpoint directly through postman I'm getting the expected `already_exists` id. However, when I'm executing the `digital_ocean_block_storage` module directly on my workstation the response coming back from `open_url` has an id of `conflict`. I need to continue to look into this further because removing the id check could result in a false state for the user.
Is the API token you're using the same with both attempts?

I ran it both with Postman and the module and received the id='conflict', but there may be other headers set in Postman producing a difference besides the token too maybe?


Sorry for the delay, yes the same API token is being used. The only application responding with the "already_exists" id is Postman. I've opened another ticket with Digital Ocean to confirm this change in the API or provide more information on why this has occurred. I've looked through the request headers and nothing is standing out.
**Update:** I've been working with Digital Ocean to trace a number of calls. They've identified my calls are going to two different backend systems. This pointed out that they have two different proxy configurations. 

When sending a request to "https://api.digitalocean.com/v2/volumes" the request is routed to the Volumes subsystem which responds with a 409 status code and id of "conflict". When the request is sent to "https://api.digitalocean.com/v2/volumes/" the request is routed to a different backend system that is responding with a status code of 409 and id of "already_exists".  I'm waiting on confirmation if this will be fixed and which direction they will be heading. Based on the direction chosen I will update the module if required.

Resolving patch:
diff --git a/lib/ansible/modules/cloud/digital_ocean/digital_ocean_block_storage.py b/lib/ansible/modules/cloud/digital_ocean/digital_ocean_block_storage.py
--- a/lib/ansible/modules/cloud/digital_ocean/digital_ocean_block_storage.py
+++ b/lib/ansible/modules/cloud/digital_ocean/digital_ocean_block_storage.py
@@ -199,7 +199,7 @@ def create_block_storage(self):
         json = response.json
         if status == 201:
             self.module.exit_json(changed=True, id=json['volume']['id'])
-        elif status == 409 and json['id'] == 'already_exists':
+        elif status == 409 and json['id'] == 'conflict':
             self.module.exit_json(changed=False)
         else:
             raise DOBlockStorageException(json['message'])


Test patch: