Repository: ansible/ansible-modules-core

Issue:
docker_service not creating docker image
<!--- Verify first that your issue/request is not already reported in GitHub -->
##### ISSUE TYPE

<!--- Pick one below and delete the rest: -->
- Bug Report
##### COMPONENT NAME

<!--- Name of the plugin/module/task -->

docker_service 
##### ANSIBLE VERSION

<!--- Paste verbatim output from “ansible --version” between quotes below -->

```
ansible 2.3.0 (devel d09f57fb3a) last updated 2016/10/04 08:46:08 (GMT +000)
  lib/ansible/modules/core: (devel 0ee774ff15) last updated 2016/10/04 09:19:29 (GMT +000)
  lib/ansible/modules/extras: (devel 5cc72c3f06) last updated 2016/10/04 09:20:09 (GMT +000)
  config file = /etc/ansible/roles/myrole/ansible.cfg
  configured module search path = Default w/o overrides

```
##### CONFIGURATION

<!---
Mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->
##### OS / ENVIRONMENT

<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say “N/A” for anything that is not platform-specific.
-->

Ubuntu 14.04
##### SUMMARY

<!--- Explain the problem briefly -->

docker_service module doesn't create a docker image from the docker-compose file, it gives an error and crashes.  The error was: compose.service.NoSuchImageError: Image 'myproject_web' not found.  
  However, if a docker image is present, it will proceed without error.

On the other hand, what is expected from docker-compose (and also docker_service) is it will create an image if one isn't there.  Especially if the --build flag is present.    During testing, "docker-compose up" does this successfully, whether an image exists or not.   
##### STEPS TO REPRODUCE

<!---
For bugs, show exactly how to reproduce the problem.
For new features, show how the feature would be used.
-->

<!--- Paste example playbooks or commands between quotes below -->

```
- name: start with docker-compose
  docker_service:
    project_src: "{{ deploy_dir }}"
    build: yes
    state: present
```

check the current docker images:

```
docker images
```

make a backup copy of the relevant image, if desired, and then delete the image.   

```
docker tag image_name image_name.bck
docker rmi image_name
```

then run ansible-playbook.   

<!--- You can also paste gist.github.com links for larger files -->
##### EXPECTED RESULTS

<!--- What did you expect to happen when running the steps above? -->

TASK [djangoapp_docker : start with docker-compose] ****************************
changed: [appserver]

PLAY RECAP *********************************************************************
appserver                  : ok=4    changed=1    unreachable=0    failed=0
##### ACTUAL RESULTS

<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->

```
TASK [djangoapp_docker : start with docker-compose] ****************************
task path: /etc/ansible/roles/myrole/roles/djangoapp_docker/tasks/main.yml:9
Using module file /opt/github/ansible/lib/ansible/modules/core/cloud/docker/docker_service.py
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: root
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1475573103.01-30232119310425 `" && echo ansible-tmp-1475573103.01-30232119310425="` echo $HOME/.ansible/tmp/ansible-tmp-1475573103.01-30232119310425 `" ) && sleep 0'
<localhost> PUT /tmp/tmpai39S1 TO /root/.ansible/tmp/ansible-tmp-1475573103.01-30232119310425/docker_service.py
<localhost> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1475573103.01-30232119310425/ /root/.ansible/tmp/ansible-tmp-1475573103.01-30232119310425/docker_service.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python /root/.ansible/tmp/ansible-tmp-1475573103.01-30232119310425/docker_service.py; rm -rf "/root/.ansible/tmp/ansible-tmp-1475573103.01-30232119310425/" > /dev/null 2>&1 && sleep 0'
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_zG7JHD/ansible_module_docker_service.py", line 929, in <module>
    main()
  File "/tmp/ansible_zG7JHD/ansible_module_docker_service.py", line 924, in main
    result = ContainerManager(client).exec_module()
  File "/tmp/ansible_zG7JHD/ansible_module_docker_service.py", line 575, in exec_module
    result = self.cmd_up()
  File "/tmp/ansible_zG7JHD/ansible_module_docker_service.py", line 630, in cmd_up
    result.update(self.cmd_build())
  File "/tmp/ansible_zG7JHD/ansible_module_docker_service.py", line 773, in cmd_build
    image = service.image()
  File "/usr/local/lib/python2.7/dist-packages/compose/service.py", line 316, in image
    raise NoSuchImageError("Image '{}' not found".format(self.image_name))
compose.service.NoSuchImageError: Image 'myproject_web' not found

fatal: [appserver]: FAILED! => {
    "changed": false,
    "failed": true,
    "invocation": {
        "module_name": "docker_service"
    },
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_zG7JHD/ansible_module_docker_service.py\", line 929, in <module>\n    main()\n  File \"/tmp/ansible_zG7JHD/ansible_module_docker_service.py\", line 924, in main\n    result = ContainerManager(client).exec_module()\n  File \"/tmp/ansible_zG7JHD/ansible_module_docker_service.py\", line 575, in exec_module\n    result = self.cmd_up()\n  File \"/tmp/ansible_zG7JHD/ansible_module_docker_service.py\", line 630, in cmd_up\n    result.update(self.cmd_build())\n  File \"/tmp/ansible_zG7JHD/ansible_module_docker_service.py\", line 773, in cmd_build\n    image = service.image()\n  File \"/usr/local/lib/python2.7/dist-packages/compose/service.py\", line 316, in image\n    raise NoSuchImageError(\"Image '{}' not found\".format(self.image_name))\ncompose.service.NoSuchImageError: Image 'myrole_web' not found\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE"
}
        to retry, use: --limit @/etc/ansible/roles/myrole/deploy_docker.retry

```



Discussion:
<!-- @sdarwin: Thanks for filing this bug report! -->

@ansible ping, this issue is waiting for your response.
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)

<!--- boilerplate: issue_bug_new --->

needs_info

@sdarwin please share a copy of your docker-compose.yml so that we can reproduce and troubleshoot what you're seeing.

> please share a copy of your docker-compose.yml

/opt/test/docker-compose.yml -

```
version: '2'
services:
  web:
    build: .
  db:
    image: mysql
```

/opt/test/Dockerfile -

```
FROM python:3.5.2
RUN mkdir -p /usr/src/app
```

/opt/test/main.yml -

```
---
- hosts: localhost
  become: yes
  tasks:
  - name: task1
    docker_service:
      project_src: /opt/test
      build: yes
      state: present
```

check the docker image doesn't exist, or delete it -

```
docker images
docker rmi
```

run ansible -

```
ansible-playbook main.yml
```

@sdarwin I see the issue now. We're not catching the NoSuchImageError in cmd_build(). Will fix shortly.


Resolving patch:
diff --git a/cloud/docker/docker_service.py b/cloud/docker/docker_service.py
--- a/cloud/docker/docker_service.py
+++ b/cloud/docker/docker_service.py
@@ -464,7 +464,7 @@
 try:
     from compose import __version__ as compose_version
     from compose.cli.command import project_from_options
-    from compose.service import ConvergenceStrategy
+    from compose.service import ConvergenceStrategy, NoSuchImageError
     from compose.cli.main import convergence_strategy_from_opts, build_action_from_opts, image_type_from_opt
     from compose.const import DEFAULT_TIMEOUT
 except ImportError as exc:
@@ -484,7 +484,7 @@
     u'tls_verify': u'--tlsverify'
 }
 
-
+    
 class ContainerManager(DockerBaseClass):
 
     def __init__(self, client):
@@ -598,11 +598,11 @@ def _get_auth_options(self):
         return options
 
     def cmd_up(self):
-
+    
         start_deps = self.dependencies
         service_names = self.services
         detached = True
-        result = dict(changed=False, actions=dict(), ansible_facts=dict())
+        result = dict(changed=False, actions=[], ansible_facts=dict())
 
         up_options = {
             u'--no-recreate': False,
@@ -624,26 +624,29 @@ def cmd_up(self):
         self.log("convergence strategy: %s" % converge)
 
         if self.pull:
-            result.update(self.cmd_pull())
+            pull_output = self.cmd_pull()
+            result['changed'] = pull_output['changed']  
+            result['actions'] += pull_output['actions']
 
         if self.build:
-            result.update(self.cmd_build())
+            build_output = self.cmd_build()
+            result['changed'] = build_output['changed']  
+            result['actions'] += build_output['actions']
 
         for service in self.project.services:
             if not service_names or service.name in service_names:
                 plan = service.convergence_plan(strategy=converge)
                 if plan.action != 'noop':
                     result['changed'] = True
-                if self.debug or self.check_mode:
-                    if not result['actions'].get(service.name):
-                        result['actions'][service.name] = dict()
-                    result['actions'][service.name][plan.action] = []
+                    result_action = dict(service=service.name) 
+                    result_action[plan.action] = []                      
                     for container in plan.containers:
-                        result['actions'][service.name][plan.action].append(dict(
+                        result_action[plan.action].append(dict(
                             id=container.id,
                             name=container.name,
                             short_id=container.short_id,
                         ))
+                    result['actions'].append(result_action)
 
         if not self.check_mode and result['changed']:
             try:
@@ -661,13 +664,19 @@ def cmd_up(self):
                 self.client.fail("Error starting project - %s" % str(exc))
 
         if self.stopped:
-            result.update(self.cmd_stop(service_names))
+            stop_output = self.cmd_stop(service_names)
+            result['changed'] = stop_output['changed']  
+            result['actions'] += stop_output['actions']
 
         if self.restarted:
-            result.update(self.cmd_restart(service_names))
+            restart_output = self.cmd_restart(service_names)
+            result['changed'] = restart_output['changed']  
+            result['actions'] += restart_output['actions']
 
         if self.scale:
-            result.update(self.cmd_scale())
+            scale_output = self.cmd_scale()
+            result['changed'] = scale_output['changed']  
+            result['actions'] += scale_output['actions']
 
         for service in self.project.services:
             result['ansible_facts'][service.name] = dict()
@@ -729,105 +738,131 @@ def cmd_up(self):
     def cmd_pull(self):
         result = dict(
             changed=False,
-            actions=dict(),
+            actions=[],
         )
 
         if not self.check_mode:
             for service in self.project.get_services(self.services, include_deps=False):
+                if 'image' not in service.options:
+                    continue 
+
                 self.log('Pulling image for service %s' % service.name)
                 # store the existing image ID
-                image = service.image()
-                old_image_id = None
-                if image and image.get('Id'):
-                    old_image_id = image['Id']
+                old_image_id = ''
+                try:
+                    image = service.image()
+                    if image and image.get('Id'):
+                        old_image_id = image['Id']
+                except NoSuchImageError:
+                    pass
+                except Exception as exc:
+                    self.client.fail("Error: service image lookup failed - %s" % str(exc))
 
                 # pull the image
-                service.pull(ignore_pull_failures=False)
+                try:
+                    service.pull(ignore_pull_failures=False)
+                except Exception as exc:
+                    self.client.fail("Error: pull failed with %s" % str(exc)) 
 
                 # store the new image ID
-                image = service.image()
-                new_image_id = None
-                if image and image.get('Id'):
-                    new_image_id = image['Id']
+                new_image_id = '' 
+                try:
+                    image = service.image()
+                    if image and image.get('Id'):
+                        new_image_id = image['Id']
+                except NoSuchImageError as exc:
+                    self.client.fail("Error: service image lookup failed after pull - %s" % str(exc))    
 
                 if new_image_id != old_image_id:
                     # if a new image was pulled
                     result['changed'] = True
-                    result['actions'][service.name] = dict()
-                    result['actions'][service.name]['pulled_image'] = dict(
-                        name=service.image_name,
-                        id=service.image()['Id']
-                    )
+                    result['actions'].append(dict(
+                        service=service.name,
+                        pulled_image=dict(
+                            name=service.image_name,
+                            id=new_image_id
+                        )
+                    ))
         return result
 
     def cmd_build(self):
         result = dict(
             changed=False,
-            actions=dict(),
+            actions=[]
         )
         if not self.check_mode:
             for service in self.project.get_services(self.services, include_deps=False):
-                self.log('Building image for service %s' % service.name)
                 if service.can_be_built():
+                    self.log('Building image for service %s' % service.name)
                     # store the existing image ID
-                    image = service.image()
-                    old_image_id = None
-                    if image and image.get('Id'):
-                        old_image_id = image['Id']
+                    old_image_id = ''
+                    try:
+                        image = service.image()
+                        if image and image.get('Id'):
+                            old_image_id = image['Id']
+                    except NoSuchImageError:
+                        pass
+                    except Exception as exc:
+                        self.client.fail("Error: service image lookup failed - %s" % str(exc))
 
                     # build the image
-                    new_image_id = service.build(pull=True, no_cache=self.nocache)
+                    try:
+                        new_image_id = service.build(pull=True, no_cache=self.nocache)
+                    except Exception as exc:
+                        self.client.fail("Error: build failed with %s" % str(exc)) 
 
                     if new_image_id not in old_image_id:
                         # if a new image was built
                         result['changed'] = True
-                        result['actions'][service.name] = dict()
-                        result['actions'][service.name]['built_image'] = dict(
-                            name=service.image_name,
-                            id=service.image()['Id']
-                        )
+                        result['actions'].append(dict(
+                            service=service.name, 
+                            built_image=dict(
+                                name=service.image_name,
+                                id=new_image_id
+                            )
+                        ))
         return result
 
     def cmd_down(self):
         result = dict(
             changed=False,
-            actions=dict(),
+            actions=[]
         )
-
         for service in self.project.services:
             containers = service.containers(stopped=True)
             if len(containers):
                 result['changed'] = True
-            if self.debug or self.check_mode:
-                result['actions'][service.name] = dict()
-                result['actions'][service.name]['deleted'] = [container.name for container in containers]
-
+            result['actions'].append(dict(
+                service=service.name,
+                deleted=[container.name for container in containers]
+            ))
         if not self.check_mode and result['changed']:
             image_type = image_type_from_opt('--rmi', self.remove_images)
             try:
                 self.project.down(image_type, self.remove_volumes, self.remove_orphans)
             except Exception as exc:
                 self.client.fail("Error stopping project - %s" % str(exc))
-
         return result
 
     def cmd_stop(self, service_names):
         result = dict(
             changed=False,
-            actions=dict()
+            actions=[]
         )
         for service in self.project.services:
             if not service_names or service.name in service_names:
-                result['actions'][service.name] = dict()
-                result['actions'][service.name]['stop'] = []
+                service_res = dict(
+                    service=service.name,
+                    stop=[]
+                ) 
                 for container in service.containers(stopped=False):
                     result['changed'] = True
-                    if self.debug:
-                        result['actions'][service.name]['stop'].append(dict(
-                            id=container.id,
-                            name=container.name,
-                            short_id=container.short_id,
-                        ))
+                    service_res['stop'].append(dict(
+                        id=container.id,
+                        name=container.name,
+                        short_id=container.short_id
+                    ))
+                result['actions'].append(service_res)
 
         if not self.check_mode and result['changed']:
             try:
@@ -840,22 +875,24 @@ def cmd_stop(self, service_names):
     def cmd_restart(self, service_names):
         result = dict(
             changed=False,
-            actions=dict()
+            actions=[]
         )
 
         for service in self.project.services:
             if not service_names or service.name in service_names:
-                result['actions'][service.name] = dict()
-                result['actions'][service.name]['restart'] = []
+                service_res = dict(
+                    service=service.name,
+                    restart=[]
+                )
                 for container in service.containers(stopped=True):
                     result['changed'] = True
-                    if self.debug or self.check_mode:
-                        result['actions'][service.name]['restart'].append(dict(
-                            id=container.id,
-                            name=container.name,
-                            short_id=container.short_id,
-                        ))
-
+                    service_res['restart'].append(dict(
+                        id=container.id,
+                        name=container.name,
+                        short_id=container.short_id
+                    ))
+                result['actions'].append(service_res)
+         
         if not self.check_mode and result['changed']:
             try:
                 self.project.restart(service_names=service_names, timeout=self.timeout)
@@ -867,22 +904,25 @@ def cmd_restart(self, service_names):
     def cmd_scale(self):
         result = dict(
             changed=False,
-            actions=dict()
+            actions=[]
         )
 
         for service in self.project.services:
             if service.name in self.scale:
-                result['actions'][service.name] = dict()
+                service_res = dict(
+                    service=service.name,
+                    scale=0
+                )
                 containers = service.containers(stopped=True)
                 if len(containers) != self.scale[service.name]:
                     result['changed'] = True
-                    if self.debug or self.check_mode:
-                        result['actions'][service.name]['scale'] = self.scale[service.name] - len(containers)
+                    service_res['scale'] = self.scale[service.name] - len(containers)
                     if not self.check_mode:
                         try:
                             service.scale(int(self.scale[service.name]))
                         except Exception as exc:
                             self.client.fail("Error scaling %s - %s" % (service.name, str(exc)))
+                    result['actions'].append(service_res) 
         return result
 
 


Test patch: