Repository: ansible/ansible-modules-extras

Issue:
Ansible 2.2.0.0 DNF module cannot manage package groups
##### ISSUE TYPE
- Bug Report

##### COMPONENT NAME
dnf

##### ANSIBLE VERSION
```console
$ ansible --version
ansible 2.2.0.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
```

##### CONFIGURATION
<!---
Mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->
`/etc/ansible.cfg` is untouched. It is as provided by the Arch Linux package. See: https://www.archlinux.org/packages/community/any/ansible/

No Ansible-related variables are set. Verified with:

```console
$ env | grep -i ansible
$
```

##### OS / ENVIRONMENT

My control host is an up-to-date Arch Linux system. It was updated yesterday evening, which is less than 24 hours ago.

My managed hosts are Fedora 23 and Fedora 24 VMs, "server" spin. These VMs are pretty bare-bones. Ignoring snapshots and clones, each system is built by doing the following:

1. Download the "Server" spin of Fedora 24 from [here](https://getfedora.org/en/server/download/) (and from a similar page for F23).
2. Create a VM, and install an up-to-date system from the images.
3. Set the system's hostname and install SSH keys.

That's about it. The important bit is not that I'm managing Fedora systems per se, but rather that I'm using the DNF package managers on said systems. Here's the version of DNF installed on the F23 system:

```console
# dnf --version
1.1.10
  Installed: dnf-0:1.1.10-1.fc23.noarch at 2016-10-10 16:03
  Built    : Fedora Project at 2016-08-18 14:43

  Installed: rpm-0:4.13.0-0.rc1.13.fc23.x86_64 at 2016-10-10 16:01
  Built    : Fedora Project at 2016-04-25 13:50
```

Here's the version of DNF installed on the F24 system:

```console
# dnf --version
1.1.10
  Installed: dnf-0:1.1.10-1.fc24.noarch at 2016-10-10 19:55
  Built    : Fedora Project at 2016-08-18 14:39

  Installed: rpm-0:4.13.0-0.rc1.27.fc24.x86_64 at 2016-10-10 19:55
  Built    : Fedora Project at 2016-04-25 13:45
```

##### SUMMARY
<!--- Explain the problem briefly -->

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem.
For new features, show how the feature would be used.
-->

To reproduce the issue, use Ansible's DNF module to install a package group. For example:

```console:
$ ansible all -i fedora-23-pulp-2-11, -m dnf -a 'name=@LibreOffice state=present'
fedora-23-pulp-2-11 | FAILED! => {
    "changed": false, 
    "failed": true, 
    "module_stderr": "Shared connection to fedora-23-pulp-2-11 closed.\r\n", 
    "module_stdout": "Traceback (most recent call last):\r\n  File \"/tmp/ansible_fMteBp/ansible_module_dnf.py\", line 375, in <module>\r\n    main()\r\n  File \"/tmp/ansible_fMteBp/ansible_module_dnf.py\", line 369, in main\r\n    ensure(
module, base, params['state'], params['name'])\r\n  File \"/tmp/ansible_fMteBp/ansible_module_dnf.py\", line 280, in ensure\r\n    for group in (g.strip() for g in groups):\r\n  File \"/tmp/ansible_fMteBp/ansible_module_dnf.py\", line 280,
 in <genexpr>\r\n    for group in (g.strip() for g in groups):\r\n  File \"/usr/lib/python2.7/site-packages/dnf/comps.py\", line 183, in __getattr__\r\n    return getattr(self._i, name)\r\nAttributeError: 'libcomps.Group' object has no att
ribute 'strip'\r\n", 
    "msg": "MODULE FAILURE"
}
```

Downgrading to Ansible 2.1 solves the issue:

```console
$ ls /var/cache/pacman/pkg/ | grep -i ansible
ansible-2.1.2.0-1-any.pkg.tar.xz
ansible-2.2.0.0-1-any.pkg.tar.xz
$ sudo pacman -U /var/cache/pacman/pkg/ansible-2.1.2.0-1-any.pkg.tar.xz
loading packages...
warning: downgrading package ansible (2.2.0.0-1 => 2.1.2.0-1)
resolving dependencies...
looking for conflicting packages...

Packages (1) ansible-2.1.2.0-1

Total Installed Size:  22.09 MiB
Net Upgrade Size:      -9.07 MiB

:: Proceed with installation? [Y/n] y
(1/1) checking keys in keyring                                                                                                                  [########################################################################################] 100%
(1/1) checking package integrity                                                                                                                [########################################################################################] 100%
(1/1) loading package files                                                                                                                     [########################################################################################] 100%
(1/1) checking for file conflicts                                                                                                               [########################################################################################] 100%
(1/1) checking available disk space                                                                                                             [########################################################################################] 100%
:: Processing package changes...
(1/1) downgrading ansible                                                                                                                       [########################################################################################] 100%
$ ansible all -i fedora-23-pulp-2-11, -m dnf -a 'name=@LibreOffice state=present'
fedora-23-pulp-2-11 | SUCCESS => {
    "changed": true, 
    "results": [
        "Installed: cairo-1.14.2-2.fc23.x86_64", 
        "Installed: google-crosextra-caladea-fonts-1.002-0.6.20130214.fc23.noarch", 
        snip!
        "Installed: mesa-libgbm-11.1.0-4.20151218.fc23.x86_64", 
        "Installed: mesa-libglapi-11.1.0-4.20151218.fc23.x86_64"
    ]
}
```

This issue is not specific to the LibreOffice package group. The error also occurs when using Ansible to install nightly builds of [Pulp](http://pulpproject.org/). See a full log [here](http://pastebin.com/FhtXkBaZ). Importantly, `ansible all -i fedora-23-pulp-2-11, -m dnf -a 'name=@pulp-server-qpid state=present'` fails when Ansible 2.2.0.0 is installed and succeeds when Ansible 2.1.2.0 is installed.

##### EXPECTED RESULTS

Ansible's DNF module should be able to install package groups.

##### ACTUAL RESULTS

Here's another demo of what happens. This time, I'm using a Fedora 24 VM. (This ensures that the issue isn't specific to Fedora 23.) 

```console
$ ansible --version
ansible 2.2.0.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
$ ansible all -i fedora-24-test, -m ping
fedora-24-test | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}
$ ansible all -i fedora-24-test, -m raw -a 'dnf -y install python2-dnf'
fedora-24-test | SUCCESS | rc=0 >>
Last metadata expiration check: 5:31:59 ago on Thu Nov  3 09:54:51 2016.
Dependencies resolved.
================================================================================
 Package               Arch        Version                   Repository    Size
================================================================================
Installing:
 pyliblzma             x86_64      0.5.3-15.fc24             fedora        53 k
 python-six            noarch      1.10.0-2.fc24             fedora        34 k
 python2-dnf           noarch      1.1.10-1.fc24             updates      444 k
 python2-hawkey        x86_64      0.6.3-6.fc24              updates       46 k
 python2-iniparse      noarch      0.4-19.fc24               fedora        45 k
 python2-libcomps      x86_64      0.1.7-4.fc24              fedora        47 k
 python2-librepo       x86_64      1.7.18-2.fc24             fedora        56 k
 python2-pygpgme       x86_64      0.3-18.fc24               updates       90 k
 rpm-python            x86_64      4.13.0-0.rc1.27.fc24      fedora       102 k

Transaction Summary
================================================================================
Install  9 Packages

Total download size: 919 k
Installed size: 3.2 M
Downloading Packages:
(1/9): pyliblzma-0.5.3-15.fc24.x86_64.rpm       299 kB/s |  53 kB     00:00
(2/9): python2-iniparse-0.4-19.fc24.noarch.rpm  226 kB/s |  45 kB     00:00
(3/9): python2-libcomps-0.1.7-4.fc24.x86_64.rpm 1.0 MB/s |  47 kB     00:00
(4/9): rpm-python-4.13.0-0.rc1.27.fc24.x86_64.r 1.0 MB/s | 102 kB     00:00
(5/9): python2-dnf-1.1.10-1.fc24.noarch.rpm     1.1 MB/s | 444 kB     00:00
(6/9): python2-pygpgme-0.3-18.fc24.x86_64.rpm   1.0 MB/s |  90 kB     00:00
(7/9): python-six-1.10.0-2.fc24.noarch.rpm      567 kB/s |  34 kB     00:00
(8/9): python2-hawkey-0.6.3-6.fc24.x86_64.rpm   305 kB/s |  46 kB     00:00
(9/9): python2-librepo-1.7.18-2.fc24.x86_64.rpm 122 kB/s |  56 kB     00:00
--------------------------------------------------------------------------------
Total                                           427 kB/s | 919 kB     00:02
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Installing  : python2-hawkey-0.6.3-6.fc24.x86_64                          1/9
  Installing  : python-six-1.10.0-2.fc24.noarch                             2/9
  Installing  : python2-iniparse-0.4-19.fc24.noarch                         3/9
  Installing  : python2-pygpgme-0.3-18.fc24.x86_64                          4/9
  Installing  : rpm-python-4.13.0-0.rc1.27.fc24.x86_64                      5/9
  Installing  : python2-librepo-1.7.18-2.fc24.x86_64                        6/9
  Installing  : python2-libcomps-0.1.7-4.fc24.x86_64                        7/9
  Installing  : pyliblzma-0.5.3-15.fc24.x86_64                              8/9
  Installing  : python2-dnf-1.1.10-1.fc24.noarch                            9/9
  Verifying   : python2-dnf-1.1.10-1.fc24.noarch                            1/9
  Verifying   : pyliblzma-0.5.3-15.fc24.x86_64                              2/9
  Verifying   : python2-iniparse-0.4-19.fc24.noarch                         3/9
  Verifying   : python2-libcomps-0.1.7-4.fc24.x86_64                        4/9
  Verifying   : python2-librepo-1.7.18-2.fc24.x86_64                        5/9
  Verifying   : rpm-python-4.13.0-0.rc1.27.fc24.x86_64                      6/9
  Verifying   : python2-pygpgme-0.3-18.fc24.x86_64                          7/9
  Verifying   : python-six-1.10.0-2.fc24.noarch                             8/9
  Verifying   : python2-hawkey-0.6.3-6.fc24.x86_64                          9/9

Installed:
  pyliblzma.x86_64 0.5.3-15.fc24          python-six.noarch 1.10.0-2.fc24
  python2-dnf.noarch 1.1.10-1.fc24        python2-hawkey.x86_64 0.6.3-6.fc24
  python2-iniparse.noarch 0.4-19.fc24     python2-libcomps.x86_64 0.1.7-4.fc24
  python2-librepo.x86_64 1.7.18-2.fc24    python2-pygpgme.x86_64 0.3-18.fc24
  rpm-python.x86_64 4.13.0-0.rc1.27.fc24

Complete!
Shared connection to fedora-24-test closed.


$ ansible all -vvvv -i fedora-24-test, -m dnf -a 'name=@LibreOffice state=present'
Using /etc/ansible/ansible.cfg as config file
Loading callback plugin minimal of type stdout, v2.0 from /usr/lib/python2.7/site-packages/ansible/plugins/callback/__init__.pyc
Using module file /usr/lib/python2.7/site-packages/ansible/modules/extras/packaging/os/dnf.py
<fedora-24-test> ESTABLISH SSH CONNECTION FOR USER: None
<fedora-24-test> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/ichimonji10/.ansible/cp/ansible-ssh-%h-%p-%r fedora-24-test '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1478201327.35-116696318694544 `" && echo ansible-tmp-1478201327.35-116696318694544="` echo $HOME/.ansible/tmp/ansible-tmp-1478201327.35-116696318694544 `" ) && sleep 0'"'"''
<fedora-24-test> PUT /tmp/tmp5wtflb TO /root/.ansible/tmp/ansible-tmp-1478201327.35-116696318694544/dnf.py
<fedora-24-test> SSH: EXEC sftp -b - -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/ichimonji10/.ansible/cp/ansible-ssh-%h-%p-%r '[fedora-24-test]'
<fedora-24-test> ESTABLISH SSH CONNECTION FOR USER: None
<fedora-24-test> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/ichimonji10/.ansible/cp/ansible-ssh-%h-%p-%r fedora-24-test '/bin/sh -c '"'"'chmod u+x /root/.ansible/tmp/ansible-tmp-1478201327.35-116696318694544/ /root/.ansible/tmp/ansible-tmp-1478201327.35-116696318694544/dnf.py && sleep 0'"'"''
<fedora-24-test> ESTABLISH SSH CONNECTION FOR USER: None
<fedora-24-test> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/ichimonji10/.ansible/cp/ansible-ssh-%h-%p-%r -tt fedora-24-test '/bin/sh -c '"'"'/usr/bin/python /root/.ansible/tmp/ansible-tmp-1478201327.35-116696318694544/dnf.py; rm -rf "/root/.ansible/tmp/ansible-tmp-1478201327.35-116696318694544/" > /dev/null 2>&1 && sleep 0'"'"''
fedora-24-test | FAILED! => {
    "changed": false,
    "failed": true,
    "invocation": {
        "module_name": "dnf"
    },
    "module_stderr": "OpenSSH_7.3p1, OpenSSL 1.0.2j  26 Sep 2016\r\ndebug1: Reading configuration data /home/ichimonji10/.ssh/config\r\ndebug1: /home/ichimonji10/.ssh/config line 21: Applying options for fedora-24-test\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 6986\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 0\r\nShared connection to fedora-24-test closed.\r\n",
    "module_stdout": "Traceback (most recent call last):\r\n  File \"/tmp/ansible_lGlW4D/ansible_module_dnf.py\", line 375, in <module>\r\n    main()\r\n  File \"/tmp/ansible_lGlW4D/ansible_module_dnf.py\", line 369, in main\r\n    ensure(module, base, params['state'], params['name'])\r\n  File \"/tmp/ansible_lGlW4D/ansible_module_dnf.py\", line 280, in ensure\r\n    for group in (g.strip() for g in groups):\r\n  File \"/tmp/ansible_lGlW4D/ansible_module_dnf.py\", line 280, in <genexpr>\r\n    for group in (g.strip() for g in groups):\r\n  File \"/usr/lib/python2.7/site-packages/dnf/comps.py\", line 183, in __getattr__\r\n    return getattr(self._i, name)\r\nAttributeError: 'libcomps.Group' object has no attribute 'strip'\r\n",
    "msg": "MODULE FAILURE"
}
```

Here's what the `module_stderr` line says:

```
OpenSSH_7.3p1, OpenSSL 1.0.2j  26 Sep 2016
debug1: Reading configuration data /home/ichimonji10/.ssh/config
debug1: /home/ichimonji10/.ssh/config line 21: Applying options for fedora-24-test
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: auto-mux: Trying existing master
debug2: fd 3 setting O_NONBLOCK
debug2: mux_client_hello_exchange: master version 4
debug3: mux_client_forwards: request forwardings: 0 local, 0 remote
debug3: mux_client_request_session: entering
debug3: mux_client_request_alive: entering
debug3: mux_client_request_alive: done pid = 6986
debug3: mux_client_request_session: session request sent
debug1: mux_client_request_session: master session id: 2
debug3: mux_client_read_packet: read header failed: Broken pipe
debug2: Received exit status from master 0
Shared connection to fedora-24-test closed.
```

Here's what the `module_stout` line says:

```
Traceback (most recent call last):
  File "/tmp/ansible_lGlW4D/ansible_module_dnf.py", line 375, in <module>
    main()
  File "/tmp/ansible_lGlW4D/ansible_module_dnf.py", line 369, in main
    ensure(module, base, params['state'], params['name'])
  File "/tmp/ansible_lGlW4D/ansible_module_dnf.py", line 280, in ensure
    for group in (g.strip() for g in groups):
  File "/tmp/ansible_lGlW4D/ansible_module_dnf.py", line 280, in <genexpr>
    for group in (g.strip() for g in groups):
  File "/usr/lib/python2.7/site-packages/dnf/comps.py", line 183, in __getattr__
    return getattr(self._i, name)
AttributeError: 'libcomps.Group' object has no attribute 'strip'
```


Discussion:
<!-- @Ichimonji10: Thanks for filing this bug report! -->

@ignatenkobrain, @DJMuggs, ping. This issue is waiting on your response.
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)

<!--- boilerplate: issue_bug_new --->


Resolving patch:
diff --git a/packaging/os/dnf.py b/packaging/os/dnf.py
--- a/packaging/os/dnf.py
+++ b/packaging/os/dnf.py
@@ -297,14 +297,14 @@ def ensure(module, base, state, names):
         filenames = [f.strip() for f in filenames]
         groups = []
         environments = []
-        for group_spec in group_specs:
+        for group_spec in (g.strip() for g in group_specs):
             group = base.comps.group_by_pattern(group_spec)
             if group:
-                groups.append(group.strip())
+                groups.append(group)
             else:
-                environment = base.comps.environments_by_pattern(group_spec)
+                environment = base.comps.environment_by_pattern(group_spec)
                 if environment:
-                    environments.extend((e.id.strip() for e in environment))
+                    environments.append(environment.id)
                 else:
                     module.fail_json(
                         msg="No group {} available.".format(group_spec))
@@ -369,11 +369,21 @@ def ensure(module, base, state, names):
                 module.fail_json(
                     msg="Cannot remove paths -- please specify package name.")
 
-            installed = base.sack.query().installed()
             for group in groups:
-                if installed.filter(name=group.name):
+                try:
                     base.group_remove(group)
+                except dnf.exceptions.CompsError:
+                    # Group is already uninstalled.
+                    pass
 
+            for envioronment in environments:
+                try:
+                    base.environment_remove(environment)
+                except dnf.exceptions.CompsError:
+                    # Environment is already uninstalled.
+                    pass
+
+            installed = base.sack.query().installed()
             for pkg_spec in pkg_specs:
                 if installed.filter(name=pkg_spec):
                     base.remove(pkg_spec)


Test patch: