Repository: ansible/ansible

Issue:
APT module behavior changed in 2.4
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report

##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->
modules/packaging/os/apt.py

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes below -->
```
ansible 2.4.0.0
  config file = /Users/kevin.prettre/work/deploy/ansible.cfg
  configured module search path = [u'/Users/kevin.prettre/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/kevin.prettre/.pyenv/versions/2.7/envs/ansible/lib/python2.7/site-packages/ansible
  executable location = /Users/kevin.prettre/.pyenv/versions/ansible/bin/ansible
  python version = 2.7 (r27:82500, Sep 15 2017, 22:21:11) [GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)]
```

##### CONFIGURATION

```
ANSIBLE_SSH_ARGS(/Users/kevin.prettre/work/deploy/ansible.cfg) = -o ControlMaster=no -o UserKnownHostsFile=/dev/null
ANSIBLE_SSH_RETRIES(/Users/kevin.prettre/work/deploy/ansible.cfg) = 2
DEFAULT_CALLBACK_WHITELIST(/Users/kevin.prettre/work/deploy/ansible.cfg) = ['profile_tasks']
DEFAULT_FORKS(/Users/kevin.prettre/work/deploy/ansible.cfg) = 15
DEFAULT_GATHERING(/Users/kevin.prettre/work/deploy/ansible.cfg) = explicit
HOST_KEY_CHECKING(/Users/kevin.prettre/work/deploy/ansible.cfg) = False
```

##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.
-->
Running from: Darwin - MacOS Serria 10.12.6 (16G29) OR debian jessie (Python 2.7)
Targeting:  Debian jessie (Python 2.7.9)
apt  1.0.9.8.4
aptitude  0.6.11-1+b1
dpkg 1.17.27

##### SUMMARY
<!--- Explain the problem briefly -->
https://github.com/ansible/ansible/blob/174cb1f33cdac126adc11dea81043914e8b127e0/lib/ansible/modules/packaging/os/apt.py#L384

https://github.com/ansible/ansible/blob/174cb1f33cdac126adc11dea81043914e8b127e0/lib/ansible/modules/packaging/os/apt.py#L494-L503

since the release of 2.4, that particular ligne, combined with lines from 494 to 503 (in the same file) leads to a wrong behavior of the apt module

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->


<!--- Paste example playbooks or commands between quotes below -->
```yaml
# playbooks/nginx.yaml
 ---
 - hosts: webserver01
    name: install nginx
    tasks:
    - apt:
         name: nginx={{version}}
         state: installed
         update_cache: yes
         only_upgrade: yes
         force: yes
```
`$ ansible-playbook playbooks/nginx.yml --extra-vars version="1.12.1-1~dotdeb+8.1"`

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
Let's nginx `1.6.2-5+deb8u5` is intalled and I want to upgrade to `1.12.1-1~dotdeb+8.1`

The playbook should detect that the package is installed, that the version installed mismatches the version requested, and install the newer version.

BUT, according to line 384, `package_installed = fnmatch.fnmatch(installed_version, version)` which is always False when you want to install a specific version.

You can reproduce this if clause  on the target machine quite easily in a python console

```python
>>> import apt
>>> import fnmatch
>>> cache = apt.Cache()
>>> pkg = cache['nginx']
>>> ll_pkg = cache._cache['nginx']
>>> installed = ll_pkg.current_state == 6 # that's apt_pkg.CURSTATE_INSTALLED value
>>> print(installed)
True
>>> cur_version = pkg.installed.version
>>> print(cur_version)
1.6.2-5+deb8u5
>>> target_version = '1.12.1-1~dotdeb+8.1'
>>> print(target_version)
1.12.1-1~dotdeb+8.1
>>> package_is_installed = fnmatch.fnmatch(cur_version, target_version)
>>> print(package_is_installed)
False
>>> print(fnmatch.fnmatch(cur_version, '*'))
True
```

TASK [install nginx] ***************************************************************************************************************
Wednesday 20 September 2017  16:08:29 +0200 (0:00:00.029)       0:00:00.029 ***
changed: [webserver01]

And nginx installed in it's newer version
<!--- What did you expect to happen when running the steps above? -->

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->
```
TASK [install nginx] ***************************************************************************************************************
Wednesday 20 September 2017  16:08:29 +0200 (0:00:00.029)       0:00:00.029 ***
ok: [webserver01]

```

nginx version did not change

Thanks for you work, I'll stick in 2.3 for now.

Bye.

APT module behavior changed in 2.4
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report

##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->
modules/packaging/os/apt.py

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes below -->
```
ansible 2.4.0.0
  config file = /Users/kevin.prettre/work/deploy/ansible.cfg
  configured module search path = [u'/Users/kevin.prettre/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/kevin.prettre/.pyenv/versions/2.7/envs/ansible/lib/python2.7/site-packages/ansible
  executable location = /Users/kevin.prettre/.pyenv/versions/ansible/bin/ansible
  python version = 2.7 (r27:82500, Sep 15 2017, 22:21:11) [GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)]
```

##### CONFIGURATION

```
ANSIBLE_SSH_ARGS(/Users/kevin.prettre/work/deploy/ansible.cfg) = -o ControlMaster=no -o UserKnownHostsFile=/dev/null
ANSIBLE_SSH_RETRIES(/Users/kevin.prettre/work/deploy/ansible.cfg) = 2
DEFAULT_CALLBACK_WHITELIST(/Users/kevin.prettre/work/deploy/ansible.cfg) = ['profile_tasks']
DEFAULT_FORKS(/Users/kevin.prettre/work/deploy/ansible.cfg) = 15
DEFAULT_GATHERING(/Users/kevin.prettre/work/deploy/ansible.cfg) = explicit
HOST_KEY_CHECKING(/Users/kevin.prettre/work/deploy/ansible.cfg) = False
```

##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.
-->
Running from: Darwin - MacOS Serria 10.12.6 (16G29) OR debian jessie (Python 2.7)
Targeting:  Debian jessie (Python 2.7.9)
apt  1.0.9.8.4
aptitude  0.6.11-1+b1
dpkg 1.17.27

##### SUMMARY
<!--- Explain the problem briefly -->
https://github.com/ansible/ansible/blob/174cb1f33cdac126adc11dea81043914e8b127e0/lib/ansible/modules/packaging/os/apt.py#L384

https://github.com/ansible/ansible/blob/174cb1f33cdac126adc11dea81043914e8b127e0/lib/ansible/modules/packaging/os/apt.py#L494-L503

since the release of 2.4, that particular ligne, combined with lines from 494 to 503 (in the same file) leads to a wrong behavior of the apt module

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->


<!--- Paste example playbooks or commands between quotes below -->
```yaml
# playbooks/nginx.yaml
 ---
 - hosts: webserver01
    name: install nginx
    tasks:
    - apt:
         name: nginx={{version}}
         state: installed
         update_cache: yes
         only_upgrade: yes
         force: yes
```
`$ ansible-playbook playbooks/nginx.yml --extra-vars version="1.12.1-1~dotdeb+8.1"`

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
Let's nginx `1.6.2-5+deb8u5` is intalled and I want to upgrade to `1.12.1-1~dotdeb+8.1`

The playbook should detect that the package is installed, that the version installed mismatches the version requested, and install the newer version.

BUT, according to line 384, `package_installed = fnmatch.fnmatch(installed_version, version)` which is always False when you want to install a specific version.

You can reproduce this if clause  on the target machine quite easily in a python console

```python
>>> import apt
>>> import fnmatch
>>> cache = apt.Cache()
>>> pkg = cache['nginx']
>>> ll_pkg = cache._cache['nginx']
>>> installed = ll_pkg.current_state == 6 # that's apt_pkg.CURSTATE_INSTALLED value
>>> print(installed)
True
>>> cur_version = pkg.installed.version
>>> print(cur_version)
1.6.2-5+deb8u5
>>> target_version = '1.12.1-1~dotdeb+8.1'
>>> print(target_version)
1.12.1-1~dotdeb+8.1
>>> package_is_installed = fnmatch.fnmatch(cur_version, target_version)
>>> print(package_is_installed)
False
>>> print(fnmatch.fnmatch(cur_version, '*'))
True
```

TASK [install nginx] ***************************************************************************************************************
Wednesday 20 September 2017  16:08:29 +0200 (0:00:00.029)       0:00:00.029 ***
changed: [webserver01]

And nginx installed in it's newer version
<!--- What did you expect to happen when running the steps above? -->

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->
```
TASK [install nginx] ***************************************************************************************************************
Wednesday 20 September 2017  16:08:29 +0200 (0:00:00.029)       0:00:00.029 ***
ok: [webserver01]

```

nginx version did not change

Thanks for you work, I'll stick in 2.3 for now.

Bye.



Discussion:
cc @mgwilliams
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
The version is not part of the package name. See apt-cache show nginx. They are two separate meta data "Package:" and "Version:". Can't you achieve your desired result with "state: latest"?
Hi, 

I'm expecting to pin-point a specific version, that's the goal of my playbook. 

I've debug print the whole process, and I can assure you that you can't move to a specific version of an installed package anymore.


cc @mgwilliams
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
The version is not part of the package name. See apt-cache show nginx. They are two separate meta data "Package:" and "Version:". Can't you achieve your desired result with "state: latest"?
Hi, 

I'm expecting to pin-point a specific version, that's the goal of my playbook. 

I've debug print the whole process, and I can assure you that you can't move to a specific version of an installed package anymore.



Resolving patch:
diff --git a/lib/ansible/modules/packaging/os/apt.py b/lib/ansible/modules/packaging/os/apt.py
--- a/lib/ansible/modules/packaging/os/apt.py
+++ b/lib/ansible/modules/packaging/os/apt.py
@@ -350,20 +350,21 @@ def package_status(m, pkgname, version, cache, state):
                 if provided_packages:
                     is_installed = False
                     upgradable = False
+                    version_ok = False
                     # when virtual package providing only one package, look up status of target package
                     if cache.is_virtual_package(pkgname) and len(provided_packages) == 1:
                         package = provided_packages[0]
-                        installed, upgradable, has_files = package_status(m, package.name, version, cache, state='install')
+                        installed, version_ok, upgradable, has_files = package_status(m, package.name, version, cache, state='install')
                         if installed:
                             is_installed = True
-                    return is_installed, upgradable, False
+                    return is_installed, version_ok, upgradable, False
                 m.fail_json(msg="No package matching '%s' is available" % pkgname)
             except AttributeError:
                 # python-apt version too old to detect virtual packages
                 # mark as upgradable and let apt-get install deal with it
-                return False, True, False
+                return False, False, True, False
         else:
-            return False, False, False
+            return False, False, False, False
     try:
         has_files = len(pkg.installed_files) > 0
     except UnicodeDecodeError:
@@ -381,6 +382,7 @@ def package_status(m, pkgname, version, cache, state):
             # assume older version of python-apt is installed
             package_is_installed = pkg.isInstalled
 
+    version_is_installed = package_is_installed
     if version:
         versions = package_versions(pkgname, pkg, cache._cache)
         avail_upgrades = fnmatch.filter(versions, version)
@@ -391,8 +393,8 @@ def package_status(m, pkgname, version, cache, state):
             except AttributeError:
                 installed_version = pkg.installedVersion
 
-            # Only claim the package is installed if the version is matched as well
-            package_is_installed = fnmatch.fnmatch(installed_version, version)
+            # check if the version is matched as well
+            version_is_installed = fnmatch.fnmatch(installed_version, version)
 
             # Only claim the package is upgradable if a candidate matches the version
             package_is_upgradable = False
@@ -409,7 +411,7 @@ def package_status(m, pkgname, version, cache, state):
             # assume older version of python-apt is installed
             package_is_upgradable = pkg.isUpgradable
 
-    return package_is_installed, package_is_upgradable, has_files
+    return package_is_installed, version_is_installed, package_is_upgradable, has_files
 
 
 def expand_dpkg_options(dpkg_options_compressed):
@@ -519,10 +521,10 @@ def install(m, pkgspec, cache, upgrade=False, default_release=None,
 
         name, version = package_split(package)
         package_names.append(name)
-        installed, upgradable, has_files = package_status(m, name, version, cache, state='install')
-        if (not installed and not only_upgrade) or (upgrade and upgradable):
+        installed, installed_version, upgradable, has_files = package_status(m, name, version, cache, state='install')
+        if (not installed and not only_upgrade) or (installed and not installed_version) or (upgrade and upgradable):
             pkg_list.append("'%s'" % package)
-        if installed and upgradable and version:
+        if installed_version and upgradable and version:
             # This happens when the package is installed, a newer version is
             # available, and the version is a wildcard that matches both
             #
@@ -684,8 +686,8 @@ def remove(m, pkgspec, cache, purge=False, force=False,
     pkgspec = expand_pkgspec_from_fnmatches(m, pkgspec, cache)
     for package in pkgspec:
         name, version = package_split(package)
-        installed, upgradable, has_files = package_status(m, name, version, cache, state='remove')
-        if installed or (has_files and purge):
+        installed, installed_version, upgradable, has_files = package_status(m, name, version, cache, state='remove')
+        if installed_version or (has_files and purge):
             pkg_list.append("'%s'" % package)
     packages = ' '.join(pkg_list)
 


Test patch: