Repository: ansible/ansible

Issue:
Ansible 2.4 CallBackModule error?
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report

##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->
- CallBackModule
##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes below -->

```
$ ansible --version
ansible 2.4.0.0
  config file = /Users/<REDACTED>/ansible.cfg
  configured module search path = [u'/Users/<REDACTED>/ansible-lib/library']
  ansible python module location = /usr/local/lib/python2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.13 (default, Aug  7 2017, 22:02:14) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.28.1)]
```

##### CONFIGURATION
```
ANSIBLE_SSH_CONTROL_PATH(/Users/<REDACTED>/ansible.cfg) = %(directory)s/%%h-%%r
ANSIBLE_SSH_PIPELINING(/Users/<REDACTED>/ansible.cfg) = False
DEFAULT_ACTION_PLUGIN_PATH(/Users/<REDACTED>/ansible.cfg) = [u'/Users/<REDACTED>/ansible-lib/action_plugins']
DEFAULT_CALLBACK_PLUGIN_PATH(/Users/<REDACTED>/ansible.cfg) = [u'/Users/<REDACTED>/ansible-lib/callback_plugins']
DEFAULT_CALLBACK_WHITELIST(/Users/<REDACTED>/ansible.cfg) = ['log_plays_capture', 'mail', 'actionable', 'skippy', 'task_timer', 'version_recorder']
DEFAULT_CONNECTION_PLUGIN_PATH(/Users/<REDACTED>/ansible.cfg) = [u'/Users/<REDACTED>/ansible-lib/connection_plugins']
DEFAULT_FILTER_PLUGIN_PATH(/Users/<REDACTED>/ansible.cfg) = [u'/Users/<REDACTED>/ansible-lib/filter']
DEFAULT_HASH_BEHAVIOUR(/Users/<REDACTED>/ansible.cfg) = merge
DEFAULT_LOOKUP_PLUGIN_PATH(/Users/<REDACTED>/ansible.cfg) = [u'/Users/<REDACTED>/ansible-lib/lookup_plugins']
DEFAULT_MODULE_PATH(/Users/<REDACTED>/ansible.cfg) = [u'/Users/<REDACTED>/ansible-lib/library']
DEFAULT_REMOTE_TMP(/Users/<REDACTED>/ansible.cfg) = $HOME/.ansible/tmp
DEFAULT_STDOUT_CALLBACK(/Users/<REDACTED>/ansible.cfg) = skippy
DEFAULT_TRANSPORT(/Users/<REDACTED>/ansible.cfg) = smart
DEPRECATION_WARNINGS(/Users/<REDACTED>/ansible.cfg) = False
DISPLAY_SKIPPED_HOSTS(/Users/<REDACTED>/ansible.cfg) = False
HOST_KEY_CHECKING(/Users/<REDACTED>/ansible.cfg) = False
```

##### OS / ENVIRONMENT
```
$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.13
BuildVersion:	17A362a
```

##### SUMMARY
After upgrading from Ansible 2.3.2 to Ansible 2.4.0.0 (via pip2), We have a bunch of custom libraries, plugins (action, filter and lookup); I see exceptions in playbook runs.

```
the full traceback was:

Traceback (most recent call last):
  File "/usr/local/bin/ansible-playbook", line 106, in <module>
    exit_code = cli.run()
  File "/usr/local/lib/python2.7/site-packages/ansible/cli/playbook.py", line 130, in run
    results = pbex.run()
  File "/usr/local/lib/python2.7/site-packages/ansible/executor/playbook_executor.py", line 90, in run
    self._tqm.load_callbacks()
  File "/usr/local/lib/python2.7/site-packages/ansible/executor/task_queue_manager.py", line 203, in load_callbacks
    callback_obj .set_options(C.config.get_plugin_options('callback', callback_plugin._load_name))
AttributeError: 'CallbackModule' object has no attribute 'set_options'

```

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->
```
  $ ansible-playbook -i env/qa -u <REDACTED> provision.yml --extra-vars="state=reloaded release=e4" --ask-become-pass --become --become-user=<REDACTED> --vault-password-file=~/.vault_pass --limit=<REDACTED> -vvvv
  ansible-playbook 2.4.0.0
    config file = /Users/<REDACTED>/ansible.cfg
    configured module search path = [u'/Users/<REDACTED>/ansible-lib/library']
    ansible python module location = /usr/local/lib/python2.7/site-packages/ansible
    executable location = /usr/local/bin/ansible-playbook
    python version = 2.7.13 (default, Aug  7 2017, 22:02:14) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.28.1)]
  Using /Users/<REDACTED>/ansible.cfg as config file
  SUDO password:
  setting up inventory plugins
  Parsed /Users/<REDACTED>/env/qa/inventory inventory source with ini plugin
  Loading callback plugin skippy of type stdout, v2.0 from /usr/local/lib/python2.7/site-packages/ansible/plugins/callback/__init__.pyc
  ERROR! Unexpected Exception, this is probably a bug: 'CallbackModule' object has no attribute 'set_options'
  the full traceback was:

  Traceback (most recent call last):
    File "/usr/local/bin/ansible-playbook", line 106, in <module>
      exit_code = cli.run()
    File "/usr/local/lib/python2.7/site-packages/ansible/cli/playbook.py", line 130, in run
      results = pbex.run()
    File "/usr/local/lib/python2.7/site-packages/ansible/executor/playbook_executor.py", line 90, in run
      self._tqm.load_callbacks()
    File "/usr/local/lib/python2.7/site-packages/ansible/executor/task_queue_manager.py", line 203, in load_callbacks
      callback_obj .set_options(C.config.get_plugin_options('callback', callback_plugin._load_name))
  AttributeError: 'CallbackModule' object has no attribute 'set_options'
```



Discussion:
I can confirm that I get a similar error running Ansible 2.4 built from `git` source tree, on Debian Jessie host:
```
ERROR! Unexpected Exception, this is probably a bug: 'CallbackModule' object has no attribute 'set_options'
the full traceback was:

Traceback (most recent call last):
  File "/usr/bin/ansible-playbook", line 106, in <module>
    exit_code = cli.run()
  File "/usr/lib/python2.7/dist-packages/ansible/cli/playbook.py", line 130, in run
    results = pbex.run()
  File "/usr/lib/python2.7/dist-packages/ansible/executor/playbook_executor.py", line 90, in run
    self._tqm.load_callbacks()
  File "/usr/lib/python2.7/dist-packages/ansible/executor/task_queue_manager.py", line 203, in load_callbacks
    callback_obj .set_options(C.config.get_plugin_options('callback', callback_plugin._load_name))
AttributeError: 'CallbackModule' object has no attribute 'set_options'
```

Resolving patch:
diff --git a/lib/ansible/executor/task_queue_manager.py b/lib/ansible/executor/task_queue_manager.py
--- a/lib/ansible/executor/task_queue_manager.py
+++ b/lib/ansible/executor/task_queue_manager.py
@@ -176,7 +176,12 @@ def load_callbacks(self):
                 raise AnsibleError("Invalid callback for stdout specified: %s" % self._stdout_callback)
             else:
                 self._stdout_callback = callback_loader.get(self._stdout_callback)
-                self._stdout_callback.set_options(C.config.get_plugin_options('callback', self._stdout_callback._load_name))
+                try:
+                    self._stdout_callback.set_options(C.config.get_plugin_options('callback', self._stdout_callback._load_name))
+                except AttributeError:
+                    display.deprecated("%s stdout callback, does not support setting 'options', it will work for now, "
+                                       " but this will be required in the future and should be updated,"
+                                       " see the 2.4 porting guide for details." % self._stdout_callback._load_name, version="2.9")
                 stdout_callback_loaded = True
         else:
             raise AnsibleError("callback must be an instance of CallbackBase or the name of a callback plugin")
@@ -200,7 +205,12 @@ def load_callbacks(self):
                     continue
 
             callback_obj = callback_plugin()
-            callback_obj .set_options(C.config.get_plugin_options('callback', callback_plugin._load_name))
+            try:
+                callback_obj .set_options(C.config.get_plugin_options('callback', callback_plugin._load_name))
+            except AttributeError:
+                    display.deprecated("%s callback, does not support setting 'options', it will work for now, "
+                                       " but this will be required in the future and should be updated, "
+                                       " see the 2.4 porting guide for details." % self._stdout_callback._load_name, version="2.9")
             self._callback_plugins.append(callback_obj)
 
         self._callbacks_loaded = True


Test patch: