Repository: ansible/ansible

Issue:
"webdocs" make target fails under Python 3
<ins>This bug was originally filed against version 2.0.0-0.7.rc2. However, this bug is also reproductible on the "devel" branch, and probably other branches too. Keep that in mind when reading the remainder of this issue.</ins>


**Issue Type**

Bug Report

**Ansible Version**

I'm running Ansible 2.0.0-0.7.rc2:

```
$ ansible --version
ansible 2.0.0 (detached HEAD cc98528ecb) last updated 2015/12/07 14:44:25 (GMT -400)
  lib/ansible/modules/core:  not found - use git submodule update --init lib/ansible/modules/core
  lib/ansible/modules/extras:  not found - use git submodule update --init lib/ansible/modules/extras
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides
```

More precisely:

```
$ git status
HEAD detached at v2.0.0-0.7.rc2
nothing to commit, working directory clean
```

**Ansible Configuration**

I have changed nothing. I am running directly from the git repository.

**Environment**

I'm running an up-to-date version of Arch Linux. However, that shouldn't matter much, given that Ansible itself is running in a virtualenv. The virtualenv setup procedure is as follows:

``` sh
# create a virtualenv
N=2
virtualenv --python python$N ~/.virtualenvs/ansible$N
source ~/.virtualenvs/ansible$N/bin/activate

# install ansible from source
git clone git@github.com:ansible/ansible.git
cd ansible
git checkout v2.0.0-0.7.rc2
pip install -e .
```

I do the same for Python 3 by setting `N=3` and repeating the procedure. The Python versions as reported by `python --version` are `Python 2.7.10` and `Python 3.5.0`.

**Summary**

The `webdocs` make target fails for Ansible v2.0.0-0.7.rc7 under Python 3. It fails due to a variety of syntax errors, such as the use of `except Foo, e` and `print 'foo'`.

**Steps To Reproduce**

Create a virtualenv and install ansible from source, as outlined in the "Environment" section. Execute `make webdocs`.

**Expected Results**

`make webdocs` completes and produces usable documentation.

**Actual Results**

`make webdocs` completes under Python 2, but _not_ Python 3.

**Proposed Fix**

According to Community Information & Contributing → Ansible Users → [Contributing Code (Features or Bugfixes)](http://docs.ansible.com/ansible/community.html#contributing-code-features-or-bugfixes), "code developed for Ansible needs to support Python 2.6+". According to that standard, it should be OK to do things like replace a `except Foo, e` with `except Foo as e`.

In my environment (Python 2.7.10 and 3.5.0), the following change works allows me to execute `make webdocs` under Python 2 and 3. However, I'm not sure how to properly test this change to ensure it does not break anything.

``` diff
diff --git a/docsite/build-site.py b/docsite/build-site.py
index 587a189..24f9fc9 100755
--- a/docsite/build-site.py
+++ b/docsite/build-site.py
@@ -15,6 +15,7 @@
 #
 # You should have received a copy of the GNU General Public License
 # along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+from __future__ import print_function

 __docformat__ = 'restructuredtext'

@@ -24,9 +25,9 @@ import traceback
 try:
     from sphinx.application import Sphinx
 except ImportError:
-    print "#################################"
-    print "Dependency missing: Python Sphinx"
-    print "#################################"
+    print("#################################")
+    print("Dependency missing: Python Sphinx")
+    print("#################################")
     sys.exit(1)
 import os

@@ -40,7 +41,7 @@ class SphinxBuilder(object):
         """
         Run the DocCommand.
         """
-        print "Creating html documentation ..."
+        print("Creating html documentation ...")

         try:
             buildername = 'html'
@@ -69,10 +70,10 @@ class SphinxBuilder(object):

             app.builder.build_all()

-        except ImportError, ie:
+        except ImportError:
             traceback.print_exc()
-        except Exception, ex:
-            print >> sys.stderr, "FAIL! exiting ... (%s)" % ex
+        except Exception as ex:
+            print("FAIL! exiting ... (%s)" % ex, file=sys.stderr)

     def build_docs(self):
         self.app.builder.build_all()
@@ -83,9 +84,9 @@ def build_rst_docs():

 if __name__ == '__main__':
     if '-h' in sys.argv or '--help' in sys.argv:
-        print "This script builds the html documentation from rst/asciidoc sources.\n"
-        print "    Run 'make docs' to build everything."
-        print "    Run 'make viewdocs' to build and then preview in a web browser."
+        print("This script builds the html documentation from rst/asciidoc sources.\n")
+        print("    Run 'make docs' to build everything.")
+        print("    Run 'make viewdocs' to build and then preview in a web browser.")
         sys.exit(0)

     build_rst_docs()
@@ -93,4 +94,4 @@ if __name__ == '__main__':
     if "view" in sys.argv:
         import webbrowser
         if not webbrowser.open('htmlout/index.html'):
-            print >> sys.stderr, "Could not open on your webbrowser."
+            print("Could not open on your webbrowser.", file=sys.stderr)
diff --git a/hacking/module_formatter.py b/hacking/module_formatter.py
index f4ab5d7..4c94ca3 100755
--- a/hacking/module_formatter.py
+++ b/hacking/module_formatter.py
@@ -140,7 +140,7 @@ def list_modules(module_dir, depth=0):
             if os.path.isdir(d):

                 res = list_modules(d, depth + 1)
-                for key in res.keys():
+                for key in list(res.keys()):
                     if key in categories:
                         categories[key] = merge_hash(categories[key], res[key])
                         res.pop(key, None)
@@ -451,7 +451,7 @@ def main():

     categories = list_modules(options.module_dir)
     last_category = None
-    category_names = categories.keys()
+    category_names = list(categories.keys())
     category_names.sort()

     category_list_path = os.path.join(options.output_dir, "modules_by_category.rst")
```



Discussion:
ansible itself does not work with python3, we have made much progress in adding support for it, but are not there yet.

we have not even started looking at things like the build utilities for documentation or anything else.

Good time as any to start? `¯\_(ツ)_/¯`

submit as a PR, then we can test and merge if possible.

Will do!


Resolving patch:
diff --git a/docsite/build-site.py b/docsite/build-site.py
--- a/docsite/build-site.py
+++ b/docsite/build-site.py
@@ -15,6 +15,7 @@
 #
 # You should have received a copy of the GNU General Public License
 # along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+from __future__ import print_function
 
 __docformat__ = 'restructuredtext'
 
@@ -24,9 +25,9 @@
 try:
     from sphinx.application import Sphinx
 except ImportError:
-    print "#################################"
-    print "Dependency missing: Python Sphinx"
-    print "#################################"
+    print("#################################")
+    print("Dependency missing: Python Sphinx")
+    print("#################################")
     sys.exit(1)
 import os
 
@@ -40,7 +41,7 @@ def __init__(self):
         """
         Run the DocCommand.
         """
-        print "Creating html documentation ..."
+        print("Creating html documentation ...")
 
         try:
             buildername = 'html'
@@ -69,10 +70,10 @@ def __init__(self):
 
             app.builder.build_all()
 
-        except ImportError, ie:
+        except ImportError:
             traceback.print_exc()
-        except Exception, ex:
-            print >> sys.stderr, "FAIL! exiting ... (%s)" % ex
+        except Exception as ex:
+            print("FAIL! exiting ... (%s)" % ex, file=sys.stderr)
 
     def build_docs(self):
         self.app.builder.build_all()
@@ -83,9 +84,9 @@ def build_rst_docs():
 
 if __name__ == '__main__':
     if '-h' in sys.argv or '--help' in sys.argv:
-        print "This script builds the html documentation from rst/asciidoc sources.\n"
-        print "    Run 'make docs' to build everything."
-        print "    Run 'make viewdocs' to build and then preview in a web browser."
+        print("This script builds the html documentation from rst/asciidoc sources.\n")
+        print("    Run 'make docs' to build everything.")
+        print("    Run 'make viewdocs' to build and then preview in a web browser.")
         sys.exit(0)
 
     build_rst_docs()
@@ -93,4 +94,4 @@ def build_rst_docs():
     if "view" in sys.argv:
         import webbrowser
         if not webbrowser.open('htmlout/index.html'):
-            print >> sys.stderr, "Could not open on your webbrowser."
+            print("Could not open on your webbrowser.", file=sys.stderr)
diff --git a/hacking/module_formatter.py b/hacking/module_formatter.py
--- a/hacking/module_formatter.py
+++ b/hacking/module_formatter.py
@@ -140,7 +140,7 @@ def list_modules(module_dir, depth=0):
             if os.path.isdir(d):
 
                 res = list_modules(d, depth + 1)
-                for key in res.keys():
+                for key in list(res.keys()):
                     if key in categories:
                         categories[key] = merge_hash(categories[key], res[key])
                         res.pop(key, None)
@@ -451,7 +451,7 @@ def main():
 
     categories = list_modules(options.module_dir)
     last_category = None
-    category_names = categories.keys()
+    category_names = list(categories.keys())
     category_names.sort()
 
     category_list_path = os.path.join(options.output_dir, "modules_by_category.rst")


Test patch: