add bash8 tool (like pep8, but way hackier)

unlike our python code, we have no automatic style checking for
bash. For the most part, it's not a big deal, but errant whitespace
or incorrect indenting is sometimes annoying to have to -1 people's
patches for. Instead of constantly picking it up in manual review
maybe we can do better.

This is an uber hacky script which could be used to do just that.

   ./tools/bash8.py file1 file2 file3 ...

And it will show issues found with the files at hand. Lightly
tested in the existing devstack tree, it exposes a few issues that
we might want to think about.

This should be python 3 compatible, and includes argparse to provide
a basic '-h' support to explain how the command should be run.

Change-Id: I5009fa5852595c2953a548e430e5e1ce06ae94e0
add support for heredoc folding of lines

this change in the parser allows for us to have heredocs folded
into logical lines.

Change-Id: I51ebe6cd7b89b5f7194e947896f20b6750e972e3
(cherry picked from commit 02d7fe13bb714c3c8c28fbe16ecbeac472a80094)
add ability to ignore rules in bash8

Change-Id: Ia6472f4bb251bf3e9846e08e30b2f9ea30ea1c03
(cherry picked from commit 0656e12d6819f6dee671dd6200b2d0895e716c2c)
introduce if/then & for/do rules

we mostly have a consistent style on if/then & for/do in devstack,
except when we don't. This attempts to build a set of rules to
enforce this.

Because there are times when lines are legitimately long, and there
is a continuation, this starts off ignoring if and for loops with
continuations. But for short versions, we should enforce this.

Changes to make devstack pass are included. The fact that the
cleanup patch was so small is pretty solid reason that this is
actually the style we've all agreed to.

Part of a git stash from hong kong that I finally cleaned up.

Change-Id: I6376d7afd59cc5ebba9ed69e5ee784a3d5934a10
(cherry picked from commit 16dd8b3ed94d5cd217d22a26c18dca52bfca115e)
Add while/until to the for/do rule

Like 'for/do' check that the while/until operator are on
the same line with the do.

Fixes some pep8 error along the way.

Change-Id: I440afe60691263365bf35310bf4212d94f30c339
(cherry picked from commit 86a8e9767912ae957cbbf6ea20a08106011a7728)
make bash8 take a -v flag

this ensures that we actually know we are processing all the files
we believe we are.

Change-Id: I8e99b5f9dc987c946586475f374f7040ca63a478
(cherry picked from commit b93ee25b64de5d587c2e0889a9ce689c92aaa0f9)
Add end-of-file checks to bash8

Add two end-of-file checks to bash8.  Firstly, alert if heredoc hasn't
finished.  Some heredocs were done like:

---
sudo bash -c "cat <<EOF > foo
...
EOF"
---

(A better way to do this is "cat <<EOF | sudo tee ..." as it retains
the usual heredoc layout in the code).

The trailing quote was throwing the matching in bash8 off and it kept
appending the next file as if it was still part of the heredoc.  To
avoid this, we check if we're still in a heredoc when we start a new
file; if so raise an error and reset the heredoc status fresh.  We
track the state of the previous file, line and lineno so we can give a
good error.

---
E012: heredoc did not end before EOF: 'cat <<EOF'
 - lib/trove: L221
---

This includes fixes for the existing problem heredocs.

A similar EOF check is to ensure the previous file ended with a
newline.

---
E004: file did not end with a newline: '$MY_XTRACE'
 - lib/neutron_plugins/embrane: L40
---

This requires only one fix

Change-Id: I5e547d87b3921fc7ce6588c28f074e5c9f489c1f
(cherry picked from commit b8e250232ec55b946d2fd7e4237f12632408bdcc)
Enforce function declaration format in bash8

Check that function calls look like ^function foo {$ in bash8, and fix
all existing failures of that check.  Add a note to HACKING.rst

Change-Id: Ic19eecb39e0b20273d1bcd551a42fe400d54e938
(cherry picked from commit aee18c749b0e3a1a3a6907a33db76ae83b8d41d9)
Import fresh cookiecut bash8 project
Merge remote-tracking branch 'import/bash8'
Move bash8 module into bash8 package

The import dropped a single file into the root,
so we need to move that into our new bash8 package.
We can add an entry point to this later and refactor
the old module into other parts as it makes sense in
the future.
Add entry point for bash8 script

So we can call from command line after installing
the python package.