Extract some shared setup functionality into a library, and use it.

Summary:
Not all the setup.sh's have been modified to use setup_fns.sh, but the
biggest ones have.

I also started a new setup.sh, for youtube-export, to show what it's
like to create a setup script from scratch using these new functions
now.

Move tools and aws-tools to a new shared/ directory, add a README.

Get rid of directories corresponding to machines we no longer have.

I've confirmed there's now a 1-to-1 map between the ec2 machines
listed on console.aws.amazon.com, and the directories here, with the
exception of 'cloudsearch', which I don't know what it is.

Rewrite caching-proxy to use setup_fns.sh.

I added a varnish routine to setup_fns.sh, and fixed a bug where I
wasn't using CONFIG_DIR properly.

This is the first example of a config that can be done (pretty much)
entirely using setup-fns without modification!  I did have to rename
some of the files in this repo to match what setup_fns.sh expects,
though.

Finish modifying youtube-export to use setup.sh

alpert confirmed that youtube-export doesn't need nginx, so I removed
it from the setup.  I also use requirements.txt now instead of
hard-coding.  And I forgot to install the crontab before, oops...

Move sandcastle over to using setup_fns.

While doing this, I discovered another bug in setup_fns, regarding
postfix.preseed.  I already realized that not every machine had it,
but every machine should (so we can send out error-mails and the
like), so I added a postfix.preseed everywhere.

Move domain-redirect to using a simple nginx setup.

I chose nginx for the new version of domain-redirect, over the
alternatives, because nginx is the standard web proxy that we use on
other ec2 machines, and because the config for redirecting is very
straightforward.

Add a cron job that does security updates every night, to each machine.

Ubuntu keeps a list of package updates that are security updates, and
we basically get that list and install those packages unattended,
every night.

This new script lives in 'shared', and I've manually symlinked it into
each repo.  setup.sh will now manually install it as part of
install_root_config_files.  I make sure every setup.sh has an
install_root_config_files now.

In the process, I found a few more buglets in setup.sh, notably stuff
that was in 'basic install' that wasn't at all basic (php5-fpm??).

Fix a typo.  Sheesh!

Install python-dbg and gdb by default, to allow us to debug python via gdb.

We won't often want this functionality, but when we need it, it will
be nice it's available on the machine!

Try again with the domain-redirect nginx config.

This time I did some research rather than just depending on what
http://winginx.com/en/htaccess told me.  But it's still to be tested.

Test Plan:
Fingers crossed.  We don't actually really ever run the setup.sh
scripts by hand, so it's not clear that they worked before, or if they
would continue to work now.  But they're self-documenting nicely, at
least...

Fingers crossed.

None

Fingers crossed.

Fingers crossed.

Fingers crossed.

Fingers crossed.  Will try setting up a new domain-redirect machine
one day, and will make sure this works!

Fingers crossed.  Will try running all this stuff on the various ec2
machines after review.

None

Fingers crossed.

Fingers crossed.  Will figure out how to test before deploying.

Reviewers: alpert, chris

Reviewed By: chris

Differential Revision: http://phabricator.khanacademy.org/D13525