Move everything into common, client or server.

The beginnings of #101 which separates out the code into common, client or server.

It turns out pypi only can deal with setup.py files, so some further work will need to be done in order to be able to upload these separately, but this separation will be helpful regardless.

All the tests pass on my machine...