Remove library specific stuff, and various cleanups

Signed-off-by: Ken Barber <ken@bob.sh>