Aggressive refactor of backends.

Move a lot of shared logic between backends to a super class.