core: rearrange package structure

No visible external changes, but this makes it easier to plug the CFFI
backend.