Refactor our creation of CFFI FFI and libraries slightly.

Centralize everything for ease of use.