Move sources to the lib instead.

And simplify the CMake file
Simplify CMake config even more and add more options.