More selective linking of .so files to avoid useless dependencies (Fixes #52)