-l flag no longer pass to binaries if there's a library in the package

Close #814 

Unfortunately I don't really know how I'd add a test for this.
In order to try compile a library then a binary, you need to successfully compile the library, which means that you'd need to pass a valid value instead of just `-l foo`