Rearrange lib directory detection, fixes #587