Remove Darwin specific setup

Relies on pkg-config being available on all platforms.
Merge pull request #1 from immersive/fix-darwin

Remove Darwin specific setup