Update INSTALL.md for Mac OS

- instructions for homebrew
- instructions for adding non-standard header and lib directories to the project