Try to set the homebrew prefix only if brew is available.