build: don't download and extract tarball if source is already present

When building under Homebrew, we want to let Homebrew manage
downloading and extracting the tarballs. See PR #1411.

Also make sure to skip only if directory is not empty. Fix #1433.