Revert "numpy & scipy: Fix PKG_CONFIG_PATH"

This reverts commit 633035de5cfc201254a8b264f6167449e7911307.