Exclude 'spec' package from installation.

This prevents some weird behavior when running
specs for another package since mamba will load
its own spec package since is first in the PYTHONPATH.