Don't create unnecessary test configurations.
Fix the generation logic to better handle test/benchmark/package test configurations.
Closes #413