[JENKINS-13619] - Throttle matrix configurations

Integrates changes from #16 and #17