Bug 1075227 - Implement chunking logic for faster tests runs on CI

Yay, this is something easy to forget, but you didn't :D