Generate source jars for tests

Closes #6125
It's just a super big jar that doesn't match the jar with the compiled classes, would love to do it properly though.