> I usually provide tarballs with what's needed to run the software, not to develop with it / runs the tests

We've done that for all our projects so far. the tests package is always included for instance and it's the case here. 
"having a makefile to check that everything is installed" is not a use case.