tests: Ensure the karma is properly reset

https://fedorahosted.org/bodhi/ticket/388
https://fedorahosted.org/fesco/ticket/1238