Making compliance test test for failures and automatically tests all found json files