gray-inferencer: fix a bunch of memory leaks

+ still need to comb through errors reported by Valgrind