protect against invalid memory access if agg gradient lookup fails to build