Fix handling of aggregate(InMemoryFailure) so that we get a result and a
logged warn/error of the problem