Properly compute response.latencyInMillis, close #1859