I am not entirely sure but I think this streaming average corresponds to a diagonal approximation of the hessian. Maybe I should look up what the actual algorithm is we are using ^^