Keep a running diff instead of forcing a full state update