My fault, I didn't realize before that the SGVector assignment can be changed when calling m_infer_impl->inference(assignment), I only considered it as an input parameter.