Thoralf just clarified this for me on IRC, it should be fine to use the CFactorGraph object
in the stack as you do here.