That is expected behaviour. Remember #1268.
Since FactorGraph is an SGObject, please use SG_REF and SG_UNREF and avoid using them from stack.