Only use our own ReferenceResolver when references are set (use the default resolver in the other case).