Only try set the property directly if we have an object