This happens because WordPress by default is replacing all emojis with images. This is done so that emojis would work in old browsers. But, the JavaScript used for this purpose can slow down the website, and it is messing with the page loading only to show very old and outdated images instead of the pretty emojis most browsers have today.
The only way to avoid the problem with quotes and emojis is to disable WordPress emoji replacing. To do that, install one of many plugins available to disable this WordPress ‘feature’, like Disable Emoji from WordPress.org repository.