Store the loaded font in the font descriptor

This should fix #4339. We attached an explanation of the idea at the issue.