More malloc checking

Don't crash if running out of memory when allocating per-glyph data
during shaping. Also some fixes to initialization.

The caller of ass_fontconfig.c/match_fullname() actually handles a NULL
return value correctly.