It is true that RIL_UNSOL_ON_USSD is specified in ril.h as returning a UTF-8 string. But, it actually returns an UTF-16 string, as all strings from RIL. Or, it might be returning UTF-8 and only use one of the 2 bytes that are allocated by character, which would be weird. It is difficult to check, as I have not seen special characters in any USSD message at the moment.

The string returned by RIL is not a full PDU, it is just the string returned by the network with no DCS, so you have to make up a valid DCS for whatever you are returning in the callback. Jolla's solution was sloppy: it was returned 0xFF and then they modified core ussd to recognize that as a magic number and do not re-encode.

0x44=0100 0100 b ; bit 5 is 0 :)