Adds some check to treat OOM problems + RDP security fix

Malloc can fail so it will, this patch adds some check in some places
where malloc/strdup results were not checked.

This patch also contains a server side fix for RDP security (credit to nfedera).
The signature len was badly set in the GCC packet. And some other RDP security
oriented fixes are also there.