Update RLE.c and debugged encode_RLE

* clarified return errors
* syntax adapted
* debugged encode_RLE