Refactor: Use std::vector<unsigned char> for binary data instead of string

Strings should be reserved for "string-like" data.

Please also do performance test comparisons for this change.