RLHF fixes

Wow, is this document outdated now.