Article : Chess-GPT "low skill" vs "high skill" axis via control vectors!
#4
by
jukofyork
- opened
Posting this here so it doesn't get lost in the thread of doom:
https://adamkarvonen.github.io/machine_learning/2024/03/20/chess-gpt-interventions.html
He basically found a control vector that encoded the "low skill" vs "high skill" axis and then applied it to make the LLM play slightly better chess:
The previous blog article is really interesting too:
https://adamkarvonen.github.io/machine_learning/2024/01/03/chess-world-models.html
This shows that there are likely lots more uses for these outside of creative-writing, eg:
- The idea of training a "brutally critical" vs "completely uncritical" axis to help calibrate LLMs abily to give genuine feedback on stories (as opposed to just praising even absolutely shit stories like most do now).