reviews: 
Comments: Can language models capture aspects of human behaviour? In this work, these authors take a step towards this question by introducing fine-tuning an LLM (specifically LLaMA) on human behavioural data. The authors find that the resulting model – CENTaUR – is able to well-capture binary human decisions and achieves competitive performance against alternative cognitive models. In particular, the authors find that CENTaUR’s learned embeddings generalise well to new human behaviour data, raising the prospects of leveraging (finetuned) LLMs as stimuli-computable cognitive models.
Strengths: The method proposed by the authors is sweet and simple (in a good way!). The idea of finetuning on human choice data is neat and impressively improves fits to human data. I believe the CENTaUR method therefore has value to the broader community. I think this study is a good first-step towards opening doors for the use of LLMs as fits of human data which would inspire further work in the space going forwards. 

I am particularly excited by the generalization results the authors demonstrate in Section 3.4. It would be intriguing to see how performance scales when jointly training CENTaUR on many more human behavior datasets. I believe the methodology the authors introduce in the work, and the fact that they will release their code (and nicely, are using an already open-source model, in contrast to the oft used GPT series) should set their framework up well for extension.   

I also think the inclusion of a random effects component in the finetuning layer to capute individual differences is quite a nice (and seemingly novel? To my understanding?) idea; I would encourage the authors to expand on this further as a contribution, if they indeed believe the method is more generalizable and important for extending LLMs as cognitive models. 

I appreciate the authors’ rigour in their qualitative investigations (though I believe the results could be pushed on even further; see Weaknesses). I also appreciate the authors’ couching the broader implications of their work in the Discussion; the authors do a good job of cautioning against too broad a take on their findings of these models *as* cognitive models. Some of this could be brought further to the Intro (see below)).
Weaknesses: While I think the authors do a good job in couching the generality of their method and limitations in the Discussion, I think this could be set-up further in the Introduction. At times, I feel the motivation was a bit confused. There is a difference, in my opinion, between seeking to bridge the gap between out-of-the-box language models and fits to behavioral data, versus actually using LLMs *as* cognitive models to *predict and study* human behavior. I believe this work nicely supports and expands on the former, and teases the possibility of the latter – but the results at present do not fully convince me that these models are ready yet, or should, serve as stimuli-computable models of human behavior. 

Expanding on the above, it’s not clear to me that CENTaUR is better than BEAST (Fig 1c); yes, the NLL is objectively lower – but what about error bars? Is it significantly lower? The authors do a good job of highlighting qualitative strengths of CENTAuR, but it would be nice for a deeper dive on the gaps between CENTaUR and human performance as well? For instance, could the authors construct the inverse of Fig 5? What do examples of cases where CENTaUR did *not* match humans (but perhaps BEAST / hybrid did?) The authors note as well that there are 8 out of 60 individual participants not well-captured by the 

More broadly, it would be nice if the authors could include error bars in their results as much as possible (it should be feasible to obtain these, given the authors used 100-fold CV?). Further, why are the more classical cognitive models not included in the plots in 2a and b? 

As a minor stylistic weakness: I did not find it helpful to read the raw NLL values in the text. I think it’s best to keep these in the figures/tables and discuss the trends. It’s hard to tell what counts as a large or small gap between NLL; again, error bars would be helpful.
Recommendation: 8: accept, good paper
Reviewer confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness/correctness: 3 good
Clarity: 2 fair