Paper: A Cognitive Model for Learning Abstract Relational Structures from Memory-based Decision-Making Tasks (ICLR-2024)

Reviewer comment (Novelty):
There is extensive discussion of previous work, but I was left wondering what exact contributions this model makes over other models in this space like TEM. The paper discusses numerous differences, but I would like to see explicit discussion of what this model brings to the table, what specific phenomenon that this model produces that other models don't, etc. There are maybe some signs of this throughout the paper, but I didn't see any explicit discussion on it.

Author response:
We think the exact contribution in TEM space as follows.  In TEM, they proposed the learning principle of abstract relational structure as hypothesis for hippocampal computation and a particular model to perform a random-walking task in this principle that can reproduce rodent data.  Our exact contribution is to consider another model in this principle that performs a decision making and reproduce human data, thus strengthening the hypothesis.   The paper explains that this is actually a substantial undertaking requiring a number of technical developments and experimental efforts.

Reviewer comment (Presentation):
There's no limitations section/paragraph, which is an important part of any iclr paper.

Author response:
We miss the limitation section and will have it in the revision.  Right now, we think of limitation in the scalability of the relational structure.  The model finds difficulty structure larger than ~15 entities in 1D or ~5x5 entities in 2D.  Also, the long training time is a big issue.

Reviewer comment (Experiments):
Would it be possible for the model to learn these relations implicitly without specifically being rewarded for them? Sometimes for humans, abstract relational structure can often be learned in service of doing a specific task, rather than being trained specifically on finding the correct relations.

Author response:
The question on the implicit learning is extreme interesting.  We have not thought about this possibility, and our current model would not be capable without substantial extension.  Probably, the task is a lot more difficult (note that the current task is quite difficult).

Reviewer comment (Writing):
While the overall presentation of the paper is good, there are a couple of sections that are not easy to follow. The results on the two-dimensional hierarchy (section 4.2) can be challenging to understand for someone not very familiar with the findings in Park et al. (2021).

Author response:
* The intermediate representation $h$ mentioned in Sec 4.2 refers to the $h$ in eq (4). 
* The state $s$ metioned in Sec 4.2 for given stimulus x is computed by $s=infer_{M,W}(x)$ using eq (4).  Note that we first run the model in the test domain (Sec 4.2, par 3)

Reviewer comment (Writing):
The notation in the section on transitive inference (3.4) can be a bit confusing (please see questions below).

Author response:
* $\phi_a^m(s)$ means m-th iteration, namely, $\phi_a(\phi_a(...(s)..))$, where $\phi_a$ is recursively applied m-times.  There is no update in this computation since $\phi_a(s)$ is defined as $\kappa(\rho(R_a s))$ where all the parameters used there $(M, W, R_a)$ are fixed when it is performed.  Note that we perform transitive inference task for testing a learned model (the model does not learn the transitive inference task).
* c^ti is the confidence value for the relation chosen by the model: $\max_a p^+(a|s,s')$ (Sec 3.4).