Cabbage Sweeter than Cake? Analysing the Potential of Large Language Models for Learning Conceptual Spaces
Authors: Usashi Chatterjee
Amit Gajbhiye
Steven Schockaert
Venue: EMNLP 2023 Main (2023)
Keywords: conceptual spaces
large language models
knowledge representation

Abstract: The theory of Conceptual Spaces is an influential cognitive-linguistic framework for representing the meaning of concepts. Conceptual spaces are constructed from a set of quality dimensions, which essentially correspond to primitive perceptual features (e.g. hue or size). These quality dimensions are usually learned from human judgements, which means that applications of conceptual spaces tend to be limited to narrow domains (e.g. modelling colour or taste). Encouraged by recent findings about the ability of Large Language Models (LLMs) to learn perceptually grounded representations, we explore the potential of such models for learning conceptual spaces. Our experiments show that LLMs can indeed be used for learning meaningful representations to some extent. However, we also find that fine-tuned models of the BERT family are able to match or even outperform the largest GPT-3 model, despite being 2 to 3 orders of magnitude smaller.

Cabbage Sweeter than Cake? Analysing the Potential of Large Language Models for Learning Conceptual Spaces
Usashi Chatterjee, Amit Gajbhiye, Steven Schockaert CardiffNLP, Cardiff University, UK (chatterjeeu.gajbbiyee,,,,, schockerrss Jecardi ac..difffcc..ukkk
Abstract
Most applications of conceptual spaces are thus lim- ited to narrow domains, where meaningful repre- sentations can learned from ratings provided by human participants (Paradis, 2015; Zwarts, 2015; Chella, 2015).
The theory of Conceptual Spaces is influ- ential cognitive-linguistc framework for repre- senting the meaning concepts. Conceptual spaces constructed from set of quality dimensions, which essentially correspond to primitive perceptual features (e.g. hue size). These quality dimensions are usually learned from human judgements, which means that ap- plications of conceptual spaces tend to be lim- ited to narrow domains (e.g. modelling colour or taste). Encouraged by recent findings about the ability of Large Language Models (LLMs) to learn perceptually grounded representations, we explore the potential of such models for learning conceptual spaces. Our experiments show that LLMs can indeed be used for learn- ing meaningful representations some extent. However, we also find that fine-tuned models BERT family are able to match or even outperform the largest GPT-3 model, despite being 2 to 3 orders of magnitude smaller.'
In this paper, we explore whether Large Lan- guage Models (LLMs) could be used for learn- ing conceptual spaces. This research question is closely related to the ongoing debate about the ex- tent to which Language Models (LMs) can learn perceptually grounded representations (Bender and Koller, 2020; Abdou et al., 2021; Patel and Pavlick, 2022; Søgaard, 2023). Recent work seems to sug- gest this might indeed be possible, at least for the colour domain. For instance, Abdou et al. (2021) found that LMs are able to Iearn representations of colour terms which are isomorphic perceptual colour spaces. When it comes predicting the typical colour of objectes, Paik al. (2021) found that the predictions of LMs are hcavily skewed by surface co-occurrence statistics, which are un- reliable for colours due to reporting bias (Gor- don and Durme, 2013), i.c. the fact that obvious colours are rarely mentioned in text. However, Liu et al. (2022a) found the effects of reporting bias to largely disappear in recent LLMss These find- ings suggest that it may now be possible to distill meaningful conceptual space representations from LLMs, as long as sufficiently large models are used. However, existing analyses are limited in two ways:
1 Introduction
Conceptual spaces (Gärdenfors, 2000) represent concepts in terms of cognitively meaningful fea- tures, called quality dimensions. For example, conceptual space of colour is composed of three quality dimensions, representing hue, saturation and intensity. Conceptual spaces provide an elegant framework for explaining various cognitive and linguistic phenomena (Gärdenfors, 2014). Within Artificial Intelligence (AI), the role of conceptual spaces is essentially to act intermediate rep- resentation layer, in between neural and symbolic representations (Gärdenfors, 2004). As such, con- ceptual spaces could play a central role in the devel- opment of explainable AI systems. Unfortunately, such representations are difficult to learn from data.
Several works have explored the colour do- main, and visual domains more generally (Li et al., 2023), but little is known about the abil- ities of LLMs in other perceptual domains.
Most analyses focus on classifying concepts, e.g. predicting colour terms or the materials from which objects are made, rather than on evaluating the underlying quality dimensions.
'Our datasets and evaluation scripts are available at https: //github..tttt/...//. comp/ExperimenttsLL/WWMM PotentialOLLLLLLannnnnooemeepp
We address the first limitation by including an eval- uation in the taste domain. To address the second

---

limitation, rather than considering discrete labels (e.g. sweet), we use LLMs rank concepts accord- ing to the degree to which they have a particular feature (e.g. sweetness).
sist of 500 pairwise judgements (e.g. an ant is larger than a bird). Note that unlike for the other datasets, no complete ranking is provided.
Methods
2 Datasets
We experiment with number of different models.
The primary focus of our experiments is the taste domain, which has not yet been considered in this context, despite having a number of impor- tant advantages. For instance, the relevant quality dimensions are well-established and have a linear structure (unlike hue in the colour domain). This domain also seems particularly challenging, as the typical terms which are used to describe taste only upply to extreme cases. For instance, we can assert that grapefruit is bitter and bananas are sweet, but it is less clear how language model would learn whether chicken is sweeter than cheese. As ground truth, we rely on the ratings that were collected by Martin et al. (2014). They rated a total of 590 food items along six dimensions: sweet, salty, sour, bitter, umami and fat. The ratings were obtained by a panel of twelve assessors who were experienced in sensory profiling. They scored the food items during an eight month measurement phase, after having received 55 hours of training in labora- tory. We manually rephrased some of the names of the items in this dataset, to make them more natu- ral. For instance, cherry (fresh fruit) was changed to cherry and hake (grilled with lemon juice) was changed to grilled hake with lemon juice.
Ranking with GPT-3 We use GPT-3 models of four different sizes* ada, babbage, curie and davinci. To rank items according to a given dimen- sion, we use prompt that contains the name of that dimension as the final word, e.g. for sweetness could use "It is known that [food item] tastes sweer". We then use the probability of this final word, conditioned on the rest of the prompt, to rank the item: the higher the probability of sweet, the more we assume the item to be sweet.
Pairwise Comparisons with GPT-3 To predict pairwise judgements, we consider two approaches. First, again use conditional probabilities. For instance, to predict whether an ant is larger than bird, we would get the conditional probability of large the sentences an ant is large and a bird is large. If the conditional probability we get from the first sentence is lower than the probability from the second sentence, we would predict that the claim that an ant is larger than a bird is false. Sec- ond, we use prompt that asserts the statement to be true (e.g. "An ant is larger than bird") and prompt that asserts the opposite (e.g. "A bird is larger than an ant"). We compute the perplexity of both statements and predict the version with the lowest perplexity to be the correct one.
We complement our analysis in the taste domain with experiments on three basic physical domains: mass, size and height. These were found to be particularly challenging by Li et al. (2023), with LLMs often failing to outperform random guessing. As the ground truth for mass, we use the house- hold dataset from Standley et al. (2017), which specifies the mass of 56 houschold objects. The original dataset includes images of each object. We removed 7 items which were not meaningful without the image, namely big elephant, small ele- phant, Ivan's 's phone, Ollie the monkey, Marshy the elephant, boy doll and Dali Clock, resulting in a dataset of 49 objects. We treat this problem as a ranking problem. Li et al. (2023) also created binary classification version of this dataset, which involves judging pairwise comparisons (e.g. is red lego brick heavier than a hammer?). For size and height, we use the datasets created by Liu et al. (2022b). These size and height datasets each con-
Ranking with ChatGPT and GPT-4 ChatGPT and GPT-4 are more difficult to than GPT- because the OpenAl API does not allow us compute conditional probabilities for these mod- els. Instead, to use these conversational models, we directly ask them to rank a set of items, using a prompt such as: Rank the following items accord- ing to their size, from the largest to the smallest, followed by a list of items to be ranked.
Baseline: DeBERTa We consider two baselines. First, we use DeBERTa-v3-large model (He et al.. 2021), which we fine-tuned to predict the com- monsense properties of concepts. To this end, we
'The exact model sizes have not been made public, but were estimated to be 350M parameters for ada, 1.3B parameters for babbage, 6.7B parameters for curie and 175B parameters for davinci; https: wallog.eleutttt eleuther. .ai/ gpt3 gpt3-moded model -sizes/-s

---

used the extended McRae dataset (McRae al.. 2005) introduced by Forbes et al. (2019) and the augmented version of CSLB introduced by Misra et al. (2022). Together, these two datasets contain 19,410 positive and 31,901 negative examples of (concept.propery pairs. We fine-tune the model on these examples using the following prompt: can concept be described as property? <MASK>, For instance, for the example (banana.yellow), the corresponding prompt would be: can banana be described as yellow? <MASK>. The probability that the concept has the property then predicted using a linear classifier that takes the final-layer embedding of the <MASK> token as input. We use the resulting model for the different evaluations, without any further fine-tuning.
Umami
Sweet
Salty
Sour
Bitter
Fatty
Ada
17.5
8.5
12.2
16.4
22.5
10.7
PROMPT
Babbage
19.5
51.1
20.2
22.0
22.6
16.0
Curie
36.0
46.3
32.8
23.2
22.6
31.7
Davinci
55.0
63.2
33.3
27.2
$7.0
52.0
Ada
23.1
11.9
8.5
16.8
-6.4
9.9
WOMMPT
Babbage
27.9
55.7
19.3
23.9
29.7
34.0
Curie
35.4
47.9
30.3
22.7
25.4
37.5
Davinci
50.2
54.4
34.6
28.3
49.8
42.1
DeBERTa
69.1
67.0
43,9
34.4
64.0
Bi-eneMsco-ORI
27.4
-4.4
15.4
14.4
-11.8
Bi-enectatirr
60.3
47.1
40.7
40.2
Table I: Ranking using conditional probability (Spear-
man p%)). Prompt1: "[food item] tastes [property]".
Prompt 2: "it is known that [food item] tastes [prop- crty]".
Baseline: Bi-encoder As the second baseline, use two variants of the bi-encoder model from Gajbhiye al. (2022). First, use the original BaRg-large model from Gajbhiye et al. (2022) that was trained on data from Microsoft Concept Graph (Ji et al., 2019) and Ge (Bhakthavatsalamm et al., 2020). However, as these training sets are not specifically focused on commonsense knowledge, we used ChatGPT to construct dataset of 109K (concept.property) pairs, since no existing dataset of sufficient size and quantity available. The key to obtain high-quality examples to ask the model suggest properties that are shared by sev- eral concepts, and to vary the examples that were provided as part of a few-shot prompting strategy. More details on how we collected this dataset us- ing ChatGPT are provided in Appendix A. We then trained the BERT-large bi-encoder on this dataset.
Item
Gold
Davinci
Cracker with Nutella spread Chocolate with nut Sweet pancake with maple syrup Fruit cake
5
324
15
201
34
279
41
265
Sweet conkies with chocolate Cracker with jam
60
233
67
268
Cooked bell pepper Redeurrant
252
282
36
Radish
431
97
Mascarpone cheese Cooked green cabbage Saint-agur Choese
477
38
512
49
584
61
Table 2:
Qualitative analysis of the predictions by davinci for sweetness, using prompt The table shows the rank positions of several food items, when ranking the items from the sweetest (rank 1) to the least sweet (rank 590), according to the ground truth and the predic- tions obtained with the Davinci model.
4 Experiments
the GPT-3 models, between model size and perfor- mance, with the best results achieved by davinci. Similar as observed for the colour domain by Liu et al. (2022a), there seems to be quali- tative change in performance between LLMs such as davinei and smaller models. While there are some differences between the two prompts, similar patterns are observed for both choices. ChatGPT and GPT-4 were not able to provide a ranking of the 590 items, and are thus not considered for this experiment. We also tried ChatGPT on subset of 50 items, but could not achieve results which were consistently better than random shuffling.
Taste Domain Table summarises the main re- sults on the taste dataset. For this experiment, each model was used to produce ranking of the 590 food items, which was then compared with the ground truth in terms of Spearman's 's rank corre- lation (p). For the LMs, we experimented with two prompts. Prompt
of the form "[food item] tastes [sweet]", e.g. "apple tastes sweet". Prompt is of the form "it is known that ffood item] tastes [sweet]". For the DeBERTa model and the bi- encoders, we verbalised the sweetness property "tastes sweet", and similar for the others. The results in Table show strong correlation, for
Surprisingly, we find that the DeBERTa model outperforms davinci most cases, and often by a substantial margin. This is despite the fact that
httpss:/ccllb.psthhhee aaychol.cam.... ac. uk/propnorms

---

Mass
Mass
Height
Size
for the properties heavy, tall and large.
P
Acc
Acc
Acc
The correlation between model size and per- formance is far from obvious here, except that ada clearly underperforms the three larger mod- els. However, among the GPT-3 models, babbage actually achieves the best results in several cases. The results based conditional probabilities are consistently better than those based on perplexity. ChatGPT and GPT-4 were difficult to use with the ranking prompt, as some items were missing, some were duplicated, and many items were paraphrased in the ranking. The results in Table 3 were obtained after manually correcting these issues. With this caveat in mind, it is nonetheless clear that GPT-4 performs exceptionally well in this experiment. In accordance with our findings in the taste domain, DeBERTa performs very well on the height and size properties, outperforming all GPT-3 models by a clear margin. For mass, however, DeBERTa failed completely, even achieving a negative cor- relation. The bi-encoder models perform well on height and size, although generally underperform- ing the largest GPT-3 models. For mass, the bi- encoder trained ChatGPT examples performs poorly, while the model trained on Microsoft Con- cept Graph and GenericsKB was more robust. It is notable that the results Table are considerably higher than those obtained by Li et al. (2023) using OPT (Zhang et al., 2022). For mass, for instance, even the largest OPT model (175B) was not able to do better than random guessing.
RANK PERPLENIEY EXPLEXITY COND. PROB.
Ada
23.0
47.8
68.7
59.1
Babbage
48.9
80.9
67.9
76.4
Curie
30.6
65.1
77.6
86.4
Duvinci
36.2
76.8
76.4
80.4
Ada
49.0
49.7
36.5
Babbage
55.0
59.1
66.7
Curie
56.6
43.3
45.5
Davinci
70.8
54.7
51.1
ChaIGPT
28.6
68.3
89.9
84.3
GPT-47
58.6
84.9
99.1
99.1
DeBERTa -8.9
42.8
86.6
93.9
Bi-eneMsco-ORE
31.1
69.2
69.7
71.9
Bi-enechaGrt 11.8
67.6
77.2
60.3
Table 3: Results for physical properties, viewed as a ranking problem (mass) and as pairwise judgment problem (mass, height and size). Prompt: "In terms of [mass/height/size]] it known that a typical [concept] is [heavy/tal//large]. Results with required manual post-processing of predictions.
the model is 2 to 3 orders of magnitude smaller. Furthermore, we can see a large performance gap between the two variants of the bi-encoder model, with the model trained ChatGPT examples out- performing curie, and even davinci in two cases.
Table 2 presents some examples of the predic- tions that were made by davinci for sweetness (us- ing prompt I), comparing the ranks according to the ground truth (gold) with the ranks according to the davinci predictions. The table focuses on some of the most egregious mistakes. As can be seen, davinci fails identify the sweetness of common foods such as chocolate, fruit cake and jam. Con- versely, the model significantly overestimates the sweetness of different cheeses and vegetables.
In Table 3, the pairwise judgments about mass were assessed by predicting the probability of the word heavy (for the GPT-3 models) or by predict- ing the probability that the property heavy was satisfied (for the baselines). Another possibility is to use the word/property light instead, or to com- bine the two probabilities. Let us write Pheavy to denote the probability obtained for heavy (i.c. the conditional probability of the word, as pre- dicted by the language model, the probability that the property is satisfied, as predicted by a base- line model), and similar for Plighr Then we can also predict the relative mass of items based on the value Pheavy (1 Plight) or based on the Plight. Pheavy/Plight-
Physical Properties Table 3 summarises the re- sults for the physical properties. For mass, con- sider both the problem of ranking all objects, evalu- ated using Spearman p%, and the problem of evalu- ating pairwise judgments, evaluated using accuracy, Height and size can only be evaluated in terms of pairwise judgments. To obtain conditional proba- bilities from the GPT-3 models, we used prompt of the form "In terms of [mass/heigh//size), it is known that typical [concept] is [heavy/talllargee]" We also tried a few variants, which performed worse. To compute perplexity scores, for evalu- ating pairwise judgements, we used a prompt of the form "[concept 1] is heavier/ualler/largerr than [concept 21". For the baselines, we obtained scores
These different possibilities are evaluated in Table As can be seen, there is no variant that consistently outperforms the others.
value Pheavy/
A nalysis of Training Data Overlap For the base- lines, we may wonder to what extent their knowl- edge comes from the pre- trained language model,

---

ada
babbage
curic
davinci
DeBERTa
Bi-enCMscG-CKR
Bi-enCehatrrt
Pheavy
46.6
81.7
64.7
76.8
67.7
69.2
42.9
1- Plight
49.8
51.3
41.5
53.4
43.4
61.9
53.4
Pheavy (1 plight)
46.6
81.7
64.7
76.8
47.8
69.2
48.1
Pheavy Plight
61.4
68.3
52.1
65.9
63.2
75.7
46.5
Table 4: Analysis of alternative strategies for predicting pairwise judgements about mass (accuracy).
Bitter
Sour
Mass
Height Acc
Size
tions about taste, but also showed that a fine-tuned DeBERTa model, and in some cases even fine- tuned BERT-large bi-encoder, can outperform GPT-
p
p
P
Acc
Full training
24.7
43.9
-8.9
86.6
93.9
Filtered training
24.8
35.0
30.7
82.0
90.8
3. The performance of these smaller models cru- cially depends on the quality of the available train- ing data. For this reason, we explored the idea of collecting training data from ChatGPT, using new prompting strategy. We complemented our exper- iments in the taste domain with evaluation of physical properties, where we achieved consider- ably better results than those reported in the litera- ture (Li et al., 2023). Whereas previous work was essentially aimed at understanding the limitations of language models, our focus was more practical, asking the question: can high-qualt conceptual space representations be distilled from LLMs? Our experiments suggest that the answer is essentially positive, but that new approaches may be needed to optimally take advantage of the knowledge that can be extracted from such models.
Table 5: Comparison of the DeBERTa model two settings: the full training setting, where the McRae and CSLB datasets are used for finettunig, and a filtered setting, where relevant properties are omitted.
and to what extent it has been injected during the fine-tuning step. For this analysis, we focus in particular on the DeBERTa model, which was fine- tuned on the McRae and CSLB datasets. These datasets indeed cover a number of physical prop- erties, as well as some properties from the taste domain. Table 5 summarises how the performance of the DeBERTa model is affected when removing the most relevant properties from the McRae and CSLB training sets, which we refer to as filtered training in the table. For instance, for the property bitter, in the filtered setting we omit all training examples involving the properties "bitter" and "can be bitter in taste"; for sour we remove the prop- crties "sour" and "can be sour in taste"; for mass we remove the properties "heavy", "light", "light weight" and "can be lightweight": for height we remove the properties "short", "can be short", "tall" and "can be tall"; and for size we remove the prop- erties "large" and "small". Note that the McRae and CSLB datasets do not cover any properties that are related to sweetness, saltiness, umami and fat- tiness. The results in Table 5 show that filtering the training data indeed has an effect results, although the performance of the model overall re- mains strong. Interestingly, in the case of mass, the filtered setting leads to clearly improved results.
Limitations
It is difficult to draw definitive conclusions about the extent to which cognitively meaningful rep- resentations can be obtained by querying LLMs. Among others, previous work has found that per- formance may dramatically differ depending on the prompt which is used; see c.g. (Liu et al., 2022a). We have attempted to make reasonable choices when deciding on the considered prompts, through initial experiments with few variations, but clearly this is not guarantee that our prompts are close to being optimal. However, this also reinforces the conclusion that LLMs are difficult use di- rectly for learning conceptual spaces. While we be- lieve that taste represents an interesting and under- explored domain, remains to be verified to what extent LLMs are able to capture perceptual features in other domains.
5 Conclusions
We proposed the use of a dataset from the taste domain for evaluating the ability of L.MMs to learn perceptually grounded representations. We found that LLMs can indeed make meaningful predic-
This work was supported by EPSRC grant EP/V025961/I.
Acknowledgments

---

References
concept graph: Mining semantic concepts for short text understanding. Data Intell., (3):238-270)
Mostafa Abdou, Artur Kulmizev, Daniel Hershcovich, Stella Frank, Ellie Pavlick, and Anders Søgaard. 2021. Can language models encode perceptual struc- ture without grounding? a case study in color. In Computa-
Lei Li, Jingjing Xu, Qingxiu Dong, Ce Zheng, Qi Liu, Lingpeng Kong, and Xu Sun. 2023. Can language models understand physical concepts? CORR, abs/2305.14057.
Proceedings of the 25th Conference tional Natural Language Learning. pages 109-132, Online. Association for Computational Linguistics.
Fangyu Liu, Julian Eisenschlos, Jeremy Cole, and Nigel Collier. 2022a. Do ever larger octopi still amplify reporting biases? evidence from judgments of typical colour. In Proceedings of the 2nd Conference of the Asia-Pucific Chapter of the Association for Compu- rational Linguistics and the 12th International Joint Conference on Natural Language Processing ( (Vol- ume 2: Short Papers), pages 210-220, Online only. Association for Computational Linguisties.
Emily M. Bender and Alexander Koller. 2020. Climbing towards NI.U: On meaning. form, and understanding the age of data. In Proceedings of the 58th Annual Meeting of the Association for Computational Lin- guistics, pages
Online. Association for Computational Linguistics.
Sumithra Bhakthavatsalam, Chloe Anastasiades, and Peter Clark. 2020. Genericskb: A knowledge base of generic statements. CoRR, abs/2005.00660.
Xiao Liu, Da Yin. Yansong Feng, and Dongyan Zhao. 2022h. Things not written in text: Exploring spatial commonsense from visual signals. In Proceedings of the 60th Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2365-2376, Dublin, Ireland. Association for Computational Linguistics.
Antonio Chella. 2015, Cognitive Architecture for Mu- sic Perception Exploiting Conceptual Spaces, pages 187-203. Springer International Publishing, Cham.
Maxwell Forbes, Ari Holtzman, and Yejin Choi. 2019. Do neural language representations learn physical commonsense? In Proceedings of the 41th Annual Meeting of the Cognitive Science Society, CogSci 2019: Creativity + Cognition+ + Computation, Mon- treal, Canada, July 24-27, 2019. pages 1753-1759. cognifivesienncssoiiett..gg
Christophe Martin, Michel Visalli, Christine Lange, Pas- cal Schlich, and Sylvie Issanchou. 2014. Creation of food taste database using an in-home "taste" profile method. Food Quality and Preference, 36:70-80.
Ken McRae, George S Cree, Mark S Seidenberg, and Chris McNorgan. 2005. Semantic feature production norms for large set of living and nonliving things. Behavior research methods, 37(4):547 559.
Amit Gajbhiye, Luis Espinosa-Anie Anke, and Steven Schockaert. 2022. Modelling commonsense proper- ties using pre-trained bi-encoders. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3971-3983, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Kanishka Misra, Julia Taylor Rayz, and Allyson Et- tinger. 2022. A property induction framework for neural language models. CoRR, abs/2205.00911
Cory Paik, Stéphane Areca- Proca-Ouelleete., Alessandro Ron- cone, and Katharina Kann. 2021. The World of an Octopus: How Reporting Bias Influences a Language Model's Perception of Color. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 823-835, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Peter Gärdenfors. 2000. Conceptual spaces- the geom- etry of thought. MIT Press.
Jonathan Gordon and Benjamin Van Durme. 2013. Re- porting bias and knowledge acquisition. In Proceed- ings of the 2013 workshop on Automated knowledge base construction, AKBC@ CIKM 13, San Francisco, California, USA, October 27-28, 2013, pages 25-30. ACM.
Carita Paradis. 2015. Conceptual Spaces at Work in Sensory Cognition: Domains, Dimensions and Dis- tances, pages 33-55. Springer International Publish- ing, Cham.
Peter Gärdenfors. 2004. Conceptual spaces as frame- work for knowledge representation. Mind and Mat- ter, 2(2):9227.
Roma Patel and Ellie Pavlick. 2022. Mapping language models to grounded conceptual spaces. In Interna- tional Conference on Learning Representations.
Peter Gärdenfors. 2014. The Geometry of Meaning: Semantics Based on Conceptual Spaces. The MIT
Press.
Anders Søgaard, 2023. Grounding the vector space of an octopus: Word meaning from raw text. Minds Mach., 33:1553355
Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2021. Debertav3: Improving deberta using electra-style pre- training with saddient-disentangle embedding shar- ing. CoRR, abs/2111 11122111.9943..
Trevor Standley, Ozan Sener, Dawn Chen, and Silvio Savarese. 2017. image2mass: Estimating the mass of an object from its image. 1st Annual Confer- ence on Robot Learning, CoRL. 2017, Mountain View,
Lei Ji, Yujing Wang, Botian Shi, Dawei Zhang, Zhongyuan Wang, and Jun Yan. 2019. Microsoft

---

California, USA, November 13-15, 2017, Proceed- ings, volume 78 of Proceedings of Machine Learning Research, pages 324-333. PMIR.
To verify the quality of the generated dataset, we manually inspected 500 of the generated concept- property pairs. In this sample, we identified 6 er- rors, which suggests that this dataset of sufficient quality for our intended purpose. Compared to re- sources such as ConceptNet, the main limitation of the ChatGPT generated dataset is that it appears to be less diverse, in terms of the concepts and properties which covered. We leave detailed analysis of this dataset for future work.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona T. Diab. Xian Li. Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shus- ter, Daniel Simig, Punit Singh Koura, Anjali Srid- har, Tianlu Wang, and Luke Zettlemoyer. 2022. OPT: open pre-trained transformer language mod- els. CoRR, abs/2205.01068.
Joost Zwarts. 2015, Conceptual Spaces, Features, and Word Meanings: The Case of Dutch Shirts, pages 57-78. Springer International Publishing, Cham.
B Issues with ChatGPT and GPT-4
For the experiments in Table 3, we used Chat- GPT and GPT-4 to rank 49, 25 and 26 unique objects according to their mass, height and size respectively. The prompt used was follows: "Rank the following objects based on their typical [mass/height/size), from [heaviest to the lightest/ tallest to the shortest/ largest to the smallest]". fol- lowed by a list of the items. We could not directly evaluate the responses of ChatGPT and GPT-4 be- cause of the following issues:
A Collecting Propepty Pairs using ChatGPT
To obtain training data for the bi-encoder model using ChatGPT, we used the following prompt:
am interested in knowing which prop- erties are satisfied by different concepis. am specifically interested in properties, such as being green, being round, be- ing located in the kitchen or being used for preparing food, rather than in hyper- nyms. For instance, some examples of what I'm looking for are: 1. Sunflower, daffodil, banana yellow 2. Guitar, banjo, mandolin are played by strum- ming or plucking strings 3. Pillow, blan- ket, comforter are soft and provide com- fort 4. Car, scooter, train have wheels 5. Tree, log, paper are made of wood 6. Study, bathroom, kitchen are located in house. Please provide me with list of 50 such examples.
Missing objects: For instance, GPT-4 ranked 48 out of 49 objects and ChatGPT ranked 46 out of 49 objects respectively, according their mass.
While ranking, both GPT-4 and ChatGPT changed some of the the names of the objects. For instance, "wooden train track" was renamed "wooden train track piece", "gardening shears" was renamed as "garden shears".
Paraphrasing:
Duplicates: GPT-4 and ChatGPT both occa- sionally introduced duplicates in the list.
To address these issues, we removed the dupli- cates and appended the missing items at the end of the ranking. We manually corrected the modified names.
We repeated this request with the same prompt around 10 times. After this, we changed the ex- amples that are given (shown in bold above). This process was repeated until we had 287K concept- property pairs. After removing duplicates, a total of 109K such pairs remained. We found it was nec- essary to regularly change the examples provided to ensure the examples were sufficiently diverse and to avoid having too many duplicates. These ex- amples were constructed manually, ensure their accuracy and diversity. Asking for more than 50 examples with one prompt became sub-optimal, as the model tends to focus on a narrow set of similar properties when the list becomes too long.


Reviews:
[{"review_id": "PcHCB6iwnY", "reviewer": "Reviewer_BTQu", "summary": "", "questions": "* The paragraph of \"Ranking with ChatGPT and GPT-4\" (starting at line 159): Did you run GPT several times to verify the responses are consistent?\n* The paragraph of \"Baseline: Bi-encoder\" (starting at line 185): Did you verify the generated data is satisfactory?\n* Why do you think that DeBERTa is better than GPT? Maybe some more comprehensive error analysis would be better here. \n* Table 2: I have no idea what these numbers mean. Why are they not normalized? ", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their careful review.\n\nSome error analysis about the results from Table 1 is presented in Table 2. This table shows the rank positions of 590 different food items, according to the ground truth and according to the Davinci predictions. For instance, the sweetest food item would receive a rank of 1. The examples illustrate how Davinci makes substantial and surprising errors, e.g. predicting chocolate or sweet cakes to be among the least sweet food items, while predicting vegetables such as cabbage to be among the most sweet food items. We will clarify how the table has to be interpreted, and add an expanded error analysis to the supplementary materials.\n\nThe point in the conclusions section about high-quality training data is not that it is required, but rather that it is sufficient, i.e. that smaller models can be surprisingly strong when the right kind of training data is available. We will rephrase the sentence to make this clearer. \n\nNote that it is quite common for conferences such as EMNLP to publish analysis papers. We thus believe that it is unfair to criticise the paper for not having “any real contribution apart from an interesting analysis”.\n\nThe full set of ChatGPT and GPT-4 results was only generated once, because we had to manually edit the responses, as explained in Appendix C. In initial experiments we found the results to be very stable, although there are of course small differences across different runs. However, these small differences would not affect our main conclusions, which are that GPT-4 and ChatGPT are difficult to use, but seem to perform at a level that is considerably above the other models when it is possible to use them. \n\nRegarding the training set that was generated using ChatGPT, please note that we have made the entire dataset available in the supplementary materials. We found this dataset to be highly accurate upon inspection: in a sample of 500 examples from the dataset, we only found 6 errors.\n\nThe main limitation of the GPT-based models is that there is no obvious way to directly exploit their latent representations. This means that we have to resort to proxies, such as evaluating the perplexity of artificially generated statements, which appears to be sub-optimal. Our hypothesis is therefore that such models are best used for generating training examples (which is much better aligned with their training objective), while BERT-based models may still be better for learning representations.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* This paper presents an interesting exploration of yet unexplored conceptual spaces. I like this human-inspired analysis direction.\n* The authors focus on an interesting commonsense problem that suffers deeply from reporting bias, and show where it is possible to trust LLMs. \n* The paper is mostly well-written. \n* The choice of models is satisfactory. ", "reasons_to_reject": "* Table 1: Some error analysis due to the surprising results would be much better than just stating them and acknowledging they are indeed surprising.\n* As I did not fully understand Table 2, I cannot judge it. \n* The conclusion \"The key to achieving good results with such smaller models is to have access to suitable training data.\" is rather trivial. I wonder what is the real contribution of this paper besides being an interesting analysis.  ", "questions_to_authors": "* The paragraph of \"Ranking with ChatGPT and GPT-4\" (starting at line 159): Did you run GPT several times to verify the responses are consistent?\n* The paragraph of \"Baseline: Bi-encoder\" (starting at line 185): Did you verify the generated data is satisfactory?\n* Why do you think that DeBERTa is better than GPT? Maybe some more comprehensive error analysis would be better here. \n* Table 2: I have no idea what these numbers mean. Why are they not normalized? ", "paper_topic_and_main_contributions": "This paper probes the question: can high-quality conceptual space representations be distilled from LLMs?\nThe authors examine an unexplored space of taste and add the physical properties space. They present a way to utilize GPT to artificially generate data. They find that DeBERTa is at least comparable to the much larger GPT-based models. ", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "* Lines 178-179: I would add an example to help the reader understand the data better. \n* Table 1: Mark in bold the best-performing model per column."}, "extra_text": {}}, {"review_id": "wfqtzZU6tJ", "reviewer": "Reviewer_k2sy", "summary": "", "questions": "N/A see weaknesses above", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their careful review.\n\nThe reviewer has misunderstood a fundamental point about our paper. Our aim is *not* to analyse whether the representations learned by LLMs themselves can be viewed as (being isomorphic to) conceptual spaces. We fully agree that for showing the latter, a different experimental methodology would be needed. Rather, we analyse whether LLMs can be used, in some way, to learn conceptual space representations (regardless of how concepts are internally represented by LLMs).\n\n“What makes a conceptual space a conceptual space is that it is not derived from text”: different assumptions are made about conceptual spaces by different authors. If we talk about conceptual spaces exclusively as a theory for describing cognitive phenomena, then perhaps this statement has merit. But conceptual spaces have long been studied, including by Gärdenfors (already in the 1990s), as a practical tool for integrating neural and symbolic representations. From this perspective, conceptual space representations can be learned from data (including text). What separates them from other types of vector representations is their structure, and in particular the use of perceptually meaningful dimensions/features.\n\n“All we can conclude is that LLMs representations are correlated with perceptual features”: This is indeed the central point of the analysis. If we can estimate perceptual features with sufficient accuracy, then we can learn meaningful approximations of conceptual spaces, which could play an important role in areas such as explainable AI.\n\nThe point in the introduction about evaluating representations is not about evaluating the internal representations of an LLM (see above), but about evaluating learned conceptual space representations. The idea is that we should go beyond using classification tasks, and instead try to evaluate quality dimensions more directly.\n\nTo verify the impact of overlapping properties, we have now carried out a few additional experiments, where relevant properties are removed from the McRae and CSLB datasets when training the DeBERTa model. Overall, we can see some impact on the performance, but our main conclusions remain unaffected (and in some cases the results of the DeBERTa model actually improve):\n\n* For the height experiment, we now removed the following properties \"short\", \"can be short\", \"tall\" and \"can be tall\" from McRae/CSLB. After removing these properties, the performance of the DeBERTa model drops from 86.6 to 82.0\n* For the size experiment, we now removed the properties \"large\" and \"small\", after which the performance dropped from 93.9 to 90.8\n* For the mass experiment, we now removed the properties \"heavy\", \"light\", \"light weight\" and \"can be lightweight\". This, in fact, led to an increase in performance from -8.9 to 30.7 (Spearman rho).\n* For the taste experiments, the properties \"bitter\" and \"can be bitter in taste\" were removed from McRae/CSLB, leading to a slight increase from 24.7 to 24.8 for the bitterness property. When removing the \"sour\" and \"can be sour in taste\" from McRae/CSLB, we observed a drop from 43.9 to 35.0 for sourness. No overlapping properties occur in McRae/CSLB for sweetness, saltiness, fattiness and umami.\n \nWe will add this analysis, to clarify the impact of overlapping properties in the pre-training datasets. However, we would argue that the presence of relevant properties is not in itself problematic. McRae and CSLB are general purpose pre-training datasets, which were selected independent of the considered evaluation tasks. Even if the pre-training datasets contain some relevant properties, we would expect LLMs to outperform this DeBERTa baseline. The fact that this is not always the case highlights the limitations of directly prompting LLMs for learning conceptual spaces.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "There is a lot of debate and discussion right now around if/how LLMs can encoded information about the nonlinguistic world. Connecting LLMs to theory on conceptual spaces is interesting and a worthwhile direction. The authors also introduce a new dataset on taste which seems interesting and others might like to study.\n", "reasons_to_reject": "I have some concerns/confusion around both the experimental design and the theoretical connections the authors want to make.\n\nTheory:\n* I am struggling a bit with how/why we should be interpreting the results of the experiments as evidence that LLMs encode conceptual spaces, and even more so with what we should actually require in order for something to count as having a conceptual space. From my understanding of the Gardenfors work, this is a line of distributional semantics theory that is notable because the dimensions of the vectors correspond to sensory (symbolic) primatives. It is often (as I have cited it and seen it cited) contrasted directly with distributional semantics models from NLP which are derived from text, because by-definition the dimensions of the space for text-based DSMs reflect word co-occurances and not perceptual primitives. So, this is to say: in some ways, no matter what, LLMs just by-definition are not conceptual spaces. What makes a conceptual space a conceptual space is that it is not derived from text. That said, I think your project could still be enlightening if you could show that, e.g., the space that the LLMs learn is _isomorphic_ to the conceptual space, similar to the Abdou et al paper. This seems like it would be a very interesting finding. But from what I can gather from your experiments, we cannot conclude that, since the experiments themselves just measure word co-occurances/LM probabilities directly. Thus, I think from the presented experiments, all we can conclude is that LLMs representations are correlated with perceptual features. This isn't necessarily news, but still could be worth documenting. However, I would like to see the paper rewritten with more nuance around the conceptual spaces theory, especially if published without changes to the experiments.\n\nExperimental design:\n* The methods you use to get rankings are arguably not measures of the representations directly (as you claim in the intro). Rather, all your experiments rely on e.g., LM continuations or perplexity scores. These don't tell us much about the representation of a word itself (e.g., the representation of the word \"cheese\") but rather tell us about how the model uses the representation to perform the task of language modeling. To make claims about the representations themselves, e.g., to try to show an isomorphism between the LLM space and the conceptual space, I think you need to work with the word representations themselves. This is not trivial in a multilayered contextualized LM (the way it used to be easy to do with e.g., word2vec). But I think you could find a way. Then, you might consider using something like relational similarity analysis (RSA) in order to measure whether the LLM space and the conceptual space have similar geometries.\n* Roberta performs well when fine-tuned on McRae. Have you looked at whether there is overlap between the facts included in McRae and those included in your test sets? I wouldn't be shocked if McRae contains some facts about size or taste...\n", "questions_to_authors": "N/A see weaknesses above", "paper_topic_and_main_contributions": "The goal of this paper is to determine whether LLMs encode conceptual spaces. Conceptual spaces (a theory associated with Gardenfors) are a type of distributional semantics but one in which the dimensions of the vectors correspond to symbolic perceptual primitives (e.g., hue and brightness for color). This paper asks whether LLMs encode conceptual spaces, looking at a new dataset in the taste domain, as well as some existing datasets on size and mass. The authors use various prompting techniques in order to get LLMs to rank objects along a given dimension (e.g., ranking foods according to sweetness) and then report correlations with human judgments. They show that LLMs have decent correlations (often in the 0.2-0.5 range) and that fine-tuned roberta performs the best.\n\nOverall, I really like the direction of this paper, but have some issues with the experimental design, and, more so, with the theoretical framing w.r.t. conceptual spaces. I elaborate on these concerns below. I'd prefer to see the paper go through a round of revisions before being published.", "excitement": 2, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "98MsKVb8OF", "reviewer": "Reviewer_bmxE", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their careful and supportive review.\n\nNote that our focus is not on showing that LLMs can “understand” the meaning of concepts. Rather, we address a practical question: can we learn meaningful conceptual space representations with the help of LLMs.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper is a nice demonstration of the capability of LLMs to learn semantic properties of objects.\n- The paper provides potentially useful data on which properties are easier or harder to learn (eg bitterness is harder than sweetness).\n- Models are prompted in a careful way, using probabilities of completions rather than just a sample, and using a ranking and a pairwise comparison methodology.\n- The results are straightforward to interpret.", "reasons_to_reject": "- The findings are unlikely to move the needle for people who are unconvinced that LLMs can meaningfully learn semantics.\n- The authors are not able to determine exactly why the LLMs succeed whereas smaller ones failed.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper seeks to discover whether the taste properties of objects, in addition to their physical qualities, can be be recovered from large language models. The question is interesting because earlier models did not show this ability due to reporting bias in the data. The authors find that GPT-3 and GPT-4 are able to reproduce human ratings on taste and physical properties with reasonable although varying accuracy. For taste properties, the GPTs are not as good as the supervised baselines.\n\n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Line 034 -> Gärdenfors"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper probes to what extent LMs encode conceptual spaces in the sense of Peter Gardenfors’ theory, using a new dataset on the taste domain. The works is interesting and focuses on the problem of how competence of LMs can be related to interpretable semantic dimensions, such as the ones in the Conceptual Space theory. On other hand, one reviewer raises important methodological issues I agree with. These concern the experiments and especially their interpretation, and  MUST be carefully addressed in the revised version of the paper.
Decision: Accept-Main