Ranking Entities along Conceptual Space Dimensions with LLMs: An Analysis of Fine-Tuning Strategies (2024)

Abstract: Conceptual spaces represent entities in terms of their primitive semantic features. Such representations are highly valuable but they are notoriously difficult to learn, especially when it comes to modelling perceptual and subjective features. Distilling conceptual spaces from Large Language Models (LLMs) has recently emerged as a promising strategy, but existing work has been limited to probing pre-trained LLMs using relatively simple zero-shot strategies. We focus in particular on the task of ranking entities according to a given conceptual space dimension. Unfortunately, we cannot directly fine-tune LLMs on this task, because ground truth rankings for conceptual space dimensions are rare. We therefore use more readily available features as training data and analyse whether the ranking capabilities of the resulting models transfer to perceptual and subjective features. We find that this is indeed the case, to some extent, but having at least some perceptual and subjective features in the training data seems essential for achieving the best results.1

heading: 1 Introduction
text: Knowledge graphs (KGs) have emerged as the de facto standard for representing knowledge in areas such as Natural Language Processing (Schneider et al., 2022), Recommendation (Guo et al., 2022) and Search (Reinanda et al., 2020). However, much of the knowledge that is needed in applications is about graded properties, e.g. recipes being healthy, movies being original or cities being kids-friendly. Such knowledge is easiest to model in terms of rankings: we can rank recipes according to how healthy they are even if we cannot make a hard decision about which ones are healthy and which ones are not. For this reason, we argue that con-
1Our pre-processed datasets and code can be found at https://github.com/niteshroyal/RankingUsingLLMs.
ceptual spaces (Gärdenfors, 2000) should be used, alongside knowledge graphs, in many settings.
A conceptual space specifies a set of quality dimensions, which correspond to primitive semantic features. For instance, in a conceptual space of movies, we might have a quality dimensions reflecting how original a movie is. Entities are represented as vectors, specifying a suitable feature value for each quality dimension. While the framework of conceptual spaces is more general, we will essentially view quality dimensions as rankings.
Conceptual spaces have the potential to play a central role in various knowledge-intensive applications. In the context of recommendation, for instance, they could clearly complement the factual knowledge that is captured by typical KGs (e.g. modelling the style of a movie, rather than who directed it), making it easier to infer user preferences from previous ratings. They could also be used to make recommendations more controllable, as in the case of critiquing-based systems, allowing users to specify feedback of the form “like this movie, but more kids-friendly” (Chen and Pu, 2012; Vig et al., 2012). Conceptual spaces furthermore serve as a natural interface between neural and symbolic representations (Aisbett and Gibbon, 2001), and may thus enable principled explainable AI methods.
However, the task of learning conceptual spaces has proven remarkably challenging. The issue of reporting bias (Gordon and Durme, 2013), in particular, has been regarded as a fundamental obstacle: the knowledge captured by conceptual spaces is often so obvious to humans that it is rarely stated in text. For instance, the phrase “green banana” is more frequent in text than “yellow banana” (Paik et al., 2021), as the colour is typically not specified when yellow bananas are discussed. Paik et al. (2021) found that predictions of Language Models (LMs) about the colour of objects were correlated with the distribution of colour terms in text corpora, rather than with human judgements, suggesting that
7974
LMs cannot overcome the challenges posed by reporting bias. However, Liu et al. (2022a) found that larger LMs can perform much better on this task. Going beyond colour, Chatterjee et al. (2023) evaluated the ability of LLMs to predict taste-related features, such as sweetness and saltiness, obtaining mixed results: the rankings predicted by LLMs, in a zero-shot setting, had a reasonable correlation with human judgments but they were not consistently better than those produced by a fine-tuned BERT (Devlin et al., 2019) model.
In this paper, we analyse whether LLMs can be fine-tuned to extract better conceptual space representations. The difficulty is that ground truth rankings are typically not available when it comes to perceptual and subjective features, outside a few notable exceptions such as the aforementioned taste dataset. We therefore explore whether more readily available features can be used for fine-tuning the model. For instance, we can obtain ground truth rankings from Wikidata entities with numerical attributes (e.g. the length of rivers, the birth date of people, or the population of cities) and then use these rankings to fine-tune an LLM. We furthermore compare two different strategies for ranking entities with an LLM: the pointwise approach uses an LLM to assign a score to each entity, given some feature, while the pairwise approach uses an LLM to decide which among two given entities has the feature to the greatest extent. Our contributions and findings can be summarised as follows:
• We evaluate on three datasets which have not previously been used for studying language models: a dataset of rocks, a dataset of movies and books, and a dataset about Wikidata entities. We use these datasets alongside datasets about taste (Chatterjee et al., 2023) and physical properties (Li et al., 2023).
• We analyse whether fine-tuning LLMs on features from one domain (e.g. taste) can improve their ability to rank entities in different domains (e.g. rocks). We find this indeed largely to be the case, as long as the training data also contains perceptual or subjective features.
• We compare pointwise and pairwise approaches for ranking entities with LLMs. Despite the fact that pairwise approaches have consistently been found superior for LLMbased document ranking (Nogueira et al., 2019; Gienapp et al., 2022; Qin et al., 2023),
when it comes to comparing entities, we find the pointwise approach to be highly effective, although the best results overall are still obtained using a pairwise model.
• To obtain rankings from pairwise judgments, we need a suitable strategy for aggregating these judgments. We show the effectiveness of an SVM based strategy for this purpose. While this strategy is known to have desirable theoretical properties, it has not previously been considered in the context of language models, to the best of our knowledge

heading: 2 Related Work
text: LMs as Knowledge Bases Our focus in this paper is on extracting knowledge from language models. This idea of language models as knowledge bases was popularised by Petroni et al. (2019), who showed that the pre-trained BERT model captures various forms of factual knowledge, which can moreover be extracted using a simple prompt. Work in this area has focused on two rather distinct goals. On the one hand, probing tasks, such as the one proposed by Petroni et al. (2019), have been used as a mechanism for analysing and comparing different language models. On the other hand, extracting knowledge from LMs has also been studied as a practical tool for building or extending symbolic knowledge bases. This has been particularly popular for capturing types of knowledge which are not commonly found in traditional knowledge bases, such as commonsense knowledge (Bosselut et al., 2019; West et al., 2022; Yu et al., 2023). Several works have focused on distilling KGs from language models (Cohen et al., 2023). Hao et al. (2023) studied this problem for non-traditional relations such as “is capable of but not good at”. Along the same lines, Ushio et al. (2023) have focused on modelling relations that are a matter of degree, such as “is a competitor of” or “is similar to”. We can similarly think of the conceptual space dimensions that we consider in this paper as gradual properties.
Where the aforementioned approaches explicitly extract knowledge from an LM, the knowledge captured by LMs has also been used implicitly, by applying such models in a wide range of knowledgeintensive applications, including closed-book question answering (Roberts et al., 2020), knowledge graph completion (Yao et al., 2019), recommendation (Sun et al., 2019; Geng et al., 2022), entity typing (Huang et al., 2022) and ontology alignment
(He et al., 2022), to name just a few.
Conceptual Space of LMs There is an ongoing debate about the extent to which LMs can truly capture meaning (Bender and Koller, 2020; Abdou et al., 2021; Patel and Pavlick, 2022; Søgaard, 2023). Within this context, several authors have analysed the ability of LMs to predict perceptual features. As already mentioned, Paik et al. (2021) and Liu et al. (2022a) analysed the ability of LMs to predict colour terms. Abdou et al. (2021) analysed whether the representation of colour terms in LMs can be aligned with their representation in the CIELAB colour space. Patel and Pavlick (2022) similarly showed that LLMs can generate colour terms from RGB codes in a few-shot setting, even if the codes represent a rotation of the standard RGB space. They also show a similar result for terms describing spatial relations. Zhu et al. (2024) have similarly shown that LLMs can understand colour codes, by using them to generate HSL codes for everyday objects, or by asking models to choose the most suitable code among two alternatives.
Beyond the colour domain, Li et al. (2023) considered physical properties such as height or mass. While they found LLMs to struggle with such properties, Chatterjee et al. (2023) reported better results on the same datasets, especially when using GPT-4. Focusing on visual features, Merullo et al. (2023) showed that the representations of concepts in vision-only and text-only models can be aligned using a linear mapping. Chatterjee et al. (2023) focused on the taste domain, modelling properties such as sweet. They found that GPT-3 can model such properties to a reasonable extent, but not better than a fine-tuned BERT model.
Gupta et al. (2015) already considered the problem of modelling gradual properties in the context of static word embeddings, although their analysis was limited to objective numerical features. Derrac and Schockaert (2015) similarly learned conceptual space dimensions for properties such as “violent” in a semantic space of movies, while Rubinstein et al. (2015) assessed the ability to predict (gradual) commonsense properties from word embeddings. More recently, Grand et al. (2022) also used linear projections to map word embeddings to dimensions capturing gradual feature (e.g. size). Erk and Apidianaki (2024) argue for the importance of using human ratings to learn such projections. Note that all these approaches essentially learn a linear classifier or regression model for each prop-
erty independently, and can thus not generalise to new properties. A related line of work has focused on modelling the rankings associated with scalar adverbs (Lorge and Pierrehumbert, 2023) and adjectives (Samir et al., 2021).

heading: 3 Extracting Rankings
text: We consider the following problem: given a set of entities E and a feature f , rank the entities in E according to the their value for the feature f . In some cases, f will refer to a numerical attribute. For instance, E may be a set of countries and f the population of a country, where the task is then to rank the countries according to their population. In other cases, f will rather refer to a gradual property. For instance, E may be a set of food items and f may be the level of sweetness. Let us write f(e) for the value of feature f for entity e.
We consider two broad strategies for solving the considered ranking task with LLMs. First, we can use LLMs to map each entity e to some score w(e), with the assumption that w(e1) < w(e2) iff f(e1) < f(e2). This pointwise approach to learning to rank is considered in Section 3.2. Second, we can use LLMs to solve a binary classification problem: given two entities e1 and e2, decide whether f(e1) < f(e2) holds. This pairwise approach needs to be combined with a strategy for aggregating the LLM predictions into a single ranking. The main disadvantage is that a large number of judgements need to be collected for this to be effective, which means that such approaches are less efficient than pointwise strategies. However, in the context of document retrieval, pairwise approaches have been found to outperform pointwise approaches (Nogueira et al., 2019; Gienapp et al., 2022; Qin et al., 2023). We discuss pairwise and pointwise strategies in Sections 3.1 and 3.2 respectively. Finally, Section 3.3 describes how we establish baseline results using ChatGPT and GPT4.

heading: 3.1 Pairwise Model
text: The problem of predicting whether f(e1) < f(e2) holds can be straightforwardly cast as a sequence classification problem. To this end, we use a prompt of the following form:
This question is about two [ENTITY TYPE]: [Is/Does/Was] [ENTITY 1] [COMPARATIVE FEATURE] than [ENTITY 2]?
Note that the exact formulation depends on the type
of feature which is used for ranking. For instance, some instantiations of the prompt are as follows:
• This question is about two rivers: Is River Thames longer than Seine?
• This question is about two companies: Was Meta founded after Alphabet?
• This question is about two food items: Does banana taste sweeter than chicken?
In initial experiments, we used prompts with a more uniform style (e.g. “should [ENTITY 1] be ranked higher than [ENTITY 2] in terms of [FEATURE]”). However, this inevitably leads to less natural sounding prompts for certain features, which may affect performance. Moreover, such prompts were sometimes found to be ambiguous (e.g. does “ranked higher in terms of date of birth” mean younger people should be ranked highest?). To obtain judgments about entity pairs, we use a standard sequence classification approach, where a linear layer with sigmoid activation is applied to the final hidden state. The model is trained using binary cross entropy using a set of training examples.
Aggregating judgments We typically want to rank a given set of entities, rather than judging the relative position of two particular elements. This means that we need a strategy for aggregating (noisy) pairwise judgments into a single ranking. This problem has received extensive attention in the literature, where standard techniques include spectral ranking (Vigna, 2016) and maximum likelihood estimation w.r.t. an underlying statistical model. However, existing approaches often consider a stochastic setting, where we may have access to several judgments for the same entity pair (e.g. when ranking sports teams based on the outcomes of head-to-head matches).
Our setting is slightly different, as we can realistically only obtain judgments for a small sample of entity pairs. In particular, we ideally need methods with Ω(n) sample complexity, i.e. methods that can perform well with a number of judgements that is linear in the number of queries. Wauthier et al. (2013) discuss two such methods. Let us write e1, ..., en for the entities to be ranked. The first method uses a linear SVM to learn a weight vector w = (w1, ..., wn). Let xi be an n-dimensional one-hot vector, which is 1 in the ith coordinate and 0 elsewhere. If we have a pairwise judgement
f(ei) < f(ej) then this is translated into the constraint that w(xj − xi) > 0. A standard SVM can then be used to find the vector w that maximises the margin between positive and negative examples. Entity ei is ranked based on its corresponding weight wi. The second method simply scores each entity ei based on the number of pairwise comparisons where ei was ranked higher/lower. Specifically, let us define sij = 1 if entities ei and ej have been compared, and sij = 0 otherwise. Furthermore, we define cij = 1 if f(ei) > f(ej), according to a pairwise comparison that was made, and cij = −1 otherwise. Then we can choose the weights as:
wi = ∑ j ̸=i sijcij∑ j ̸=i sij
We will refer to this strategy as Count.

heading: 3.2 Pointwise Model
text: For the pointwise model, we need to learn a scoring function w : E → R. To this end, we use a prompt of the following form:
Is [ENTITY 1] among [SUPERLATIVE FEATURE] [ENTITY TYPE]?
For instance, Is River Thames among the longest rivers? For each entity ei, we obtain a score w(ei) ∈ R by applying a linear layer to the final hidden state. Intuitively, w(ei) captures the (latent) quality of ei w.r.t. the considered feature. Since we cannot obtain ground truth labels for this score, we again rely on pairwise comparisons for training the model. Specifically, we estimate the probability pij that f(ei) > f(ej) holds as:
pij = σ ( w(ei)− w(ej) )
Then we use binary cross entropy as follows:
L = − (∑
i ̸=j tij log pij + (1− tij) log(1− pij)
)
where tij = 1 if f(ei) > f(ej) and tij = 0 otherwise, and the summation ranges over all distinct entity pairs ei, ej within the given mini-batch. Note that while we use pairwise comparisons for training the model, it is still a pointwise approach as the model produces scores for individual entities.

heading: 3.3 Baselines
text: To put the performance of the fine-tuning strategies from Sections 3.1 and 3.2 into context, we compare them with two conversational models: ChatGPT (gpt-3.5-turbo) and GPT-4 (gpt-4). We use both models in a zero-shot setting. For this purpose, we use the same prompt as in Section 3.1 but append the sentence Only answer with yes or no. Despite this instruction, the models occasionally still generated a different response, typically expressing that the question cannot be answered. For such entity pairs, we replace the generated response with a randomly generated label (yes or no).2

heading: 4 Datasets
text: In our experiments, we will rely on the following datasets, either for training or for testing the models. Each dataset consists of a number of rankings, where each ranking is defined by a set of entities and a feature along which the entities are ranked.
Wikidata We have obtained 20 rankings from numerical features that are available on Wikidata3. For instance, we obtained a ranking of rivers by comparing their length.4 If there were more than 1000 entities with a given feature value, we selected the most 1000 popular entities. To estimate the popularity of an entity, we use their QRank5, which counts the number of page views of the corresponding entry in sources such as Wikipedia. For the entity type person, we limited the analysis to people born in London (which made it possible to retrieve the required information from Wikidata more efficiently). We similarly only considered museums located in Italy. For some experiments, we split the collected data in two datasets, called WD1 and WD2. This will allow us to test whether models trained on one set of features (i.e. WD1) generalise to a different set of features (i.e. WD2). WD1 contains rankings which were cut off at 1000 elements, whereas WD2 contains rankings with fewer elements. We will write WD to refer to the full dataset, i.e. WD1 and WD2 combined.
Taste Following Chatterjee et al. (2023), we use a dataset with ratings about the taste of 590 food
2Statistics about how often this was needed can be found in the appendix.
3https://www.wikidata.org/wiki/Wikidata: Main_Page
4The entity types and corresponding features are listed in the appendix.
5https://qrank.wmcloud.org
items along six dimensions: sweetness, sourness, saltiness, bitterness, fattiness and umaminess. The dataset was created by Martin et al. (2014), who used a panel of twelve experienced food assessors to rate the items. We use the version of the dataset that was cleaned by Chatterjee et al. (2023), who altered some of the descriptions of the items to make them sound more natural in prompts.6
Rocks Nosofsky et al. (2018) created a dataset of rocks, with the aim of studying how cognitively meaningful representation spaces for complex domains can be learned. A total of 30 rock types were studied (10 igneous rocks, 10 metamorphic rocks and 10 sedimentary rocks). For each type of rock, 12 pictures were obtained, and each picture was annotated along 18 dimensions. However, only 7 of the considered dimensions allow for ranking all types: lightness of colour, average grain size, roughness, shininess, organisation, variability of colour and density. For our experiments, we only considered these dimensions. The dataset from Nosofsky et al. (2018) contains ratings for each of the 12 pictures of a given rock type, where each picture was assessed by 20 annotators. To construct rankings of rock types, we average the ratings across the 12 pictures. As such, we end up with 7 rankings of 30 rock types.
Tag Genome Vig et al. (2012) collected a dataset7 of movies, called the Tag Genome, by asking annotators to what extent different tags apply to different movies, on a scale from 1 to 5. From these tags, we first selected those that correspond to adjectives and for which ratings for at least 15 movies were available. We then manually identified 38 of these adjectives which correspond to ordinal features. More recently, Kotkov et al. (2022) created a similar dataset for books. We again selected adjectives for which at least 15 items were ranked, and manually identified 32 adjectives that correspond to ordinal features. A list of the adjectives that we considered is provided in the appendix. It should be noted that most items are only judged by a single annotator, and the judgements were moreover obtained using crowdsourcing. The movies and books datasets are thus clearly noisier than the taste and rocks datasets. For this reason, we
6Available from https://github.com/ ExperimentsLLM/EMNLP2023_PotentialOfLLM_ LearningConceptualSpace.
7Available from https://grouplens.org/datasets/ movielens/tag-genome-2021/.
will only consider aggregated results across all tags when evaluating on these datasets. We will write TG to refer to the combined dataset, containing both the books and movies rankings.
Physical Properties Following Li et al. (2023), we consider three physical properties: mass, size and height. The ground truth for mass dataset was obtained from a dataset about household objects from Standley et al. (2017). Following Chatterjee et al. (2023), we removed 7 items, because their mass cannot be assessed without the associated image: big elephant, small elephant, Ivan’s phone, Ollie the monkey, Marshy the elephant, boy doll and Dali Clock. The resulting dataset has 49 items. For size and height, we use the datasets from Liu et al. (2022b) as ground truth. These datasets each consist of 500 pairwise judgements.

heading: 5 Experiments
text: We now evaluate the performance of the fine-tuning strategies on the considered datasets.8
Comparing Models Table 1 compares a number of different models. We test four different LLMs: the 7B and 13B parameter Llama 2 models, the 8B parameter Llama 3 model9, and the 7B parameter
8Our datasets, code and pre-trained models will be shared upon acceptance.
9We use the llama-2-7b-hf, llama-2-13b-hf, and meta-llama/Meta-Llama-3-8B models available from https://huggingface.co/meta-llama.
Mistral model10. We evaluate the different models in terms of their accuracy on pairwise judgements. To this end, for a given dataset, we randomly sample pairs of entities ei, ej and construct queries asking whether f(ei) > f(ej). For WD, Taste and Rocks, we sample 500 such pairs for each of the features. Since the TG dataset has a total of 70 features, we limit the test set to 100 pairs per feature. For this analysis, we have split the WD1 dataset into two parts: 80% of the entities, for each feature, are used for training the models. The remaining 20% are used as a test set. All models are fine-tuned on the training split of WD1 (apart from the baselines, which are evaluated zero-shot). This allows us to see how well the models perform on the features they were trained on (for WD1-test) and how they generalise to unseen properties.
The aim of the analysis in Table 1 is to assess whether models can be successfully fine-tuned using a relatively small training set (i.e. WD1-train), involving only well-defined numerical features. In particular, we want to test whether models which are fine-tuned on such features would also generalise to more subjective and less readily available ones, similar to the easy-to-hard generalisation that has been observed for LLMs in other tasks (Hase et al., 2024). The results show that this is indeed the case, at least to some extent. Overall, we can
10We use the mistral-7b-v0.1 model available from https://huggingface.co/mistralai/Mistral-7B-v0. 1.
see that Mistral-7B achieves the best results among the pointwise models, while Llama3-8B achieves the best results among the pairwise models. The pointwise Mistral-7B model outperforms the pairwise Mistral-7B model, which is surprising given that pairwise models generally perform better in ranking tasks. The performance of the models across different features is not always consistent. Each model achieves close to random chance on some of the features, but the features where one model performs poorly are not always the same features where other models perform poorly. However, for bitterness and roughness, the accuracy of all models is below 61%. Furthermore, sourness and organisation also stand out as being more challenging. Regarding the baselines, GPT-4 generally performs better than the fine-tuned models. Nonetheless, the pairwise Llama3-8B and pointwise Mistral-7B models outperform GPT-4 in several cases. ChatGPT performs worse on most fea-
tures, but achieves the best results for books.
Comparing Training Sets The fine-tuned models in Table 1 generally underperform GPT-4. This may be partially explained by the fact that a small training set was used, which moreover only covered numerical features. In Table 2, we evaluate the impact of using different training sets. For this analysis, we use the pairwise Llama3-8B model, the best fine-tuned model in Table 1. Our focus is on seeing whether models trained on one domain can generalise to other domains. To put the results in context, we also compare with a model that is not fine-tuned but includes three in-context demonstrations (shown as few-shot); see Appendix A for details. The results are again evaluated in terms of accuracy, using the same pairwise judgements as for Table 1. WD refers to the full dataset (including both the training and test splits of WD-1).
We can see in Table 2 that training on larger datasets indeed leads to considerably better results. While this is not unexpected, we can also make more striking observations. For instance, the model that was trained on WD+TG+Taste achieves strong results on Rocks, despite covering very different features. Similarly, the model that was trained on TG alone achieves strong results for both Taste and Rocks. This suggests that the fine-tuned models are indeed capable of generalising to unseen domains. However, to achieve strong results, it appears to be important that the training data contains subjective or perceptual features. In particular, training on WD alone leads to clearly worse results on Taste and Rocks. The best results in Table 2 generally outperform the GPT-4 results from Table 1. As the training and test sets cover disjoint domains, the results reflect the knowledge that is captured
by the LLMs themselves, rather than knowledge that was injected during the fine-tuning process. This suggests that pre-trained LLMs capture more perceptual knowledge than it may initially appear.
Comparing Ranking Strategies Table 3 compares different strategies for generating rankings. The pointwise model can be directly used for this purpose. For the pairwise model, we show results with the SVM strategy and the Count strategy. We furthermore vary the number of pairwise judgments per entity (5 or 30) for Count and SVM. For this experiment, we use the pointwise and pairwise Llama3-8B models that were trained on WD+TG+Rocks. We evaluate the different models by comparing the predicted rankings with the ground truth in terms of Spearman ρ. We can see that the pairwise approaches consistently outperform the pointwise model. The SVM method generally performs better than the Count method, especially in the case where 30 judgments per entity are obtained. We also compare with the GPT-3 results reported by Chatterjee et al. (2023), finding that the pairwise Llama model consistently performs best.
Qualitative Analysis Table 4 shows the 10 highest and lowest ranked entities for some selected features, according to the rankings from the SVM method with the pairwise Llama3-8B model. The results for sweetness and saltiness were obtained with the model that was trained on WD+TG+Rocks.
The rankings for scary and funny movies were obtained with the model that was trained on WD+Taste+Rocks. The ranking for population was obtained with the model that was trained on WD1. The table shows that the model was successful in selecting these top and bottom ranked entities. The top-ranked entities for sweetness, for instance, are all clearly sweet food items, while none of the bottom ranked entities are. Similar observations can be made for the other features. The model is sometimes less successful in distinguishing middleranked entities from bottom-ranked entities. For instance, most cheeses appear at the bottom of the ground truth ranking, whereas the model predicted these to be somewhere closer to the middle.11 For population, we can see that while the top-ranked entities are all countries with a high population, their relative ranking is not accurate.
To further test the ability of the model to generalise to unseen properties, Table 5 shows results that were obtained for a number of selected features, focusing primarily on commonsense properties. For instance, for the first example, we compared a number of items according to their suitability as a present for a 10-year-old girl. To obtain the rankings, we exhaustively compared every pair of entities (from the considered set) and used the SVM method. The results were obtained with the
11A more detailed analysis can be found in the appendix.
pairwise Llama3-8B model that was trained on WD+TG+Rocks. These examples illustrate that the model often struggles with commonsense features. For instance, spaceship is listed as the most suitable vehicle for long-distance travel with a family, the school location is considered to be the least important criterion for selecting a primary school and a bathroom renovation is considered to be more urgent than fixing a leak. The disappointing results for commonsense features are somewhat surprising, although Chatterjee and Schockaert (2023) similarly found that ChatGPT struggled with commonsense properties.

heading: 6 Conclusions
text: We have studied the problem of ranking entities along conceptual space dimensions, such as sweetness (for food), roughness (for rocks) or scary (for movies). We found that fine-tuning LLMs on data from one domain (e.g. taste) is a viable strategy for learning to extract rankings in unrelated domains (e.g. rocks), as long as both domains are perceptual. In contrast, LLMs that were fine-tuned on objective numerical features from Wikidata were
less successful when applied to perceptual domains. When comparing pairwise and pointwise strategies, surprisingly, we found that pointwise methods were as successful as pairwise methods for making pairwise judgements (i.e. should entity e1 be ranked before entity e2), although pairwise methods still had the advantage when such judgments were aggregated. Overall, our results suggest that the current generation of open-source LLMs, such as Llama and Mistral, can be effectively used for constructing high-quality conceptual space representations. However, further work is needed to construct more comprehensive training sets. Encouragingly, we found that subjective (and relatively noisy) rankings, such as those from the movies and books datasets, can also be effective, while being much easier to obtain than perceptual features.
Acknowledgments This work was supported by EPSRC grants EP/V025961/1 and EP/W003309/1.
Limitations
The performance of LLMs is highly sensitive to the prompting strategy. While we have made efforts to choose a reasonable prompt, it is likely that better
results are possible with different choices. Furthermore, while we have tested a number of different LLMs, it is possible that other (existing or future) models of similar sizes may behave qualitatively different. Care should therefore be taken when drawing any conclusions about the limitations of LLMs in general. Moreover, the limitations we have identified might be particular to the specific fine-tuning techniques that we have used, rather than reflecting limitations of the underlying LLMs. When it comes to modelling subjective features, such as those in the movies and books datasets, it is important to acknowledge that people may have different points of view. When using conceptual space representations extracted from LLMs in downstream applications, we thus need to be aware that these representations are biased and, at best, can only represent a majority opinion.

heading: A Further Details
text: Training Details To train the four base models, we used the QLoRa method, which allows converting the floating-point 32 format to smaller data types. In particular, for all three models, we used 4- bit quantization for efficient training. In the QLoRa configuration, r (the rank of the low-rank matrix
used in the adapters) was set to 32, α (the scaling factor for the learned weights) was set to 64, and dropout was set to 0.05. We applied QLoRa to all the linear layers of the models, including q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, and lm_head. The models were trained with a batch size of 8. We used 20% of the WD1 training split as a validation set. Based on this validation set, we fixed the number of training steps to 12500 for the pairwise models and 1500 for the pointwise models. Note that we need fewer training steps for the pointwise model, because each minibatch consists of 8 entities, and we consider all pairwise combinations of these entities. In contrast, for the pairwise model, each mini-batch consists of 8 pairwise combinations. We also observed that the pointwise model converges more quickly than the pairwise model.
OpenAI Models Table 6 shows for how many cases ChatGPT and GPT-4 failed to answer with yes or no, when asked about pairwise comparisons. Overall, such cases were rare. The highest number of failures was seen for TG dataset, which appears to be related to the subjective nature of the features involved.
Datasets Table 7 gives an overview of the properties that were selected for the WD1 and WD2 datasets, along with the corresponding number of entities. Table 8 and 9 similarly show the tags that have been considered for the Movies and Books datasets, along with the number of corresponding entities.
Few-shot Baseline For the few-shot configuration in Table 2, we use the pre-trained Llama3-8B model with the following prompt: Answer the following with Yes or No only. In the
worst case, if you do not know the answer then choose randomly between Yes and No.
This question is about two rivers: Is Nile longer than Indus? Yes
This question is about two countries: Is Japan more populated than India? No This question is about two countries: Is China
larger than India? Yes
Note that the same three in-context examples were provided for all datasets.

heading: B Additional Analysis
text: Analysis using Llama2-13B In the main paper, we primarily used Llama3-8B for the experiments. Tables 11 and 12 show the corresponding results for Llama2-13B, focusing respectively on analysing the impact of using different training sets and on comparing different ranking strategies. Table 14 shows the 10 highest and lowest ranked entities, according to the rankings from the SVM method with the pairwise Llama2-13B model. These examples can be contrasted with the Llama3 results from Table 4.
Error Analysis Table 10 presents an error analysis for the same five rankings that were considered in Table 14. Specifically, in Table 10 we focus on the entities where the difference between the predicted ranking position and the position of the entity in the ground truth ranking is highest. On the left, we show entities which are ranked too high (i.e. where the model predicts the entity has the feature to a greater extent than is the case according to the ground truth). On the right, we show entities which are ranked too low. In the case of sweetness, we can see that the model consistently ranks cheeses to high. They are predicted to be in ranking positions 150-250, whereas the ground truth puts them at 500-590. In the case of saltiness, we can see that sweet drinks and pastries are ranked too high. For instance, cola soda is ranked in po-
sition 108 whereas the ground truth puts it at 517 (out of 590). Overall, these results suggest that the model struggles with certain food groups. For the features scary, funny and population, clear patterns are harder to detect.
Impact of Entity Popularity The reliability of LLMs when it comes to modelling entity knowledge has been found to correlate with the popularity of the entities involved (Mallen et al., 2023). To analyse this aspect, Figure 1 compares entity popularity with prediction error, for the countries population feature from the WD2 dataset. For this analysis, we have used the pairwise Llama2-13B model that was trained on the WD1-training split. We obtained a ranking of all countries using the SVM method with 20 samples. On the X-axis, the entities are ranked from the most popular to the least popular. On the Y-axis, we show the prediction error for the corresponding entity, measured as the difference between the position of the entity in the predicted ranking and its position in the ground truth ranking. Based on this analysis, no clear correlation between entity popularity and prediction error can be observed.
Accuracy Variability Table 13 shows standard deviations across four runs for one of the bestperforming models. As can be seen, the standard deviations are relatively small, indicating that the main differences across the models are not due to random variations.

heading: Future Work
text: Furthermore, while we have tested a number of different LLMs, it is possible that other (existing or future) models of similar sizes may behave qualitatively different