Towards Olfactory Information Extraction from Text: A Case Study on Detecting Smell Experiences in Novels (2020)

Abstract: Environmental factors determine the smells we perceive, but societal factors factors shape the importance, sentiment and biases we give to them. Descriptions of smells in text, or as we call them ‘smell experiences’, offer a window into these factors, but they must first be identified. To the best of our knowledge, no tool exists to extract references to smell experiences from text. In this paper, we present two variations on a semi-supervised approach to identify smell experiences in English literature. The combined set of patterns from both implementations offer significantly better performance than a keyword-based baseline.

## introduction we rely on our senses: touch, taste, hearing, sight and smell; to complement one another in shaping our interpretation of our environment. there is shifting historical relevance placed on smell -its worthiness for attention, its association with social standing, lifestyle, emotion, science and superstitions, and other topical associations shifting with time @xcite . english language vocabulary specific to the description of smell experiences is not expansive, and, to the best of our knowledge, language technology to identify references to smell in text even less so. a topic search in the cambridge dictionary online of words categorised as relating to smells and smelling foot_0 returns fewer than 30 words, that are predominantly concerned with intensity or sentiment such as fetid and reek. other characteristics of smell are instead often described in terms of reference smell sources as similes such as there is a strange unwholesome smell upon the room, like mildewed corduroys. in this paper, we present a dataset of annotated references to smell, which we call 'smell experiences' in literature, as well as a first approach and experiments to automatically recognise references to smells in texts. the remainder of this paper is organised as follows. in section 2, we discuss related work. in section 3, we describe our corpus and its creation process. in section 4, we present our extraction approach and experiments. in section 5, we discuss the results. we conclude with section 6 in which we present our conclusions and directions for future work. our data is available at http://doi.org/10.5281/zenodo.4199996 and the code to the experiments is available at https://github.com/dhlab-nl/detecting-smell-experiences-in-novels . this work is a preliminary result of the odeuropa project which will commence formally in january 2021: https://odeuropa.eu/ . ## related work the cultural significance of smells is a niche topic in the humanities domain, but one that has recently gained more interest with a translation of muchembled's 2017 la civilisation des odeurs (xvie siècledébut xixe siècle) to english @xcite ) and @xcite smellosophy: what the nose tells the mind @xcite being reviewed in mainstream media. 2 while historians interested in olfaction such as @xcite analyse textual accounts of experienced smells, (computational) linguistic analysis of smell experiences (at least for english) has received little attention. this lack of attention may be due to the fact that western languages such as english and dutch do not contain rich vocabularies for describing odorants as opposed to some other languages. in @xcite jahai and english speakers and in @xcite jahai and dutch speakers were contrasted in describing a range of odorants. the jahai, a group of nomadic hunter-gatherers in malaysia, have over a dozen terms to describe odours. in the experiment, the jahai speakers were both more consistent and greatly more controlled in the terms they used than the english and dutch speakers relying on reference smell sources. there is evidence that consistency in the use of smell sources in english can be conditioned. croijmans and majid (2016) examined the accuracy and consistency of wine experts, coffee experts and people with no expertise in identifying smells. no group was better at naming smells outside of the domain of their expertise. however, it was apparent the domain experts had developed a toolkit of common smell sources they frequently drew upon. an experiment on predicting properties of wines from experts' wine reviews confirms this @xcite . sensorial lexicons have been developed that include terms related to smell (cf. tekiroglu et al. ( 2014 )). however, as the initial seed words to bootstrap the lexicon for smell are limited, the olfactory clusters in such lexicons are by extension also limited. a different approach is taken in @xcite , where terms related to smells are connected to their chemical compounds. while this is useful for translating olfactory information from the chemistry domain to language, it does not aid us in recognising the wide variety of expressions used in texts to describe smell experiences. ## literary smell dataset to begin to tackle the problem of recognising smell experiences, we created a unique dataset focused on such expressions in literary texts. specifically, by selecting texts from project gutenberg foot_3 that had the highest rate of occurrence of keywords derived from from the evaluation dataset, a gold standard of manually labelled sentences was assembled consisting of seven documents, each of 100 randomly assigned sentences from multiple literary texts and annotated by a single annotator. to evaluate the inter-annotator agreement between the three annotators, one additional document consisting of 100 randomly selected sentences was annotated independently by the annotators. despite having chosen the harvesting, validation and extract sets for their high frequency of annotators were asked to highlight and annotated spans according to the following criteria: the inherent subjectivity of when precisely a smell experience becomes a description is left to the perception of the annotator. additionally, two documents (of the aforementioned group of 7) were annotated with an additional set of tags: the inter-annotator agreement is measured using cohen's kappa @xcite per single gold standard document in a pairwise fashion. specifically: cohen's kappa is a metric used to measure pairwise inter-annotator agreement. a cohen's kappa of 0, denotes an even probability of agreement. landis and @xcite denote a cohen's kappa score of 0.41 to 0.60, and .61 -0.80 as representing moderate and substantial strength of agreement, respectively. a score of 0.81 to 1.0 can be considered as near perfect agreement. figure 1 shows the pairwise annotator agreement with regards the single gold standard document of 100 extracts, annotated by multiple annotators. all annotators are in substantial agreement in identifying all and any extracts that allude to smell, i.e., all spans labelled 'o' or 'd'. annotators are generally in substantial agreement in identifying extracts which describe smell experiences, i.e., extracts with a spans labelled 'd'. although, one pair of annotators are at the very upper end of moderate agreement only. finally, in identifying verbs either highly associated with smell, or associated with smell in the context, there was only moderate annotator agreement. it is reasonable to assume that human error, i.e., misreading, miscomprehending or simply skipping an extract, played some role in the observed imperfect inter-annotator agreement scores. instances of likely human error are apparent on inspection of the gold standard, in those instances where there is arguably little room for personal subjectivity, for example, one of the three annotators did not attribute either a 'd' or 'o' tag to the span: there was a smell of decaying leaves and of dog. however, a number of extracts clearly demonstrate the potential for subjectivity in smell experience interpretation, as a source for annotator disagreement, for instance, in the following extract, each of the three annotators attributed 'd', 'o' and no tag to it, respectively: seated beside her aromatic rest, in silence musing on her loveliness, her knight and troubadour. in the following extract, one of three annotators tagged it as 'o', the other two presumably thought it sufficiently descriptive to be tagged 'd': between each pair of columns an elegant table of cedar bore on its platform a bronze cup filled with scented oil, from which the cotton wicks drew an odoriferous light. as there is no known corpus annotated with smell expressions available to train a supervised smell language recogniser, we investigate pattern-based approaches to recognising such expressions. we base our approach on the concepts put forward in approaches such as the detection of hypernym-hyponym pairs @xcite , e.g., car is a type of vehicle, and iterative bootstrapping @xcite to detect author-book title pairs. we start with seed features that are matched against the harvesting dataset. the resulting matches are then manually evaluated to identify new linguistic pattens, which are in-turn used to identify new features for the next bootstrapping round. the process can be seeded by introducing known features into the lexicon, or known patterns into the pattern set. in both @xcite and @xcite , this process was used to assemble a lexicon, the pattern set being effectively a by-product of the process. however, for our purposes, it is the set of patterns that is of interest, and their potential use to identify smell experiences. the textual features targeted in prior work represented distinct real world concepts linked through a conceptual relationships. the expression of smell (in english) does not conform to a natural set of paired entities reflecting a relationship in the same way as authors and book titles do. however, surrounding adjectives, and verb and noun groups help characterise the smell experience and can therefore be targeted as complements. a difficulty of identifying smell references over hypernym-hyponym or author-book title pairs is that there is no inherent relationship defining the number of coincident complements necessary. instead it is a question of how restrictive we wish to make the criteria for matching sentences in the harvesting dataset. the greater the number of complements, the fewer extracts we can expect to retrieve. thus, a too restrictive choice may result in stalling bootstrapping process. conversely, if the choice is not restrictive enough and too many extracts unrelated to smell are returned, the process becomes uninformative. in evaluating smell related vocabulary, @xcite concluded that the most commonplace words used in smell descriptions are those that could apply to a wide range of sensory contexts. this makes intuitive sense given the heavy reliance on reference smell sources to define smell characteristics. hence, single complements are not targeted, as being too relevant outside of smell contexts. in this pilot project, we therefore focused on at least two complements in a pattern. a basic assumption in our approaches is that found complements are indicative of the presence of a smell expression. we therefore aim to detect the following types of smell expression complements: ## evaluation and discussion four complete cycles of the adjectives & nouns approach and three complete cycles of the verbs & nouns approach were performed. the adjectives & nouns approach identified 48 new identification patterns as seen in additionally, a small number of patterns identified do not involve the the verbs & nouns approach identified 31 new identification patterns as show in figure 2 shows the relative precision-recall performance of the group predictions, in respect of the gold standard, with regards the pattern sets of: approach 1; approach 2; and both combined. it is apparent that the adjectives & nouns approach (red) consistently outperformed the pattern set of verbs & nouns approach (black); and that they target complementary smell expressions (combined approach, green). the relative performance of the combined approach with that of the verbs & nouns approach. the combined approach is significantly better with a 5% significance level where patterns have a precision greater than 0.75. the combined pattern set significantly has superior recall performance at corresponding precision cut-offs. thus, we can conclude that targeting different feature pairs did result in patterns that target different smell extracts in the gold standard set. ## conclusions and future work in this paper, we set out to identify smell experiences in english literary texts. we created a gold standard dataset and our experiments demonstrated that iterative bootstrapping techniques can be used to identify smell experiences in text. whilst the overwhelming majority of identified patterns involved keywords and phrases with a high smell association, the implementations revealed a number of new phrases used in smell contexts. furthermore, we showed that at the very highest levels of precision, pattern group identification of smell experiences offers significantly better recall rates than a keyword search. the focus of application of iterative bootstrapping implementations was centred on single sentence extracts for english literary texts. it would be interesting to explore the applicability of this semi-supervised method to other textual contexts, and longer-distance relationships spanning multiple sentences. additionally, it would be informative to explore the influence of tweaking the implementation's parameters and approaches, including the pre-processing steps such as instead of using a shallow parser and experimenting with dependency grammars or semantic role labelling. with regards the precision-recall performance of the resulting pattern sets, such as exploring the influence of different seed words on the process outcomes, investigate the impact of a higher validation precision threshold for the number, and quality, of extracts returned each cycle and the corresponding identified patterns. our experiments demonstrated the potential use of patterns for identifying textual smell experiences in text. however, the number of extracts, and the quality of extracts in terms of smell experience density was identified as source of inefficiency which would benefit from being addressed further. the statistics presented in the observed high volume of low smell experience extracts may be an inherent challenge of smell experiences relying on vocabulary which is equally, or more so, applicable in other sensory contexts. i.e., the targeting and adding to the lexicon of words with a low smell association, resulting in poorer quality extracts. however, there are a number of clear, possible avenues to explore such as increasing the validation set size, ensuring more accurate precision estimates thus improving the level of smell association of lexicon entries on average and exploring the effects of using more coincident features simultaneously, i.e., pairs were selected on the basis that if one feature alone was weakly associated with smell, two together may improve the association. more coincident features may further improve the likelihood of an extract relating to smell. finally, consideration as to degree of agreement between people in their interpretation, suggested a less than perfect agreement not only of the subtly nuanced aspects of smell experiences, but even at recognition of smell experiences as a broad classification. on inspection of annotations, however, it is unclear how many of these were genuine discrepancies in terms of subjective perceptions. more annotators, supported by a more comprehensive approach to tagging, e.g., requiring the annotators to explicitly note their reasoning and deliberations would help. this would offer a window into the mind of the annotators, reinforcing any conclusions that may be drawn. whilst there is extensive scope for further study, the experiments in this paper have shown that smell experiences can be identified, and smell experience features can be extracted from text providing a useful foundation for understanding smell usage within literature.