# Analyzing User Reviews in the Tourism & Cultural Domain - The Case of the City of Athens, Greece

Tasos Papagiannis, George Ioannou, Konstantinos Michalakis, Georgios Alexandridis, George Caridakis

► To cite this version:

Tasos Papagiannis, George Ioannou, Konstantinos Michalakis, Georgios Alexandridis, George Caridakis. Analyzing User Reviews in the Tourism & Cultural Domain - The Case of the City of Athens, Greece. 19th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Jun 2023, León, Spain. pp.284-293, 10.1007/978-3-031-34171-7_22. hal-05419402

HAL Id: hal-05419402

https://inria.hal.science/hal-05419402v1

Submitted on 16 Dec 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Analyzing User Reviews in the Tourism & Cultural Domain - The Case of the City of Athens, Greece

Tasos Papagiannis<fnref n="1" />✉, George Ioannou<fnref n="1" />✉, Konstantinos Michalakis<fnref n="2" />✉,
Georgios Alexandridis<fnref n="3" />✉, and George Caridakis<fnref n="2" />✉

**Abstract.** Tourism is an important economic activity for many countries and the ability of understanding visitor needs as they evolve over time is a priority for all involved stakeholders. The analysis of textual reviews written by travelers on various online platforms may be a valuable tool in this direction. In this work, we showcase the potential of this idea by examining 8 well-known attractions in the City of Athens, Greece. After retrieving the relevant data from two popular online services, we employ a state-of-the-art transformer-based language model for two tasks; the extraction of distinctive keywords and phrases out of the free-text reviews and the assignment of a sentiment score to each review. Based on this information, we can associate certain keywords and phrases with specific sentiment values and monitor their evolution over time, in the context of specific touristic & cultural places. The analysis that follows explores the potential of this idea in more detail.

**Keywords:** sentiment-analysis · keyword extraction · free-text reviews · language models · transformers

## 1 Introduction

In many countries, tourism constitutes an important financial sector, displaying a significant potential each year. To better exploit its capacity and further maintain the observed growth rates, it is necessary to interconnect the touristic content with the interests of actual and expected visitors. Nowadays, the vast proliferation of mobile devices and the expansion of high speed data networks has turned tourists into active content creators, who take photos & videos of

<footnote n="1">¹ School of Computer and Electrical Engineering, National Technical University of Athens, Zografou, Greece
{tasos,geoioannou}@islab.ntua.gr</footnote>

<footnote n="2">² Department of Cultural Technology and Communication, University of the Aegean, Mytilene, Greece
{kmichalak,gcari}@aegean.gr</footnote>

<footnote n="3">³ Department of Digital Industry Technologies, National and Kapodistrian University of Athens, Psachna, Greece
gealexandri@uoa.gr</footnote>
the places they visit, write reviews & recommendations and in general, digitally interact with the cultural & touristic spaces.

As a consequence, vast amounts of relevant data are stored on tourist-based platforms, such as TripAdvisor, map services, such as Google Maps, online social networks, such as Instagram, Facebook and Twitter, as well as on travel blogs, forums etc. The aforementioned data may be of great value to all stakeholders involved in the tourism and cultural domain, ranging from businesses to regulation authorities and policy makers. Indeed, if properly processed, they can provide valuable insight on visitors' interests, preferences, as well as their evolution over time. It is obvious that this information can be very useful to all interested entities in addressing visitor needs and enhance the tourists' cultural experience, so as to meet their expectations. As a result, various tools have been developed over time that crawl the relevant information from the World Wide Web, apply data mining techniques and aggregate results, acting as decision support systems.

In this respect, the current work focuses on the case of the City of Athens in Greece, a place of paramount cultural & historical importance, that attracts thousands of tourists every year. The emphasis is placed on 8 well-known sites and museums with the data being crawled from two popular online platforms; TripAdvisor, which is tourist-oriented and Google Places, which is more business-oriented and is integrated into Google Maps. After an initial pre-possessing stage, the obtained reviews are fed into transformer-based language models in order to extract the most prominent keywords and phrases for each place. Additionally, the aforementioned language models are used to assign a sentiment score to each textual review [3], which, in turn, is associated with the rating provided by each visitor. Finally, positive and negative reviews are analyzed in order to uncover user/tourist preference and identify key issues, along with their evolution over time.

The rest of this paper is organized as follows; Sect. 2 overviews related work on tourism & cultural spaces-oriented data mining. Section 3 presents the methodology & the approach followed in the current work, while Sect. 4 discusses the obtained results. Finally, the paper concludes in Sect. 5.

## 2 Related Work

The application of data mining techniques on visitor reviews in the tourism and the cultural domain is an active and interdisciplinary research area [10]. The most common intention, which is also one of the objectives of the current work, is to reason on the aspects of tourist satisfaction [8,12] and explore their connection to other inter-related factors. Of course, the analysis is not limited to attractions but is also directed to hotels [9] and restaurants [18]. It could be argued that the aforementioned features constitute the three most important factors considered by visitors during their stay. Therefore, they play an important role in the hospitality industry, as well as in attraction marketing.

The examined data sources are quite diverse, ranging from travel blogs & tourism-oriented portals to domain-specific online social networks. TripAdvisor,
also considered in this work, is the most popular among them, predominately
due its specific focus and ease of access (crawling) [1,7]. Other, more rarely
used platforms include Yelp, Expedia & Booking [17]. Regarding online social
networks, Twitter is a straightforward choice, especially due to its access policy
and micro-blogging nature [4]. Researchers from China also prefer Sina Weibo,
the Chinese alternative to Twitter [5].

After retrieving the data from the selected sources, most researchers follow
a methodology similar to the one described in Sect. 3. Initially, the data are
cleaned; HTML tags, URLs, usernames, extra-white space etc are removed, in
order to obtain the useful part of the textual review. Then, especially in earlier
approaches [15], the text is further pre-possessed, with the removal of stop-words
and of very low frequency words. Following, the data are tokenized; that is they
are split into words, phrases or any other meaningful elements (tokens). At
the final stages, the extracted tokens are lemmatized/stemmed and labeled with
part-of-speech taggers prior to further analysis. On the other hand, most state-of-
the-art methodologies, directly employ language models after prepossessing [2],
an approach followed in this work as well.

Finally, a variety of techniques and methods are used for the actual analysis
of the obtained data. These may be simple, descriptive statistics [13], like those
presented in Sect. 3. Very often, keyword extraction techniques are considered,
an approach followed in the current work as well. In this respect, techniques
that identify “topics” in textual data, such as latent Dirichlet allocation [8],
are preferred. At a higher level lies text summarization that produces succinct
summaries of documents. For example, in [9] a relevant technique was used to
identify the most informative sentences in hotel reviews. Lastly, another common
modeling technique is sentiment analysis that can help determine visitor attitude
towards e.g. attractions [16] or accommodation [9].

3 Methodology

The main objective of the current work is to address knowledge extraction from
textual reviews of cultural & touristic places with respect to two different tasks;
keyword/phrase extraction and sentiment analysis. For this purpose, the relevant
review data are provided to deep learning language models in order to identify
possible correlations in-between them and draw an overall insight on visitor
opinion regarding specific popular tourist & cultural attractions.

## 3.1 Data

The data considered in this study are related to 8 well-known cultural places in
the City of Athens, Greece and they have been crawled by two popular sources
discussed in Sects. 1 and 2; TripAdvisor and Google Places. For each place, the
reviews and comments of the visitors were extracted, which consisted of ratings
(in the 5-star scale), a small comment (review in free-text form), the date of visit
and the date the comment was posted. The characteristics of the collected data
obtained from TripAdvisor are summarized on Table 1.
<table><caption>**Table 1.** Characteristics of the TripAdvisor dataset.</caption><thead><tr><td>Location</td><td>Number of reviews</td><td>Oldest Review</td><td>Newest Review</td></tr></thead><tbody><tr><td>Acropolis</td><td>20,346</td><td>June 6, 2005</td><td>January 9, 2023</td></tr><tr><td>Acropolis Museum</td><td>22,507</td><td>June 21, 2009</td><td>February 15, 2023</td></tr><tr><td>Ancient Agora of Athens</td><td>2,479</td><td>July 24, 2009</td><td>February 8, 2023</td></tr><tr><td>Benaki Museum</td><td>929</td><td>June 20, 2005</td><td>February 8, 2023</td></tr><tr><td>Museum of Cycladic Art</td><td>625</td><td>October 12, 2005</td><td>February 4, 2023</td></tr><tr><td>National Archeological Museum</td><td>4,286</td><td>April 11, 2007</td><td>February 8, 2023</td></tr><tr><td>Parthenon</td><td>9,586</td><td>January 20, 2006</td><td>January 12, 2023</td></tr><tr><td>Temple of Hephaestus</td><td>2,225</td><td>September 12, 2011</td><td>February 8, 2023</td></tr></tbody></table>

In the case of Google Places, the crawling procedure was not as straightforward, since for each attraction only 1,000 to 1,500 reviews were returned, either the most recent or the “most relevant”, as determined by the service itself. Despite this fact, we decided to consider this dataset in our analysis as well, in order to identify possible correlations in visitor behavior between the users of the two platforms.

A first observation on the characteristics outlined on Table 1 is that the obtained reviews span a large time period, which in all cases is longer than 10 years and in some cases (the Acropolis, Benaki Museum and the Museum of Cycladic Art) it is close to 20 years (the crawling took place in mid-February, 2023). This is of particular importance, as it allows the ensuing analysis to study the evolution of visitors’ beliefs and evaluations over time. Additionally, as it might have been expected, there is an obvious difference in the popularity of places, as quantified by the volume of the relevant reviews. Indeed, the number of reviews of the most popular place (the Acropolis) is two orders of magnitude bigger than those of the least popular place (Museum of Cycladic Art).

The overwhelming majority of the reviews obtained from TripAdvisor were written in the English language. As a consequence, in the data pre-processing stage, we disregarded reviews written in other languages. However, this was not the case for the data originating from Google Places, as it contained reviewers in many languages (e.g. English, Greek, French, German etc.). Nevertheless, given that the dataset obtained from Google Places was considerably smaller and therefore not as representative, we decided to keep the English language reviews in this case as well, for comparison purposes.

The final data pre-processing stage involved an initial “cleaning” of the remaining records (i.e. remove entries with no textual reviews, remove textual reviews with one or two words, etc.). Then, the ensuing data have been provided to the transformer-based language models discussed in the following Subsection.
## 3.2 Transformer-Based Language Model

For the keyword extraction and sentiment analysis tasks, state-of-the-art transformer-based language models are employed and more specifically, approaches that are extensions of the Bidirectional Encoder Representations from Transformers (BERT) architecture [6]. One such popular, pre-trained, transformer-based language model, suitable for Natural Language Processing (NLP) tasks such as question answering, text classification, and language translation is RoBERTa [11]. Its development involved several key innovations that distinguish it from its predecessors, like dynamic masking during pre-training, which allowed the model to learn more effectively from the text data. In this work, a distilled version of RoBERTa has been considered, with 6 layers, 768 dimension and 12 heads. This is inspired by DistilBERT [14], a distilled version of BERT.

Regarding the keyword extraction task, each review text is initially tokenized into words and phrases of up to four words, using *n*-grams of the corresponding length. Then, the ensuing tokens are being fed into the pre-trained model (RoBERTa) and are assigned to a numerical embedding. The same procedure is followed for the entire textual review in order to obtain the final representation (embedding). A comparison is made between the latter embedding and each of the candidate keyword embeddings, so as to calculate similarities. For this purpose, the cosine similarity metric has been considered and the phrases with a higher score were denoted as the most relevant keywords.

For the sentiment analysis task, the reviews of both datasets were fed into a pre-trained DistilBERT-based model in order to quantify visitor opinion and draw useful insights. This is accompanied by a confidence value of the latter classification. The confidence values range in [0, 1] and serve as a metric that measures the overall sentiment of a comment (for negative comments values are mapped to [-1, 0]).

## 4 Discussion

In this section, the results of the knowledge extraction process are displayed and reasoned upon, in order to interpret them in a meaningful way. The goal is to find general patterns in phrasing, as well as patterns regarding the sentiment of each comment. The analysis will be performed under two different scopes; a long term one and a seasonal one (e.g. detect periods when reviews are negative, find patterns concerning the season of the year etc.).

For this purpose, the reviews are grouped by the season they refer to and their sentiment score (in the [-1, 1] range) is extracted for the corresponding season. In order to be able to evaluate the credibility of the produced metric, sentiment scores are scaled to the [0, 5] range and are directly compared with the users' ratings (stars) for the relevant periods. The sub-figures of Fig. 1 depict the two metrics based on the reviews of each attraction. The two curves seem to have similar fluctuations and capture the overall trend per season in most cases. Also, it seems that the ratings are almost always higher, which shows that even
<figure><img src="image_3.png" /><figcaption>(a) Acropolis</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>(b) Acropolis Museum</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>(c) Ancient Agora of Athens</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>(d) Benaki Museum</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>(e) Museum Cycladic Art</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>(f) National Archaeological Museum</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>(g) Parthenon</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>(h) Temple Hephaestus</figcaption></figure>

<caption>**Fig. 1.** Ratings and adjusted sentiment scores per attraction.</caption>
<figure><img src="image_11.png" /><figcaption>(a) Summer 2011</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>(b) Summer 2014</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>(c) Autumn 2017</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>(d) Autumn 2021</figcaption></figure>

<caption>**Fig. 2.** Extracted key-phrases for different periods from Parthenon negative reviews.</caption>

though the reviews may contain negative comments, the overall assessment of the reviewer remains positive.

As stated earlier, in the knowledge extraction task the emphasis was placed on selecting the most suitable phrases (1-4 words) that better characterize the reviews received by each attraction. To combine it with the sentiment analysis task, we detect time periods when the sentiment score decreases and we search in these particular periods' reviews for negative keywords and phrases.

Specifically, in Fig. 1g we observed a substantial decrease in the sentiment score during the summer of 2011 for Parthenon. In order to identify possible causes, a sentiment analysis step has been performed to isolate negative reviews and subsequently extract knowledge from these reviews, in order to detect possible reasons for this trend. Figure 2a displays the predominant keywords in negative comments for that period, as well as some other later seasons (Figs. 2b to 2d), so as to evaluate the evolution of negative comments over time.
Regarding the summer of 2011 specifically, we can see various phrases, such as “slippery rocks”, “ppl pushing”, “broken columns”, as well as several references indicating the need for special shoes. Additional conclusions can be drawn for all periods of interest in a similar manner. In this way, knowledge extraction can assist in detecting issues raised by visitors and help the involved parties to address them.

Keyword extraction is also useful for long-term analysis, as seen, for example in Fig. 3 for the Acropolis Museum. It displays some general comments towards this specific attraction, consisting mostly of positive evaluations. An analysis like that is always helpful in quantifying the overall attitude of the people that visited an attraction.

<figure><img src="image_15.png" /><figcaption>**Fig. 3.** Most common keywords and phrases for the Acropolis Museum during 2010–2023.</figcaption></figure>

# 5 Conclusions and Future Work

In this work, an initial analysis of reviews in free-text form, posted on online platforms, of visitors of popular cultural & touristic places in the City of Athens, Greece, has been attempted. To this end, a large and representative dataset of 8 distinctive attractions has been crawled from two popular online services (TripAdvisor and Google Places) that spans several years. After pre-processing, the said dataset has been provided to RoBERTa, a state-of-the-art transformer-based language model, in order to achieve two tasks; keyword/phrase extraction
and sentiment analysis. Then, the obtained results have been used to identify
visitor opinion, both in a seasonal manner and as a whole. The presented exam-
ples showcased the insights that can be drawn from this kind of analysis, as well
as its overall potential.

The course of action presented in this work may be extended to other, less
popular touristic & cultural attractions in the city of Athens, as well as in the
broader Attica region. Of course, the consideration of even more sources, both
from online social networks and tourism websites, forums and blogs, is expected
to enrich the available datasets and form the basis for further research. Finally,
the presented approach can be a part of a larger architecture that constantly
collects data from multiple sources and functions as a decision support system
for interested parties and policymakers.

**Acknowledgements.** This research has been co-financed by the European Union and Greek national funds through the Regional Operational Program “ATTICA 2014-2020” of the National Strategic Reference Framework (NSRF) - Research Funding Program: Smart Tourism Recommendations based on Efficient Knowledge Mining on Online Platforms ATTP4-0349847.

References

1. A segmentation of online reviews by language groups: how english and non-english speakers rate hotels differently. Int. J. Hosp. Manage. **48**, 143–149 (2015). https://doi.org/10.1016/j.ijhm.2014.12.007, https://www.sciencedirect.com/science/article/pii/S0278431914001935

2. Arefeva, V., Egger, R.: When BERT started traveling: tourbert-a natural language processing model for the travel industry. Digital **2**(4), 546–559 (2022). https://doi.org/10.3390/digital2040030, https://www.mdpi.com/2673-6470/2/4/30

3. Barbieri, F., Espinosa Anke, L., Camacho-Collados, J.: XLM-T: multilingual language models in Twitter for sentiment analysis and beyond. In: Proceedings of the Thirteenth Language Resources and Evaluation Conference, pp. 258–266. European Language Resources Association, Marseille, France (2022). https://aclanthology.org/2022.lrec-1.27

4. Bordogna, G., Frigerio, L., Cuzzocrea, A., Psaila, G.: Clustering geo-tagged tweets for advanced big data analytics. In: 2016 IEEE International Congress on Big Data (BigData Congress), pp. 42–51 (2016). https://doi.org/10.1109/BigDataCongress. 2016.78

5. Cheng, M., Edwards, D.: Social media in tourism: a visual analytic approach. Curr. Issues Tourism **18**(11), 1080–1087 (2015). https://doi.org/10.1080/13683500.2015.1036009

6. Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Burstein, J., Doran, C., Solorio, T. (eds.) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2–7, 2019, Volume 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/n19-1423
7. Fang, B., Ye, Q., Kucukusta, D., Law, R.: Analysis of the perceived value of online tourism reviews: influence of readability and reviewer characteristics. Tourism Manage. 52, 498–506 (2016). https://doi.org/10.1016/j.tourman.2015.07.018, https://www.sciencedirect.com/science/article/pii/S0261517715001715

8. Guo, Y., Barnes, S.J., Jia, Q.: Mining meaning from online ratings and reviews: tourist satisfaction analysis using latent dirichlet allocation. Tourism Manage. 59, 467–483 (2017). https://doi.org/10.1016/j.tourman.2016.09.009, https://www.sciencedirect.com/science/article/pii/S0261517716301698

9. Hu, Y.H., Chen, Y.L., Chou, H.L.: Opinion mining from online hotel reviews - a text summarization approach. Inf. Process. Manage. 53(2), 436–449 (2017). https://doi.org/10.1016/j.ipm.2016.12.002, https://www.sciencedirect.com/science/article/pii/S0306457316306781

10. Li, J., Xu, L., Tang, L., Wang, S., Li, L.: Big data in tourism research: a literature review. Tourism Manage. 68, 301–323 (2018). https://doi.org/10.1016/j.tourman.2018.03.009, https://www.sciencedirect.com/science/article/pii/S0261517718300591

11. Liu, Y., et al.: Roberta: a robustly optimized BERT pretraining approach. CoRR abs/1907.11692 (2019). http://arxiv.org/abs/1907.11692

12. Liu, Y., Teichert, T., Rossi, M., Li, H., Hu, F.: Big data for big insights: investigating language-specific drivers of hotel satisfaction with 412,784 user-generated reviews. Tourism Manage. 59, 554–563 (2017). https://doi.org/10.1016/j.tourman.2016.08.012, https://www.sciencedirect.com/science/article/pii/S0261517716301534

13. Racherla, P., Friske, W.: Perceived ‘usefulness’ of online consumer reviews: an exploratory investigation across three services categories. Electron. Commerce Res. Appl. 11(6), 548–559 (2012). https://doi.org/10.1016/j.elerap.2012.06.003, https://www.sciencedirect.com/science/article/pii/S1567422312000464, information Services in EC

14. Sanh, V., Debut, L., Chaumond, J., Wolf, T.: Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter. CoRR abs/1910.01108 (2019). http://arxiv.org/abs/1910.01108

15. Xiang, Z., Schwartz, Z., Gerdes, J.H., Uysal, M.: What can big data and text analytics tell us about hotel guest experience and satisfaction? Int. J. Hosp. Manage. 44, 120–130 (2015). https://doi.org/10.1016/j.ijhm.2014.10.013, https://www.sciencedirect.com/science/article/pii/S0278431914001698

16. Xu, H., Yuan, H., Ma, B., Qian, Y.: Where to go and what to play: towards summarizing popular information from massive tourism blogs. J. Inf. Sci. 41(6), 830–854 (2015). https://doi.org/10.1177/0165551515603323

17. Xu, X., Li, Y.: The antecedents of customer satisfaction and dissatisfaction toward various types of hotels: a text mining approach. Int. J. Hosp. Manage. 55, 57–69 (2016). https://doi.org/10.1016/j.ijhm.2016.03.003, https://www.sciencedirect.com/science/article/pii/S0278431916300202

18. Zhang, Y., Cole, S.T.: Dimensions of lodging guest satisfaction among guests with mobility challenges: a mixed-method analysis of web-based texts. Tourism Manage. 53, 13–27 (2016). https://doi.org/10.1016/j.tourman.2015.09.001, https://www.sciencedirect.com/science/article/pii/S0261517715300054