# "From meta-data to research-data. How to use text analysis techniques to do Information retrieval as a propaedeutic practice to Data Literacy"

Deborah Grbac

► To cite this version:

Deborah Grbac. "From meta-data to research-data. How to use text analysis techniques to do Information retrieval as a propaedeutic practice to Data Literacy". JADT 2022, Jul 2022, Napoli, Italy. pp.453-460. hal-04643944

HAL Id: hal-04643944

https://hal.science/hal-04643944v1

Submitted on 12 Jul 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# From meta-data to research-data. How to use text analysis techniques to do Information retrieval as a propaedeutic practice to Data literacy

Deborah Grbac¹

¹Library, Università Cattolica del Sacro Cuore di Milano – deborah.grbac@unicatt.it

## Abstract

Thanks to recent evolution of information technology and computational methods, text analysis is becoming a widespread tool adopted in different sectors and research disciplines, no more confined to computational linguistics. In particular ‘distant reading’, or computer-aided tools to retrieve information from texts, can be used in libraries too. Moreover, all the different phases and skills to be implemented to do a distant reading (collecting data, analysing data, retrieving relevant information from data, visualize results, finding out trends or patterns) have the further benefit to oblige to start to work with data, or in other words, text analysis is a good example to be used by librarians to start to offer some Data literacy courses to students.

**Keywords:** Information retrieval, text analysis, data literacy, research data

## 1. (Meta-)data literacy at the library

Dealing with metadata, or data containing information about bibliographic references can be the first activity to introduce students to Data literacy, or all necessary knowledge and skills to do research using data, from a Data science perspective. In the past only software managing bibliographic data were at disposal and they were presented to students as a tool to automatically download citations. That software does not allow to work directly with metadata, which are only harvested and collected in a personal account to create a customized dataset reusable to draft bibliographies in different bibliographic (citation) styles.

Instead, nowadays, statistical text analytics, using words frequencies, applied to metadata and texts to discover contents (Information retrieval) can also be used to gather metadata to build up datasets containing bibliographic references, selected according to a specific search query. A collection of metadata, to which further texts analytics can be added, can be used as a quantitative support for scientific conclusions, or a statistically meaningful proof sustaining a theory; in a time when research is becoming more and more a data-driven one.

Corpora analytics, borrowed from linguists, such as stylometric based on ngrams calculation, normally used for authorship's attribution, can be used under the 'distant reading' approach to 'read' through statistical measures a whole database, made of references and corresponding full text, tackling it as a 'bag of words'. For distant reading we mean Franco Moretti's theory, nowadays a paradigm, or automatically reading great quantities of electronic texts (full-texts collections), through computational analyses producing statistical measures (Moretti, 2013). By 'distant reading' we can gather some ideas on the contents (preliminary analysis), but we are not yet properly analysing contents (contents are extracted instead by a corpora analysis). Looking at distance an entire database could be useful to do some information retrieval, if information needs strictly correspond to database contents, only in this case, words retrieved
by statistical measures will in some ways resonate (semantic meaning) with effective
contents.

## 1.1 Information retrieval vs. corpora analysis

Thanks to the big number of documents at disposal on electronic format nowadays, a researcher, who has selected a collection of texts, according to a stated query on a database, is able to retrieve some hints on contents, mainly on the statistical and linguistic point of view. Or through lexicometrics measures (words frequency, co-occurrence, ngrams, ...) it is possible to infer trends and evolutions made explicit by customized visualisations which present data to support or disprove a theory or a model taken from the reality, if texts are coherent and meaningful.

Those analyses became corpora analysis, or a 'corpus-based' approach, when computational assistance is applied to an assembled corpus, not only with statistically meaningfulness (as above), but a corpus build to answer a specific question and duly gathered for this purpose. Building a corpus allows to make a deeper analysis also according to contents, because corpora can be processed through for instance some cleaning techniques. The 'cleaning' allows to get rid of the 'rumours' and to make a better analysis in terms of words and their 'semantic' content. The difference between using statistic methods on texts to do a distant reading (a bird's eye approach) most of the time according to a query, or contents' analysis (meaning) done choosing every text to be part of a corpus, is that one has to go back and forth from statistical numbers to 'semantic' meaning of words and vice versa (Tuzzi, 2003).

The difference is not a minor one because when you apply 'distant reading' techniques to databases made of references and full texts, first of all contents are not picked up from a chosen corpus, but they are automatically extracted by search engines, and secondly contents are continuingly growing and diminishing. Each month new periodicals' issues may be added to the database, or subtracted, according to the library subscriptions 'moving-wall' or agreed changing availability in contents during the time. These two conditions or the adding and subtracting of texts make the dataset produced by the query very changeable day after day and because of this different than a chosen corpus.

However, different databases are recently offering text analytics to browse contents by a query, creating a little bit of puzzle around on the theoretical and practical point of view. They are using both 'topic modelling' for texts classification and TF-IDF ratio to rank results. The application of these text analysis techniques can obtain some preliminary analysis or exploring ('discover') contents and 'scoring' ('relevance') results. If some contents can be inferred without reading texts ('distant reading'), results still have to be validated, as it can also happen with the classic "exact match" approach ensured by index search done on database contents.<fnref n="1" />

Those instruments are offered with the idea of enabling to use some text analysis techniques embedded to the database, avoiding recurring to specific external software (as external "search engine", or other software doing texts analysis automatically) and implementing

<footnote n="1">¹Another example of database using text analysis techniques to suggest “key words” to users in exploring and discovering contents is the platform under construction GoTriple a multilingual and multicultural discovery solution for the social sciences and humanities. Retrievable at: https://www.gotriple.eu/.</footnote>
instead customized notebooks (to databases' contents and user's needs) hosting the coding necessary to do the statistical analysis on data directly browsed from the database.

Regardless of which data can be downloaded and which analysis can be done, all those instruments are examples of how to use data to do research or data that will produce further research data (publications, datasets, protocols). Although, under a reference perspective it is essential to help students to understand what those different instruments are offering.

## 1.2 Data literacy should take into consideration existing caveats

When doing information retrieval in an automatic way, it is also essential to verify some points, because the easiness in retrieving results must be counterbalanced by a critical approach. Some questions have to be taken into consideration. Which resources are used to do the analysis? Resources are at disposal because they are subscribed, open access? Which are the contents, databases are multi or mono disciplinary? Which is the time range coverage, are they archives or collections of current texts?

Which and if database contents can resonate to the query formulated by the user? Is the key words strategy sufficient to retrieve results? Are contents numerically enough to be statistically significant? Statistical measures are working only if there is a big quantity of data at disposal, is there enough data to make a sensible analysis? Or databases consulted are enough to give a picture of the state of the art in the topic?

Which instruments are used to do the text analysis? In the past textual analysis software packages offered an all-in-one approach to do the analysis, users had only to do the query, without knowing how the software was working. Nowadays this approach is no more feasible because if science has to be reproducible (Open Science), it is also necessary to know which analyses are done and the steps followed to come to a result (protocols, codebooks, ...). This is part of the 'research integrity' essential to do data-driven and reproducible research.

It is also important to know exactly which are the limits of the analyses that can be done because this has consequences on the results, or if there is a 'moving-wall' in contents availability, if a good OCR work has been applied to texts, if there is the opportunity to customize the analysis according to needs, for instance, linguistic ones, or the necessity to use a particular dictionary of words, if entity recognition is present or it can be modified.

Another important point is the use of authoritative sources as taxonomies to help the analysis, or the importance of *thesauri* and ontologies, all instruments that request an investment in time and money, but in case of less huge corpora, that can help in some ways to get a better result, counterbalancing a lack of significant statistical meaningfulness.

Finally, but not least, how to interpret the results? Which is the meaning of visualisations proposed? Are they clear to be interpreted by everyone? How to communicate results having particular attention to open data, but also data obtained under licence (also a CC BY licence) or data at disposal against subscription? This is also an important point in an era when digital and data skills are still lacking (also on copyright and licence issues), but the offer of data is continuously growing.

## 2. Some examples of text analysis used to do Information Retrieval
We will present in the following two instruments showcasing what can be done with text analysis for information retrieval, both instruments offer users an interface and analyses are done automatically, only results are shown, but information on their working is retrievable.

In 2017 Jstor Labs at Ithaka<fnref n="2" /> introduced one of the first instruments to do information retrieval in all the Jstor Archive database using text analysis, the instrument is called “Text Analyzer”.<fnref n="3" /> Users can simply upload a document in any format and language and retrieve further reading suggestions from the database contents. The uploaded text is read by two different topic modelling techniques an unsupervised and a supervised one. The last one recurs to a *thesaurus* (a set of humans’ tagged labels) helping to identify concepts, entity recognition is possible (named entities: persons, locations, organizations), finally, another instrument helps to manage data. The result is the extraction of a list of terms that the user can adjust according to personal information retrieval needs and further use to search on the database. Proposed readings suggestions are scored by the TF-IDF calculation and are retrieved according to the co-occurrence of reading suggestions’ terms with the terms extracted from the uploaded text (Snyder R. (2017).

A couple of years later, end 2019, a similar instrument was proposed by DESA (Department of Economic and Social Affairs) of the United Nations General Secretariat in cooperation with the United Nations Dag Hammarskjöld Library in New York. The instrument is based on the linked data technology, but it also uses some text analysis techniques.<fnref n="4" /> Text analysis techniques aim at identifying the presence of SDGs (Sustainable Development Goals)<fnref n="5" /> inside texts by a semi-supervised topic modelling trained with a set of data,<fnref n="6" /> whose analyses have been further implemented using three authoritative sources: the former UNBIS (United Nations Bibliographic Information System), the European Union’s legislation *thesaurus* called EuroVoc, and the SDGs (Sustainable Development Goals) taxonomy created by DESA (Department of Economic and Social Affairs).<fnref n="7" />

The DESA’s taxonomy is also the interface to query the SDGs’ ontology called SDGIO (Sustainable Development Goals Interface), whose aim is to draft a machine-readable representation of human knowledge in the SDGs’ domain, to guide information retrieval, data extraction and analysis.<fnref n="8" />

<footnote n="2">²Jstor Archive and Jstor Labs are part of Ithaka a not-for-profit organisation preserving scholarly records (Jstor Archive) and proposing innovative pedagogic and research methods (Jstor Labs). Jstor Archive, one of the most ancient datasets containing references and full texts, is retrievable at: https://www.jstor.org/, Jstor Labs are retrievable at: http://labs.jstor.org/.</footnote>

<footnote n="3">³The “Text Analyser” is retrievable at: https://www.jstor.org/analyze/.</footnote>

<footnote n="4">⁴The “Linked data service” is retrievable at http://metadata.un.org/?lang=en.</footnote>

<footnote n="5">⁵Sustainable Development Goals, or SDGs, are 17 targets that have been adopted by United Nations Member States in 2015 to be realized by 2030. More information on SDGs is retrievable at: https://sdgs.un.org/goals.</footnote>

<footnote n="6">⁶The first Topic modelling experiment was done on a set of 267 individual DESA publications. (LaFleur, 2019).</footnote>

<footnote n="7">⁷The UNBIS *thesaurus*, now substituted by the United Nations Digital Library, is managed, and maintained by the Dag Hammarskjöld Library and retrievable at: http://metadata.un.org/thesaurus/?lang=en. The EuroVoc *thesaurus* is managed by the European Union Publications Office and retrievable at: https://op.europa.eu/it/web/eu-vocabularies.</footnote>

<footnote n="8">⁸The ontology has been created and it is maintained by the UNEP’s (United Nations Environment Programme) SDG and Environment Statistics Unit. (Coppens & Ghafary, 2018).</footnote>
All those instruments have been applied to map and retrieve data and information on SDGs in the continuingly growing corpus consisting of all the texts produced by the United Nations. To have an idea about how it works a demo app is proposed called the “Linked SDG” service. When uploading a file or a web address, whose content is related to SDGs, the app uses controlled vocabularies, to extracts geographic entities, main concepts related to sustainable development and SDGs present in the uploaded text and it clusters them according to the 17 SDGs goals and sub-goals.<fnref n="9" />

## 2.1. Some examples of open research data and open education

At the beginning of 2022, after two years spent on working on the prototype, Jstor Labs launched the Constellate service, a platform for learning and performing text analysis, building datasets, and sharing open education analytics course material.<fnref n="10" /> When accessing to the platform everyone is free to build one's dataset up to 25,000 units (ten dataset per day). Datasets consist of bibliographic data (metadata), language statistics measures (ngrams) and full texts available in open access (after request). Data visualizations are also at disposal: presentation of world frequencies in chronological order (time series), subject categories distribution in time, categories' tree-map, words bubble for terms frequencies. At least one visualisation is free for everyone.

Besides metadata and linguistic measures (ngrams, bigrams and trigrams) downloadable for free as open research data, the most interesting feature consists in the set of open education Jupyter notebooks written to have some clear and explained command line codebooks to do some text analytics autonomously. Proposed features are extracting bibliographic references (metadata), calculating world frequencies, scoring words (TF-IDF), discovering contents (topic modelling).<fnref n="11" /> The notebooks are a set of instructions for a beginner in Digital Humanities downloadable from GitHub free for all, but also a computational space where to upload personal notebooks for more advanced users, this feature under subscription.

The Pierre Auger Observatory Open Data release.<fnref n="12" /> Launched in October 2021 the initiative is another example of data education and open research data, the release is a tool enabling users to download data and to access to open education Jupyter notebooks helping users to do some autonomous analyses with those data. This is possible by a dedicated website hosting a framework to help people to work with data with all explanations and a set of simple analyses on Python notebook (a couple of tutorials on how to use data and five notebooks as an example reproducing results).<fnref n="13" />

<footnote n="9">⁹The “LinkedSGD.org” demo app is retrievable at: http://linkedsdg.apps.officialstatistics.org/#/.</footnote>

<footnote n="10">¹⁰The Constellate platform is retrievable at: https://constellate.org/.</footnote>

<footnote n="11">¹¹ Notebooks are set by default to work with 1.500 item taken from the dataset, but it is possible to broaden the range to work on the full length of the dataset available, or 25.000 items for free and 50.000 items under subscription.</footnote>

<footnote n="12">¹²The Pierre Auger Observatory’s website: https://opendata.auger.org/#:~:text=The%20Pierre%20Auger%202021%20Open,Auger%20Collaboration%20Open%20Data%20Policy.</footnote>

<footnote n="13">¹³ Data at disposal are: 10% of the Observatory’s cosmic-ray data and 100% of weather and space data.</footnote>
### 3. Data literacy for librarians, but not only

At the beginning of this paper, we have talked about how Data literacy can help Library users, but how librarians could be trained to offer Data literacy to users? Or which is the Data literacy for librarians?

In the past an initiative called “Library Carpentry” helped librarians to start to learn how to use computational instruments and methods as: coding, organizing datasets, analyse data. At first to learn contents one has to enrol, but since 2019 course materials have been made open to all under a CCBY licence.<fnref n="14" />

More recently, an initiative coming from librarians and addressed to librarians has been proposed by the Federal Reserve Bank of St. Louis,<fnref n="15" /> Research Division “Data Literacy for Librarians”.<fnref n="16" /> An online course developed by the Library Services and Economic Education teams, offering a professional development program using the open research data service called FRED<fnref n="17" /> to build capacity “to use data effectively and accurately”.

Seven areas have been identified to develop competencies on Data literacy or in general to find and reuse data. They are Information retrieval, authoritative sources, measures and the methodology, visualization of quantitative data, communicate data, make decision with data and ethics.

All the above-mentioned areas of data literacy are part of the ‘research data life cycle’. If we deal with data, irrespective of the scientific discipline (linguistics, economics, or astronomy), the process is always the same: finding, collecting, analysis, communicate, take decision with data. Therefore (meta)Literacy is not only propaedeutic to research data, but it is part of it, and it is a knowledge that has to be known by everyone, not only librarians.

### References

Coppens L., Ghafary D. (2018) *Sustainable Developments Goals Interface Ontology (SDGIO). Progress*. 24th September 2018, retrievable at: https://unstats.un.org/unsd/unsystem/Documents-Sept2018/Presentation-SDGIdentifiers-UNEP.pdf

LaFleur M. T. (2019). *Art is long, life is short: an SDG Classification System for DESA publications*. DESA Working Paper No. 159, United Nations Symbol ST/ESA/2019/DWP/159

Moretti, F. (2013). *Distant reading*: London, New York, Verso

Snyder R. (2017). *Under the Hood of Text Analyzer*. Published on 7th March 2017, retrievable at: https://labs.jstor.org/blog/under-the-hood-of-text-analyzer-2/

<footnote n="14">¹⁴The Library Carpentry initiative website: https://librarycarpentry.org/lessons/.</footnote>

<footnote n="15">¹⁵One of the 12 regional Federal Reserve Banks part of the decentralized central bank system having reserves and decision making distributed across the United States.</footnote>

<footnote n="16">¹⁶The initiative is retrievable at: https://research.stlouisfed.org/info-services/data-literacy-for-librarians/.</footnote>

<footnote n="17">¹⁷FRED, Federal Reserve Economic Data is a free online database containing economic data time series and tools to help users “to understand, interact with, display and disseminate data”, with annotations helping to “understand and interpret data”. Retrievable at: https://fred.stlouisfed.org/#.</footnote>
Tuzzi A. (2003). *L'analisi del contenuto: introduzione ai metodi e alle tecniche di ricerca*, Roma, Carocci